Announcing: MapGuide Open Source 3.1.2

I am pleased to announce the availability of MapGuide Open Source 3.1.2

The notable changes in this release are detailed below:

Updated Web Tier Components

MapGuide Open Source bundles updated web tier components:

• Apache HTTPD 2.4.37
• Tomcat 7.0.92
• PHP 5.6.40

Since PHP 5.6 is now end-of-life, this is most likely the last release in the 3.1 branch as we work to bring PHP 7 support for the MapGuide API in the next major release.

Functional SHP FDO provider for 64-bit Linux

Over the new year, I had set out to solve one of the last remaining blockers that prevented us from having a functional 64-bit SHP FDO provider for 64-bit Linux.

I had speculated that the blocker is due to incorrect data size assumptions in the SHP spatial indexing code on 64-bit Linux which would throw off spatial index queries. Intensive debugging sessions proved this to be the case and I was finally able to land the long-awaited fixes which is now available with the SHP provider bundled with this release.

The proof that the provider was truly functional was being able to load a SHP-heavy dataset into a 64-bit MapGuide Server and being able to pan, zoom, select and buffer SHP features without any errors or features disappearing in and out at various zoom levels (due to previously out-of-whack SHP spatial index queries)

Experimental King.Oracle FDO provider for Linux (32 and 64-bit)

Since I got a major win with one Linux FDO blocker and was on a roll, I figured we can do a double-whammy with the other major Linux FDO blocker: The lack of a functional King.Oracle FDO provider on Linux.

For this one I had speculated that the problem was due to heavy usage of wchar_t for strings which (due to different size assumptions on Linux) meant that most strings going in and out of the provider on Linux were garbage, meaning the SQL queries we think we were sending to OCI for execution were completely scrambled.

Due to the lack of solid examples of using OCI with wchar_t strings on Linux, I figured the code as it currently stands will never work on Linux. So I took the page out of a reference implementation I knew to definitely work on Linux (the OCI driver for GDAL/OGR) and wholesale #ifdef'd the entire provider codebase to use narrow (char) strings for Linux and their expected OCI functions, while retaining wide (wchar_t) strings and current OCI function usage for Windows.

The ability to easily spin up an Oracle XE environment (well until recently at least) meant that I was able to quickly verify that switching to narrow strings on Linux was indeed the way to go. This also provided motivation for me to implement a basic (and repeatable) unit test suite for this provider, finally giving us an objective baseline level of validation and verification for this provider.

The end result of all this is that the King.Oracle provider now not only builds on Linux, but it now finally works!

However, I'm certain that there will be bugs in this provider for Linux waiting to be revealed, so the provider gets the "experimental" label. Use at your own risk/discretion. Please do report any issues you find.

OGR FDO Provider Improvements

The OGR provider has undergone many "quality of life" improvements:

• We've cleaned up various correctness issues around provider behavior. For example, attempting to access invalid property names in an OGR feature reader will now throw instead of silently failing before.
• The provider supports a new DefaultSchemaName property that lets you specify a default schema name other than OGRSchema
• The provider also supports a new DataSourceEncoding property to allow you declare the encoding for an OGR data source where it cannot be inferred by default by the OGR library so that properties/names/etc with non-english characters are rendered properly.

Fusion Changes

• We've introduced support for a new opt-in server-side entry point instead of accessing templates through their individual index.html files
• We've also fixed the long non-functional support for XYZ tiles.
• We've fixed Google Maps support yet again.
• The SelectionPanel widget can now support a custom date/time formatter

Other Changes

• We've added new options in webconfig.ini to allow one to apply global limits for various feature query operations in the mapagent.

Download/Release Notes

Announcing: vscode-map-preview 0.4.6

A kind user alerted me to the fact that my Map Preview extension for Visual Studio Code was using APIs that have long been deprecated and could be removed any moment in a future release, breaking this extension in the process.

Although I haven't touched this code in a while (not intentional btw. One person can only tend to so many different open source projects), getting back into VSCode extension development was a very comfortable affair:

• Update a few packages here and there. 
• Update TypeScript. 
• Read up on the new Webview API, which is what our extension should be using now.
• Replace usages of vscode.previewHtml with the new Webview API.
• Verify the extension still works
• Get re-acquainted with how to publish VSCode extensions

And ... voila! A new version of the Map Preview extension is available for your VSCode installation to auto-update to.

In terms of changes, beyond some updates to OpenLayers and a small bug fix or two, there isn't really much to write home about, it should still be the same extension as before. This release was driven primarily by the need to move away from their old vscode.previewHtml API over to their new Webview API.

Once things have settled down on my other projects, I'd like to give this project some more attention. There's still some cool ideas I want to explore with this extension. I don't know if I still want to go down the Cesium route as previously hinted as there have been many more data viz players that have arrived on the scene since then and I also sort of do not want to detract from the original goal and purpose of this extension: An easy way to preview on a Map textual content that you are already editing or viewing in VSCode.

MapGuide tidbits: Updating coordinate system dictionaries

DISCLAIMER: This procedure is unsupported. Use at your own risk.

The coordinate system dictionaries that ship with a MapGuide installation are effectively coupled with that particular version of MapGuide. Should support for new coordinate systems, datums, etc be introduced into CS-Map, ideally you should upgrade your version of MapGuide to one that uses a version of CS-Map that incorporates these new definitions.

However, since the next major of version of MapGuide will still be some time away and the next immediate release (3.1.2) is a minor release, which we're being conservative with regards to updates to upstream libraries, it means there's a potential long wait to be able to use some coordinate systems, datums, etc if they were added to CS-Map since our last major release of MapGuide.

So if you can't afford to wait, you can try this method of updating your coordinate system dictionaries. We assume you are using the current latest version of MapGuide (3.1.1).

Firstly, use a subversion client to check out the current CS-Map source code from trunk.

svn co http://svn.osgeo.org/metacrs/csmap/trunk

Once checked out, you will need to build the CS_Comp.exe dictionary compiler. To build this dictionary compiler, open the OpenSource.sln file (under any VC* folder) in Visual Studio and build. If you do not have Visual Studio to build this executable, you can download this pre-compiled binary.

Either way, make sure the CS_Comp.exe is in your Dictionaries folder.

Then open a command prompt, navigate to this folder and run the following command:

CS_Comp.exe /b /c . .

You should see output like this

This produces a whole series of files with a .csd extension

Now locate the Dictionaries folder of your MapGuide installation and copy all the .csd files and NameMapper.csv into it, overwriting all existing files. In case things go wrong, you should make a backup of the existing files first.

Some new definitions may require new supporting grid files (eg. These files are required for the new Australian GDA2020 datum). Such files have to be copied across as well.

Once you have copied these files over, (re)start your MapGuide Server.

Now there's 2 ways to verify the new coordinate systems, datums, etc are in place.

Firstly, in MapGuide Maestro, when bringing up the coordinate system picker. You should see newer definitions for your particular country.

Secondly, as a sanity test you write a simple PHP script that verifies you can transform coordinates into these new coordinate systems. Here's an example script:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

<?php MgInitializeWebTier(dirname(__FILE__)."./webconfig.ini"); $csFactory = new MgCoordinateSystemFactory(); $srcCs = $csFactory->CreateFromCode("LL84"); //Map Grid Australia, Zone 55, with new GDA2020 datum $dstCs = $csFactory->CreateFromCode("MGA/20-55"); $xform = $csFactory->GetTransform($srcCs, $dstCs); $srcX = 144.9674853; $srcY = -37.808262; $txCoord = $xform->Transform($srcX, $srcY); echo "Lng: $srcX, Lat: $srcY"; echo "<br />"; echo "X: " . $txCoord->GetX() . ", Y: " . $txCoord->GetY(); ?>

Save this and when you run it, if it prints out the transformed coordinates then your new coordinate systems should be working.

Some caveats

The key thing to watch out for here is whether a new coordinate system, datum, etc requires only a data file update or it requires both a data file update and an engine update

A new definition that only requires a data file update, will only touch one or more of the following files:

• Any file with an .asc extension
• NameMapper.csv

For example, this commit to CS-Map that introduces some new coordinate systems only touches the aforementioned data files.

New definitions that require engine updates are indicated by commits that also involve modifications to one or more *.c source files.

Such definitions would require building MapGuide against a newer version of CS-Map with these engine changes, which only happens with every new major release.

Based on my cursory testing, definitions that only require a data file update can be incorporated into your current MapGuide install with this approach. Otherwise, such a definition is unlikely to be usable until we release the next major version of MapGuide.

How would you know if a given coordinate system, datum, etc only requires a data file update or both? The only thing I can suggest is to hit up the CS-Map trac instance and search for commits involving your coordinate system, datum, etc and see if they change the above aforementioned files.

A short MapGuide poll: What you said

A month-and-a-half ago, I put up a short poll regarding DWF support in MapGuide.

I got 97 responses in the time window that I left that poll open, and this is what you said.

Regarding whether you use this DWF support in MapGuide:

I also posed a hypothetical situation where we removed DWF support from MapGuide and asked how badly you'd be affected as a result:

My motivation for this poll was simple. I believe that DWF support in MapGuide is now a technical burden that is not worth carrying on.

This feature is terminal. You won't see any DWF-related enhancements (not from me anyways). Any DWF-related issues still open will most likely never be fixed. Because I doubt that anyone would be familiar enough with the underlying DWF toolkit library to keep maintaining this support as it stands.

Also, presumably this DWF support was originally put in as value proposition from Autodesk to integrate with AutoCAD and friends. Well, since they've officially bailed out at the beginning of this year, that is yet another reason I'm cold on DWF support in MapGuide. Given my dwindling free time on anything MapGuide related, I'd rather spend it (wherever I can) on making MapGuide a better GIS/WebMap server than trying to maintain support for technologies that Autodesk themselves have given up on.

And judging by these poll results, I would gather that you (the poll respondent) would mostly agree with my sentiments.

A short MapGuide poll

If you are a MapGuide user/developer reading this post, I have a short survey I'd like you to take part in.

I'll leave submissions open until the end of this month which should be enough time to gather a good representative opinion on this matter.

Announcing: MapGuide Maestro 6.0m10 and a new build of Fusion

This announcement is a double-whammy because they are both somewhat intertwined.

Firstly, we'll start with a new build of Fusion that cleans up several aspects of the viewer framework. These changes in Fusion are slated for inclusion in a future release of MapGuide

New PHP entry point for fusion templates

This new build of Fusion introduces a new PHP entry point for the 5 fusion templates.

Instead of loading say ... the slate template like so:

http://servername/mapguide/fusion/templates/mapguide/slate/index.html

You can now load it like so:

http://servername/mapguide/fusion/templates/mapguide/index.php?template=slate

This entry point supports all the same query string parameters as the original templates but with support for the following additional parameters:

• template (required) - The name of the template to load 
• debug (optional) - If set to 1, the entry point will use fusion.js instead of fusionSF-compressed.js, making the whole debug/development process way more simpler.

The entry point also fetches the application definition and writes it out to the underlying template as JSON, avoiding the need for the initial round trip to fetch this document among other things.


Fixing external base layer support 

The way external base layers are currently supported in Fusion is a bit crufty which this build also addresses:

• Fusion no longer attempts to append script tags to support Google Maps / Bing / OSM / Stamen. The way we do it is unsafe according to modern browsers. That is now the responsibility of the new entry point.
• For Bing / OSM / Stamen, we no longer load external helper scripts at all. This is because such support is already present in OpenLayers itself and the external scripts are merely convenience wrappers that we can easily implement ourselves in Fusion.
• Finally, XYZ layers are now properly supported in Fusion. This means you can do things like consuming OSM tiles from your own OSM tile server, or maybe use non-watermarked CycleMap/TransportMap layers, or you can finally be able to consume your own MapGuide XYZ Tile Set Definitions.

Now the reason this announcement is a double-header is because these changes in Fusion really needs tooling assistance to best take advantage of these changes, so here's also a new release of MapGuide Maestro as well. Here's the notable changes.

Fusion XYZ layer editor support

Now that we have proper XYZ tile layer support in Fusion we now have support for consuming external XYZ tile sets.

 
Because MapGuide XYZ Tile Sets Definitions can now be consumable with Fusion, you can specify a tile set URL for such a Tile Set Definition by clicking the new Add from XYZ Tile Set Definition toolbar button.

If you want to consume OSM tiles from your own OSM tile server you can use this editor and point it to your OSM tile server with the requisite ${x}, ${y} and ${z} placeholders.

RtMapInspector tool improvements 

The other major feature of this new release is improvements to the RtMapInspector.

For a refresher, the RtMapInspector tool was introduced way back in a beta release of Maestro 5.0 and the purpose of this tool was to easily inspect the state of any runtime map if you know its session id and map name so you can easily debug map manipulation and state updates. Did your MapGuide application code correctly added your new layer? You can use this tool to inspect the map state and find out.

Having looked at this tool more recently, I've come to realise that I'm only skimming the surface of what this tool is capable of and with this release, the capabilities have been significantly expanded.

For example, the tool now lets you see the map image for the current map state

But it also occurred to me if we're inspecting a map, we can (and should be able to) also inspect its layers (and their feature sources) too! So when you select any layer in this tool, a new Inspect Layer button is enabled.

Clicking it will bring up an inspection dialog that shows the layer's XML and be able to interact with its feature source using the same components as the local feature source preview.

Other Changes

• Now requires .net Framework 4.7.1. The windows installer will check for this (and install if required)
• Maestro API now uses the latest stable releases of NetTopologySuite and GeoAPI
• The MgTileSeeder tool now targets .net Core 2.1

Download MapGuide Maestro
Download test build of Fusion

Introducing: New experimental bindings for the MapGuide API

I haven't been quiet on the MapGuide front. I've just been deeply entrenched in my lab conducting an important experiment whose results are finally coming into fruition that I am back here to make this exciting announcement.

That experiment was: Can we generate bindings for the MapGuide API using a vanilla version of SWIG? Yes we can!

Background

This is an important question that we needed an answer for and we wanted that answer to be "Yes". 

We currently provide bindings for the MapGuide API in:

• Java
• PHP 5.x
• .net (full framework)

However, we currently use an ancient and heavily modified version of SWIG, whose unclear audit trail of modifications and changes means that being able to support newer versions of PHP (ie. PHP 7.x) or variants of .net like .net Core is nigh-impossible, which puts us in a bit of a pickle because:

• PHP 5.6 (our current bundled version of PHP) will be end-of-life on December 2018. Bundling and targeting an EOL version of PHP is not a good look. To bundle the current version of PHP (7.x) we need to be able to generate a compatible PHP extension for it. We can't do that with current internal copy of SWIG as the zend extension APIs have massively breaking changes from PHP5 to PHP7.
• .net Core is where the action is at in the .net space, and not having a presence in this space diminishes our story of being able to build MapGuide applications in .net because as time goes on, that definition of ".net" will assume to mean both the (windows-only) full framework and (cross-platform) .net core.
• We may want to add support for additional programming languages in the future. Can't do it with our super-modified copy of SWIG again because of the unclear history of changes made to this tool.

Given these factors, and the untenable situation we currently find ourselves in technology-wise, we needed to explore the possibility of generating the bindings using a vanilla (un-modified) version of SWIG. If we're going to go vanilla, we want the latest and greatest, which supports generating bindings for PHP7, and can support .net core with the right amount of SWIG typemap elbow-grease, so all the more motivation to get this working!

What we now have

2 months since the decision to embark on this journey, the mission to get functional MapGuide bindings using vanilla SWIG has been a success! We now have the following bindings for the MapGuide API:

• A Java binding modeled on the non-crufty official Java binding. Requires Java 7 or higher.
• A (currently windows-only) PHP extension for PHP 7.1
• A netstandard2.0-compatible binding for .net that works on both .net Core and full framework and is also cross-platform for platforms where both .net Core and official MapGuide binary packages are available for. For Linux, that means this .net binding works in Ubuntu 14.04 64-bit (where .net Core also has packages available). The nuget package for this binding is fully self-contained and includes the necessary native dependencies (both 32 and 64-bit) needed for the .net library to work. For .net full framework, it includes an MSBuild .targets file to ensure all the required native dependencies are copied out to your application's output directory.

Where to get it

You can grab the bits from the releases page of the mapguide-api-bindings GitHub repo.

For .net, you will have to setup a local package source and drop the nuget package there in order to consume in your library/application.

You will need MapGuide Open Source 3.1.1 installed as this is the only version of MapGuide I am generating these bindings for and testing against. Please observe the current supported platform matrix to see what combinations of technology stacks work with this new set of bindings. Please also observe the respective notes on the .net, PHP and Java bindings to observe what changes and adjustments you need to make in your MapGuide application should you want to try out these bindings.

Sample applications (to make sure this stuff works)

As proof that these bindings work, here's a sample asp.net core application using the new MapGuide .net binding. As a testament to what targeting .net Core gives us, you could bypass building the sample application from source and perhaps give the self-contained package a go. Thanks to the powerful publishing capabilities provided by the dotnet CLI, we can publish a self-contained .net core application with zero external dependencies. In the case of this sample application, you can download the zip, extract it to a windows or Ubuntu 14.04 64-bit machine with a standard MapGuide Open Source 3.1.1 install, run the MvcCoreSample executable within, go to http://localhost:5000 and your MapGuide application is up and running!

For Java and PHP, I'm still cooking up some sample applications in the same vein as the asp.net core one (ie. Porting across the MapGuide Developer's Guide samples), but for now the only verification that these bindings work is that our current binding test suite run to completion (with some failures, but these failures are known failures that are also present in our current bindings).

Where to from here?

I intend for the mapguide-api-bindings project to serve as an incubation area where we can iron out any show-stopping problems before planning for the eventual inclusion into MapGuide proper and supplementing (and in the case of PHP, replacing) our current bindings because eventually, we have to. We cannot keep bundling and targeting PHP 5.x forever. We need to be able to target newer versions of these programming languages, and maybe in some cases new programming languages.

mapguide-api-bindings project repo
asp.net core sample application repo