Wednesday 18 November 2020

Experimental Azure PaaS support in the SQL Server FDO provider

Do you use MapGuide? Do you also use SQL Server on Azure and have lamented for the longest time the inability of the SQL Server FDO provider to read/write data to a Azure-PaaS-hosted SQL Server db? 

Then boy, do I have news for you!

Thanks to some day-job dogfooding of FDO Toolbox and a test Azure account on hand I have finally tackled this 7 year old FDO ticket and landed experimental support for SQL Server on Azure PaaS.

You can download the 7zip archive of the patched provider from here.

Download, extract and overwrite the existing provider dll in your MapGuide Open Source 4.0 preview or FDO Toolbox 1.5.1 installation.

I have so far verified this provider can perform most provider operations needed for use in a MapGuide context.

The only operation I couldn't verify was whether the FDO CreateDataStore API worked for this provider in Azure PaaS due to insufficient permissions in the test account I was given. The CreateDataStore API is not used in the MapGuide context, so if this API doesn't work properly on Azure PaaS, it is not a deal breaker for use in MapGuide.

I'd be interested from anyone who tries this patched provider to let me know if this particular API works on Azure PaaS (ie. You can create a new FDO data store on Azure PaaS via FDO Toolbox with this patched provider)

Many thanks to Crispin Hoult of TrueViewVisuals (FKA LinkNode) for the original patch, which I had tweaked so the azure db detection is a runtime check instead of support you have to explicitly compile into the provider dll. If we had been using Azure PaaS earlier, the patch wouldn't have taken 7 years to finally be applied!

Sunday 15 November 2020

MapGuide dev diary: The beginnings of clearing the final hurdle

I've stated many times in this long and arduous MapGuide Open Source 4.0 development cycle that the final hurdle that must be cleared before 4.0 could ever be considered final is when we can finally generate language bindings for .net, Java and PHP with a vanilla and un-modified version of SWIG.

The reasons for needing to do this were already explained in my previous introductory post to these new bindings, but to re-iterate the cliff notes version:

  • We need to support and bundle PHP 7. This is non-negotiable. The current bundled PHP 5.6 is too old and long past EOL and it is a bad look to have to bundle this version of PHP for a production MapGuide deployment/installation.
  • The latest release of SWIG can generate bindings for PHP 7
  • The cross-platform .net core has grown in leaps and bounds over traditional windows-only .net Framework in terms of adoption. The just released .net 5.0 is a sign that the current windows-only .net Framework is dead/legacy and the future of .net is a cross-platform one.
  • As a result, if we're going to be supporting .net in MapGuide, we should be generating a .net binding that can work in both Windows and Linux.
  • And if we need to do that, we might as well do it with the latest release of SWIG
  • And if 2/3 languages require vanilla SWIG, we might as well go for the trifecta and generate our Java binding with it as well!

As this final hurdle involves many steps, I figure this journey is worth documenting with its own mini dev diary series.

So what has changed since the initial announcement of these experimental bindings?

Firstly, I have decided to bring the current binding work into the official MapGuide source in a new vanilla_swig sandbox branch. All development work will continue in this branch. The previous GitHub repo housing this work will no longer be maintained and I will eventually archive/delete this repo. Going from Git back to SVN might sound like a downgrade (technically yes), but my developer "inner loop" has sped up a lot by having everything in the same repo and not having to coordinate files/changes across 2 different repos in 2 different locations. Maybe one day we'll permanently migrate the MapGuide source on GitHub, but today is not that day.

Secondly, before I tackle the PHP 7 support, I wanted to see whether the .net/Java bindings were still functional and what other final improvements we can make before all attention is fully diverted to the PHP 7 binding.

For Java, after some minor fix ups, the binding and its test suite were still A-OK. So onto the .net binding.

When I introduced these new experimental bindings, the .net one was back to a single monolithic assembly (MapGuideDotNetApi). I wasn't fully comfortable with the monolithic assembly as it would hamper any attempts to write code that could work in both MapGuide and mg-desktop. The mg-desktop .net support was hanging off of the currently Foundation/Geometry/PlatformBase split assemblies. Having our new .net binding go back to a monolithic assembly would hamper our ability to write such shared code, so if it was possible we should try to replicate the Foundation/Geometry/PlatformBase/MapGuideCommon/Web split layout in the new .net binding.

Using the current .net binding as a point of reference, splitting the monolithic MapGuideDotNetApi assembly back to the 5 constituent parts was a relatively simple affair. Thanks to the dramatically simplified csproj format we now have 5 hand-crafted C# projects targeting netstandard2.0 that reference each other and that SWIG dumps all its generated C# source into (without having to add each .cs file into the project itself) for easy compilation that automatically publishes out to respective nuget packages like so.

And because our 5 projects reference each other, those dependencies are also expressed in the nuget packages themselves. That is to say, if you install the MapGuideCommon package, it will automatically install the PlatformBase, Geometry and Foundation packages as well as they were defined as project dependencies of the MapGuideCommon C# project file.

And the final cherry on top? These nuget packages are still self-contained and bundle the native dlls that the .net binding is wrapping. The current nuget packages are already self-contained, but they are only consumable in legacy .net Framework, are windows-only and require kludgy powershell hacks to make sure all the native dlls are copied out to the project's output directory. Our new nuget packages take advantage of the fact that native libraries are now first class citizens in the .net core packaging world.

By adding such dlls to the runtimes/win-x64/native folder of a C# project, they will automatically be bundled into any nuget package created and the .net core project system knows to automatically copy these dlls out to the right location where the .net assembly can P/invoke them. 

Now for a multi-platform .net binding to work, we have to get SWIG to generate the same C++ glue code, but this time to be compiled on Linux but with the same library names so our SWIG-generated C# code will correctly P/Invoke into the respective Windows .dll or Linux .so, and pack those compiled .so files into the runtimes/linux-x64/native folder of our 5 C# projects.for automatic bundling into our nuget package.

How we are able to do this will be the topic of a future post once I've figured it all out.

Thursday 5 November 2020

MapGuide 4.0 Showcase: Making WFS/WMS support beyond a box ticking exercise

For the longest time, MapGuide's support for WFS and WMS was nothing too special. The level of support was the bare-minimum enough so that we could say "We support WFS/WMS"

For MapGuide 4.0, the WFS and WMS support has been enhanced in the following areas:

GeoJSON format support

As I've previously mentioned, if we're going to serve feature data in a JSON format, we should just go straight to GeoJSON and not bother with anything else.

This now also applies for WFS and WMS operations that return feature data. Namely:
  • GetFeatures for WFS
  • GetFeatureInfo for WMS
For both these operations, specifying application/json as the requested format will return the data in GeoJSON. This support is most useful for WMS GetFeatureInfo as due to the ubiquity of GeoJSON support, a WMS GetFeatureInfo response in GeoJSON format can be used as a convenient "selection overlay" to display selected features when clicking on a WMS map.

Configurable geometry output for WMS GetFeatureInfo

Sometimes, one may not wish to have geometry output for certain WMS GetFeatureInfo requests. So for MapGuide 4.0, there is a new _EnableGeometry metadata option for Layer Definition resources, that determines if WMS GetFeatureInfo requests against this layer should return geometry data or not.

The next release of MapGuide Maestro lets you toggle this setting in the UI without having to mess around with resource header XML.



This setting is only applicable if the Layer Definition itself has been set to be queryable for WMS.

WFS Support for hit count

The spec for WFS GetFeatures defines a special mode where one can request a hit count (ie. A raw total) instead of the actual raw feature data. MapGuide did not implement this part of the WFS spec (it is optional). For MapGuide 4.0, this is now implemented.

If you pass resultType=hits to your WFS request, you now get a total instead of the feature data.


As an aside, if you use VSCode on the regular, I highly recommend you install the REST client extension. It has replaced Postman for my HTTP/REST API testing. As evidenced by the above screenshot, testing HTTP requests is dead simple.

Special thanks to OpenSpatial for their assistance in testing out this feature.

Viewer representation for WMS GetMap

As of the 4.0 preview 2 release, the mapagent now also supports a new viewer representation for WMS layers, giving you a built-in way to easily preview any published WMS layer in MapGuide by simply specifying a format of application/openlayers, which is a new option in the GetMap test page


In this format, a HTML page is returned which contains an OpenLayers viewer set up to the WMS layer(s) in question.


No more needing to fire up a client GIS application like Gaia or QGIS to preview such layers, MapGuide now provides the means to preview such layers out of the box!