MapGuide dev diary: It is done!

In the previous installment, we had gotten a test MapGuide installation with bundled PHP 8.1 up and running, and we were able to successfully produce a PHP error when hitting the Site Administrator.

This PHP error referenced previously was a milestone because it meant that our PHP 8.1 setup (FastCGI on Apache via mod_fcgid) was working, our PHP code was actually running and so the actual errors is the result of the vast swaths of our current PHP web tier applications needing to be migrated across to work against this new PHP 8.1 binding for the MapGuide API.

And so for the next few months I did just that, not just for PHP, but also for Java and .net. You can consider this post to be a preview of what you'll need to do yourself if you want to migrate your PHP/Java/.net MapGuide applications to MGOS 4.0.

PHP Migration Overview

The PHP binding was the one I expect to be the most work because besides finally supporting PHP 8.1, the major feature of this new PHP binding is that constants.php is no longer required! This is because with vanilla SWIG we now can bake all the constants of the MapGuide API into the PHP binding extension itself! So I expect a lot of  "include 'constants.php'" references needing to be removed.

Once all the constants.php references are removed, we found the prime issue with this new PHP binding. PHP didn't like some of our C++ classes had overloaded methods whose signatures did not exist in the parent class. This manifested in the form of fatal PHP errors like the following:

PHP Fatal error:  Declaration of MgCoordinateSystemMeasure::GetDistance(MgCoordinate|float|null $arg1, MgCoordinate|float|null $arg2, float $arg3, float $arg4): float must be compatible with MgMeasure::GetDistance(?MgCoordinate $arg1, ?MgCoordinate $arg2): float in Unknown on line 0

In this case, our MgMeasure class has a GetDistance method of the following signature:

double GetDistance(MgCoordinate* coord1, MgCoordinate* coord2)

In the derived MgCoordinateSystemMeasure, it has a new overload of GetDistance that has this signature:

double GetDistance(double x1, double y1, double x2, double y2)

However, when this converted to PHP proxy classes by SWIG, PHP doesn't like this class setup because under its inheritance model, it is expecting the GetDistance overload with 4 double parameters to also exist in the base MgMeasure class. This is not the case, and thus PHP throws the above fatal error.

To workaround this problem, we had to use the SWIG %rename directive to rename the conflicting overload signature in MgCoordinateSystemMeasure in PHP to the following:

double GetDistanceSimple(double x1, double y1, double x2, double y2)

With this rename, there is no longer a signature conflict in the generated proxy class and PHP no longer throws a fatal error. Fortunately, only 2 classes in the MapGuide API have this problem, so the amount of method renaming is minimal.

Once this issue was addressed, I tried firing up the PHP implementation of the AJAX viewer and I got an interactive map! Everything seemed to be working until I tried to generate a map plot, and found my second problem. I was getting PHP fatal errors like this:

PHP Fatal error:  Uncaught TypeError: No matching function for overloaded 'MgRenderingService_RenderMap' in quickplotgeneratepicture.php:115

Fortunately, this one was easier to explain and fix. In PHP 8.1 (maybe even earlier in the PHP 7.x series), the type checking became more stricter which meant int parameters must take integers, double parameters must take doubles, etc, etc, you couldn't pass ints as doubles or vice versa, and for a method like RenderMap of MgRenderingService, there are lots of overloads that take many different combinations of int and double parameters.

Our map plotting code was passing in strings where int/double parameters were expected. In PHP 5.6 this was allowed because the type checking was evidently more lax. Now such cases cause the above PHP fatal error. This was easy enough to fix, we just use the intval and doubleval functions to make sure ints are being passed into int parameters and doubles are being passed into double parameters.

And with that, the rest of the changes involving fixing up our exception handling code due to a major change with how MapGuide applications should be handling exceptions from the MapGuide API. As part of this SWIG binding work, we've flattened the MapGuide exception hierarchy into a single MgException class, and introduced a new exception code property to allow handling MapGuide exceptions on a case-by-case basis.

So if you were handling specific MapGuide exceptions like this:

1 2 3 4 5 6 7 8

try { //Some code that could throw } catch (MgUnauthorizedAccessException $e) { ... }

You would now rewrite them like this:

1 2 3 4 5 6 7 8 9 10

try { //Some code that could throw } catch (MgException $e) { if ($e->GetExceptionCode() == MgExceptionCodes::MgUnauthorizedAccessException) { ... } }

The reason for flattening the exception hierarchy was:

• To make wrapping exceptions simpler (we now only need to wrap the sole MgException class in SWIG) and not have to handle exception class inheritance chains in a consistent manner across all 3 language bindings.
• Most of the example MapGuide API code pretty much only caught MgException anyways and rarely catches any of its derived exception classes (and I imagine that this is the case in your MapGuide applications as well). Any code that cared to handle specific exception cases, we can just include the relevant sub-classification as a property of MgException itself as the above code example shows.

Once this final change was made, the AJAX viewer was fully functional. Porting Fusion to PHP 8 was a similar process.

Java Migration Overview

This migration I expect to be a cakewalk because although this binding is now also being generated by vanilla SWIG, it is 99% identical to the existing MapGuideJavaApiEx.jar that we have been generating and shipping for many releases of MapGuide.

So all I expect is just to:

• Fix up references to MapGuideJavaApiEx and rename them to MapGuideJavaApi
• Update exception handling blocks to take action based on the captured exception code in the caught MgException instead of catching for specific subclasses of MgException
• Replace MapGuideApiEx.jar with MapGuideApi.jar (we're using the old jar name for the new binding) in our MG installation (and effectively, going back full circle to the way things were for Java in MapGuide)

And ... it went just exactly what I said above! This was by far the easiest migration effort of the lot.

.net Migration Overview

This migration was the one I had been dreading the most. Not because I feared this binding was going to be fragile, because we already had an exhaustive test suite which this binding passed with flying colors.

But rather, I had been dreading this one because all of our .net code that is going to use this binding (AJAX viewer, code samples, etc) are all legacy pre-historic aspx webforms and I wasn't sure if such code would accept the brand new .net development story I had planned for it.

Consider the current .net development story.

1. You would reference the 5 OSGeo.MapGuide.* assemblies from mapviewernet/bin in your MapGuide application.
2. You would then have to manually copy the remaining dlls from mapviewernet/bin to your MapGuide application's output directory so that the .net MapGuide API binding doesn't fail due to missing native dll dependencies

The alternative to this is to use the NuGet package, which make this story more seamless, but the process to build this NuGet package was a bespoke affair, with hand-crafted powershell scripts that trigger on nuget package installation to set up the necessary project build events to copy the native dlls needed by the OSGeo.MapGuide.* assemblies to the right location. Such functionality is tightly-coupled to Visual Studio, so if you were installing this NuGet package and building your MapGuide application outside of Visual Studio, none of the required post-build events would fire and the result is a broken MapGuide .net application because the native dll dependencies were not being copied to your application's output directory.

For this new .net binding, we build each OSGeo.MapGuide.* assembly as a separate SDK-style csproj project files. This new csproj file format has several benefits:

• You don't have to reference individual C# source files that need to be compiled. Any C# source fil in the same directory as the csproj file is implied to be compiled as part of the project. This is great because it means we can run SWIG to generate the .cs files straight into the project and build that project straight away afterwards.
• This csproj file format supports NuGet packages as first-class project output
• The NuGet packages produced have first-class support for native dependencies. This is the real killer feature because it means in terms of packaging, we just have to include these native dlls in a well known location and they will be bundled up automatically as part of NuGet package creation. Such a package when consumed will have its native dependencies automatically copied to the right place and loaded from the right spot without any custom post-build events to set this stuff up!
• And finally, it means instead of targeting .net Framework, we can target netstandard2.0

What does netstandard2.0 support imply? It implies your MapGuide application using these packages can work on all of these platforms. Now practically speaking, despite now being netstandard2.0 packages, these packages will only work on platforms where the underlying OS is Windows and (maybe) Linux as those are the platforms where we can actually compile the underlying supporting native libraries needed by these nuget packages. So no Mac OSX, no Xamarin, etc. 

In practical terms, it means you are no longer shackled to legacy .net framework for building MapGuide .net applications. You can now use .net core from its earliest netstandard2.0-supported iterations all the way to the latest .net 6.0 (as of this post). 

That's great and all, but going back to the original issue: Can the current suite of aspx webforms code accept this new way of consuming the .net MapGuide API and come along for the ride? I hope so! Because the alternative is to rewrite all of this code with more modern .net web technologies (razor pages maybe?), and while such a rewrite has merit and probably warranted, it is not warranted right now because that would add many more months of dev work to my already time-poor schedule. We have bigger fish to fry! We just hope this current codebase will cooperate with our new .net packaging paradigm with minimal effort.

So let's start with the basic facts.

• MapGuide's .net integration requires IIS and .net framework to already be installed
• We can assume that for the purpose of being able to use this netstandard2.0 library, that the installed .net framework version must .net framework 4.8. Building your own MapGuide applications for .net core and .net 5.0+ is something you can opt-in to, but it is not something to be demanded by our existing .net web tier code.

With these facts established we have our first hurdle, and it is one of setup/deployment.

The AJAX viewer and sample code have no Visual Studio solution/project files! How do these things ever get built?

The AJAX viewer for .net is a series of raw .aspx files, will be "compiled" to .net assemblies on first request to IIS. As part of this compilation, it will check its respective "bin" directory for any references. That's why the mapviewernet/bin has the OSGeo.MapGuide.* assemblies in there because that is what is being referenced when the .aspx files get compiled. The .net sample code also follows the same pattern.

So we have a bunch of folders of .aspx files and we need to get the correct set of dlls from inside our brand new shiny nuget packages into these folders. How would we go about this without needing to disruptively set up solution/project files for them?

Here's my approach. We setup a stub SDK-style project that targets net4.8 and references the 5 OSGeo.MapGuide.* nuget packages produced from our new .net binding project setup.

As part of the main build, we perform a framework-dependent publish of this project. Because of the first-class native dependency support, the publish output of this project would be the project's assembly, the 5 OSGeo.MapGuide.* assemblies and (importantly) all of their native dll dependencies in one single output directory. Once we done the framework-dependent publish, we can then copy all the dll files in this publish output folder (except for the stub project) into the bin directory of our AJAX viewer and sample directories.

It turns out this approach does result in a functional .net AJAX viewer and code samples. What was needed in addition to using a stub project to setup the required dll file list, is that the .net AJAX viewer and code samples need a web.config file that references the netstandard assembly due to our OSGeo.MapGuide.* assemblies now target netstandard2.0

Is this a complete and utter hack? Totally!

But this approach gives us a functional .net AJAX viewer and code samples. Considering the alternative solutions and my current timelines, this is a workable approach and sometimes ...

So that was the ugly setup/deployment aspect, but what about the code itself? Well that was relatively simple. Like PHP and Java before it, we only needed to fix up the exception handling code to match to our new pattern of checking for specific exception codes in the caught MgException to handle for certain error cases.

Getting this to work on Linux

With the bindings now all being generated by vanilla SWIG and all working on Windows, it was a case of getting the build system fully working again on Linux with these new bindings and updated web tier components.

Fortunately, on the binding front, most of the CMake configurations added in the initial phases of this work only needed minor adjustments, so the bulk of the work was actually building PHP 8.1 and  integrating this into the Apache httpd server, which we are also building from source and updating our various httpd/php config file templates to work with this new version of PHP.

Where we stand now

We now finally have MapGuide API bindings generated with vanilla, unmodified SWIG that work on both Windows and Linux. This has been a long and arduous journey and I can finally see the light at the end of this tunnel!

A new RFC has been posted for PSC discussion/voting. I hope there isn't strong opposition/dissent on these changes, because I truly believe that this work is absolutely necessary for MapGuide going forward. Newer versions of .net/Java/PHP will always get released and inevitably we will need our MapGuide API bindings to work on these newer versions. Our current infrastructure to keep up with newer .net/Java/PHP versions is just not maintainable or tenable.

If/when this RFC is adopted, the long overdue Preview 4 release should drop not too long after!