Announcing: mapguide-react-layout 0.13.3

Was I lying when I said I will be taking a short break from all things open source after releasing MapGuide Open Source 4.0 Preview 3?

Technically speaking, no.

This 0.13.3 release is just a few important fixes made since the 0.13.2 release that actually landed a few months ago, but I haven't "pressed the release button" until now. The fixes are:

• A fix for the initialElementVisibility mount option not working for the Aqua template
• A very important fix for inconsistent layer/group state if refreshing the legend after doing some server-side layer/group changes

This release is browser bundle (viewer.zip) only. There is no npm module release for this. 

Now back to my self-imposed break.

Download

Announcing: MapGuide Open Source 4.0 Preview 3

Here is the long awaited third preview release of MapGuide Open Source 4.0.

Refer to the release notes for download links and an overview of what's new and change since the last preview release.

For MapGuide users on Linux who use Java, you will be glad to hear that Java support is back and fully operational with this release (no more 403 errors from Tomcat). Restoring Java support on Linux came at a small price, we now no longer ship MapGuideApi.jar as the Java wrapper to the MapGuide API. We now only ship MapGuideApiEx.jar which has been in all MapGuide Open Source releases since 2.5 and this wrapper fixes most of the "cruftiness" of the original Java wrapper as described in this RFC where it was first introduced.

For those who like to swap the bundled GDAL dll on windows with a version with an expanded suite of raster/vector driver support, this release will make this process more seamless:

• Our internal GDAL version is now 2.4.4, the latest in the 2.x series.
• Our internal xalan/xerces dlls now have a "fdo" suffix in their dll name, so you are no longer forced to overwrite these dlls when overlaying an external GDAL dll from gisinternals and introduce potential instability as a result.
• All of our FDO providers that link to GDAL/OGR now only use their C API to avoid any potential ABI incompatibilities from replacing our internal GDAL dll with an external one that may not be built with the same MSVC compiler version as ours.

Other than that, Preview 3 is mostly a roll up of fixes to MapGuide/FDO since the last preview release to tide things over while work is still ongoing to support PHP7 in our MapGuide API to clear the obstacle to a final 4.0 release.

But before then, as of this release I will be taking a short break from all things open source. I am mildly burned out all things considered and a good solid month or two away from all things open source should hopefully be enough time to recharge the batteries.

Announcing: vscode-map-preview 0.5.8

This minor update fixes display of spatial data that spans the international date line by turning off the flag that causes OpenLayers to auto-wrap such features

Announcing: vscode-map-preview 0.5.7

This final release of anything from me for 2020 fixes our KML content scrubbing code to no longer trash KML icon styles. OpenLayers didn't support KML icons properly when this extension was first created which necessitated scrubbing said content out when previewing KML files so that at least something will render in the preview instead of nothing.

That is no longer the case, so now KML icon styles are preserved when previewing. Case in point below, we now get cutlery icons instead of the standard pin marker.

One small caveat needs to be observed: Due to content security restrictions on the HTML generated by this extension, the KML icon URLs must be https otherwise nothing will render.

This release also updates OpenLayers to 6.5.0 and ol-layerswitcher to 3.8.3

My new rig

This past week, I made the final decision to retire my old PC used for gaming and dev on my various open source projects and start shopping out for a new PC.

The main reason for the decision was my daily YouTube feed being subliminally inserted with many videos praising the (just-released) AMD Ryzen 5000 series of CPUs and the AMD Radeon RX 6000 series of GPU and how they were absolutely trouncing Intel and Nvidia in their respective CPU and GPU performance.

I also haven't been a fan of Intel CPUs since the advent of the meltdown and spectre vulnerabilities and Linus Torvalds' remarks about Nvidia have long stuck with me and my open source sensibilities, so I knew my next PC was going to have an AMD CPU and an AMD GPU. Now for me personally, I'm more of a coder than a gamer so while most of the competition-trouncing benchmarks thrown around were gaming-oriented, I was more interested in any benchmarks on developer-style workloads. One particular benchmark on one particular video caught my attention and told me that not only will my next PC be all AMD, I must get this next PC yesterday!

That's right, the new top-of-the-line AMD Ryzen 5950x compiles the Chromium browser (the software whose C++ codebase is most notorious for long build times) in around 45 minutes! Since the C++ codebases I work with are way smaller than Chromium's, but still took a considerable time to build on my old 2012 rig I was salivating at the thought of being able to compile MapGuide/FDO/GDAL/etc in 5-10 minutes on my brand new Ryzen-powered beast.

So then I started to look around to see who would offer such a beast of a PC in my home country of Australia. This eventually lead me to PC Case Gear (PCCG) which offered pre-built Intel and AMD Ryzen Gaming PCs at various pricing tiers. Some people may tell me that I should buy the components and build the PC myself, but I'm more of a software person than a hardware person so when it comes to a new PC, I prefer to go a with a pre-built one than risk screwing thing up building a new PC myself.

My personal budget for this new PC was to be no more than $5000 AUD. This sadly put any pre-built systems with a Ryzen 5950x out of my price range, so I settled on their Elite 6800 XT gaming system, whose main components of interest were:

• AMD Ryzen 9 5900x
• AMD Radeon RX 6800 XT
• 4TB (2x2) SSD
• 32 GB of DDR4 RAM
• Windows 10 Home pre-installed

So I placed my order and gleefully waited in anticipation for this box to arrive. Then reality hit.

These AMD CPUs were selling faster than toilet paper during the early stages of this pandemic. Demand so wildly exceeded supply that you even had dedicated YouTube livestreams tracking availability of AMD Ryzen stock! And so a few days after placing the order, I sadly got the phone call from PCCG that they run out of stock on the Ryzen 9 5900x and for reasons unknown (miscalculation of inventory perhaps?) their website had erroneously reported the PC I ordered as being in stock. In light of this, they offered to replace the out-of-stock Ryzen 9 5900x with a less-but-still-powerful Ryzen 7 5800x with a $300 AUD discount. My dev-oriented mindset at this point was "well... it still compiles Chromium in under 90 minutes! So it should still be great for developer workloads relatively-speaking", so I accepted their revised offer.

A few days later, my doorbell rings and it arrived.

PCCG obviously took great care in packaging by wrapping the PC in so much foam and bubble wrap that it could double as a padded cell for a psychiatric hospital. Several minutes of cutting all the wrapping and unboxing later, my new rig was ready to be powered on and ready to rock!

Now for some developer-oriented first impressions/observations.

Windows 10 Home observations

This PC (like most computers these days) came with Windows 10 Home pre-installed and is my first computing device with this particular edition installed. My experiences with Windows 10 thus far have either been from playing with their preview builds or using the Pro edition on computers at work. I've known that home editions of past Windows releases really gimped out on the features that I needed or would be of interest to me as a developer. But Windows 10 home edition was a real surprise on this front.

Firstly, it includes IIS. So I can install MapGuide on this with IIS/.net configuration. Previous versions of Windows left IIS as a pro-only feature.

The other big surprise was Windows 10 Home supports WSL2. This one is a game-changer and was a feature I seriously did not expect to be appear in the Home edition. As someone who builds software for both Windows and Linux, being able to build (and debug) for both OSes from a single host OS without needing to spin up separate virtual machines is a massive productivity boost!

And by supporting WSL2, it means I can also spin up docker containers as Docker for Windows uses WSL2 for its container backend. I can run the new docker-based MapGuide/FDO build system completely inside an Ubuntu WSL2 distro, while building MapGuide/FDO for Windows at the same time!

MapGuide/FDO observations

So with my new rig up and running, the first item of order was obviously to get my dev environment all set up and see how fast MapGuide/FDO will take to build from a clean SVN checkout.

• FDO takes 17 minutes (release build, windows)
• MapGuide takes 15 minutes (release build, windows)
• Linux numbers TBD, but I'm expecting comparable numbers

I have never seen C++ compiler output whiz by so fast! I can only imagine how much faster this would be if I my new PC had the original Ryzen 9 5900x (or even better, if the Ryzen 9 5950x was in stock and not so expensive to fit within my budget)

When all the compiler output is whizzing by, you start to notice the slower parts of the build because their compiler output isn't ticking along as fast. In the case of FDO, it noticeably slowed down when building the internal OpenSSL library. It turns out that the OpenSSL build system is woefully un-parallel on Windows, using only 16% of my available CPU during the entire build.

Overall though, I really like these numbers, which only go down once we start doing actual dev work where we won't be building thirdparty libs and certain projects over and over.

In closing

So far, I am very happy with my new purchase. PCCG were very speedy in their delivery and very helpful in their communications.

My last PC lasted a solid 8 years. I'm certain this new PC will last me a good decade. The only slight disappointment was not being able to get the Ryzen CPU I originally wanted, but then again many others can't get the Ryzen CPU they want either!

Experimental Azure PaaS support in the SQL Server FDO provider

Do you use MapGuide? Do you also use SQL Server on Azure and have lamented for the longest time the inability of the SQL Server FDO provider to read/write data to a Azure-PaaS-hosted SQL Server db? 

Then boy, do I have news for you!

Thanks to some day-job dogfooding of FDO Toolbox and a test Azure account on hand I have finally tackled this 7 year old FDO ticket and landed experimental support for SQL Server on Azure PaaS.

You can download the 7zip archive of the patched provider from here.

Download, extract and overwrite the existing provider dll in your MapGuide Open Source 4.0 preview or FDO Toolbox 1.5.1 installation.

I have so far verified this provider can perform most provider operations needed for use in a MapGuide context.

The only operation I couldn't verify was whether the FDO CreateDataStore API worked for this provider in Azure PaaS due to insufficient permissions in the test account I was given. The CreateDataStore API is not used in the MapGuide context, so if this API doesn't work properly on Azure PaaS, it is not a deal breaker for use in MapGuide.

I'd be interested from anyone who tries this patched provider to let me know if this particular API works on Azure PaaS (ie. You can create a new FDO data store on Azure PaaS via FDO Toolbox with this patched provider)

Many thanks to Crispin Hoult of TrueViewVisuals (FKA LinkNode) for the original patch, which I had tweaked so the azure db detection is a runtime check instead of support you have to explicitly compile into the provider dll. If we had been using Azure PaaS earlier, the patch wouldn't have taken 7 years to finally be applied!

MapGuide dev diary: The beginnings of clearing the final hurdle

I've stated many times in this long and arduous MapGuide Open Source 4.0 development cycle that the final hurdle that must be cleared before 4.0 could ever be considered final is when we can finally generate language bindings for .net, Java and PHP with a vanilla and un-modified version of SWIG.

The reasons for needing to do this were already explained in my previous introductory post to these new bindings, but to re-iterate the cliff notes version:

• We need to support and bundle PHP 7. This is non-negotiable. The current bundled PHP 5.6 is too old and long past EOL and it is a bad look to have to bundle this version of PHP for a production MapGuide deployment/installation.
• The latest release of SWIG can generate bindings for PHP 7
• The cross-platform .net core has grown in leaps and bounds over traditional windows-only .net Framework in terms of adoption. The just released .net 5.0 is a sign that the current windows-only .net Framework is dead/legacy and the future of .net is a cross-platform one.
• As a result, if we're going to be supporting .net in MapGuide, we should be generating a .net binding that can work in both Windows and Linux.
• And if we need to do that, we might as well do it with the latest release of SWIG
• And if 2/3 languages require vanilla SWIG, we might as well go for the trifecta and generate our Java binding with it as well!

As this final hurdle involves many steps, I figure this journey is worth documenting with its own mini dev diary series.

So what has changed since the initial announcement of these experimental bindings?

Firstly, I have decided to bring the current binding work into the official MapGuide source in a new vanilla_swig sandbox branch. All development work will continue in this branch. The previous GitHub repo housing this work will no longer be maintained and I will eventually archive/delete this repo. Going from Git back to SVN might sound like a downgrade (technically yes), but my developer "inner loop" has sped up a lot by having everything in the same repo and not having to coordinate files/changes across 2 different repos in 2 different locations. Maybe one day we'll permanently migrate the MapGuide source on GitHub, but today is not that day.

Secondly, before I tackle the PHP 7 support, I wanted to see whether the .net/Java bindings were still functional and what other final improvements we can make before all attention is fully diverted to the PHP 7 binding.

For Java, after some minor fix ups, the binding and its test suite were still A-OK. So onto the .net binding.

When I introduced these new experimental bindings, the .net one was back to a single monolithic assembly (MapGuideDotNetApi). I wasn't fully comfortable with the monolithic assembly as it would hamper any attempts to write code that could work in both MapGuide and mg-desktop. The mg-desktop .net support was hanging off of the currently Foundation/Geometry/PlatformBase split assemblies. Having our new .net binding go back to a monolithic assembly would hamper our ability to write such shared code, so if it was possible we should try to replicate the Foundation/Geometry/PlatformBase/MapGuideCommon/Web split layout in the new .net binding.

Using the current .net binding as a point of reference, splitting the monolithic MapGuideDotNetApi assembly back to the 5 constituent parts was a relatively simple affair. Thanks to the dramatically simplified csproj format we now have 5 hand-crafted C# projects targeting netstandard2.0 that reference each other and that SWIG dumps all its generated C# source into (without having to add each .cs file into the project itself) for easy compilation that automatically publishes out to respective nuget packages like so.

And because our 5 projects reference each other, those dependencies are also expressed in the nuget packages themselves. That is to say, if you install the MapGuideCommon package, it will automatically install the PlatformBase, Geometry and Foundation packages as well as they were defined as project dependencies of the MapGuideCommon C# project file.

And the final cherry on top? These nuget packages are still self-contained and bundle the native dlls that the .net binding is wrapping. The current nuget packages are already self-contained, but they are only consumable in legacy .net Framework, are windows-only and require kludgy powershell hacks to make sure all the native dlls are copied out to the project's output directory. Our new nuget packages take advantage of the fact that native libraries are now first class citizens in the .net core packaging world.

By adding such dlls to the runtimes/win-x64/native folder of a C# project, they will automatically be bundled into any nuget package created and the .net core project system knows to automatically copy these dlls out to the right location where the .net assembly can P/invoke them. 

Now for a multi-platform .net binding to work, we have to get SWIG to generate the same C++ glue code, but this time to be compiled on Linux but with the same library names so our SWIG-generated C# code will correctly P/Invoke into the respective Windows .dll or Linux .so, and pack those compiled .so files into the runtimes/linux-x64/native folder of our 5 C# projects.for automatic bundling into our nuget package.

How we are able to do this will be the topic of a future post once I've figured it all out.