Announcing: vscode-map-preview 0.5.8

This minor update fixes display of spatial data that spans the international date line by turning off the flag that causes OpenLayers to auto-wrap such features

Announcing: vscode-map-preview 0.5.7

This final release of anything from me for 2020 fixes our KML content scrubbing code to no longer trash KML icon styles. OpenLayers didn't support KML icons properly when this extension was first created which necessitated scrubbing said content out when previewing KML files so that at least something will render in the preview instead of nothing.

That is no longer the case, so now KML icon styles are preserved when previewing. Case in point below, we now get cutlery icons instead of the standard pin marker.

One small caveat needs to be observed: Due to content security restrictions on the HTML generated by this extension, the KML icon URLs must be https otherwise nothing will render.

This release also updates OpenLayers to 6.5.0 and ol-layerswitcher to 3.8.3

My new rig

This past week, I made the final decision to retire my old PC used for gaming and dev on my various open source projects and start shopping out for a new PC.

The main reason for the decision was my daily YouTube feed being subliminally inserted with many videos praising the (just-released) AMD Ryzen 5000 series of CPUs and the AMD Radeon RX 6000 series of GPU and how they were absolutely trouncing Intel and Nvidia in their respective CPU and GPU performance.

I also haven't been a fan of Intel CPUs since the advent of the meltdown and spectre vulnerabilities and Linus Torvalds' remarks about Nvidia have long stuck with me and my open source sensibilities, so I knew my next PC was going to have an AMD CPU and an AMD GPU. Now for me personally, I'm more of a coder than a gamer so while most of the competition-trouncing benchmarks thrown around were gaming-oriented, I was more interested in any benchmarks on developer-style workloads. One particular benchmark on one particular video caught my attention and told me that not only will my next PC be all AMD, I must get this next PC yesterday!

That's right, the new top-of-the-line AMD Ryzen 5950x compiles the Chromium browser (the software whose C++ codebase is most notorious for long build times) in around 45 minutes! Since the C++ codebases I work with are way smaller than Chromium's, but still took a considerable time to build on my old 2012 rig I was salivating at the thought of being able to compile MapGuide/FDO/GDAL/etc in 5-10 minutes on my brand new Ryzen-powered beast.

So then I started to look around to see who would offer such a beast of a PC in my home country of Australia. This eventually lead me to PC Case Gear (PCCG) which offered pre-built Intel and AMD Ryzen Gaming PCs at various pricing tiers. Some people may tell me that I should buy the components and build the PC myself, but I'm more of a software person than a hardware person so when it comes to a new PC, I prefer to go a with a pre-built one than risk screwing thing up building a new PC myself.

My personal budget for this new PC was to be no more than $5000 AUD. This sadly put any pre-built systems with a Ryzen 5950x out of my price range, so I settled on their Elite 6800 XT gaming system, whose main components of interest were:

• AMD Ryzen 9 5900x
• AMD Radeon RX 6800 XT
• 4TB (2x2) SSD
• 32 GB of DDR4 RAM
• Windows 10 Home pre-installed

So I placed my order and gleefully waited in anticipation for this box to arrive. Then reality hit.

These AMD CPUs were selling faster than toilet paper during the early stages of this pandemic. Demand so wildly exceeded supply that you even had dedicated YouTube livestreams tracking availability of AMD Ryzen stock! And so a few days after placing the order, I sadly got the phone call from PCCG that they run out of stock on the Ryzen 9 5900x and for reasons unknown (miscalculation of inventory perhaps?) their website had erroneously reported the PC I ordered as being in stock. In light of this, they offered to replace the out-of-stock Ryzen 9 5900x with a less-but-still-powerful Ryzen 7 5800x with a $300 AUD discount. My dev-oriented mindset at this point was "well... it still compiles Chromium in under 90 minutes! So it should still be great for developer workloads relatively-speaking", so I accepted their revised offer.

A few days later, my doorbell rings and it arrived.

PCCG obviously took great care in packaging by wrapping the PC in so much foam and bubble wrap that it could double as a padded cell for a psychiatric hospital. Several minutes of cutting all the wrapping and unboxing later, my new rig was ready to be powered on and ready to rock!

Now for some developer-oriented first impressions/observations.

Windows 10 Home observations

This PC (like most computers these days) came with Windows 10 Home pre-installed and is my first computing device with this particular edition installed. My experiences with Windows 10 thus far have either been from playing with their preview builds or using the Pro edition on computers at work. I've known that home editions of past Windows releases really gimped out on the features that I needed or would be of interest to me as a developer. But Windows 10 home edition was a real surprise on this front.

Firstly, it includes IIS. So I can install MapGuide on this with IIS/.net configuration. Previous versions of Windows left IIS as a pro-only feature.

The other big surprise was Windows 10 Home supports WSL2. This one is a game-changer and was a feature I seriously did not expect to be appear in the Home edition. As someone who builds software for both Windows and Linux, being able to build (and debug) for both OSes from a single host OS without needing to spin up separate virtual machines is a massive productivity boost!

And by supporting WSL2, it means I can also spin up docker containers as Docker for Windows uses WSL2 for its container backend. I can run the new docker-based MapGuide/FDO build system completely inside an Ubuntu WSL2 distro, while building MapGuide/FDO for Windows at the same time!

MapGuide/FDO observations

So with my new rig up and running, the first item of order was obviously to get my dev environment all set up and see how fast MapGuide/FDO will take to build from a clean SVN checkout.

• FDO takes 17 minutes (release build, windows)
• MapGuide takes 15 minutes (release build, windows)
• Linux numbers TBD, but I'm expecting comparable numbers

I have never seen C++ compiler output whiz by so fast! I can only imagine how much faster this would be if I my new PC had the original Ryzen 9 5900x (or even better, if the Ryzen 9 5950x was in stock and not so expensive to fit within my budget)

When all the compiler output is whizzing by, you start to notice the slower parts of the build because their compiler output isn't ticking along as fast. In the case of FDO, it noticeably slowed down when building the internal OpenSSL library. It turns out that the OpenSSL build system is woefully un-parallel on Windows, using only 16% of my available CPU during the entire build.

Overall though, I really like these numbers, which only go down once we start doing actual dev work where we won't be building thirdparty libs and certain projects over and over.

In closing

So far, I am very happy with my new purchase. PCCG were very speedy in their delivery and very helpful in their communications.

My last PC lasted a solid 8 years. I'm certain this new PC will last me a good decade. The only slight disappointment was not being able to get the Ryzen CPU I originally wanted, but then again many others can't get the Ryzen CPU they want either!

Experimental Azure PaaS support in the SQL Server FDO provider

Do you use MapGuide? Do you also use SQL Server on Azure and have lamented for the longest time the inability of the SQL Server FDO provider to read/write data to a Azure-PaaS-hosted SQL Server db? 

Then boy, do I have news for you!

Thanks to some day-job dogfooding of FDO Toolbox and a test Azure account on hand I have finally tackled this 7 year old FDO ticket and landed experimental support for SQL Server on Azure PaaS.

You can download the 7zip archive of the patched provider from here.

Download, extract and overwrite the existing provider dll in your MapGuide Open Source 4.0 preview or FDO Toolbox 1.5.1 installation.

I have so far verified this provider can perform most provider operations needed for use in a MapGuide context.

The only operation I couldn't verify was whether the FDO CreateDataStore API worked for this provider in Azure PaaS due to insufficient permissions in the test account I was given. The CreateDataStore API is not used in the MapGuide context, so if this API doesn't work properly on Azure PaaS, it is not a deal breaker for use in MapGuide.

I'd be interested from anyone who tries this patched provider to let me know if this particular API works on Azure PaaS (ie. You can create a new FDO data store on Azure PaaS via FDO Toolbox with this patched provider)

Many thanks to Crispin Hoult of TrueViewVisuals (FKA LinkNode) for the original patch, which I had tweaked so the azure db detection is a runtime check instead of support you have to explicitly compile into the provider dll. If we had been using Azure PaaS earlier, the patch wouldn't have taken 7 years to finally be applied!

MapGuide dev diary: The beginnings of clearing the final hurdle

I've stated many times in this long and arduous MapGuide Open Source 4.0 development cycle that the final hurdle that must be cleared before 4.0 could ever be considered final is when we can finally generate language bindings for .net, Java and PHP with a vanilla and un-modified version of SWIG.

The reasons for needing to do this were already explained in my previous introductory post to these new bindings, but to re-iterate the cliff notes version:

• We need to support and bundle PHP 7. This is non-negotiable. The current bundled PHP 5.6 is too old and long past EOL and it is a bad look to have to bundle this version of PHP for a production MapGuide deployment/installation.
• The latest release of SWIG can generate bindings for PHP 7
• The cross-platform .net core has grown in leaps and bounds over traditional windows-only .net Framework in terms of adoption. The just released .net 5.0 is a sign that the current windows-only .net Framework is dead/legacy and the future of .net is a cross-platform one.
• As a result, if we're going to be supporting .net in MapGuide, we should be generating a .net binding that can work in both Windows and Linux.
• And if we need to do that, we might as well do it with the latest release of SWIG
• And if 2/3 languages require vanilla SWIG, we might as well go for the trifecta and generate our Java binding with it as well!

As this final hurdle involves many steps, I figure this journey is worth documenting with its own mini dev diary series.

So what has changed since the initial announcement of these experimental bindings?

Firstly, I have decided to bring the current binding work into the official MapGuide source in a new vanilla_swig sandbox branch. All development work will continue in this branch. The previous GitHub repo housing this work will no longer be maintained and I will eventually archive/delete this repo. Going from Git back to SVN might sound like a downgrade (technically yes), but my developer "inner loop" has sped up a lot by having everything in the same repo and not having to coordinate files/changes across 2 different repos in 2 different locations. Maybe one day we'll permanently migrate the MapGuide source on GitHub, but today is not that day.

Secondly, before I tackle the PHP 7 support, I wanted to see whether the .net/Java bindings were still functional and what other final improvements we can make before all attention is fully diverted to the PHP 7 binding.

For Java, after some minor fix ups, the binding and its test suite were still A-OK. So onto the .net binding.

When I introduced these new experimental bindings, the .net one was back to a single monolithic assembly (MapGuideDotNetApi). I wasn't fully comfortable with the monolithic assembly as it would hamper any attempts to write code that could work in both MapGuide and mg-desktop. The mg-desktop .net support was hanging off of the currently Foundation/Geometry/PlatformBase split assemblies. Having our new .net binding go back to a monolithic assembly would hamper our ability to write such shared code, so if it was possible we should try to replicate the Foundation/Geometry/PlatformBase/MapGuideCommon/Web split layout in the new .net binding.

Using the current .net binding as a point of reference, splitting the monolithic MapGuideDotNetApi assembly back to the 5 constituent parts was a relatively simple affair. Thanks to the dramatically simplified csproj format we now have 5 hand-crafted C# projects targeting netstandard2.0 that reference each other and that SWIG dumps all its generated C# source into (without having to add each .cs file into the project itself) for easy compilation that automatically publishes out to respective nuget packages like so.

And because our 5 projects reference each other, those dependencies are also expressed in the nuget packages themselves. That is to say, if you install the MapGuideCommon package, it will automatically install the PlatformBase, Geometry and Foundation packages as well as they were defined as project dependencies of the MapGuideCommon C# project file.

And the final cherry on top? These nuget packages are still self-contained and bundle the native dlls that the .net binding is wrapping. The current nuget packages are already self-contained, but they are only consumable in legacy .net Framework, are windows-only and require kludgy powershell hacks to make sure all the native dlls are copied out to the project's output directory. Our new nuget packages take advantage of the fact that native libraries are now first class citizens in the .net core packaging world.

By adding such dlls to the runtimes/win-x64/native folder of a C# project, they will automatically be bundled into any nuget package created and the .net core project system knows to automatically copy these dlls out to the right location where the .net assembly can P/invoke them. 

Now for a multi-platform .net binding to work, we have to get SWIG to generate the same C++ glue code, but this time to be compiled on Linux but with the same library names so our SWIG-generated C# code will correctly P/Invoke into the respective Windows .dll or Linux .so, and pack those compiled .so files into the runtimes/linux-x64/native folder of our 5 C# projects.for automatic bundling into our nuget package.

How we are able to do this will be the topic of a future post once I've figured it all out.

MapGuide 4.0 Showcase: Making WFS/WMS support beyond a box ticking exercise

For the longest time, MapGuide's support for WFS and WMS was nothing too special. The level of support was the bare-minimum enough so that we could say "We support WFS/WMS"

For MapGuide 4.0, the WFS and WMS support has been enhanced in the following areas:

GeoJSON format support

As I've previously mentioned, if we're going to serve feature data in a JSON format, we should just go straight to GeoJSON and not bother with anything else.

This now also applies for WFS and WMS operations that return feature data. Namely:

• GetFeatures for WFS
• GetFeatureInfo for WMS

For both these operations, specifying application/json as the requested format will return the data in GeoJSON. This support is most useful for WMS GetFeatureInfo as due to the ubiquity of GeoJSON support, a WMS GetFeatureInfo response in GeoJSON format can be used as a convenient "selection overlay" to display selected features when clicking on a WMS map.

Configurable geometry output for WMS GetFeatureInfo

Sometimes, one may not wish to have geometry output for certain WMS GetFeatureInfo requests. So for MapGuide 4.0, there is a new _EnableGeometry metadata option for Layer Definition resources, that determines if WMS GetFeatureInfo requests against this layer should return geometry data or not.

The next release of MapGuide Maestro lets you toggle this setting in the UI without having to mess around with resource header XML.

This setting is only applicable if the Layer Definition itself has been set to be queryable for WMS.

WFS Support for hit count

The spec for WFS GetFeatures defines a special mode where one can request a hit count (ie. A raw total) instead of the actual raw feature data. MapGuide did not implement this part of the WFS spec (it is optional). For MapGuide 4.0, this is now implemented.

If you pass resultType=hits to your WFS request, you now get a total instead of the feature data.

As an aside, if you use VSCode on the regular, I highly recommend you install the REST client extension. It has replaced Postman for my HTTP/REST API testing. As evidenced by the above screenshot, testing HTTP requests is dead simple.

Special thanks to OpenSpatial for their assistance in testing out this feature.

Viewer representation for WMS GetMap

As of the 4.0 preview 2 release, the mapagent now also supports a new viewer representation for WMS layers, giving you a built-in way to easily preview any published WMS layer in MapGuide by simply specifying a format of application/openlayers, which is a new option in the GetMap test page

In this format, a HTML page is returned which contains an OpenLayers viewer set up to the WMS layer(s) in question.

No more needing to fire up a client GIS application like Gaia or QGIS to preview such layers, MapGuide now provides the means to preview such layers out of the box!

Announcing: vscode-map-preview 0.5.6

This release updates OpenLayers to the latest 6.4.3, ol-layerswitcher to the latest 3.7.0 and adds support for opacity properties of the mapbox simple style spec for GeoJSON features.