mapguide-react-layout dev diary part 23: Adding more than just WMS layers

In the tail end of part 21, I did say that the next release (0.13) will have some actual new features besides some important under-the-hood updates and for this post we'll be talking about one such feature.

Way back in the 0.11 release, a new component was introduced to allow adding external WMS layers to your current map.

I knew at that point in time that this component had much more room for improvement as the wide array of formats that OpenLayers can support means we can add more than just WMS layers.

For the upcoming 0.13 release we've re-designed and re-built this component to allow adding a wider array of external layers to your map. When you bring up the external layer manager, you are now greeted with a choice of what kind of layer to add:

1. A local file-based layer
2. A remote URL-based layer

With a local file-based layer, simply drag a supported file type into the specified drop-zone or click the drop-zone itself to be prompted for a file to load. Once loaded, specify the projection of the data source (default will be EPSG:4326) and the layer will be added to the map.

Several things to note with local file-based layers.

Firstly, the files you pick are not uploaded to any server. The "upload" UI is merely a means to obtain a HTML5 FileReader so we can load the vector features client-side into the map directly.

Secondly, the following file formats are supported in this mode:

• GeoJSON/TopoJSON
• KML
• GPX
• IGC

Finally, the available projection list in the following dropdown:

Are built-in projections provided by the proj4js library we're using along with any additional projections registered while discovering map definitions or projections registered up-front.

The other mode is remote URL-based layers, which is how you add WMS and WFS layers through this re-designed UI.

UI for adding WFS layers is near identical, so it is not shown here.

Loaded layers can be viewed and managed through the Manage Layers tab, which itself has been re-designed as well.

Every layer in this list:

• Can have their visibility toggled via the switch
• Can have their opacity changed via the slider
• Can have their draw-order (relative to the MapGuide map) changed through the up/down arrow buttons
• Can be removed later on via the button with the trash icon

For WMS layers that support the LegendURL sub-capability, you can click the info button to show the WMS legend inline.

For added vector layers, you can zoom to the extents of that layer.

And for vector layers that aren't KML files, you can edit the style for these features. For KML files, the style is intrinsically part of the file itself, so the style is effectively "locked in" for such layers and is not editable as a result.

So in closing, the revised external layer manager now supports adding the following external data sources to your current map:

• Local files: GeoJSON/TopoJSON, KML, GPX, IGC
• Remote sources: WMS, WFS

We could actually add many more formats, but this revised external layer manager took a significant toll on our production bundle size and adding more formats would've blown up the bundle size to unacceptable levels. This list of formats I have determined to be a "good enough" list given our bundle size constraints.

Thanks to the storybook support, you can also see a live demo of this external layer manager here.

mapguide-react-layout dev diary part 22: It's story time!

With the major under-the-hood updates out of the way, it was time to tackle another long-standing item on my todo list. The ability to showcase the key components of mapguide-react-layout in storybook.

From storybook's home page introduction:

Storybook is a user interface development environment and playground for UI components. The tool enables developers to create components independently and showcase components interactively in an isolated development environment.

In the case of mapguide-react-layout, the motivation was to be able to leverage our existing gh-pages branch that currently hosts the project landing page and API docs to also host storybook to showcase the various react components that make up mapguide-react-layout as an [interactive playground / component documentation / pseudo-demo site] on GitHub Pages.

The major challenge to storybook adoption

Storybook has existed for quite some time now, so what was the major blocker to adopting storybook in mapguide-react-layout?

Namely, mapguide-react-layout would require a running MapGuide Server in order to properly showcase the viewer components. While I have no problems pointing storybook to a demo MapGuide Server if there was no other options, it shouldn't need to be a hard requirement. If we can intercept and where possible, mock the expected responses from the MapGuide Server, then it means it would simplify our ability to host storybook on GitHub Pages.

It turns out that it is indeed possible to "mock out" the MapGuide Server dependency:

• The requests to the mapagent are done through a dedicated class. We just needed a mechanism to register an alternate implementation that can just return canned response data for certain requests.
• The API for OpenLayers image sources allows us to register a custom "image load function". We can register our own image load function that intercepts the mapagent rendering request URL and using the HTML5 Canvas API, render out an alternate image that simply dumps out the key request parameters of note and export the rendered image out to a data URI to be assigned to the image element that OpenLayer provides.

The end result of this, is that it means we can showcase our map viewer components, but with the MapGuide Server communication bits mocked out and canned response data returned where needed.

We can showcase the map viewer component without dependency to a running MapGuide Server with the canned replacement image still providing useful information about what would happen on a real MapGuide Server.

Our test app def is set up with Stamen and OSM maps so that even though we're rendering a textual placeholder in place of the actual MapGuide-rendered map image, you still have some "real world context" that the Stamen/OSM base layer provides.

We can showcase the legend component by providing a canned CREATERUNTIMEMAP response of the Sheboygan map.

We can showcase our selection panel by providing a canned QUERYMAPFEATURES response.

And so on, and so on.

In closing ...

Storybook for mapguide-react-layout is now live on GitHub Pages. If you ever wanted to see or explore the components that make up this mapguide-react-layout and how they work without the need to spin up your own dev environment and/or a running MapGuide Server, we now have storybook for that.

It will be periodically kept up to date I'm guessing, with each new release in the future.

mapguide-react-layout dev diary part 21: Some long overdue updates and elbow grease

The previous blog series title is too long to type out, so I've shortened this blog series to be just called "mapguide-react-layout dev diary". It's much easier to type :)

So for this post, I'll be outlining some of the long overdue updates that have been done for the next (0.13) release (and why these updates have been held off for so long)

Due to the long gap between 0.11 and 0.12 releases I didn't want to rock the boat with some of the disruptive changes I had in the pipeline, choosing to postpone this work until 0.12 has settled down. Now that 0.12 is mostly stable (surely 8 bug fix releases should attest to that!), we can now focus on the original disruptive work I had planned and postponing the planned hiatus I had for this project.

Updating OpenLayers (finally!)

For the longest time, mapguide-react-layout was using OpenLayers 4.6.5. The reason we were stuck on this version was because this was the last version of OpenLayers where I was able to automatically generate a full-API-surface TypeScript d.ts definition file from the OpenLayers sources, through a JSDoc plugin that I built for this very purpose. This d.ts file provides the "intellisense" when using OpenLayers and type-checking so that we were actually using the OpenLayers API the way it was documented.

Up until the 4.6.5 release, this JSDoc plugin did its job very well. After this release, OpenLayers completely changed its module format for version 5.x onwards, breaking my ability to have updated d.ts files and without up-to-date d.ts files, I was not ready to update and given the expansiveness of the OpenLayers API surface, it was going to be a lot of work to generate this file properly for newer version of OpenLayers.

What brought me to originaly write this JSDoc plugin myself was that TypeScript compiler supported vanilla JavaScript (through the --allowJs flag), but for the longest time did not work in combination with the --declarations flag that allowed the TypeScript compiler to generate d.ts files from vanilla JS sources that we're properly annotated with JSDoc.

When I heard that this long standing limitation was finally going to be addressed in TypeScript 3.7, I took this as the time to see if we can upgrade to OpenLayers (now at version 6.x) and use the --allowJs + --declarations combination provided by TypeScript 3.7 to generate our own d.ts files for OpenLayers.

Sadly, it seems that the d.ts files generated through this combination aren't quite usable still, which was deflating news and I was about to put OL update plans on ice again until I learned of another typings effort for OpenLayers 5.x and above. As there were no other viable solutions, I decided to given these d.ts files a try. Despite lacking inline API documentation (which my JSDoc plugin was able to preserve when generating the d.ts files), these typings did accurately cover most of the OpenLayers API surface which gave me the impetus to make the full upgrade to OpenLayers 6.1.1, the latest release of OpenLayers as of this writing.

Updating Blueprint

Also for the longest time, mapguide-react-layout was using Blueprint 1.x. What previously held us off from upgrading, besides dealing with the expected breaking changes and fixing our viewer as a result, was that Blueprint introduced SVG icons as replacement for their font icons. While having SVG icons is great, having the full kitchen sink of Blueprint's SVG icons in our viewer bundle was not as that blew up our viewer bundle sizes to unacceptable levels.

For the longest time, this had been a blocker to fully upgrading Blueprint until I found someone suggesting a creative use of webpack's module replacement plugin to intercept the original full icon package and replace it with our own stripped-down subset. This workaround meant brought our viewer bundle size back to acceptable levels (ie. Only slightly larger than the 0.12.8 release). With this workaround in place, it was finally safe to upgrade to the latest version of Blueprint, which is 3.22 as of this writing.

Resizable Modal Dialogs!

So we finally upgraded Blueprint, but our Blueprint-styled modal dialogs were still fixed size things whose inability to be resized really hampered the user experience of features that spawned modal dialogs or made heavy use of them (eg. The Aqua viewer template). Since we're on the theme of doing things that are long overdue, I decided to tackle the problem of making these things resizable.

My original mental notes were to check out the react-rnd library and see how hard it was to integrate this into our modal dialogs. It turns out, this was actually not that hard at all! The react-rnd library was completely un-intrusive and as a bonus was lightweight as well meaning our bundle sizes weren't going to blow out significantly as well.

So say hello to the updated Aqua template, with resizable modal dialogs!

Now unfortunately, we didn't win everything here. The work to update Blueprint and make these modals finally resizable broke our ability to have modal dialogs with a darkened backdrop like this:

This was due to overlay changes introduced with Blueprint. My current line of thinking around this is to ... just remove support for darkened backdrops. I don't think losing this support is such a big loss in the grand scheme of things.

Hook all of the react components

The other long overdue item was upgrading our react-redux package. We had held on to our specific version (5.1.1) for the longest time because we had usages of its legacy context API to be able to dispatch any redux action from toolbar commands. The latest version removed this legacy context API which meant upgrading would require us to re-architect how our toolbar component constructed its toolbar items.

We were also using the connect() API, which combined with our class-based container components produced something that required a lot of pointless type-checking and in some cases forced me to fall back to using the any type to describe things

It turns out that the latest version of react-redux offered a hooks-based alternative for its APIs and having been sold on the power of hooks in react in my day job, I took this upgrade as an opportunity to convert all our class-based components over to functional ones using hooks and the results were most impressive.

Moving away from class-based container components and using the react-redux hooks API meant that we no longer needed to type state/dispatch prop interfaces for all our container components. These interfaces had to have all optional props as they're not required when rendering out a container component, but were set as part of when the component is connect()-ed to the redux store. This optionality infected the type system, meaning we had to do lots of pointless null checks in our container components for props that could not be null or undefined but we had to check anyway because of our state/dispatch interfaces saying so.

Using the hooks API mean that state/dispatch interfaces are no longer required as they are now all implementation details of the container component through the new useDispatch and useSelector hook APIs. It means that we no longer need to do a whole lot of pointless checks for null or undefined. Moving to functional components with hooks means we no longer need to use the connect() API (we just default export the functional component itself) and having to use the "any" type band-aid as well.

To see some visual evidence of how much cleaner and more compact our container components are, consider one of our simplest container components, the "selected features" counter:

• Before: From the 0.12.8 release
• After: Current master

A useful property of hooks is that they are composable so from the low-level useSelector hook that react-redux gives us, we can build a series of specialized and reusable hooks on top to access common viewer and application state across all our container components. The useViewerLocale hook in the linked master example above is just one of many reusable hooks that's been built that all our container components use now. The useSelector API encourages us to return scalar values instead of objects (as objects would require custom equality comparisons for testing re-rendering) and you see that reflected in most of the hooks that have been written, which mostly return single values. 

Usage of the new hooks API provided some valuable insights when doing the needed re-architecting of how our toolbar component constructs its toolbar items. In our original implementation, we pulled the full application state for determining if toolbar items are selected/disabled/etc. This meant that even the most innocuous change of state like change of mouse coordinates would cause our toolbars to re-render themselves (because we were listening to the full application state), causing the react devtools to light up like a christmas tree when highlighting re-renders was enabled.

In re-architecting our toolbar, it forced us to take a look at what part of the application state we actually cared about when determining if a given toolbar item should be selected/disabled/etc. The end result is that we really only cared about 6 bits of state and so we now have a custom hook that only returned this subset and only triggering re-renders when any part of this subset has actually changed. The end result of this being: The UI is more responsive because the toolbars aren't constantly re-rendering due to updates to state that is not relevant to toolbar items.

In closing ...

The main objective of the next 0.13 release was to carry out some long overdue updates to key libraries we were using, which we've passed with rousing success.

But despite it being our main objective, that doesn't mean 0.13 is going to be released immediately, there's still actual features I want to get into this release, which will (of course) be the topic of various dev diary entries in the future.

Announcing: mapguide-react-layout 0.12.8

This bugfix release fixes additional localization holes in:

• The feature tooltip prompt text when it contains a hyperlink
• The share link to view component.

Also for any commands that open a modal dialog, it now uses the label of the command as the dialog title instead of the command name.

This release will be the last one in the 0.12.x series as I move full steam ahead with the next 0.13 release. Yeah ... slight change of plans about putting this project on short hiatus. 

It turns out mapguide-react-layout needs some major updates for several key libraries its using, so it's not worth holding off on this any longer.

The journey to this next release is worth a long overdue blog post to talk about it as well.

Project Home Page

Download

mapguide-react-layout on npm

Making the MySQL FDO provider ... geospatially useful

For the longest time, the MySQL FDO provider was of limited utility, though not to the fault of the provider itself. The last time this provider saw serious development, the latest version of MySQL was around 5.0/5.1 and the spatial capabilities of MySQL at that point in time left a lot to be desired.

While you could store spatial data, querying this data out spatially was another matter. MySQL at this point in time only offered bounding-box-based spatial predicates which manifests in user-facing viewer behavior like this:

Why would such a box selection query select that line? Because their minimal bounding boxes spatially intersect according to MySQL's limited spatial predicates. While MySQL 5.6 finally introduced proper spatial capabilities, the provider itself was still working against the pre-5.6 feature set until recently.

After receiving signs that people still use MySQL for spatial data, I've finally decided to tackle this long standing annoyance. If you have MapGuide Open Source 3.1.2 64-bit installed, you can download the patched MySQL Provider to unlock the full set of spatial capabilities if you are connected to MySQL 5.6 or higher, the end result is that map selections against MySQL data sources now actually make sense!

That line is no longer selected out of nowhere! You have to actually box select on the line, like an actual ST_Intersects spatial predicate should!

Since MySQL has long been forked into another popular and highly-compatible fork called MariaDB, I did some testing to make sure this provider works against MariaDB as well. It turns out that the [MySQL version is >= 5.6] check the provider does to determine whether to unlock the full spatial capabilities in the provider is not quite correct when working with MariaDB. The problem was that the version checking APIs provided by the MySQL/MariaDB client return "5.5.5" when connected to MariaDB, which breaks the version check as 5.5 < 5.6.

This left me scratching my head for a bit as why would MariaDB 10.4 (the version of MariaDB I was testing) return a version of "5.5.5"? It turns out this version number has special meaning as a versioning hack to support replication compatibility with MySQL. The real version can be obtained by getting the version string and if it contains "mariadb", check for the "5.5.5-" prefix and if it's present, parse the version number following that prefix for the real version number. Because there was never a release of MySQL 5.5.5, the presence of this "5.5.5-" prefix in the version string gives us 99.9% certainty that we're actually dealing with MariaDB. With this change, the provider will now have full spatial capabilities when connected to MariaDB as well.

These changes will be rolled into FDO trunk and will be part of the MySQL FDO Provider that ships with the next preview release of MapGuide Open Source 4.0

Announcing: mapguide-react-layout 0.12.7

This bugfix release plugs more localization holes in:

• The display of measurement segments
• The loading screen text (NOTE: The loading text will still be in english until the non-english string bundle has been fetched and registered successfully. This moment should be brief)

The release now auto-closes the Task Pane menu (if open) if the Task Pane is made invisible/hidden.

Project Home Page
Download
mapguide-react-layout on npm

Announcing: mapguide-react-layout 0.12.6

Another release? So quickly?

So between these various releases, I had updated webpack and it had somehow introduced some behavioral changes such that my "production" webpack config no longer produced a minified production bundle.

This meant that for the past few bugfix releases, the production bundle was weighing around 5MB! Not good.

It didn't help that our CIs can't detect such a problem easily. It would be great if there was a service that integrates with TravisCI/AppVeyor and allows us to track and monitor production bundle file sizes and preferably raise an alert or fail the build if the new production bundle size increases by an abnormal amount. If such a service exists, and is free and can integrate with TravisCI/AppVeyor, I'd like to know about it.

This release fixes our production webpack config so that the production viewer bundle is properly minified again and back down to its normal expected size.

In addition, this release adds new APIs for our OL factory so that task pane content has a greater range of types of OL styles to work with when creating/managing client-side vector features.

Project Home Page
Download
mapguide-react-layout on npm