Tuesday 17 December 2019

mapguide-react-layout dev diary part 22: It's story time!

With the major under-the-hood updates out of the way, it was time to tackle another long-standing item on my todo list. The ability to showcase the key components of mapguide-react-layout in storybook.

From storybook's home page introduction:
Storybook is a user interface development environment and playground for UI components. The tool enables developers to create components independently and showcase components interactively in an isolated development environment.
In the case of mapguide-react-layout, the motivation was to be able to leverage our existing gh-pages branch that currently hosts the project landing page and API docs to also host storybook to showcase the various react components that make up mapguide-react-layout as an [interactive playground / component documentation / pseudo-demo site] on GitHub Pages.

The major challenge to storybook adoption

Storybook has existed for quite some time now, so what was the major blocker to adopting storybook in mapguide-react-layout?

Namely, mapguide-react-layout would require a running MapGuide Server in order to properly showcase the viewer components. While I have no problems pointing storybook to a demo MapGuide Server if there was no other options, it shouldn't need to be a hard requirement. If we can intercept and where possible, mock the expected responses from the MapGuide Server, then it means it would simplify our ability to host storybook on GitHub Pages.

It turns out that it is indeed possible to "mock out" the MapGuide Server dependency:
  • The requests to the mapagent are done through a dedicated class. We just needed a mechanism to register an alternate implementation that can just return canned response data for certain requests.
  • The API for OpenLayers image sources allows us to register a custom "image load function". We can register our own image load function that intercepts the mapagent rendering request URL and using the HTML5 Canvas API, render out an alternate image that simply dumps out the key request parameters of note and export the rendered image out to a data URI to be assigned to the image element that OpenLayer provides.
The end result of this, is that it means we can showcase our map viewer components, but with the MapGuide Server communication bits mocked out and canned response data returned where needed.

We can showcase the map viewer component without dependency to a running MapGuide Server with the canned replacement image still providing useful information about what would happen on a real MapGuide Server.



Our test app def is set up with Stamen and OSM maps so that even though we're rendering a textual placeholder in place of the actual MapGuide-rendered map image, you still have some "real world context" that the Stamen/OSM base layer provides.

We can showcase the legend component by providing a canned CREATERUNTIMEMAP response of the Sheboygan map.



We can showcase our selection panel by providing a canned QUERYMAPFEATURES response.



And so on, and so on.

In closing ...

Storybook for mapguide-react-layout is now live on GitHub Pages. If you ever wanted to see or explore the components that make up this mapguide-react-layout and how they work without the need to spin up your own dev environment and/or a running MapGuide Server, we now have storybook for that.

It will be periodically kept up to date I'm guessing, with each new release in the future.

Sunday 15 December 2019

mapguide-react-layout dev diary part 21: Some long overdue updates and elbow grease

The previous blog series title is too long to type out, so I've shortened this blog series to be just called "mapguide-react-layout dev diary". It's much easier to type :)

So for this post, I'll be outlining some of the long overdue updates that have been done for the next (0.13) release (and why these updates have been held off for so long)

Due to the long gap between 0.11 and 0.12 releases I didn't want to rock the boat with some of the disruptive changes I had in the pipeline, choosing to postpone this work until 0.12 has settled down. Now that 0.12 is mostly stable (surely 8 bug fix releases should attest to that!), we can now focus on the original disruptive work I had planned and postponing the planned hiatus I had for this project.

Updating OpenLayers (finally!)

For the longest time, mapguide-react-layout was using OpenLayers 4.6.5. The reason we were stuck on this version was because this was the last version of OpenLayers where I was able to automatically generate a full-API-surface TypeScript d.ts definition file from the OpenLayers sources, through a JSDoc plugin that I built for this very purpose. This d.ts file provides the "intellisense" when using OpenLayers and type-checking so that we were actually using the OpenLayers API the way it was documented.

Up until the 4.6.5 release, this JSDoc plugin did its job very well. After this release, OpenLayers completely changed its module format for version 5.x onwards, breaking my ability to have updated d.ts files and without up-to-date d.ts files, I was not ready to update and given the expansiveness of the OpenLayers API surface, it was going to be a lot of work to generate this file properly for newer version of OpenLayers.

What brought me to originaly write this JSDoc plugin myself was that TypeScript compiler supported vanilla JavaScript (through the --allowJs flag), but for the longest time did not work in combination with the --declarations flag that allowed the TypeScript compiler to generate d.ts files from vanilla JS sources that we're properly annotated with JSDoc.

When I heard that this long standing limitation was finally going to be addressed in TypeScript 3.7, I took this as the time to see if we can upgrade to OpenLayers (now at version 6.x) and use the --allowJs + --declarations combination provided by TypeScript 3.7 to generate our own d.ts files for OpenLayers.

Sadly, it seems that the d.ts files generated through this combination aren't quite usable still, which was deflating news and I was about to put OL update plans on ice again until I learned of another typings effort for OpenLayers 5.x and above. As there were no other viable solutions, I decided to given these d.ts files a try. Despite lacking inline API documentation (which my JSDoc plugin was able to preserve when generating the d.ts files), these typings did accurately cover most of the OpenLayers API surface which gave me the impetus to make the full upgrade to OpenLayers 6.1.1, the latest release of OpenLayers as of this writing.

Updating Blueprint

Also for the longest time, mapguide-react-layout was using Blueprint 1.x. What previously held us off from upgrading, besides dealing with the expected breaking changes and fixing our viewer as a result, was that Blueprint introduced SVG icons as replacement for their font icons. While having SVG icons is great, having the full kitchen sink of Blueprint's SVG icons in our viewer bundle was not as that blew up our viewer bundle sizes to unacceptable levels.

For the longest time, this had been a blocker to fully upgrading Blueprint until I found someone suggesting a creative use of webpack's module replacement plugin to intercept the original full icon package and replace it with our own stripped-down subset. This workaround meant brought our viewer bundle size back to acceptable levels (ie. Only slightly larger than the 0.12.8 release). With this workaround in place, it was finally safe to upgrade to the latest version of Blueprint, which is 3.22 as of this writing.

Resizable Modal Dialogs!

So we finally upgraded Blueprint, but our Blueprint-styled modal dialogs were still fixed size things whose inability to be resized really hampered the user experience of features that spawned modal dialogs or made heavy use of them (eg. The Aqua viewer template). Since we're on the theme of doing things that are long overdue, I decided to tackle the problem of making these things resizable.

My original mental notes were to check out the react-rnd library and see how hard it was to integrate this into our modal dialogs. It turns out, this was actually not that hard at all! The react-rnd library was completely un-intrusive and as a bonus was lightweight as well meaning our bundle sizes weren't going to blow out significantly as well.

So say hello to the updated Aqua template, with resizable modal dialogs!


Now unfortunately, we didn't win everything here. The work to update Blueprint and make these modals finally resizable broke our ability to have modal dialogs with a darkened backdrop like this:


This was due to overlay changes introduced with Blueprint. My current line of thinking around this is to ... just remove support for darkened backdrops. I don't think losing this support is such a big loss in the grand scheme of things.

Hook all of the react components

The other long overdue item was upgrading our react-redux package. We had held on to our specific version (5.1.1) for the longest time because we had usages of its legacy context API to be able to dispatch any redux action from toolbar commands. The latest version removed this legacy context API which meant upgrading would require us to re-architect how our toolbar component constructed its toolbar items.

We were also using the connect() API, which combined with our class-based container components produced something that required a lot of pointless type-checking and in some cases forced me to fall back to using the any type to describe things


It turns out that the latest version of react-redux offered a hooks-based alternative for its APIs and having been sold on the power of hooks in react in my day job, I took this upgrade as an opportunity to convert all our class-based components over to functional ones using hooks and the results were most impressive.

Moving away from class-based container components and using the react-redux hooks API meant that we no longer needed to type state/dispatch prop interfaces for all our container components. These interfaces had to have all optional props as they're not required when rendering out a container component, but were set as part of when the component is connect()-ed to the redux store. This optionality infected the type system, meaning we had to do lots of pointless null checks in our container components for props that could not be null or undefined but we had to check anyway because of our state/dispatch interfaces saying so.

Using the hooks API mean that state/dispatch interfaces are no longer required as they are now all implementation details of the container component through the new useDispatch and useSelector hook APIs. It means that we no longer need to do a whole lot of pointless checks for null or undefined. Moving to functional components with hooks means we no longer need to use the connect() API (we just default export the functional component itself) and having to use the "any" type band-aid as well.

To see some visual evidence of how much cleaner and more compact our container components are, consider one of our simplest container components, the "selected features" counter:

A useful property of hooks is that they are composable so from the low-level useSelector hook that react-redux gives us, we can build a series of specialized and reusable hooks on top to access common viewer and application state across all our container components. The useViewerLocale hook in the linked master example above is just one of many reusable hooks that's been built that all our container components use now. The useSelector API encourages us to return scalar values instead of objects (as objects would require custom equality comparisons for testing re-rendering) and you see that reflected in most of the hooks that have been written, which mostly return single values. 

Usage of the new hooks API provided some valuable insights when doing the needed re-architecting of how our toolbar component constructs its toolbar items. In our original implementation, we pulled the full application state for determining if toolbar items are selected/disabled/etc. This meant that even the most innocuous change of state like change of mouse coordinates would cause our toolbars to re-render themselves (because we were listening to the full application state), causing the react devtools to light up like a christmas tree when highlighting re-renders was enabled.

In re-architecting our toolbar, it forced us to take a look at what part of the application state we actually cared about when determining if a given toolbar item should be selected/disabled/etc. The end result is that we really only cared about 6 bits of state and so we now have a custom hook that only returned this subset and only triggering re-renders when any part of this subset has actually changed. The end result of this being: The UI is more responsive because the toolbars aren't constantly re-rendering due to updates to state that is not relevant to toolbar items.

In closing ...

The main objective of the next 0.13 release was to carry out some long overdue updates to key libraries we were using, which we've passed with rousing success.

But despite it being our main objective, that doesn't mean 0.13 is going to be released immediately, there's still actual features I want to get into this release, which will (of course) be the topic of various dev diary entries in the future.

Thursday 5 December 2019

Announcing: mapguide-react-layout 0.12.8

This bugfix release fixes additional localization holes in:

  • The feature tooltip prompt text when it contains a hyperlink
  • The share link to view component.
Also for any commands that open a modal dialog, it now uses the label of the command as the dialog title instead of the command name.

This release will be the last one in the 0.12.x series as I move full steam ahead with the next 0.13 release. Yeah ... slight change of plans about putting this project on short hiatus

It turns out mapguide-react-layout needs some major updates for several key libraries its using, so it's not worth holding off on this any longer.

The journey to this next release is worth a long overdue blog post to talk about it as well.

Thursday 28 November 2019

Making the MySQL FDO provider ... geospatially useful

For the longest time, the MySQL FDO provider was of limited utility, though not to the fault of the provider itself. The last time this provider saw serious development, the latest version of MySQL was around 5.0/5.1 and the spatial capabilities of MySQL at that point in time left a lot to be desired.

While you could store spatial data, querying this data out spatially was another matter. MySQL at this point in time only offered bounding-box-based spatial predicates which manifests in user-facing viewer behavior like this:



Why would such a box selection query select that line? Because their minimal bounding boxes spatially intersect according to MySQL's limited spatial predicates. While MySQL 5.6 finally introduced proper spatial capabilities, the provider itself was still working against the pre-5.6 feature set until recently.

After receiving signs that people still use MySQL for spatial data, I've finally decided to tackle this long standing annoyance. If you have MapGuide Open Source 3.1.2 64-bit installed, you can download the patched MySQL Provider to unlock the full set of spatial capabilities if you are connected to MySQL 5.6 or higher, the end result is that map selections against MySQL data sources now actually make sense!


That line is no longer selected out of nowhere! You have to actually box select on the line, like an actual ST_Intersects spatial predicate should!


Since MySQL has long been forked into another popular and highly-compatible fork called MariaDB, I did some testing to make sure this provider works against MariaDB as well. It turns out that the [MySQL version is >= 5.6] check the provider does to determine whether to unlock the full spatial capabilities in the provider is not quite correct when working with MariaDB. The problem was that the version checking APIs provided by the MySQL/MariaDB client return "5.5.5" when connected to MariaDB, which breaks the version check as 5.5 < 5.6.

This left me scratching my head for a bit as why would MariaDB 10.4 (the version of MariaDB I was testing) return a version of "5.5.5"? It turns out this version number has special meaning as a versioning hack to support replication compatibility with MySQL. The real version can be obtained by getting the version string and if it contains "mariadb", check for the "5.5.5-" prefix and if it's present, parse the version number following that prefix for the real version number. Because there was never a release of MySQL 5.5.5, the presence of this "5.5.5-" prefix in the version string gives us 99.9% certainty that we're actually dealing with MariaDB. With this change, the provider will now have full spatial capabilities when connected to MariaDB as well.

These changes will be rolled into FDO trunk and will be part of the MySQL FDO Provider that ships with the next preview release of MapGuide Open Source 4.0

Monday 18 November 2019

Announcing: mapguide-react-layout 0.12.7

This bugfix release plugs more localization holes in:

  • The display of measurement segments
  • The loading screen text (NOTE: The loading text will still be in english until the non-english string bundle has been fetched and registered successfully. This moment should be brief)
The release now auto-closes the Task Pane menu (if open) if the Task Pane is made invisible/hidden.


Project Home Page
Download
mapguide-react-layout on npm

Thursday 7 November 2019

Announcing: mapguide-react-layout 0.12.6

Another release? So quickly?

So between these various releases, I had updated webpack and it had somehow introduced some behavioral changes such that my "production" webpack config no longer produced a minified production bundle.

This meant that for the past few bugfix releases, the production bundle was weighing around 5MB! Not good.

It didn't help that our CIs can't detect such a problem easily. It would be great if there was a service that integrates with TravisCI/AppVeyor and allows us to track and monitor production bundle file sizes and preferably raise an alert or fail the build if the new production bundle size increases by an abnormal amount. If such a service exists, and is free and can integrate with TravisCI/AppVeyor, I'd like to know about it.

This release fixes our production webpack config so that the production viewer bundle is properly minified again and back down to its normal expected size.

In addition, this release adds new APIs for our OL factory so that task pane content has a greater range of types of OL styles to work with when creating/managing client-side vector features.

Project Home Page
Download
mapguide-react-layout on npm

Announcing: mapguide-react-layout 0.12.5

People keep reporting reproducible bugs, so I keep making bug fix releases :)

This release fixes the following:

  • Fix clicked hyperlinks in tooltip content not opening in Task Pane
  • Default initial active viewer tool to "Pan" instead of being not set
  • Fix default format string for mouse coordinates component missing localized units display
  • Fix drawing task pane sample

Friday 18 October 2019

500 posts!

The state of my admin panel before I hit publish on this post


And given the amount of posts still in the draft queue in various states of completed-ness, it is clear that there's still plenty of things for me to talk about in the realm of MapGuide, FDO, general GIS and software/tech trickery.

So here's to the next 500 posts!


Announcing: mapguide-react-layout 0.12.4

Since porting the Maestro API over to target .net standard and making the library truly cross-platform, the SDK story of the Maestro API was left in an somewhat undefined state. If you wanted to use the Maestro API for your own applications, where do you start?

For many milestones since that migration, I have intentionally side-stepped from answering that question until now. For the next milestone of MapGuide Maestro we'll be re-activating the Maestro SDK story in the form of a separate repository that contains:

  • Various code samples using the Maestro API
  • Source code template for the "Local Native" connection provider that wraps the official MapGuide .net API with instructions on how to roll your own build of this provider
So as a result, I've been re-vamping some of the code samples, especially usage in your own web applications. The last time I touched this code, it was all ASP.net Web Forms, not exactly a shining example of how best to use the Maestro API in your own web application in our current year.

Obviously, this means we need code samples that demonstrate usage of the Maestro API in ASP.net core. So in the process of building this modern code sample, I found some interesting challenges. 

Firstly, an ASP.net core application runs on its own webserver (Kestrel or IIS express). Our mapagent and built-in viewers are hosted on Apache or your local IIS. Even though in development they would both be on localhost, the web servers listen on different ports, meaning the same-origin policy is in play, complicating integration.

After some deliberation, I've decided that rather than try to embed the existing AJAX viewer in an iframe (which is what the old webforms sample did), I'll eat my own dogfood and just use the mapguide-react-layout as the viewer for the new asp.net core sample and set it up to talk to the existing mapagent of your MapGuide installation. This approach requires enabling CORS to make the mapagent accessible from a different origin. I figured out how this is actually done, which I'll show you in another blog post.

So with this main challenge addressed, I launched the work-in-progress asp.net core sample with the mapguide-react-layout viewer, and I get this problem.

For some context, this code sample does the following:
  1. Create a new session id
  2. Create a new web layout on-the-fly that sets up the task pane to point to our custom landing page of code samples
  3. Save this web layout into a session repository and then render the MVC view instructing the embedded mapguide-react-layout viewer to load from this weblayout and session id
It turns out that mapguide-react-layout misinterpreted this initial state and tries to load the viewer as though the user manually refreshed the browser (the URL state recovery feature introduced in 0.11) and assumes that a runtime map already exists

This was clearly a bug and since I needed this viewer to behave properly in the context of a new code sample for a re-activated SDK story for the next milestone release of MapGuide Maestro, this has been the longest block of text to segue into ...

... Announcing a new bugfix release of mapguide-react-layout :)

This release also fixes the font icons so they aren't a scrambled mess and fixes some minor warnings around some of the libraries we're using.

Saturday 28 September 2019

Announcing: mapguide-react-layout 0.12.3

This bugfix release fixes a bug where the initial map view (if specified in a flexible layout) is parsed incorrectly, breaks the viewer as a result.

Project Home Page
Download
mapguide-react-layout on npm

Thursday 26 September 2019

MapGuide 4.0 Showcase: Transform all the things!

Did you know that MapGuide uses CS-Map, a coordinate system transformation library with support for many thousands of coordinate systems out of the box? No matter how esoteric the map projection your data is in, MapGuide can re-project geospatial data in and out of it thanks to this library.

So it is quite a shame that for the longest time, MapGuide's powerful coordinate system transformation capabilities has zero representation in any part of the mapagent (the HTTP API provided by MapGuide).

To use MgCooordinateSystem and friends (the classes that wrap the CS-Map library) requires building a your own MapGuide application using the MapGuide Web API in one of our supported languages (.net, Java or PHP) and having your client application call into your own MapGuide application. There is nothing out of the box in the mapagent for a client map viewer application to do a basic HTTP request for transforming coordinates or requesting feature data to be transformed to a certain coordinate system.

For MapGuide 4.0, we've exposed the coordinate system transformation capabilities in APIs where it makes sense. Such as our SELECTFEATURES APIs for returning feature data. In my previous showcase post, I've shown how appending VERSION=4.0.0 and CLEAN=1 now gives you an intuitive GeoJSON result for this API.



The coordinates are based on the coordinate system of the data source it comes from. In the above screenshot, this is latitude/longitudes (code: LL84, epsg: 4326).

Suppose you want this data in web mercator (code: WGS84.PseudoMercator, epsg: 3857) so you can easily plonk this GeoJSON onto a slippy map with OpenStreetMap or your own XYZ tiles, which would also be in web mercator. With MapGuide 4.0, you can now also include a TRANSFORMTO=WGS84.PseudoMercator parameter to your HTTP request and that GeoJSON data will be transformed to web mercator.


The other major capability gap is that the mapagent offers no API over the mapagent for transforming a series of coordinates from one coordinate system to another. For MapGuide 4.0, there's now a dedicated API for doing this: The CS.TRANSFORMCOORDINATES operation.

With this operation, simply feed it:

  • A comma-separated list of space-separated coordinate pairs (COORDINATES)
  • The CS-Map code of the source coordinate system the above coordinates are in (SOURCE)
  • The CS-Map code of the target coordinate system to transform to (TARGET)
  • Your desired output format of JSON or XML (FORMAT)
  • If you want JSON, whether you want the pretty JSON version (CLEAN)
Invoking the operation will give you back a collection of transformed coordinates.



A place where this new API is sorely needed is MapGuide Maestro, which previously required kludgy workarounds for transforming bounding boxes in Map Definitions (when adding layers in different coordinate systems, or re-computing extents):
The next milestone of MapGuide Maestro will now take advantage of this new API for transformation when connected to a MapGuide 4.0 or newer server.

Tuesday 24 September 2019

Now *THIS* is progress!

So in my previous post, it turns out I had mis-configured the tile grid property of the OL vector tile source in this example, namely the tile size. Is it not the default 256x256, but 4096x4096.

Once I fixed the tile size, everything beautifully snaps into place!





It is still progress

Even if the mapbox vector tiles are completely out of alignment, progress is still progress.

I'm seeing stuff!


Saturday 21 September 2019

MapGuide 4.0 showcase: JSON that's easier to work with.

If you've ever worked with the HTTP-based mapagent API, you should be aware that most of the APIs responses in the mapagent generally come back in 2 forms:

  • XML
  • JSON
However, if you ever look at the JSON responses, you'll be accustomed to dealing with ugly monstrosities like this (eg. A JSON version of a Layer Definition):



Or heaven forbid, you want to explore the structure of a Feature Source and so you need the JSON version of a FDO feature schema:



Good luck trying to parse and comprehend that!

Why is the JSON so horrible? The answer is XML is the canonical response format for all non-image/binary mapagent operations (MapGuide Open Source's inception was in 2004 after all, when XML was king and JSON probably didn't exist yet), so the JSON version is a literal translation of whatever its XML form would be. While there is nothing wrong with that, the way this JSON translation was done is the most lowest-common-denominator approach:
  • Treat every XML element as a JSON array.
  • Treat every XML element text body as a JSON string.
As a result, as evidenced by the above screenshots that means that all JSON responses are nothing but a series of JSON arrays (of possible more JSON arrays) of strings.

But there is a reason for such laziness. The JSON translation knows nothing about the content model of whatever XML it's trying to translate, so it really has no choice in the matter. Such content models are defined in the various XML schemas that are shipped with MapGuide. But working with XML schemas in C++ sounds like a nightmare in and of itself, so there is another way to determine the content model: Just manually hard-code the list of all possible XML xpaths that are:
  • Repeating (ie. Should be converted to actual JSON arrays)
  • Not a string
Using the existing XML schemas as the reference, this list only required a one-off painstaking translation of this xpath list. The end result isn't exactly pretty, but it gets the job done. With this hard-coded list now in our JSON converter, it means we now have the means to output JSON that is much easier to comprehend and intuitive! 

For MGOS 4.0, for any mapagent operation that returns JSON, if you specify VERSION=4.0.0 and CLEAN=1 in the request parameters you will now get a JSON format that actually makes sense!



Now for some mapagent operations, finally having a "clean" JSON version still isn't enough because the clean JSON response is still mostly clunky and unusable. Consider the FDO feature schema example. The original JSON version is an unworkable monstrosity, and the "clean" version would've just been lipstick on a pig because at the end of the day, it is still a literal JSON translation of an XML schema! 



So as a result, for this set of APIs, a new simpler XML response format is introduced for schema and class definitions by including SIMPLE=1 to your VERSION=4.0.0 request:



From which its JSON equivalent (I hope you'll agree) is much more easier to comprehend!



There is another class of operations that have received a similar treatment. Operations that return geometric/feature data. Consider the XML form of selecting features (the ugly schema section collapsed for brevity):



This is what the "ugly" JSON version would've looked like (schema parts again collapsed for brevity). 



Even if we prettied this up with content-model-aware conversion, the JSON is still mostly unremarkable. Because, if we're serving JSON feature data out to clients, then there's really only one format we should care about supporting:


GeoJSON is the de-facto format for serving out JSON-based feature data, and is universally supported by any mapping/GIS product/tool/library/viewer worth a damn, so if we have the opportunity for MapGuide to return clean JSON data for geometries/features, forget about the literal conversion from XML approach! Just go ahead and make GeoJSON output a first-class citizen

So for MGOS 4.0, if you run any mapagent operation that returns geometry/feature data and you ask for VERSION=4.0.0 and CLEAN=1 you will now get GeoJSON!



Which (due to GeoJSON's ubiquity) can be plugged directly into your web mapping library of choice for display if you so choose.

If you are building your own server-side MapGuide applications with the MapGuide Web API, there's a new MgGeoJsonWriter class is available for you to convert MgFeatureReaders to GeoJSON data.

With MapGuide Open Source 4.0, JSON support is no longer an afterthought, it's actually usable!

Friday 20 September 2019

Announcing: mapguide-react-layout 0.12.2

This bugfix release plugs remaining localization gaps in the measurement tool and tooltips for various OpenLayers controls we are using.

Project Home Page
Download
mapguide-react-layout on npm

Sunday 15 September 2019

Announcing: mapguide-react-layout 0.12.1

This bug fix release fixes the broken WMS layer manager that was discovered after making the 0.12 release and also refines the selection sub-highlighting capability check to exclude 4.0 preview 1 and older servers as the 4.0 preview 1 release had a broken implementation of QUERYMAPFEATURES 4.0.0.

Project Home Page
Download
mapguide-react-layout on npm

Announcing: mapguide-react-layout 0.12.0

As promised, here's the long overdue new release of mapguide-react-layout.

As stated in my revised plans, this release marks the point where this project will go on a short development hiatus as I give MapGuide Open Source maximum development priority for the next few months. I may put out a bugfix release or two if the need arises.

Here's what's new and notable in this release.

Support for XYZ tile sets in Flexible Layouts

Fusion in MapGuide was recently able to finally support XYZ tile sets defined in a Flexible Layout (aka. Application Definition) document. That same support has been extended to this release

XYZ tile sets can come from many different sources. It could be your own XYZ tile set definitions or a custom OSM flavor tile set, or Here Maps (as shown below)


Or many more!

Selections now live through browser refreshes

Map selection state is now stashed into browser local storage and recovered from on startup upon a browser refresh


Sub-selection highlighting now only enabled for MapGuide Open Source 4.0 (and newer) servers

Our neat little trick of sub-selecting selected features actually has a major flaw. It is built on the erroneous assumption that selection keys and selection attributes in a QUERYMAPFEATURES response are rendered in the same order (we used the current index of the selected feature's attributes to pick up the selection key at the same index to request a sub-selection for). This turned out to be sadly not the case.

Rectifying this required a new version of the QUERYMAPFEATURES API that will also render selection keys alongside attributes (to avoid the same-index selection key lookup). This has been implemented for the MGOS 4.0 preview 1 release.

But what that means is that sadly we have to disable sub-selection highlighting for older MapGuide Servers. This feature is now only enabled for MGOS 4.0 preview 1 (and newer) servers.

Share Link To View component can now copy to clipboard

This component now includes a button to copy the link to the clipboard, saving you the need to select the URL and copy yourself.


NOTE: This component will not include the map selection if another user opens the shared link on another browser/computer as that selection state lives in the user's browser local storage and is too big to reliably encode into the URL itself for sharing.

Smaller JS bundle size

The viewer JS bundle previously bundled all the necessary CSS within the JS. For this release, that CSS has been extracted out to a separate viewer.css file. This has 2 benefits.

  1. The JS bundle is now smaller
  2. The viewer CSS can be modified if you so please.

Compare the bundle size of the previous 0.l1.2 release

Against this 0.12 release

Other Changes
  • Plug remaining i18n holes in measure and WMS layer manager components.
  • Don't word wrap layer/group node labels in Legend component and provide tooltips for these labels
  • Remove non-geodesic measure option as it is mostly inaccurate.
  • Fix init failure if appdef has an empty widget container
  • Fix line measurement total showing units in m^2
  • Fix measure tool starting measuring when switching measurement type
  • Fix memory leak due to dangling measure components being held on by the measure context after unmount
  • Various components updated to use componentDidUpdate instead of componentWillReceiveProps
  • Made redux actions more strongly-typed
  • Updated Blueprint to 1.40.0
  • Updated TypeScript to 3.6.3
  • Updated React to 16.9.0
  • Replaced query-string module with qs
  • Now built with webpack 4.x

Project Home Page
Download
mapguide-react-layout on npm

Tuesday 3 September 2019

MapGuide 4.0 showcase: Supercharged tile sets

As promised, this is the first (of many) blog posts showcasing the new features of the just released MapGuide Open Source 4.0 Preview 1. Before we dive in, here's a little recap for some extra context.

Way back in MapGuide Open Source 3.0, we introduced the concept of a Tile Set Definition, a new resource type that allows one to define a shareable, re-usable tile set that was free of (most) server-imposed global default settings and is decoupled from any particular map or application. Before the introduction of shareable tile sets, a tile cache could only be defined within a Map Definition and was thus coupled to any application consuming this particular map. This setup also made the respective tile cache extremely prone to invalidation upon any minute changes to the Map Definition. Tile settings like size and image format were global (in serverconfig.ini) making it impossible to have tile sets of various sizes and image formats.

With the introduction of shareable tile sets, it means that tile sets no longer need to be defined on the Map Definition. Instead, it can be defined in an external Tile Set Definition resource and linked into any Map Definition (hence the shareable aspect). Being a linked resource, also means any changes to a Map Definition will not errantly flush cached tiles for the associated tile set as we have solid guarantees that editing a Map Definition that links to a tile set would not alter or compromise the layer/group structure of a tile set being linked that would normally necessitate invalidating the tile cache by deleting all previously cached tile images.

Because tile sets are now their own separate resources, it also opens up a design space that allows us to do things with these tiles that were previously not possible. Each Tile Set Definition can:
  • Specify their own tile sizes
  • Specify their own image formats
  • Specify their own storage location (internally managed by MapGuide or an explicit directory)
  • ... and much more
Tile Set Definitions also support a mini-FDO-style provider model that allows us to support different tile access semantics. This is how we're able to support XYZ tiles, while retaining the same method of accessing such tiles without any API changes.

For MapGuide 4.0, we've expanded the "...and much more" bullet point above by building on top of the shareable tile set foundation with new capabilities.

UTFGrid Tile Support

UTFGrid is a specification for rasterized interaction data. UTFGrid allows us to pre-render tooltip interaction data into tiles of textual content, which when superimposed on top of your existing map, allows us to have rich tooltips without having to send a single mapagent request for tooltip data! 

If you installed 4.0 Preview 1 with samples, there is a new sample demonstrating UTFGrid tiles in action. Notice the distinct lack of mapagent requests (beyond the requests for viewer assets, tile images and UTFGrid tiles).

UTFGrid files follow the XYZ tile access scheme, and thus are supported in Tile Set Definitions using the XYZ Tile Provider with the new tile output format of UTFGRID


With the introduction of UTFGrid tiles, the 4.0 preview 1 release opens up a new viable and potent interactive web mapping combination that was previously not possible.
  • Base map imagery provided by XYZ tiles (whether from an XYZ tile set served from MapGuide or from external services like OpenStreetMap)
  • Overlaid with tooltip interaction data provided by UTFGrid tiles served from another MapGuide XYZ tile set.
This combination requires zero MapGuide sessions to operate. If you recall my now-somewhat-ancient thoughts on scalability in MapGuide, session repositories (and their creation/maintenance/teardown) is a main contributing factor to load in a MapGuide Server.

You can pre-cache all the required content up-front through stateless tile request APIs. You can also choose to output generated tiles to a custom directory of your choosing (by editing the TilePath setting) and front that tile cache directory as a source of static file content in a HTTP web server of your choosing for even more scalability/cacheability. If your mapping interactivity requirements are simple, this new combination of XYZ + UTFGrid tiles served from MapGuide may be a viable combination for you.

(...and it just occurred to me in the process of writing the above blocks of text that this combination opens up brand new map publishing workflows and capabilities that were previously not possible! Here's one such idea I'm brewing for a future release of MapGuide Maestro)

Beyond the provided sample, there is nothing out of the box in the Preview 1 release that currently takes advantage of UTFGrid tile support. UTFGrid support exists in OpenLayers, both in:
So it is possible for some kind of out-of-the-box viewer support for UTFGrid tiles to appear in a future 4.0 preview release (no promises right now!)

Meta-Tiling Support

Meta-tiling is the process of rendering a bigger tile and slicing it down into sub-tiles of the original requested size. Here's a visual explanation:



By rendering bigger tiles and slicing the result down to sub-tiles for tile cache storage, we improve tile generation throughput via:
  • Less queries for feature data (because we're rendering for bigger tiles)
  • Requesting 1 tile and getting 2^n cached tiles for free for other requests because we cache those other sliced sub-tiles as well.
Meta-tiling was a feature that was actually implemented many years ago in a sandbox branch, but it was never merged back into trunk to be made available in a future release. With the advent of shareable tile sets introduced in MGOS 3.0, it was somewhat impractical to merge this work back into trunk as-is, so the existing meta-tiling support was refactored by yours truly to make it easy to retrofit into the new shareable tile set infrastructure.

As a result, in the 4.0 preview 1 release, meta-tiling support is available for Tile Set Definitions in both the Default and XYZ tile providers through 2 new properties:
  • MetaTileFactor
  • MetaTileLockMethod
The MetaTileFactor is a value > 1 that indicates how big our meta-tile to render should be. The specified value will be the multiple of the size of the tile we're requesting. For example, if we're requesting a 256x256 tile from a tile set with a MetaTileFactor of 2, MapGuide will render a 512x512 tile for division into four 256x256 tiles.

The MetaTileLockMethod is an optional setting that controls the locking behavior during this whole process.

  • 1 - Uses a mutex to ensure thread-safe tile writing
  • 0 or unset - Uses a file-based lock for the same effect
Generally speaking, this is a setting you don't really need to tweak and can leave as unset.


Hi-DPI (Retina) tiles

For Tile Set Definitions using the XYZ provider, a new RetinaScale property is available that allows you to generate Retina tiles. Retina tiles are (256 * n) x (256 * n) tiles where n is the specified RetinaScale value. Such tiles provide higher visual fidelity on high-resolution devices such as iPhones, iPads and high-end Android devices.

Our current out-of-the-box viewer offerings do not take advantage of Retina tiles and there is no plans to add such support. This feature is made available primarily for custom viewer offerings (like mapguide-react-layout) to take advantage of.

Other Loose Ends

The previously global TileExtentOffset property can now be configured on a per-tileset basis through a tile set property of the same name. This value is used to internally "buffer" a tile by a certain amount to be rendered to allow label placement algorithms more "breathing space" when determining optimal label placement.

We've also removed the previous restriction that you cannot create a new MgMap from a Map Definition that links to a XYZ tile set (NOTE: Maestro still enforces this restriction in the UI, but this restriction will be removed if we're connecting to a 4.0 server). To see what removing this restriction can give you, there's a new code sample that demonstrates that we can now view any conceivable Map Definition or Tile Set Definition.


Announcing: MapGuide Open Source 4.0 Preview 1.1

When I announced the availability of the 1st Preview release of MapGuide Open Source 4.0, I discovered shortly afterwards, a nasty show-stopping bug in one of the new APIs that breaks interaction with the bleeding edge version of mapguide-react-layout (the sole consumer of this new API).

As this is somewhat of a blocker in terms of what I'm am currently working with, I've put out a new 1.1 Preview release that includes a fix for this bug.

Download/Release Notes

If you don't use mapguide-react-layout, this bug probably doesn't affect you and you can stick with the existing Preview 1 release, if you have already downloaded and installed that.

And since we're here, the Linux binaries will be delayed until Preview 2 as my revised packaging scheme still needs more incubation time.

Wednesday 21 August 2019

Announcing: MapGuide Open Source 4.0 Preview 1

Time to get the first part of my plan in motion: Get a new release of MapGuide out there!

This 1st preview release of MapGuide Open Source 4.0 is the culmination of 2+ years (Wow! Time flies!) of trunk development since the release of MapGuide Open Source 3.1. Competing interest in other side projects like mapguide-react-layout undoubtedly contributing to the long stretch.

A cliff-notes summary of what's new in this release (along with the download links) can be found on the 4.0 Preview 1 release notes wiki page. This blog will showcase all the new features in an upcoming series of posts.

Only 64-bit windows binaries are available as of this post. Linux binaries will soon follow as I am still experimenting with a new packaging mechanism for the Linux builds.

And if the "Preview" designation does not give it away, this release represents a work-in-progress and is not indicative of what the 4.0 final release will look like. There is still plenty of work to be done and more preview releases to follow.

Friday 26 July 2019

Plans and stuff

Since I'm back, I'll start with some rough plans for MapGuide and my related projects.

The next version of MapGuide Open Source will be: 4.0

The next version of MapGuide Open Source was originally slated to be 3.3, but due to the volume of features implemented for this milestone and with some planned future work that will have "breaking change" ramifications (namely dropping support for PHP5 and supporting PHP7) means that the next release is better represented as 4.0

I'll be putting out a 4.0 preview release shortly that showcases all of the features implemented thus far, all of which I will be blogging about here. This will still support PHP5 for the PHP flavor of the MapGuide API for now.

Supporting PHP7 is a non-negotiable blocker towards a final 4.0 release, MapGuide Open Source 4.0 will only be final once we have stable and reliable PHP7 support.

I have an experimental testbed on GitHub which has been somewhat dormant since I first announced it, but will soon be re-activated and most likely incorporated back into the MapGuide sources where I'll continue this work on an experimental sandbox branch.

That's not to say that PHP7 support is the only remaining feature between the 4.0 preview and the final release. There may still be new features implemented in the meantime.

I currently can't tell you how long this journey will take (I honestly do not know how long this work to support PHP7 will take), but at least you know what the final destination will be.

MapGuide Maestro

The main theme for the next milestone of MapGuide Maestro is to support or take advantage of the capabilities/features introduced in MapGuide Open Source 4.0. Check this issue for details.

mapguide-react-layout

I will shortly put out a new 0.12 release which will bridge the 750+ commit gap since the last 0.11.2 release (the bulk of these commits, undoubtedly greenkeeper-assisted pull requests)

After this release, this project will be on a short hiatus as I give MapGuide Open Source 4.0 development maximum priority

mapguide-rest

I will eventually sunset this project with the release of MapGuide Open Source 4.0. The final pieces (which will finally culminate in a 1.0 final release) are:

  • Knocking off the remaining items on this list
  • Blocker: Being fully operational under MapGuide Open Source 4.0
Having mapguide-rest fully operational under PHP7 (that we assume MGOS 4.0 final will be shipping with) will be my final barometer of "it is done" for this project.

Monday 22 July 2019

Ping replied in 10095780000ms

This blog has been quiet as of late. Not because I have run out of things to blog about, merely I've been on a self-imposed blogging hiatus to get a serious chunk of MapGuide development time in.

And the time is right to start talking about some of the stuff that has landed into MapGuide in the past few months as well as a some updates on my other MapGuide/FDO-related projects.

Stay tuned!

Thursday 28 March 2019

Announcing: MapGuide Open Source 3.1.2

I am pleased to announce the availability of MapGuide Open Source 3.1.2

The notable changes in this release are detailed below:

Updated Web Tier Components

MapGuide Open Source bundles updated web tier components:

  • Apache HTTPD 2.4.37
  • Tomcat 7.0.92
  • PHP 5.6.40
Since PHP 5.6 is now end-of-life, this is most likely the last release in the 3.1 branch as we work to bring PHP 7 support for the MapGuide API in the next major release.

Functional SHP FDO provider for 64-bit Linux

Over the new year, I had set out to solve one of the last remaining blockers that prevented us from having a functional 64-bit SHP FDO provider for 64-bit Linux.

I had speculated that the blocker is due to incorrect data size assumptions in the SHP spatial indexing code on 64-bit Linux which would throw off spatial index queries. Intensive debugging sessions proved this to be the case and I was finally able to land the long-awaited fixes which is now available with the SHP provider bundled with this release.

The proof that the provider was truly functional was being able to load a SHP-heavy dataset into a 64-bit MapGuide Server and being able to pan, zoom, select and buffer SHP features without any errors or features disappearing in and out at various zoom levels (due to previously out-of-whack SHP spatial index queries)


Experimental King.Oracle FDO provider for Linux (32 and 64-bit)

Since I got a major win with one Linux FDO blocker and was on a roll, I figured we can do a double-whammy with the other major Linux FDO blocker: The lack of a functional King.Oracle FDO provider on Linux.

For this one I had speculated that the problem was due to heavy usage of wchar_t for strings which (due to different size assumptions on Linux) meant that most strings going in and out of the provider on Linux were garbage, meaning the SQL queries we think we were sending to OCI for execution were completely scrambled.

Due to the lack of solid examples of using OCI with wchar_t strings on Linux, I figured the code as it currently stands will never work on Linux. So I took the page out of a reference implementation I knew to definitely work on Linux (the OCI driver for GDAL/OGR) and wholesale #ifdef'd the entire provider codebase to use narrow (char) strings for Linux and their expected OCI functions, while retaining wide (wchar_t) strings and current OCI function usage for Windows.

The ability to easily spin up an Oracle XE environment (well until recently at least) meant that I was able to quickly verify that switching to narrow strings on Linux was indeed the way to go. This also provided motivation for me to implement a basic (and repeatable) unit test suite for this provider, finally giving us an objective baseline level of validation and verification for this provider.

The end result of all this is that the King.Oracle provider now not only builds on Linux, but it now finally works!


However, I'm certain that there will be bugs in this provider for Linux waiting to be revealed, so the provider gets the "experimental" label. Use at your own risk/discretion. Please do report any issues you find.

OGR FDO Provider Improvements

The OGR provider has undergone many "quality of life" improvements:
  • We've cleaned up various correctness issues around provider behavior. For example, attempting to access invalid property names in an OGR feature reader will now throw instead of silently failing before.
  • The provider supports a new DefaultSchemaName property that lets you specify a default schema name other than OGRSchema
  • The provider also supports a new DataSourceEncoding property to allow you declare the encoding for an OGR data source where it cannot be inferred by default by the OGR library so that properties/names/etc with non-english characters are rendered properly.
Fusion Changes
Other Changes


Friday 15 March 2019

Announcing: vscode-map-preview 0.4.6

A kind user alerted me to the fact that my Map Preview extension for Visual Studio Code was using APIs that have long been deprecated and could be removed any moment in a future release, breaking this extension in the process.

Although I haven't touched this code in a while (not intentional btw. One person can only tend to so many different open source projects), getting back into VSCode extension development was a very comfortable affair:

  • Update a few packages here and there. 
  • Update TypeScript. 
  • Read up on the new Webview API, which is what our extension should be using now.
  • Replace usages of vscode.previewHtml with the new Webview API.
  • Verify the extension still works
  • Get re-acquainted with how to publish VSCode extensions
And ... voila! A new version of the Map Preview extension is available for your VSCode installation to auto-update to.


In terms of changes, beyond some updates to OpenLayers and a small bug fix or two, there isn't really much to write home about, it should still be the same extension as before. This release was driven primarily by the need to move away from their old vscode.previewHtml API over to their new Webview API.

Once things have settled down on my other projects, I'd like to give this project some more attention. There's still some cool ideas I want to explore with this extension. I don't know if I still want to go down the Cesium route as previously hinted as there have been many more data viz players that have arrived on the scene since then and I also sort of do not want to detract from the original goal and purpose of this extension: An easy way to preview on a Map textual content that you are already editing or viewing in VSCode.