Saturday, 28 September 2019

Announcing: mapguide-react-layout 0.12.3

This bugfix release fixes a bug where the initial map view (if specified in a flexible layout) is parsed incorrectly, breaks the viewer as a result.

Project Home Page
mapguide-react-layout on npm

Thursday, 26 September 2019

MapGuide 4.0 Showcase: Transform all the things!

Did you know that MapGuide uses CS-Map, a coordinate system transformation library with support for many thousands of coordinate systems out of the box? No matter how esoteric the map projection your data is in, MapGuide can re-project geospatial data in and out of it thanks to this library.

So it is quite a shame that for the longest time, MapGuide's powerful coordinate system transformation capabilities has zero representation in any part of the mapagent (the HTTP API provided by MapGuide).

To use MgCooordinateSystem and friends (the classes that wrap the CS-Map library) requires building a your own MapGuide application using the MapGuide Web API in one of our supported languages (.net, Java or PHP) and having your client application call into your own MapGuide application. There is nothing out of the box in the mapagent for a client map viewer application to do a basic HTTP request for transforming coordinates or requesting feature data to be transformed to a certain coordinate system.

For MapGuide 4.0, we've exposed the coordinate system transformation capabilities in APIs where it makes sense. Such as our SELECTFEATURES APIs for returning feature data. In my previous showcase post, I've shown how appending VERSION=4.0.0 and CLEAN=1 now gives you an intuitive GeoJSON result for this API.

The coordinates are based on the coordinate system of the data source it comes from. In the above screenshot, this is latitude/longitudes (code: LL84, epsg: 4326).

Suppose you want this data in web mercator (code: WGS84.PseudoMercator, epsg: 3857) so you can easily plonk this GeoJSON onto a slippy map with OpenStreetMap or your own XYZ tiles, which would also be in web mercator. With MapGuide 4.0, you can now also include a TRANSFORMTO=WGS84.PseudoMercator parameter to your HTTP request and that GeoJSON data will be transformed to web mercator.

The other major capability gap is that the mapagent offers no API over the mapagent for transforming a series of coordinates from one coordinate system to another. For MapGuide 4.0, there's now a dedicated API for doing this: The CS.TRANSFORMCOORDINATES operation.

With this operation, simply feed it:

  • A comma-separated list of space-separated coordinate pairs (COORDINATES)
  • The CS-Map code of the source coordinate system the above coordinates are in (SOURCE)
  • The CS-Map code of the target coordinate system to transform to (TARGET)
  • Your desired output format of JSON or XML (FORMAT)
  • If you want JSON, whether you want the pretty JSON version (CLEAN)
Invoking the operation will give you back a collection of transformed coordinates.

A place where this new API is sorely needed is MapGuide Maestro, which previously required kludgy workarounds for transforming bounding boxes in Map Definitions (when adding layers in different coordinate systems, or re-computing extents):
The next milestone of MapGuide Maestro will now take advantage of this new API for transformation when connected to a MapGuide 4.0 or newer server.

Tuesday, 24 September 2019

Now *THIS* is progress!

So in my previous post, it turns out I had mis-configured the tile grid property of the OL vector tile source in this example, namely the tile size. Is it not the default 256x256, but 4096x4096.

Once I fixed the tile size, everything beautifully snaps into place!

It is still progress

Even if the mapbox vector tiles are completely out of alignment, progress is still progress.

I'm seeing stuff!

Saturday, 21 September 2019

MapGuide 4.0 showcase: JSON that's easier to work with.

If you've ever worked with the HTTP-based mapagent API, you should be aware that most of the APIs responses in the mapagent generally come back in 2 forms:

  • XML
  • JSON
However, if you ever look at the JSON responses, you'll be accustomed to dealing with ugly monstrosities like this (eg. A JSON version of a Layer Definition):

Or heaven forbid, you want to explore the structure of a Feature Source and so you need the JSON version of a FDO feature schema:

Good luck trying to parse and comprehend that!

Why is the JSON so horrible? The answer is XML is the canonical response format for all non-image/binary mapagent operations (MapGuide Open Source's inception was in 2004 after all, when XML was king and JSON probably didn't exist yet), so the JSON version is a literal translation of whatever its XML form would be. While there is nothing wrong with that, the way this JSON translation was done is the most lowest-common-denominator approach:
  • Treat every XML element as a JSON array.
  • Treat every XML element text body as a JSON string.
As a result, as evidenced by the above screenshots that means that all JSON responses are nothing but a series of JSON arrays (of possible more JSON arrays) of strings.

But there is a reason for such laziness. The JSON translation knows nothing about the content model of whatever XML it's trying to translate, so it really has no choice in the matter. Such content models are defined in the various XML schemas that are shipped with MapGuide. But working with XML schemas in C++ sounds like a nightmare in and of itself, so there is another way to determine the content model: Just manually hard-code the list of all possible XML xpaths that are:
  • Repeating (ie. Should be converted to actual JSON arrays)
  • Not a string
Using the existing XML schemas as the reference, this list only required a one-off painstaking translation of this xpath list. The end result isn't exactly pretty, but it gets the job done. With this hard-coded list now in our JSON converter, it means we now have the means to output JSON that is much easier to comprehend and intuitive! 

For MGOS 4.0, for any mapagent operation that returns JSON, if you specify VERSION=4.0.0 and CLEAN=1 in the request parameters you will now get a JSON format that actually makes sense!

Now for some mapagent operations, finally having a "clean" JSON version still isn't enough because the clean JSON response is still mostly clunky and unusable. Consider the FDO feature schema example. The original JSON version is an unworkable monstrosity, and the "clean" version would've just been lipstick on a pig because at the end of the day, it is still a literal JSON translation of an XML schema! 

So as a result, for this set of APIs, a new simpler XML response format is introduced for schema and class definitions by including SIMPLE=1 to your VERSION=4.0.0 request:

From which its JSON equivalent (I hope you'll agree) is much more easier to comprehend!

There is another class of operations that have received a similar treatment. Operations that return geometric/feature data. Consider the XML form of selecting features (the ugly schema section collapsed for brevity):

This is what the "ugly" JSON version would've looked like (schema parts again collapsed for brevity). 

Even if we prettied this up with content-model-aware conversion, the JSON is still mostly unremarkable. Because, if we're serving JSON feature data out to clients, then there's really only one format we should care about supporting:

GeoJSON is the de-facto format for serving out JSON-based feature data, and is universally supported by any mapping/GIS product/tool/library/viewer worth a damn, so if we have the opportunity for MapGuide to return clean JSON data for geometries/features, forget about the literal conversion from XML approach! Just go ahead and make GeoJSON output a first-class citizen

So for MGOS 4.0, if you run any mapagent operation that returns geometry/feature data and you ask for VERSION=4.0.0 and CLEAN=1 you will now get GeoJSON!

Which (due to GeoJSON's ubiquity) can be plugged directly into your web mapping library of choice for display if you so choose.

If you are building your own server-side MapGuide applications with the MapGuide Web API, there's a new MgGeoJsonWriter class is available for you to convert MgFeatureReaders to GeoJSON data.

With MapGuide Open Source 4.0, JSON support is no longer an afterthought, it's actually usable!

Friday, 20 September 2019

Announcing: mapguide-react-layout 0.12.2

This bugfix release plugs remaining localization gaps in the measurement tool and tooltips for various OpenLayers controls we are using.

Project Home Page
mapguide-react-layout on npm

Sunday, 15 September 2019

Announcing: mapguide-react-layout 0.12.1

This bug fix release fixes the broken WMS layer manager that was discovered after making the 0.12 release and also refines the selection sub-highlighting capability check to exclude 4.0 preview 1 and older servers as the 4.0 preview 1 release had a broken implementation of QUERYMAPFEATURES 4.0.0.

Project Home Page
mapguide-react-layout on npm

Announcing: mapguide-react-layout 0.12.0

As promised, here's the long overdue new release of mapguide-react-layout.

As stated in my revised plans, this release marks the point where this project will go on a short development hiatus as I give MapGuide Open Source maximum development priority for the next few months. I may put out a bugfix release or two if the need arises.

Here's what's new and notable in this release.

Support for XYZ tile sets in Flexible Layouts

Fusion in MapGuide was recently able to finally support XYZ tile sets defined in a Flexible Layout (aka. Application Definition) document. That same support has been extended to this release

XYZ tile sets can come from many different sources. It could be your own XYZ tile set definitions or a custom OSM flavor tile set, or Here Maps (as shown below)

Or many more!

Selections now live through browser refreshes

Map selection state is now stashed into browser local storage and recovered from on startup upon a browser refresh

Sub-selection highlighting now only enabled for MapGuide Open Source 4.0 (and newer) servers

Our neat little trick of sub-selecting selected features actually has a major flaw. It is built on the erroneous assumption that selection keys and selection attributes in a QUERYMAPFEATURES response are rendered in the same order (we used the current index of the selected feature's attributes to pick up the selection key at the same index to request a sub-selection for). This turned out to be sadly not the case.

Rectifying this required a new version of the QUERYMAPFEATURES API that will also render selection keys alongside attributes (to avoid the same-index selection key lookup). This has been implemented for the MGOS 4.0 preview 1 release.

But what that means is that sadly we have to disable sub-selection highlighting for older MapGuide Servers. This feature is now only enabled for MGOS 4.0 preview 1 (and newer) servers.

Share Link To View component can now copy to clipboard

This component now includes a button to copy the link to the clipboard, saving you the need to select the URL and copy yourself.

NOTE: This component will not include the map selection if another user opens the shared link on another browser/computer as that selection state lives in the user's browser local storage and is too big to reliably encode into the URL itself for sharing.

Smaller JS bundle size

The viewer JS bundle previously bundled all the necessary CSS within the JS. For this release, that CSS has been extracted out to a separate viewer.css file. This has 2 benefits.

  1. The JS bundle is now smaller
  2. The viewer CSS can be modified if you so please.

Compare the bundle size of the previous 0.l1.2 release

Against this 0.12 release

Other Changes
  • Plug remaining i18n holes in measure and WMS layer manager components.
  • Don't word wrap layer/group node labels in Legend component and provide tooltips for these labels
  • Remove non-geodesic measure option as it is mostly inaccurate.
  • Fix init failure if appdef has an empty widget container
  • Fix line measurement total showing units in m^2
  • Fix measure tool starting measuring when switching measurement type
  • Fix memory leak due to dangling measure components being held on by the measure context after unmount
  • Various components updated to use componentDidUpdate instead of componentWillReceiveProps
  • Made redux actions more strongly-typed
  • Updated Blueprint to 1.40.0
  • Updated TypeScript to 3.6.3
  • Updated React to 16.9.0
  • Replaced query-string module with qs
  • Now built with webpack 4.x

Project Home Page
mapguide-react-layout on npm

Tuesday, 3 September 2019

MapGuide 4.0 showcase: Supercharged tile sets

As promised, this is the first (of many) blog posts showcasing the new features of the just released MapGuide Open Source 4.0 Preview 1. Before we dive in, here's a little recap for some extra context.

Way back in MapGuide Open Source 3.0, we introduced the concept of a Tile Set Definition, a new resource type that allows one to define a shareable, re-usable tile set that was free of (most) server-imposed global default settings and is decoupled from any particular map or application. Before the introduction of shareable tile sets, a tile cache could only be defined within a Map Definition and was thus coupled to any application consuming this particular map. This setup also made the respective tile cache extremely prone to invalidation upon any minute changes to the Map Definition. Tile settings like size and image format were global (in serverconfig.ini) making it impossible to have tile sets of various sizes and image formats.

With the introduction of shareable tile sets, it means that tile sets no longer need to be defined on the Map Definition. Instead, it can be defined in an external Tile Set Definition resource and linked into any Map Definition (hence the shareable aspect). Being a linked resource, also means any changes to a Map Definition will not errantly flush cached tiles for the associated tile set as we have solid guarantees that editing a Map Definition that links to a tile set would not alter or compromise the layer/group structure of a tile set being linked that would normally necessitate invalidating the tile cache by deleting all previously cached tile images.

Because tile sets are now their own separate resources, it also opens up a design space that allows us to do things with these tiles that were previously not possible. Each Tile Set Definition can:
  • Specify their own tile sizes
  • Specify their own image formats
  • Specify their own storage location (internally managed by MapGuide or an explicit directory)
  • ... and much more
Tile Set Definitions also support a mini-FDO-style provider model that allows us to support different tile access semantics. This is how we're able to support XYZ tiles, while retaining the same method of accessing such tiles without any API changes.

For MapGuide 4.0, we've expanded the "...and much more" bullet point above by building on top of the shareable tile set foundation with new capabilities.

UTFGrid Tile Support

UTFGrid is a specification for rasterized interaction data. UTFGrid allows us to pre-render tooltip interaction data into tiles of textual content, which when superimposed on top of your existing map, allows us to have rich tooltips without having to send a single mapagent request for tooltip data! 

If you installed 4.0 Preview 1 with samples, there is a new sample demonstrating UTFGrid tiles in action. Notice the distinct lack of mapagent requests (beyond the requests for viewer assets, tile images and UTFGrid tiles).

UTFGrid files follow the XYZ tile access scheme, and thus are supported in Tile Set Definitions using the XYZ Tile Provider with the new tile output format of UTFGRID

With the introduction of UTFGrid tiles, the 4.0 preview 1 release opens up a new viable and potent interactive web mapping combination that was previously not possible.
  • Base map imagery provided by XYZ tiles (whether from an XYZ tile set served from MapGuide or from external services like OpenStreetMap)
  • Overlaid with tooltip interaction data provided by UTFGrid tiles served from another MapGuide XYZ tile set.
This combination requires zero MapGuide sessions to operate. If you recall my now-somewhat-ancient thoughts on scalability in MapGuide, session repositories (and their creation/maintenance/teardown) is a main contributing factor to load in a MapGuide Server.

You can pre-cache all the required content up-front through stateless tile request APIs. You can also choose to output generated tiles to a custom directory of your choosing (by editing the TilePath setting) and front that tile cache directory as a source of static file content in a HTTP web server of your choosing for even more scalability/cacheability. If your mapping interactivity requirements are simple, this new combination of XYZ + UTFGrid tiles served from MapGuide may be a viable combination for you.

(...and it just occurred to me in the process of writing the above blocks of text that this combination opens up brand new map publishing workflows and capabilities that were previously not possible! Here's one such idea I'm brewing for a future release of MapGuide Maestro)

Beyond the provided sample, there is nothing out of the box in the Preview 1 release that currently takes advantage of UTFGrid tile support. UTFGrid support exists in OpenLayers, both in:
So it is possible for some kind of out-of-the-box viewer support for UTFGrid tiles to appear in a future 4.0 preview release (no promises right now!)

Meta-Tiling Support

Meta-tiling is the process of rendering a bigger tile and slicing it down into sub-tiles of the original requested size. Here's a visual explanation:

By rendering bigger tiles and slicing the result down to sub-tiles for tile cache storage, we improve tile generation throughput via:
  • Less queries for feature data (because we're rendering for bigger tiles)
  • Requesting 1 tile and getting 2^n cached tiles for free for other requests because we cache those other sliced sub-tiles as well.
Meta-tiling was a feature that was actually implemented many years ago in a sandbox branch, but it was never merged back into trunk to be made available in a future release. With the advent of shareable tile sets introduced in MGOS 3.0, it was somewhat impractical to merge this work back into trunk as-is, so the existing meta-tiling support was refactored by yours truly to make it easy to retrofit into the new shareable tile set infrastructure.

As a result, in the 4.0 preview 1 release, meta-tiling support is available for Tile Set Definitions in both the Default and XYZ tile providers through 2 new properties:
  • MetaTileFactor
  • MetaTileLockMethod
The MetaTileFactor is a value > 1 that indicates how big our meta-tile to render should be. The specified value will be the multiple of the size of the tile we're requesting. For example, if we're requesting a 256x256 tile from a tile set with a MetaTileFactor of 2, MapGuide will render a 512x512 tile for division into four 256x256 tiles.

The MetaTileLockMethod is an optional setting that controls the locking behavior during this whole process.

  • 1 - Uses a mutex to ensure thread-safe tile writing
  • 0 or unset - Uses a file-based lock for the same effect
Generally speaking, this is a setting you don't really need to tweak and can leave as unset.

Hi-DPI (Retina) tiles

For Tile Set Definitions using the XYZ provider, a new RetinaScale property is available that allows you to generate Retina tiles. Retina tiles are (256 * n) x (256 * n) tiles where n is the specified RetinaScale value. Such tiles provide higher visual fidelity on high-resolution devices such as iPhones, iPads and high-end Android devices.

Our current out-of-the-box viewer offerings do not take advantage of Retina tiles and there is no plans to add such support. This feature is made available primarily for custom viewer offerings (like mapguide-react-layout) to take advantage of.

Other Loose Ends

The previously global TileExtentOffset property can now be configured on a per-tileset basis through a tile set property of the same name. This value is used to internally "buffer" a tile by a certain amount to be rendered to allow label placement algorithms more "breathing space" when determining optimal label placement.

We've also removed the previous restriction that you cannot create a new MgMap from a Map Definition that links to a XYZ tile set (NOTE: Maestro still enforces this restriction in the UI, but this restriction will be removed if we're connecting to a 4.0 server). To see what removing this restriction can give you, there's a new code sample that demonstrates that we can now view any conceivable Map Definition or Tile Set Definition.

Announcing: MapGuide Open Source 4.0 Preview 1.1

When I announced the availability of the 1st Preview release of MapGuide Open Source 4.0, I discovered shortly afterwards, a nasty show-stopping bug in one of the new APIs that breaks interaction with the bleeding edge version of mapguide-react-layout (the sole consumer of this new API).

As this is somewhat of a blocker in terms of what I'm am currently working with, I've put out a new 1.1 Preview release that includes a fix for this bug.

Download/Release Notes

If you don't use mapguide-react-layout, this bug probably doesn't affect you and you can stick with the existing Preview 1 release, if you have already downloaded and installed that.

And since we're here, the Linux binaries will be delayed until Preview 2 as my revised packaging scheme still needs more incubation time.