We have a pulse

After 5+ years of slumber, FDO Toolbox is getting some long overdue quality-of-life improvements

The finer details will be blogged out in due course.

MapGuide tidbits: An unconventional way to bulk convert features to GeoJSON

Recently in my regular day job I had a spatial data processing task that needed to be carried out in a timely manner.

I needed the current Australian Federal Electorates, which come from the AEC in the form of a single SHP file or MapInfo TAB file and I needed this dataset split up so that each of the 151 federal electorates existed as separate GeoJSON files named based on some combination of attributes in each feature.

After deliberating on possible solutions (I absolutely did not want to have to write a throwaway program that uses GDAL/OGR or FDO to do this conversion unless there was no other choice!), I came up with a somewhat unconventional solution using mapguide-rest and its data publishing feature that I've decided to share in this particular post. Here's what I did.

First, I loaded the electorates SHP file into the MapGuide Server via Maestro. The feature source can be either connected to the SHP file loaded into the resource data files or connect to an external SHP path or aliased path. It ultimately doesn't matter and it's up to you how you prefer to manage your SHP feature sources.

This MapGuide installation already has mapguide-rest installed, so I use the REST Explorer in Maestro to connect to it. Once connected, I proceeded to create 2 data configurations.

The first one is a CSV representation of the SHP feature source, that returns the identity and a computed expression of my desired output file name.

The resulting REST configuration looks like this:

We then set up a 2nd REST configuration to provide a GeoJSON representation of our electorates.

Whose REST configuration looks like this

With these 2 rest configs set up, we now have new URLs available to us to access the electorates in our desired representations. In relation to what I want to achieve, the URLs of interest are:

GET http://servername/mapguide/rest/data/electorateinfo/.csv

To get the id and desired filename of all electorates in a CSV, whose content looks like this:

The other URL is:

GET http://servername/mapguide/rest/data/electorate/{id}.geojson

To get the individual electorate feature by id in GeoJSON format.

With these 2 URLs, we now just need to write a simple script that:

1. Downloads the CSV of all electorates
2. Loops through each line in the CSV (except header) to request a GeoJSON download of the electorate for the given id and download it as the requested file name.

Which can be done with a Powershell script like this:

1 2 3 4 5 6 7 8 9 10 11 12 13

$csvFile = [System.IO.Path]::GetTempFileName(); $workingDirectory = Convert-Path (Resolve-Path -path ".") $outDir = "$workingDirectory\electorates_geojson" New-Item -ItemType Directory -Force -Path $outDir Invoke-WebRequest -Uri "http://localhost/mapguide/rest/data/electorateinfo/.csv" -OutFile $csvFile $electorates = Import-Csv $csvFile foreach ($elec in $electorates) { $id = $elec.FeatId; $fileName = $elec.filename + ".geojson"; $outPath = [System.IO.Path]::Combine($outDir, $fileName); Invoke-WebRequest -Uri "http://localhost/mapguide/rest/data/electorate/$id.geojson" -OutFile $outPath Write-Host "Saved electorate ($id) to: $outPath" }

Run the script, and a minute or 2 later, all 151 electorate boundary features have been split off and converted into individual GeoJSON files!

I hope this post is as useful to you as it was for me.

Announcing: mapguide-react-layout 0.13

Here it is. The last release of mapguide-react-layout in its current form. There will be bugfix releases as required but from this release onwards, mapguide-react-layout will be re-branded as something else, and when I figure out what that will be called, you will find out here first!

Here's the highlights of what's new and changed in this release.

Revamped External Layer Manager

Already showcased in previous dev diary entries, the 0.13 release features a revamped external layer manager allowing you to add external layers from file-based or remote WFS/WMS data sources.

The External Layer Manager also has the following new changes since my dev diary posts.

We've tweaked the WMS select tool to now also display the feature in our client-side selection overlay if the WMS GetFeatureInfo response contains geometry data.

You'll also notice that the WMS feature tooltip is styled differently (to distinguish it from the regular feature tooltip) and is also now also close-able (indicated by the clickable [x])

This WMS Query tool is a separate widget, which you can add to your application by including the following widget definition in your Application Definintion and referencing that widget from a toolbar or menu.

1 2 3 4 5 6 7 8 9 10 11 12

<Widget xsi:type="UiWidgetType"> <Name>WmsQuery</Name> <Type>WmsQuery</Type> <Location /> <Extension /> <ImageUrl>images/icons.png</ImageUrl> <ImageClass>select</ImageClass> <Label>WMS Query</Label> <Tooltip>Click to query for features in WMS layers at the point you click</Tooltip> <StatusText /> <Disabled>false</Disabled> </Widget>

Also for vector layers, you can now style points with image icons through the style editor.

Storybook-based demo

This release is the first one to be showcased in storybook.

Legend Search Filtering

The legend now includes a search box to easily drill down big layer/group structures if you need to rapidly locate a specific layer for toggling. Useful for maps with very large layer/group structures.

Template Changes - Aqua

Thanks to the use of react-rnd, we now finally have resizable dialogs in the Aqua Template!

Template Changes - Other

For all templates, making a map selection will now automatically give focus to the selection panel.

The only exception to this behavior is the sidebar template in mobile view, where rather than automatically flying out a full screen selection panel, we just flash a red indicator over the selection panel toggle to let you know there are selected features to look at.

Removed "Invalid Task Pane" warning

If you work with multi-map configurations, you may have noticed this warning on Task Pane content when switching between maps.

This warning is shown when the passed MAPNAME and SESSION parameters for any loaded URL in the Task Pane does not match the mapname/session of the map you're currently looking at.

Upon further reflection, this warning is overzealous as any Task Pane content that needs to know the actual map name and session id can simply request this information from the viewer APIs in javascript. So this warning has been removed in this release.

SVG icon replacements

Some of the image based icons used by the viewer have been replaced with SVG icon equivalents from the blueprint icon set. This is to enable the possibility of scalable versions of these user interfaces in future releases.

 

Other Changes

There is a whole load of other fixes and minor improvements which you can find in the full 0.13 release notes in the download link below.

Download

mapguide-react-layout dev diary part 25: There is a general purpose map viewer in there.

As I am tidying up the remaining loose ends for the upcoming 0.13 release, I am going to be giving the project a slight re-branding and revised mission statement after this release.

Because although I've touted the mapguide-react-layout project as a "modern map viewer for MapGuide", the fact of the matter is that only a small portion of the viewer codebase concerns integration with the services provided by MapGuide. The rest of the codebase builds on top of the already solid foundations provided OpenLayers, React and TypeScript that can also work with other web mapping servers and services.

I haven't come up with the specifics yet, but my current thoughts so far are:

• This project will be renamed to something else (name TBD)
• The npm package will have a different name. The mapguide-react-layout packages up to 0.13 will still remain on npm, but when the time is right I'll put a deprecation warning on mapguide-react-layout to let you all know about the new package and that you should migrate over to it.
• MapGuide-specific functionality and features of the viewer will be extracted and separated out. How this separation will be done TBD.

With this move, there is potential for some nice cross-pollination:

• I could pivot the vscode-map-preview extension to be based on this viewer (and its kitchen sink of built-in functionality) instead of raw OpenLayers strung together with other libraries and re-creating functionality I already take for granted in mapguide-react-layout.
• I'm parking my static map publishing idea for Maestro because the prospect of having a mapguide-react-layout viewer that need not necessarily have MapGuide Server dependencies is a more attractive proposition to build this idea on top of.
• There's opportunities for adoption of this viewer in other projects where the hard MapGuide Server dependency would've nixed such opportunities.

So when 0.13 finally drops, that will be the end of mapguide-react-layout in its current form. Not because I'm done with the project, but because it's time for this project to evolve.

Announcing: vscode-map-preview 0.5.1

This release restricts the display of the new preview UI buttons to only files whose extension we know to be preview-able

The preview buttons now only display for the following file extensions:

• .geosjon
• .csv
• .kml
• .gml
• .gpx
• .igc

Although preview-able content may exist within .txt, .xml and .json files, these formats are too generic to reliably determine ahead of time if this is actually the case (before the preview command is run).

The preview commands can still be run the "old fashioned" way for all other cases.

Announcing: vscode-map-preview 0.5.0

Having figured out the technique for seamlessly loading CSV data in mapguide-react-layout, it was clear that there was another OpenLayers-based tool of mine that could benefit as well, hence a new release of the VSCode Map Preview extension.

Here's what's new in this release.

CSV Support

Obviously!

Rather than bombard you with excessive UI prompts for what column is the X coordinate and what column is the Y coordinate, we just check for a list of common column pairs:

• X/Y
• Lon/Lat
• Lng/Lat
• Long/Lat
• Longitude/Latitude
• Easting/Northing

If you want to add more column pairs to search for, there is a new configuration setting to define these pairs.

Preview buttons

Having learned about VSCode menu contribution points, we now finally have buttons in the editor UI to invoke the preview commands!

Custom Projection Support

The extension now supports custom projections allowing you to preview data from projections other than 4326 or 3857. A new configuration property is available to let you define additional custom projections:

To find/register your custom projection, visit epsg.io and get the proj.4 or proj4js definition for your particular EPSG code and add a new entry with this EPSG code and definition as per the above fragment.

Because of this support, the Map Preview (with projection) command now presents a list of known projections instead of requiring you to enter the EPSG code manually

Other Changes

Vertex styling for line/polygon features can now be toggled on/off through a new configuration property.

OpenLayers has also been updated to the latest release (6.1.1) and the extension fixes some corner cases when previewing certain types of KML and GeoJSON content.

mapguide-react-layout dev diary part 24: Adding *even* more than just WMS layers

Since I introduced the revamped external layer manager, the creativity floodgates have opened as I have realised that there are many other things that we could implement on top. So for this post, I'm going to cover even more cool things you can do with the revamped external layer manager.

Revised file-loading process

The UI for loading file-based layers has been tweaked a bit. This means that the first step is picking your file as usual. Note that we've changed the "uploader" UI back to a plain blueprint-styled file input due to bundle size budget constraints that required us to drop react-dropzone

After picking the file, we now give you the ability to specify the projection of this file you just picked, which is now a number picker instead of a dropdown. If you enter an EPSG code not known to the viewer, it will now do an epsg.io lookup and register the found projection on-the-fly before adding the vector layer. The design that drives this UI has been refactored to be more modular and flexible, meaning we now can identify the "type" of the file based on the first successful driver to properly parse the file content.

This refactored design also finally allows us to add ...

CSV Support!

When looking at what types of files to support for external vector layers, CSV seemed like a no-brainer to support. Point data is after-all, commonly defined in CSV files.

The problem with CSV files is that there is no established standard as to how point coordinates are defined. The ideal solution would involve having to attempt peek and introspect the file and build the UI to allow picking the correct point coordinate columns. But this UI would complicate our 2-step UI for file-based layers. How would we know the file we're about to add is indeed a CSV file to then be able to present a UI for the user to pick the columns for point coordinate data?

In the interest of pragmatism and avoid having to answer that hard question altogether, our approach is to look for the most common column labels for point coordinates on the first line of the file we encounter:

• X/Y
• Lon/Lat
• Lng/Lat
• Longitude/Latitude
• Easting/Northing

If the file you add has a CSV header and it contains one of the above pairs of columns and the row data referenced by these columns checks out as numbers, it will be accepted as a CSV file.

Added Layers UI Changes

The UI for added layers has been compressed to maximize screen real estate

The opacity slider took up a lot of space (relatively speaking) and toggles a setting I deem to be non-essential to overall functionality, so that slider is now hidden behind a layer options panel (toggled by the new cog icon)

Finally for WMS layers, we now register their source load events so they can contribute to the viewer busy state, meaning that we can properly communicate busy loading state for any added WMS layers.

WMS UI changes

After loading the WMS capabilities URL, for each layer we find you now have the ability to add the WMS layer in as a single-tile or tiled WMS layer.

WFS Support

It seems strange that this external layer manager supports WMS services but does not support the other major W*S service. This was a glaring omission the viewer now addresses. WFS is available as a new layer type:

Added WFS layers are vector layers and assumes the UI of added file-based layers.

Due to viewer bundle size budget constraints the WFS layer UI will only care about layers in the loaded capabilities document that offer GeoJSON output. We could add support for GML2 and GML3 (the formats commonly associated with WFS), but it is not worth blowing up our bundle size to add format drivers for such an esoteric and verbose format.

Select tool support for external layers

So now our revamped external layer manager can add data from a whole assortment of file formats and remote WFS/WMS services. But these data sources are not just for show. They generally have intelligence attached in the form of attribute data for vector data or in the case of WMS layers, APIs to get at such data (WMS GetFeatureInfo), so to put the cherry on top of this revamped external layer manager, the viewer should provide some means of interacting with these layers after adding them.

To this end, our default select tool can now select vector features from external layers and display their attributes in an overlay tooltip.

Vector features take precedence over MapGuide layers. That is to say, the select tool will now try to select vector features first (and display its attributes if found). Only when no vector feature are selected does the viewer proceed with the normal MapGuide layer selection logic.

For WMS layers, we introduce a new active tool dedicated for sending GetFeatureInfo requests against any WMS layers on the current map. The effect is still the same, click on a WMS layer and if we get data back from a WMS GetFeatureInfo response, we'll show the data in a overlay tooltip.

The choice to do WMS "selection" is done through a separate active tool instead of adding on to the default select tool behavior was because adding WMS layer selection logic to the existing select tool would complicate the control flow significantly given that WMS GetFeatureInfo requests are asynchronous and it is hard to maintain a selection priority when dealing with a map that has vector, WMS and MapGuide layers together.

In Closing

The revamped external layer manager is the last major feature for the upcoming 0.13 release. Remaining work before release will primarily be knocking off any remaining bugs and loose ends and also tidying up documentation.