Monday, 30 March 2015

Announcing: MapGuide Open Source 3.0 Beta 1

After a month or two lost due to mucking around with packer and vagrant to get the perfect repeatable and disposable build environment for both Windows and Linux builds of MapGuide/FDO, I'm now happy to announce the availability of MapGuide Open Source 3.0 Beta 1.

Who knew debugging build infrastructure would be such a PITA? And who also knew that vagrant and windows is like oil and water? Ugh! I gave up on that part, so as it stands our current build infrastructure for putting out releases of MapGuide is still Jenkins for windows and (slightly improved) vagrant for Linux.

Stay tuned for the (promised) series of posts talking about what's new in MapGuide Open Source 3.0


Friday, 6 March 2015

Announcing: mapguide-rest 1.0 RC1

I've decided that mapguide-rest has enough features to lock in the 1.0 version number.

Here's what's new in this release.

Cleaner JSON responses

If you've used JSON output from mapguide-rest or MapGuide proper, you'll notice that the JSON is ugly and cumbersome to use:
  • Every JSON property is an array
  • All property values are strings
The main reason this JSON is so ugly is because of the way that XML is converted to JSON. The conversion process knows nothing about the content model of the XML document it is trying to convert, so as a lowest-common-denominator solution, it forces all converted XML elements to JSON array properties and all XML text elements as string values. While this allows for rudimentary usage by client applications, it's not a pleasant experience.

For this release, we've set out to solve this problem. The JSON produced which would've looked like this:

Now looks like this:

How did we achieve this? Simply put, we now maintain a hard-coded list of known XML element paths (compiled from all the MapGuide XML schemas) and make note of:
  • Which paths must always be processed as a JSON array
  • Which paths have values whose data type is not a string
With this list, we now know which XML elements need to be converted to JSON arrays and which elements need to be converted to JSON property values, and the proper data type to convert such values to.

The end result is that the new JSON responses are much more natural and intuitive to work with. Not to mention these new JSON responses are smaller than their previous counterparts. Those extra [ and " characters do add up.

This is a PHP-level solution. With this release, all JSON conversion/output is now handled by mapguide-rest. While slower than doing it natively in MapGuide, it gives us more usable JSON structures and this increased usability is worth it for the slower performance.

As for the old JSON response structure, that is going away in mapguide-rest. This new structure will be used for all APIs in mapguide-rest that return JSON. If you want the old JSON structured responses, you'll have to go back to the mapagent.

In one particular case, we eschew the convert-to-JSON-from-XML approach and define a new dedicated JSON response format. 

Ever tried to describe the schema of a Feature Source? This is the XML response:

Have you seen the JSON version of this monstrosity?

How on earth would any sane person know how to use this response in their applications? Yeah me neither.

So for schemas and class definitions that are returned as JSON, we introduce a more cleaner and intuitive JSON format

Isn't that a more intuitive and self-explanatory JSON response?

This new response structure is only applicable for JSON schemas and class definitions. The XML version of these schemas and class definitions will still be its same ugly and verbose self. From my perspective, XML is the "legacy" response format in mapguide-rest. JSON is the preferred response format for any client application and so we should strive to make consuming such data in this format as simple and clean as possible. With this release, hopefully we've achieved that objective.

While we're still on the subject of JSON, for certain REST APIs that previously accepted XML request bodies, such APIs now support JSON-equivalent request bodies. For example, consider this route to set resource content:

POST /rest/library/{resourcePath}/content.xml

To set resource content, you'd make a POST request to this route and include the resource XML as part of the request body.

You can now set the same resource content via its JSON representation

POST /rest/library/{resourcePath}/content.json

By making a POST request to this route and include the equivalent JSON as part of the request body. Because our JSON formats now structurally follow their XML counterparts, knowing what JSON to construct should be easy to figure out. On the server-side such JSON bodies will be converted back to its XML counterpart before passing down to the underlying MapGuide APIs (because sadly XML is still the canonical format for storage and data transfer throughout the official MapGuide APIs)

Authoring APIs for restcfg.json files

We've added a set of new APIs in the data publishing framework to allow for restcfg.json and any supporting files to be uploaded from a mapguide-rest client without needing physical access to the directory where the restcfg.json files are stored.

These authoring APIs allow you to:

  • Upload a restcfg.json file to a desired subdirectory within mapguide-rest based on the given URI part
  • Upload any supporting files to the subdirectory mapped to the given URI part
  • Delete any restcfg.json configuration
  • Delete any supporting file of a restcfg.json configuration
Because URI parts are mapped to physical subdirectories within a mapguide-rest installation, there could be a opportunities for malicious activity without proper security precautions. To this effect, we do the following:
  • Such APIs require Administrator or Author-level roles to be assigned to the authenticated user. The MapGuide Anonymous users will not have such roles, and thus any attempt at accessing these APIs will be denied.
  • Any URI parts and filename parameters are sanitized to strip off any semblance of relative paths and any other path navigation tricks that would cause mapguide-rest to save/delete a file or directory outside of its installation directory.

You can find out how to use these new APIs in the built-in interactive API reference.

API additions and cleanups

APIs that previously returned simple primitive values now return a XML or JSON boxed version of that value. Simply put, if an API returned a boolean value like this:


It will now return a boxed value, which looks like this for XML:

And looks like this for JSON:

{ "PrimitiveValue": { "Type": "Boolean", "Value": true } }

While this looks more verbose, it is simply to provide a consistent developer experience. If you have been expecting XML or JSON responses for 99% of the APIs, why make an exception for this 1% of APIs that return basic primitive values? They should be XML or JSON respectively as well and in order for that to happen, we have to box such values.

In addition many new APIs have been added to cover missing pieces and loose ends in our REST API:

  • Creating Feature Sources
  • Setting and deleting resource data items
  • Listing Long Transactions of a supported Feature Source
  • Getting the coordinate system base library
  • Validating Coordinate System WKT
  • Converting Coordinate System WKT to EPSG codes
  • Testing the connectivity of a Feature Source
  • Enumerating unmanaged (aliased) data
  • Getting site information status/statistics
  • DescribeSchema with class name hint
  • Setting content and header of any resource in a single request
  • Fusion layout authoring support APIs
  • Interrogating and setting the edit-ability of a Feature Source with the REST API

You can find out how to use these new APIs in the built-in interactive API reference.

(Geo)JSON CRUD support in restcfg.json

The data publishing framework now supports CRUD for JSON representations. The JSON representation is much more simpler and compact in its request payloads. The requests to create, update and delete features are structurally the same as the XML one (using the same XML-to-JSON pattern for most of our JSON data structures)

The existing Editable Properties example has been modified to use the JSON representation instead of the XML one. For doing CRUD through the data publishing framework, I'm sure you'll find it much easier to work with this JSON representation than the XML one.

Customizable HTML representation

The HTML representation for features are now customizable and localizable as the previous bits of hardcoded HTML that were rendered out by the HTML renderer, have all been extracted out into Smarty templates, where you can customize the HTML to your own liking.

MapGuide 3.0 support

This release of mapguide-rest supports new features introduced in MapGuide Open Source 3.0.
  • Your tile service routes can also work with Tile Set Definitions
  • XYZ tile requests will pass through to the MapGuide API on a MGOS 3.0 install
  • Runtime Map responses use the v3.0.0 response schema
Other changes
Where to from here?

From now to the final 1.0 release of mapguide-rest, it will be nothing but bug fixes and (really) minor enhancements.

After 1.0 as I've somewhat hinted, if mapguide-rest as a project should have a continued evolution, I strongly believe it should be on the 5.0 platform. Given 5.0 won't see light for many more months, I'll have plenty of time to play and experiment with many ideas that are possible with 5.0

Thursday, 12 February 2015

A small IronPython console improvement

I wasn't going to put out a new release of MapGuide Maestro without crossing at least this item off my list. So after several weeks of coder's block about how to solve this particular problem, I finally cracked it.

So I proudly present: Support for capturing user input from stdin in the IronPython console!

And yes, you can store the result as variables

You can also use input() for capturing user input from stdin, but it has the caveat of auto-evaluating whatever you enter as a Python expression. What this means is that if the user tried the same thing as above with input(), it will fail.

This is because the input of Jackie Ng is not a string literal, which causes Python to throw the SyntaxError. With input() if you are capturing strings, you have to quote them.

The real power of input() is allowing you to input raw python expressions, like capturing an array as input.

If you just want to capture string input, follow the advice of the Python documentation and use raw_input() instead.

The code to get this working doesn't look pretty IMO, but that's better than what we had before, which was nothing! So here's to more flexible python scripts in the next release of Maestro.

As for that next release, still some more pending items to tick off first.

Monday, 26 January 2015

MapGuide on Windows 10

Since Microsoft released a new preview build of Windows 10, I'd thought I give it a spin on VirtualBox.

My first impressions are simply that I am so happy to see the return of the Start Menu!

The other pleasant surprise was to see that the command prompt was finally resizable!

But my main motivation for giving Windows 10 a spin was to see if MapGuide will run on it. So I downloaded the 3.0 preview release.

I ran the installer and got this.

Okay, this normally means I have to manually "unblock" the executable in the file properties before retrying, so I did exactly that.

Except unblock does nothing! Is this a bug?

So before I chew out Microsoft for their "UAC v2.0" over-zealousness with this SmartScreen feature, I decided to click the "More Information" link on the SmartScreen dialog to see if it does anything and lo and behold, it gives me the options to finally run the thing!

Terrible UX there Microsoft. The "Run Anyway" option should be visible IMO and not concealed in the "More Information" link.

Nevertheless, after negotiating this little obstacle the installer ran after the UAC prompt. I decided to try the IIS/.net install option since:

  • PHP and its MapGuide API bindings is installed regardless (it's why mapguide-rest will/should just work out of the box on any supported version of MapGuide), so we can also verify the PHP bits are working by loading a package through the Site Administrator
  • This is the configuration that we always want to confirm and verify when testing support for newer versions of windows. The Apache/PHP/Java configurations are mostly self contained and can be generally trusted to work on any windows platform where there is a matching Visual C++ runtime library and a working Java SDK/RE for it.
So after waiting the normal amount of time for a MapGuide install to complete.

The Site Administrator runs without issues, allowing me to load my sample data package. Firing up the AJAX viewer (that is bound to the .net implementation due to my installation choice), shows that it is indeed functional

So there you have it. MapGuide 3.0 works on Windows 10. 

Given its current Technical Preview status, we cannot realistically have Windows 10 as a supported platform for MapGuide 3.0, but should you want to try, chances are MapGuide should work as evidenced by this post.

Saturday, 17 January 2015

MapGuide 3.0 feature showcase: Bumping the version number

So now you know. The next major release of MapGuide will be MapGuide Open Source 3.0

Why the major version bump to 3.0? It's mainly for us to signify to you (the MapGuide application developer) that we are breaking free from some of the cruft and baggage that has accumulated in MapGuide over the years.

This is the cruft that you will no longer see in MapGuide Open Source 3.0

Goodbye blank MgMap constructor

The blank MgMap constructor existed in the MapGuide API from the very beginning. A new overload of the MgMap constructor (introduced with RFC 9) allows access to various convenience APIs in MgMap, but such APIs can only be available with the use of the correct constructor: The one that is not blank.

Having two different behaviours based on which constructor you use kind of violates the principle of least astonishment. Even though we have deprecated the blank MgMap constructor and try to leverage Java/C# language features and having a dedicated blog post about it to try to communicate this fact to you, there's nothing actually stopping you from using the blank MgMap constructor and having convenience APIs in MgMap throw exceptions back at you as a result.

With the 3.0 release, we removed the blank MgMap constructor and associated verbose APIs, eliminating this confusion once and for all. The convenience APIs will now always work because there will now only be one MgMap constructor to use instead of two.

Goodbye DWF eMap support

DWF eMap is basically the DWF analogue to the MapGuide 6.5 ActiveX viewer of yore. It allows your maps to be viewed in the AJAX viewer using a DWF viewer ActiveX plugin that can consume eMap data from MapGuide.

The problem (beside the ugly ActiveX) is that eMap support has long since been removed in the DWF Toolkit. You needed Autodesk Design Review 2009 or older (AFAIK) to have a DWF viewer ActiveX control that supports eMap data. We've had to stick with an ancient version of the DWF Toolkit in order to support this feature. As a side effect of an ancient DWF Toolkit, this presents DWF interoperability issues with Autodesk products with newer versions of the DWF Toolkit. We've had stories of DWF files plotted from MapGuide crashing current versions of Autodesk Design Review.

With the 3.0 release, we've upgraded the DWF Toolkit to the latest version (7.7) and removed DWF eMap API and viewer support. If you are still somehow using this eMap functionality, this is the end of the line for you. You should seriously consider migrating to our current viewer offerings, or stay on the 2.6 series if this is not an option.

For more details of what APIs have been cut, you can consult MapGuide RFC 139

Isn't it ironic?

Nope, that title is nothing more than a segway into this post :)

I've just discovered an awesome language implementation library called Irony, and have been playing around with some grammar that you might be familiar with

This has positive implications for Maestro if I can get Irony to do what I want it to do.

So far that looks very promising!

Tuesday, 13 January 2015

Announcing: MapGuide Open Source 3.0 Preview

It's that time of the year where we kick release cycle for the next version of MapGuide into gear.

Grab the installers for MapGuide 3.0 here. Linux builds will follow shortly.

Stay tuned for what's new in MapGuide 3.0. I think you'll like what's in store.