Bootstrap map viewer templates

So this past week, I attended another hackathon and our hack was yet-another-twitter-bootstrap-with-openlayers concoction. The problem it seems, every time I go down this path, I lose several critical hours re-creating my desired responsive layout of a map viewer with sidebar with bootstrap CSS every time. You'd think that having done this many times now, I'd have memorised the necessary HTML markup and CSS to do this by now.

But that is just not the case, so after the event (and a good long post-hackathon sleep), I fired up Sublime Text and set out to solve this problem once and for all: A set of starter bootstrap templates that should serve a useful foundation to build a bootstrap-based map viewer web application on top of. Several hours later, there's two templates available.

A 2-column template. A full-screen map viewer with a sidebar to the left:

And a 3-column template, which has sidebars to the left and right which is reminiscent of the classic MapGuide AJAX viewer.

A key feature of both templates is that the floating sidebars are both collapsible.

On small displays, sidebars start off initially collapsed, but can be brought out through their respective toggle buttons.

You can check out the templates below:

• 2-column template
• 3-column template

Do you have ideas for improving my initial design and have some web design skills? Send me some pull requests.

I have a dream

I have a dream

Where MapGuide and FDO source code are hosted and/or mirrored on GitHub.

Where by the virtue of being hosted on GitHub, these repositories are set up to take advantage of every free service available to improve our code quality and developer workflow:

• TravisCI for Continuous Integration of Linux builds
• AppVeyor for Continuous Integration of Windows builds
• CoverityScan for static code analysis
• Coveralls for code coverage analysis
• What other awesome services can we hook on here? Please enlighten me. I'd really want to know.

Where a single commit (from svn or git) can start an avalanche of cloud-based services that will immediately tell me in several hours time (because C++ code builds so fast doesn't it?):

1. If the build is OK (thanks to Travis and AppVeyor)
2. Where we should look to improve our test coverage (thanks to coveralls)
3. Areas in our codebase where we should look to change/tweak/refactor (thanks to coverity scan)
4. Other useful reports and side-effects.

Now the difference between dream and reality is that there are clear obstacles preventing our dream from being realised. Here's some that I've identified.

1. Git presents a radically different workflow than Subversion

Yes, we're still using subversion for MapGuide and FDO (har! har! Welcome to two-thousand-and-late!). Moving to GitHub means not only a change of tools, but a change of developer workflows and mindset.

So in this respect, rather than a full migration, an automated process of mirroring svn trunk/branches (and any commits made to them) to GitHub would be a more practical solution. Any pointers on how to make this an idiot-proof process?

2. Coverage/support is not universal

MapGuide/FDO are multi-platform codebases. Although TravisCI can cover the Ubuntu side and AppVeyor can cover the Windows side, it does leave out CentOS. I've known enough from experience (or plain ignorance) that CentOS and Ubuntu builds need their own separate build/test/validate cycles.

And actually, Travis VMs being 64-bit Ubuntu Linux doesn't help us either. Our ability to leverage Travis would hinge on whether we can either get 64-bit builds working on Linux or am able to cross-compile and run 32-bit MapGuide on 64-bit Ubuntu, something that has not been tried before.

Also most service hooks (like coveralls and CoverityScan) target Travis and not AppVeyor, meaning whatever reports we get back about code quality and test coverage may have a Linux-biased point of view attached to them.

3. The MapGuide and FDO repositories are HUGE!

The repositories of MapGuide and FDO not only contain the source code of MapGuide and FDO respectively, but the source code of every external thirdparty library and component that MapGuide/FDO depends on, and there's a lot of third-party libraries we depend on.

If we transfer/mirror the current svn repositories to GitHub as-is, I'm guessing we'd probably be getting some nice friendly emails from GitHub about why our repos are so big in no time.

Also would Travis and AppVeyor let us get away with such giant clones/checkouts happening every time a build is triggered in response to a commit? I probably don't think so. Then again, I do live in a country where bandwidth doesn't grow on trees and our current government has destroyed our dreams of faster internet. What do I know?



So what do you think? Is this dream something worth pursuing?

Announcing: CentOS build of MapGuide Open Source 2.6

As mentioned in my previous post, the CentOS blocker is now resolved.

This means we now once again have a functional CentOS build of MapGuide Open Source 2.6 which is now available for download on the 2.6 release notes page.

I'm still scratching my head as to how aclocal/libtoolize/automake/autoconf on a thirdparty MapGuide component somehow makes iconv_open() fail!

Binary Search Algorithm: Applied in real life

When I announced the availability of MapGuide Open Source 2.6, we did not have a CentOS build available due to a serious blocking issue where iconv APIs would mysteriously fail within MapGuide.

The main symptoms of this iconv failure are:

• Under default build settings, mgserver will fail on start up with a "could not load a transcoding service" error from the xerces library. The default build settings use iconv APIs to transcode strings within the xerces library.
• The SHP FDO provider will throw a "memory allocation failed" error when attempting to connect to any SHP feature sources. The SHP provider also uses iconv APIs for narrow <-> wide string conversions.

While the first problem was worked around (by building xerces to use a different transcoder), the second one was truly back breaking. I didn't want to release 2.6 for CentOS where support for the most ubiquitous spatial data format was broken out of the box.

As a first priority after the 2.6 release, I went to see when this issue first cropped up and what was the offending component.

The 2.5.2 release for CentOS did not have this issue, so to eliminate FDO as the offending component I built the 2.5.2 release against the FDO 3.9 branch. Fortunately, the iconv failure did not show up, so we now knew that FDO was not the culprit. It was going to either be MapGuide or one of its Oem components that has caused this breakage.

Knowing that FDO was not the culprit, it was time to start identifying the svn revision in MapGuide that brought us this mess. Unfortunately, there's been quite a lot of revisions between 2.5.2 and 2.6 and knowing how long it takes to build and verify a single revision of MapGuide (because ... C++ code), it would be painstaking to build and verify every single revision.

So to take a logical shot in the dark as a means of reducing the set of svn revisions to identify, I picked the very first working revision of the 2.6 branch to see if this issue exists. It didn't (yay!), meaning our problem space is now reduced to 60 commits in the 2.6 branch.

One of these 60 revisions broke the CentOS build. Rather than wasting time building and verifying 60 individual revisions to identify the breaking revision, I took a more systematic approach and picked the closest "mid-point" revision that affected files in the Server/Oem/Common/Web directories.

If the problem showed up there, we can reduce the problem space to revisions older than that one being tested (ie. some revision older than the tested one introduced the problem), otherwise we can reduce it to revisions newer than that one being tested (ie. some revision newer than the tested one introduced the problem) and repeat the process, until our problem space becomes a single revision that fails. That revision is the revision that broke our CentOS build.

As the tale of my Trello card to track this problem can attest to, finding the offending revision was pretty quick.

And if the title of this post didn't give it away, this systematic process has a name: It's called a Binary Search Algorithm

Though in our case it's not a true binary search, more like a "biased" binary search in that although our problem space was 60 revisions, some of these revisions did not touch any part of the MapGuide Server/Oem/Web/Common code base, so such svn revisions can be excluded from our problem set. Also when the candidate revision to test landed beside a "big merge" revision, we tested that "big merge" revision as well just so we can immediately rule it out.

So there you have it. Knowing Binary Search is not just for passing your Computer Science exam or that Software Developer Job Interview or to implement various data structures, it has real life applications too like hunting down what commit broke your build.

Some might say, wouldn't a Continuous Integration system have caught this? Indeed it would've, but as I've talked about previously about how we make our builds of MapGuide, the Windows builds of MapGuide are built under Jenkins (which can detect and flag broken/unstable builds thanks to its rich ecosystem of plugins), but the Linux builds are not. Linux builds although they are mostly automated now, still require manual invocation (to start the vagrant provisioning process) and manual review of the various log files produced to see if anything broke. The offending revision obviously slipped through the radar.

The Linux build system obviously has much more room for improvement, something that we can now have the opportunity to explore now that 2.6 is out the door.

But before we go about that, let's put out that overdue 2.6 build for CentOS.

Using PHP composer with MapGuide's bundled PHP

If you're building PHP applications for MapGuide (like I am currently doing with mapguide-rest) you should strongly consider using PHP composer for installing and managing all your external PHP libraries.

PHP composer is basically the PHP version of NuGET (for .net) and simplifies and accelerates development of PHP applications by allowing you to install and manage third-party PHP libraries and frameworks with ease.

No dependency hell or wondering what PHP files to include/require. PHP composer will sort all of that out, leaving you to simply "require 'vendor/autoload.php'" to start using your libraries and like NuGet.org, the Packagist website will help you find the PHP library you need to solve your particular problem.

So how do you make sure when installing PHP composer that it will work with MapGuide's bundled PHP? In the windows installer, simply set the PHP path to where MapGuide's PHP is installed.

For Linux or if you didn't use the Windows installer, you should make sure the php that is running when you invoke composer is the one from your MapGuide installation and not a system-based PHP if you so happen to have one installed.

Once installed, you can start to enjoy a (less painful and more exciting) PHP development experience.

MapGuide tidbits: RenderTile (or: Build Your Own Tile Management system)

Here's a little tidbit about tiles in MapGuide.

Are you not satisfied with MapGuide's tile management? The fact that saving a Map Definition blows away your precious tiles? The fact that you have no control where rendered tiles are stored?

Well it just so happens that the functional primitive for rendering tiles is also available in the MapGuide API: The RenderTile() method of MgRenderingService. This API only renders the tile, it does not store it in MapGuide's pre-defined tile location. The tile storage and management is completely up to you to implement. You can think of the GetTile() method of MgTileService as simply being RenderTile() with tile access/management built in. If you don't like how MapGuide does its tile management, you can implement this part yourself, and call RenderTile() yourself for the actual tiles.

Although we don't use this API in mapguide-rest for XYZ tiles, we follow the same concept of doing the tile access and management ourselves at the PHP level, but still tapping into the actual MapGuide API itself to render the actual XYZ "tiles".

This approach does have its own downsides. The automatic tile cache invalidation that MapGuide does when dependent upstream resources are saved and some users find annoying won't be at play here. You'll have to manually purge these cached tiles yourself, otherwise you may find that your cached tiles are horribly visually out-of-sync with respect to the current Map Definition and its layer structure and styles. This is what the automatic tile cache invalidation is trying to prevent, only it does it with the surgical precision of a nuclear warhead.

GovHack 2014 post-post-mortem

This past Sunday was the awards ceremony for GovHack 2014.

After a month of Judging and voting on hacks from over 150 different teams across Australia, it was time to find out who would take the prizes and the brownie points.

In the tense moments of anticipation, our team "CreativeDrought" was mentioned.

Not once.

Not twice.

But three times!

Expecting this haul was way beyond what I was expecting. Just winning one award would've been a nice feather on my cap, but winning 3 awards ... my cap has become a peacock!

This was not the first time I won a prize at a hackathon. My previous win was at RHoK 2012 once again with MapGuide playing a major part in the solution. That's 2 out of 2 MapGuide-assisted hackathon victories. Not bad if I say so myself!

Now that GovHack and the judging is over, I'll guess I'll leave our demo site up for another few weeks before I decide what to do with it.

Move over Sheboygan! There's a new MapGuide sample dataset in town!

For the longest time, the Sheboygan dataset has been our reference dataset for demonstrating the data publishing capabilities of MapGuide as well as being the focal point for all our sample code demonstrating the use of the MapGuide API.

Unfortunately, the actual spatial data in the Sheboygan dataset does not lend itself for MapGuide to demonstrate its capabilities to its fullest. Short of the Parcels layer, everything else in the dataset is just "dumb" spatial data, lacking any real attributes of substance that we could use to demonstrate features of MapGuide with.

MapGuide deserves a more comprehensive and intelligent sample dataset to show off its features and capabilities, and I think I found it.

Say hello to my home city of Melbourne

This MapGuide dataset was made possible by the various bits of open spatial data made available under City of Melbourne's open data platform. I discovered this site the night before GovHack 2014 as City of Melbourne was one of the "data custodians" for the GovHack event. Though I didn't use any of their data over the hackathon weekend, I did make a mental note about the rich and varied volume of data that they had available and that there might be enough stuff in there to make a much more compelling sample dataset for MapGuide than what we currently have with Sheboygan.

So a few weeks after GovHack, I paid another visit to that site to check out their data offerings in more detail and see what kind of dataset we can make in MapGuide with their available datasets. The above screenshot is the current work in progress. The MapGuide dataset was assembled together very quickly with the help of MapGuide Maestro. I guess that's a testament to the many years of development work put into Maestro to make it the most efficient authoring tool for MapGuide (not trying to toot my own horn here honest :)).

Most of the spatial data was in SHP format making loading into MapGuide a dead simple affair. Some of the spatial data was in CSV (huh?), so these were converted to SQLite with the help of OGR and its awesome virtual format feature. One particular dataset they had which sparked my initial interest in building this MapGuide dataset was the Building Footprints, which contains actual height attributes!

This means demonstrating MapGuide's KML support is much more exciting now because we can actually tap into the elevation and extrustion support to give actual height to our KML buildings exported from MapGuide.

As mentioned before, this MapGuide dataset is a work-in-progress which you can check out on my GitHub. If you want the actual package, you can get it from the releases page. If you are interesting in seeing a more comprehensive sample dataset for MapGuide, I encourage you to contribute to this repo. If you are so inclined, I'd also encourage you to have a look at the City of Melbourne data portal and see what other interesting datasets we could integrate.

It would be nice to also have a nice set of sample code that works against this dataset as well. Aside from porting the developer's guide samples across to work against this dataset, if you have any interesting ideas we could explore with this dataset, I'm all ears.

None of this would be possible if the City of Melbourne didn't open up their various datasets. So full kudos to them for not only opening up these datasets, but also for licensing their data under Creative Commons and not some "open" license that's entangled under various legal spiderwebs.

#opendata ftw!

Announcing: mapguide-rest 0.9

Here's a new release of mapguide-rest, packed with some new toys.

PDF Support

The existing URL routes for DWF plotting now also support PDF output.

For example, a DWF plot URL like this:

http://localhost/mapguide/rest/library/Samples/Sheboygan/Maps/Sheboygan.MapDefinition/plot.dwf?x=-87.73&y=43.74&scale=8000

Can be turned into a PDF plot like this just by replacing the .dwf in the URL to .pdf:

http://localhost/mapguide/rest/library/Samples/Sheboygan/Maps/Sheboygan.MapDefinition/plot.pdf?x=-87.73&y=43.74&scale=8000

When plotting from a session-based runtime map, you also have the option of producing a layered PDF. To produce a layered PDF, simply append layeredpdf=1 to a PDF plot URL on a session-based runtime map. This takes longer to plot than a normal PDF, but the resulting PDF will have a toggle-able layer structure

Queryable Layer Definitions

The existing URL routes to query feature data from Feature Sources can now also be applied to Layer Definitions.

For example, suppose you originally were querying Parcels from its Feature Source

GET http://localhost/mapguide/rest/library/Samples/Sheboygan/Data/Parcels.FeatureSource/features.xml/SHP_Schema/Parcels?maxfeatures=500

You can now also query the same data via a Layer Definition that points to it

GET http://localhost/mapguide/rest/library/Samples/Sheboygan/Layers/Parcels.LayerDefinition/features.xml?maxfeatures=500

You'll notice that querying via Layer Definitions means you don't have to include the fully-qualified FDO class name in the URL as that will be inferred from the Layer Definition document. Also when querying via Layer Definitions, it will also include evaluated tooltip and hyperlink expressions for each feature (as MG_TOOLTIP and MG_HYPERLINK properties)

The Layer Definition query route supports the same parameters as the Feature Source one and supports GeoJSON output as well.

New Samples

We've added a new OpenLayers 3 example, demonstrating how to consume dynamic GeoJSON vector layers from the REST API

And a new Cesium example, also consuming GeoJSON from the REST API

Other Changes

• GeoJSON features now include id attributes containing the identity property value. Previously the identity property value was written out as a regular feature property.
• Improved XYZ tile cache resiliency
• Feature query routes (ie. selecting features) no longer require an authentication challenge and can be done anonymously.
• Feature query responses are no longer internally buffered, reducing response wait times.

Download

Announcing: MapGuide Open Source 2.6 Final

Here it is! The final release of MapGuide Open Source 2.6

Here's the changes from RC2:

• Fix: PHP deprecation warning in schema report feature/data preview
• Fix: Cannot select search results in Fusion Search widget

It is with regret that we have to pull the CentOS 6.x build from this final release. There will not be a CentOS build of MapGuide 2.6 until we have found a solution for this issue of non-functional iconv APIs. While this issue is present, the CentOS build will not have a functional SHP FDO provider and that is too much of a deal-breaker. If you are on CentOS, you should either stay on the 2.5.2 release or help assist us in finding a solution for this issue.

A note for those who want to use custom GDAL binaries from gisinternals for ECW/MrSID support on Windows: You should probably hold off from upgrading to 2.6. MapGuide 2.6 was built with Visual C++ 2012, and so requires custom GDAL binaries that were also built with MSVC 2012. These binaries don't exist yet on gisinternals. You should keep an eye on this issue.

Since is the final release, I will also talk a bit in this post about the mg-desktop release that also coincides with this release.

mg-desktop now requires the Visual C++ 2012 runtime library as a pre-requisite for any application that you build with this release of mg-desktop. In terms of changes, there's only two real changes:

• Configuration Documents with external path aliases are now supported
• Fix: Layer watermarks will not be rendered for layers that are not visible

Download MapGuide 2.6 Final

Download mg-desktop

mapguide-api-web at NuGet Gallery

mg-desktop at NuGet Gallery