Friday, 24 June 2016

Let's introduce the other major player

My plans for the VSCode map extension now extend beyond just OpenLayers



At this point there are some small oddities:

  • The timeline button SVG has broken hrefs. You can see it from the missing play/pause/rewind graphics.
  • The Cesium InfoBox doesn't work.
This is probably due to a combination of:
  • Running cesium from the local file system
  • The VSCode/Electron webview or Cesium itself imposing sandboxing restrictions that breaks the InfoBox
Still, for the purposes of being able to quick and easily preview CZML content, this is a very nice starting point.

Friday, 17 June 2016

Announcing: vscode-map-preview 0.4.0

Today I pushed out a new version (0.4.0) of the vscode-map-preview extension. You might have already gotten a prompt from Visual Studio Code to auto-update to this version.

Here's some new features in this version.

Configuration Options

This version adds a bucket-load of configuration options around:
  • Controlling the style of vector features in your preview layer
  • Controlling the style of selected features in your preview layer
  • Controlling the display of the mouse coordinate tracker
  • Controlling the default base layer to use
Once saved to your user settings, these updated settings take effect on the next preview



Preview (with specific projection)

When previewing a file format, the source project can generally be inferred. For example, KML will always be EPSG:4326, for GeoJSON it should figure it out if you have a crs property on there.

But then you have formats like Well-Known Text (WKT), which we can render a preview of, but without proper contextual information such as projection, the data may not be placed where you expect it to be.

For example if we preview WKT as-is:


Because there is no projection context in the data itself, the coordinates are assumed to already be in EPSG:3857, and means the point lies just off the shores of Null Island, which in actuality, the above coordinates are actually supposed to be EPSG:4326 and is supposed to be the central business district of Melbourne, Australia.

To fix this, we need a way to declare that the data we're previewing is actually of a specific projection, which is what the new Map Preview (with projection) command gives us. It's just like the regular Map Preview command, but provides an input prompt for entering the proper EPSG code for this data.


Which will then be used as the projection of the data about to be previewed, ensuring the data is placed at the right location.


As an aside, you can put away that SQL Server Management Studio. VSCode with this extension is way more quicker and lightweight way for easy WKT visualization.

One caveat is that for this new command the only supported projections are:

  • EPSG:4326
  • EPSG:3857
I'll need to add custom projection support in order to be able to add support for more projections beyond the above two projections.


Other Changes

This version includes fixes to allow for subsequent map previews to work better and fixes various CSS issue with the map preview.

If you haven't already, check out and install the extension on the Visual Studio marketplace.

Wednesday, 8 June 2016

Introducing: Map Preview extension for Visual Studio Code

From an idea, to an MVP, to now a published extension on the Visual Studio Marketplace

I am proud to announce the availability of the Map Preview extension for Visual Studio Code

Just to recap for those who don't know, this extension allows you to view any plain-text based geospatial data file or content as vector features on an interactive map (powered by OpenLayers), allowing for quick and easy visualization of plain-text geographic content without firing up a dedicated GIS application to view it or pasting the content into a GitHub gist.



All rendered features are selectable on the map and any useful attribute information is displayed as a popup. The map also provides plenty of available base layers for giving your preview data a real-world contextual backdrop.

The initial release supports formats that are supported by OpenLayers, which is currently:
If your document content checks out as any of the above formats, it can be previewed on a map with this extension.

You can install this extension with the following command:

ext install vscode-map-preview

The source code for this extension is available on my GitHub repo

Feel free to send feedback, pull requests, etc, etc.

Thursday, 2 June 2016

Because sometimes making a gist on github is just too tedious

It started from a conversation today at work.

"You know how you can preview markdown files as HTML in Visual Studio Code?"
"Yeah"
"What if there was an extension that could do this previewing as well, but with non-binary geospatial data files like GeoJSON? So you don't have to ... you know ... go to github, make a gist, paste your GeoJSON and preview?"
"That would be awesome!"

So once I got home, I did some research to answer some questions that will test whether this idea is feasible:

That confidence boosting set of findings gave me enough motivation to fire up VSCode, yo generate a new TypeScript VSCode extension project, wait for npm to install 50 bajillion packages and set out to hack out an MVP extension for VSCode.

2 and a half hours later ...


I love it when you have an idea, the idea shows feasibility after some encouraging results from research and after some hacking around, you reach a point like the above gif that tells you ... yes this idea definitely has some legs and you should keep on going!

Friday, 27 May 2016

Announcing: MapGuide Open Source 3.1 Beta 1

After several months of messing with vagrant for a more streamlined build pipeline and clearing some last-minute roadblocks, I am happy to finally announce the availability of the 1st beta release of MapGuide Open Source 3.1.

In terms of new features, well ... 3.1 can mostly be considered a "bugfix" release. The major features I had hoped to get implemented before this release cycle did not materialize in time, so I'll spare you the usual showcase series of blog posts every time we do a new major release, because you can fit it all in this one post.

New Layer Definition Cache

When MapGuide renders maps, we have to fetch Layer Definition documents from the repository to know how to render and style feature data. Until now, those documents aren't being cached, so we have to fetch the same Layer Definition documents over and over every time MapGuide renders a map image.

3.1 introduces a new server-side Layer Definition cache so we can avoid repeated fetching of Layer Definitions from the repository (and converting the XML to its object-oriented form) every time we render maps. For maps with a very large layer count, some noticeable gains in rendering performance should be noticeable.

Updated Web Tier Components

MapGuide Open Source 3.1 Beta 1 includes the following updated Web Tier components.

  • Apache HTTPD 2.4.18
  • PHP 5.5.33
  • Tomcat 7.0.68

Linux support

The Ubuntu build of MapGuide Open Source 3.1 Beta 1 now supports Ubuntu 14.04 LTS.

The Linux builds in general have a new and improved service control script for:

  • Restarting the mgserver daemon
  • Checking the running status of the mgserver daemon
  • Extra warnings when attempting to start an already started mgserver daemon or stopping an already stopped mgserver daemon
Many thanks to Gabriele Monfardini for providing this new service control script.

Like 3.0, the 64-bit Linux builds should still be considered experimental/preview for 3.1 and should still only be used for testing and feedback. Use in production at your own risk.


Fusion

Fusion has some important and long overdue fixes for Google Maps support and various widgets


For more information, consult the release notes in the link below.

Download

400 posts!

... about the release of MapGuide Open Source 3.1 Beta 1?

Nah, we'll save that one for the post after this one.

For this is the 400th published post on this blog!




This year has slowed down a bit in blog-age, but as I've mentioned previously, it's not due to lack of content. Like a CPU at 100% usage, I've been busy with stuff. But that should be hopefully fleeting and I'll be resuming normal blogging operations soon.

Thank you all for your continued eyeball patronage.

Thursday, 26 May 2016

MapGuide Open Source 3.1 pre-flight check part 3 (a wild roadblock appeared!)

Stop the presses. We ran into a blocker!

A particular nasty one, that affects not just MapGuide on CentOS, but on Ubuntu as well.

Something slipped into MapGuide or the WMS FDO provider that causes mgserver to segfault when one does a GETFEATUREPROVIDERS request and a WFS/WMS FDO connection is created when inspecting its capabilities. Previously, I thought it was a quirk with the internal FDO source copy of OpenSSL and so FDO was built using the system-installed copy of OpenSSL (like we did for Ubuntu), but this problem still persisted on CentOS, and it turned out it was also segfaulting on Ubuntu as well!

Running this under gdb didn't help us at all. It only shows that the SIGSEGV happens at a function named ?? in libWMSProvider.so on CentOS, and at a function named _init in libWMSProvider.so on Ubuntu. Not very helpful.

Even the venerable valgrind couldn't help me here. Putting mgserver under valgrind and triggering the segfault gives me this incomprehensible gibberish

==11521== Thread 13:
==11521== Jump to the invalid address stated on the next line
==11521==    at 0x6E8A6: ???
==11521==    by 0x4275D9F: FdoConnectionManager::CreateConnection(wchar_t const*) (ConnectionManager.cpp:334)
==11521==    by 0x4BBB543: MgServerGetFeatureProviders::AddConnectionProperties(xercesc_3_1::DOMElement*, wchar_t const*) (in /usr/local/mapguideopensource-3.1.0/server/lib/libMgServerFeatureService-3.1.0.so)
==11521==    by 0x4BBADD3: MgServerGetFeatureProviders::CreateFeatureProvidersDocument() (in /usr/local/mapguideopensource-3.1.0/server/lib/libMgServerFeatureService-3.1.0.so)
==11521==    by 0x4BB9E23: MgServerGetFeatureProviders::GetFeatureProviders() (in /usr/local/mapguideopensource-3.1.0/server/lib/libMgServerFeatureService-3.1.0.so)
==11521==    by 0x4B67285: MgServerFeatureService::GetFeatureProviders() (in /usr/local/mapguideopensource-3.1.0/server/lib/libMgServerFeatureService-3.1.0.so)
==11521==    by 0x4B303A4: MgOpGetFeatureProviders::Execute() (in /usr/local/mapguideopensource-3.1.0/server/lib/libMgServerFeatureService-3.1.0.so)
==11521==    by 0x4B19AE6: MgFeatureServiceHandler::ProcessOperation() (in /usr/local/mapguideopensource-3.1.0/server/lib/libMgServerFeatureService-3.1.0.so)
==11521==    by 0x809768A: MgOperationThread::ProcessOperation(MgServerStreamData*) (OperationThread.cpp:397)
==11521==    by 0x8095C1B: MgOperationThread::ProcessMessage(ACE_Message_Block*) (OperationThread.cpp:226)
==11521==    by 0x8094536: MgOperationThread::svc() (OperationThread.cpp:90)
==11521==    by 0x623521B: ACE_Task_Base::svc_run(void*) (in /usr/local/mapguideopensource-3.1.0/lib/libACE.so)
==11521==  Address 0x6e8a6 is not stack'd, malloc'd or (recently) free'd
==11521== 
==11521== 
==11521== Process terminating with default action of signal 11 (SIGSEGV)
==11521==  Bad permissions for mapped region at address 0x6E8A6
==11521==    at 0x6E8A6: ???
==11521==    by 0x4275D9F: FdoConnectionManager::CreateConnection(wchar_t const*) (ConnectionManager.cpp:334)
==11521==    by 0x4BBB543: MgServerGetFeatureProviders::AddConnectionProperties(xercesc_3_1::DOMElement*, wchar_t const*) (in /usr/local/mapguideopensource-3.1.0/server/lib/libMgServerFeatureService-3.1.0.so)
==11521==    by 0x4BBADD3: MgServerGetFeatureProviders::CreateFeatureProvidersDocument() (in /usr/local/mapguideopensource-3.1.0/server/lib/libMgServerFeatureService-3.1.0.so)
==11521==    by 0x4BB9E23: MgServerGetFeatureProviders::GetFeatureProviders() (in /usr/local/mapguideopensource-3.1.0/server/lib/libMgServerFeatureService-3.1.0.so)
==11521==    by 0x4B67285: MgServerFeatureService::GetFeatureProviders() (in /usr/local/mapguideopensource-3.1.0/server/lib/libMgServerFeatureService-3.1.0.so)
==11521==    by 0x4B303A4: MgOpGetFeatureProviders::Execute() (in /usr/local/mapguideopensource-3.1.0/server/lib/libMgServerFeatureService-3.1.0.so)
==11521==    by 0x4B19AE6: MgFeatureServiceHandler::ProcessOperation() (in /usr/local/mapguideopensource-3.1.0/server/lib/libMgServerFeatureService-3.1.0.so)
==11521==    by 0x809768A: MgOperationThread::ProcessOperation(MgServerStreamData*) (OperationThread.cpp:397)
==11521==    by 0x8095C1B: MgOperationThread::ProcessMessage(ACE_Message_Block*) (OperationThread.cpp:226)
==11521==    by 0x8094536: MgOperationThread::svc() (OperationThread.cpp:90)

==11521==    by 0x623521B: ACE_Task_Base::svc_run(void*) (in /usr/local/mapguideopensource-3.1.0/lib/libACE.so)

I admit to not being an expert-level C++ developer, so when I can't find the usual cases for a SIGSEGV (eg. A null pointer de-reference) in gdb/valgrind output, I am often left scratching my head. Google searches on these error messages weren't of much help.

The good thing was that this problem only happens on FDO since 4.0 was branched and released (we're targeting FDO 4.1 here) so in the absence of expert-level C++ knowledge it was time to be systematic and track down the offending FDO revision since 4.0 branched that introduced this breakage. Time to apply that binary search again! After several wasted days of trial and error compilation/testing, we found the offending revision.

At face value, this commit looks innocuous enough, a few new files and some modifications. Nothing in the new and modified files that sounds like dangerous C++ code to my padawan-tier knowledge.


But looking at the overall list of files, something caught my eye. In this commit was a project file update, but it was only for windows (the modified WMSProvider.vcxproj file). Normally if new files are to be added, not only should the windows vcxproj file be updated as well (to include the new .h and .cpp files), but the corresponding Makefile.am files for Linux as well, and this was conspicuously absent.

This lit a light-bulb in my head, I had a look at the Makefile.am file for the WMS provider and lo-and-behold, it didn't have the new .cpp and .h files in this commit. Then it all came together.

The WMS provider was segfaulting because it was being compiled with undefined symbols (to the new FdoWmsGetFeatureInfoFormats class). Updating the Makefile.am with the missing files and re-compiling the WMS provider made the segfault go away.

And that was the tale of how I cleared the final roadblock to releasing MapGuide Open Source 3.1 (smoke tests with this change all pass, the beta 1 binaries are being built as this blog post is being published). Now on Windows with the MSVC compiler, such a problem would've manifested at the compiler phase as LNK2001 unresolved external symbol errors, so it surprised me big time that gcc or ld did not error out about undefined symbols and immediately break the build which would've caught this problem much sooner!

It turns out gcc/ld erroring out is a flag you have to opt-in to. The more you know. Thanks to Johan for the heads up.

So you can now expect the next post to be about ...