About that grand plan of 2016 ...

Not all plans go as intended. In my case, it was my grand plan for this year.

If you thought this was the mapguide-react-layout project, then I'm afraid you were mistaken. It was actually something else entirely (I did stress in that post that the project won't be MapGuide-specific). However, due to external circumstances, this plan never and probably will not materialize because I no longer see the need for it.

So since I don't see this grand plan going ahead in the near future, I might as well talk about what it was going to be. Basically, I was wanting to make a Cesium-based replacement for the Google Earth desktop application, using the stack of:

• Electron
• Cesium (possibly in conjunction with OpenLayers)
• React
• TypeScript

Why did I want to build this application (which strangely sounds like some chemistry/physics experiment, what's with these projects co-opting their terminology?). I wanted a replacement for Google Earth. 

Google Earth was good for only one thing: Visualizing KML files referenced against good quality satellite imagery.

It was absolutely horrible in all other aspects:

• I can't mash KML together other geospatial data sources (SHP files, GeoJSON, etc)
• KML is a horrible data interchange format not helped by the fact that Google Earth can't import/export to-and-from different geospatial data formats out of the box! You need a 10+ step data transformation pipeline involving OGR, QGIS and friends to get spatial data in and out of KML. OGR having 2 KML drivers adds an extra layer of confusion to the mix with one guaranteeing to trash any semblance of intelligence by outputting dumb KML geometry.
• Creating KML files is a chore because of GE's cumbersome drawing tools
• Modifying KML files is an even greater chore because GE lacks basic GIS-y tools like buffering, split, etc.

Going all in on QGIS wasn't desirable either because while it had the kitchen sink of GIS tools and functionality and rich support for countless geospatial data formats, it lacked the rich satellite imagery that GE provided and I've had a mixed experience with the various plugins available for QGIS that allowed me to drop-in OSM/Bing/GoogleMaps for real-world context when creating/editing features.

My hypothetical application was intended to be the sweet spot in between GE and QGIS and would've addressed most of these pain points with Cesium/OpenLayers doing most of the heavy lifting (Cesium has strong KML visualization capabilities and OpenLayers is the swiss army knife supporting many vector data sources and has rich editing capabilities) and having it all housed within an Electron host for a desktop experience.

But as I said, due to external circumstances, this idea has been mostly abandoned now. Those circumstances being mainly: I've found something that already addresses most of my pain points with Google Earth.

It's called geojson.io. Despite its deceptive name^ it solved most of my grievances with Google Earth:

• I can bring in KML files and can easily visualize its geographic form
• It provides serviceable tools for easily editing these geographic features. Creating new features is similarly dead simple. As these loaded features are now GeoJSON I can also easily hand-edit the GeoJSON itself if required.
• Once done, I can export the data back out as KML
• The provided mapbox vector/satellite layers provide a decent backdrop for geographic reference when drawing new features or editing existing ones.

So here's something that did 80-90% of what I needed from Google Earth without any of the pain and hassle of using Google Earth itself. The remaining percentages can be easily filled out with QGIS for editing beyond simple tracing of polygons and using GeoJSON or SHP as the interchange format (instead of KML) when I need to sling data between geojson.io and QGIS. The only time I needed to touch KML was when I am importing it into geojson.io or exporting it out. As a result, there was no longer such a need for this hypothetical replacement for Google Earth. I had what I wanted.

Now having said that, this wasn't ultimately all for nothing. I did get some fruits from this aborted plan. The mapguide-react-layout project is one (it uses most of the same technologies that I had originally intended to use for this Google Earth replacement). Another project (which I'll reveal in due course) was also born from the initial exploration phase of this idea.

But hey, if you think we could use an Electron/Cesium-based replacement for the Google Earth desktop application, by all means feel free to take this idea and run with it.

^ Seriously, I had known about the existence of this site for ages, but because of the name of the site I always assumed it only dealt in GeoJSON and glossed over the fact that I could import in KML and export back out as KML. This was the game-changing feature for me where one of my job workflows was having to produce KML files for certain geographic areas and the discovery of this feature made me scrap this grand plan. May I suggest a different name? Like ... GeoFiddle? That's what they call all online data playgrounds.

React-ing to the need for a modern MapGuide viewer (Part 8): I figured out how to make an accordion

Previously, I wrote about how the introduction of Blueprint as our UI toolkit gave us a nice solid set of UI components that not only replaces a whole bunch of existing (and disjoined) react components, but also provides all the necessary components to port over all of the remaining Fusion templates across.

Well that statement was half true, we got 3 of the 5 Fusion templates ported across. The remaining templates (Slate and Maroon) required an accordion-like component to dock the Legend/TaskPane/SelectionPanel in its respective sidebar.

Sadly, the Blueprint toolkit did not offer an accordion component out of the box. But based on some encouraging words, I set out to build my own accordion using the Collapse component as the foundation. A few hours later, I got a basic accordion component working, except I ran into that age old problem of how to get an element to fill 100% of its available height.

What this meant was that the "expanded" content needed either a fixed height (not acceptable) or some way where we can pre-calcuate the maximum available height for the "expanded" content when rendering the "expanded" content in the accordion. Fortunately, it turns out that there was a component that can do this legwork for us, it's called react-measure and using it was dead simple: Just wrap it around the component you need the height for and it will automagically calculate it for you.

And with that, we got a functional accordion to house our various components.

And if the look of that accordion looks familiar, you're right. Because now that we have a functional accordion, we can (and have) ported across the remaining Fusion templates. So say hello to the fully ported set of Fusion templates for mapguide-react-layout.

Slate

Maroon

Aqua

TurquoiseYellow

LimeGold

Expect a new release of mapguide-react-layout in the New Year with these Fusion templates and many other goodies in store.

My Oracle XE setup adventure with Docker

NOTE: This blog post was left in draft limbo for several months. Some things in this post may feel disjointed and some things are now obviously being referenced in the past tense. 

DISCLAIMER: This is not an authoritative Oracle installation guide. Neither is this post the opinions of someone with expert knowledge of Oracle. This is just a blogger's tale of getting an installation of Oracle XE up and running with Docker with the minimum of pain and unnecessary administration so he can easily have an Oracle XE instance available for testing with MapGuide/FDO and other geospatial software.

So before I start the release cycle for the first beta release of MapGuide Open Source 3.1, I'd thought I'd try to investigate (and try to knock off) some long standing issues with Oracle with the King FDO provider.

Naturally, this requires I have an installation of Oracle (XE at a minimum) lying around, which I don't. So that means I've to install it. Having had the displeasure of working with Oracle in the past and encountering its horrible, UI-from-the-last-millennium setup process and administrative UIs, I really didn't want to go down that path.

And in YEAR(GETDATE()), I should no longer have to install Oracle to a bare metal OS. I shouldn't even have to do it inside a clean test Virtual Machine. We have docker, and there's bound to be an Oracle docker image on the docker hub, that I can pull down, spin up a container and have an Oracle installation ready to go.

So that's what I set out to do. Since Windows is the first priority platform of focus for any MapGuide/FDO issues, I needed to get docker installed on Windows. Unlike Linux, Docker support on Windows is not first-class (they're working on that ^) and currently requires a virtualization layer (eg. VirtualBox) for docker on windows to interface with. Fortunately, docker provides the Docker Toolbox package which makes setting up Docker on Windows or Mac an easy one click installer affair with everything to get docker up and running on Windows or Mac

• Docker engine
• Docker machine
• Docker compose
• VirtualBox to interface with the barebones Linux VM hosting the docker engine
• Kitematic, for downloading of docker images and spinning up of docker containers through an easy to use GUI

With docker installed, it was now a case of finding an appropriate Oracle docker image on the docker hub, and Kitematic makes this dead simple. So after typing in "oracle xe" a few images were available, I gravitated towards the one with the one with the most likes and downloads.

Clicking the Create button starts the downloading process

Once downloaded, Kitematic automatically spins up a running container for the freshly downloaded docker image. Kitematic has a Web Preview option, which I presume points to the Oracle APEX manager, which I recall from my frail memory was the web-based admin for an Oracle XE installation. I don't know what black magic Kitematic does to auto-magically know this container was web-previewable, but the fact this was available was much appreciated. So clicking the preview button opened up Chrome to what I presumed to be the URL of Oracle APEX.

Well, at this point I got an unexpected http authentication prompt. Okay, so I tried the documented login for this container with no luck. What the ...? Okay, so I cancelled the prompt and tried refreshing the URL and then it suddenly takes me to the Oracle APEX login page. Weird!

So anyways, now that I'm at the login page, I login with the given credentials and was greeted with the visual confirmation that I now had a running Oracle XE installation via docker!

Except, this is not the APEX web admin, it's actually the database home page. Turns out, if I put in this address manually in the web browser (http://192.168.99.100:32768/apex) it will take me to APEX, if I open this through the web preview option in Kitematic, it takes me to the database home page. Weird!

Still at the end of the day, I had a running Oracle XE instance. Much more pleasant than my previous Oracle installation experiences!

Now it's time to bootstrap this Oracle database with some spatial data, so in the interest of eating my own dogfood, I gave FDO Toolbox a spin to see if I can load data into the fresh Oracle XE instance. So first things first, we check the running container in Kitematic for the IP and ports of this container and the ports that are exposed.

Then I proceed to use FDO Toolbox to try and create an Oracle data store via the King Oracle provider

But then ...

No dice, I don't know what happened here. I swore recalling some time ago in the long distant past that it should've been possible to bootstrap a clean Oracle database from scratch with the King Oracle FDO provider and FDO Toolbox, but try as I might, I just couldn't get it to work this time round!

So when FDO failed, it was time to make the hot tag and bring OGR into the fold. Running the venerable ogr2ogr utility brought the SHP file data into oracle flawlessly. Except, it is excruciatingly s l o w to get anything out of this Oracle database! A basic spatial table listing from (eg. A FDO feature class listing) takes an eternity.

It was a this point I paged a resident Oracle expert and realized that one shouldn't try connecting and loading in data as the SYS/SYSTEM user as that it means we bring in EVERYTHING when doing the simplest thing like listing spatially-enabled tables from SDO_GEOM_METADATA

So I fire up SQLPlus and connect to this XE container using the documented admin login

sqlplus system/oracle@//192.168.99.100:32769/xe

I then make a MapGuide oracle user

SQL> create user mapguide identified by mapguide;

Then give it sufficient privileges

SQL> grant CREATE SESSION, ALTER SESSION, CREATE DATABASE LINK, -
>   CREATE MATERIALIZED VIEW, CREATE PROCEDURE, CREATE PUBLIC SYNONYM, -
>   CREATE ROLE, CREATE SEQUENCE, CREATE SYNONYM, CREATE TABLE, -
>   CREATE TRIGGER, CREATE TYPE, CREATE VIEW, UNLIMITED TABLESPACE -
>   to mapguide;

Then I exit SQLPlus and proceed to ogr2ogr my sample SHP file into this user's oracle schema

ogr2ogr -f OCI OCI:mapguide/mapguide@192.168.99.100:32769/xe Parcels.shp

Now when I connect with FDO Toolbox again (with emphasis on specifying my oracle user's schema, otherwise it's going to I presume, hit ALL_SDO_GEOMETRY_METADATA un-filtered, which performance-wise means we'd be back to slow square one!)

It is now connecting and performing at the performance level we are expecting.

And if we can connect to this Oracle XE database with FDO Toolbox, it means we can also connect to it with MapGuide proper, which is a case of setting up a new Feature Source, with the same connection parameters.

And from here, it's the standard MapGuide authoring process of creating layers pointing to this Feature Source, styling them up and composing them together on a map.

So what have I learned from this exercise?

1. Docker is just plain awesome. I didn't want to touch the Oracle installation process and I also didn't want to "taint" my host environment with an Oracle installation either. Docker made spinning up a repeatable and disposable Oracle XE environment dead simple. Even docker on windows, which requires a Virtual Machine intermediate layer is still a seamless experience thanks to the Docker Toolbox suite of integrated tools.
2. Either something regressed or FDO Toolbox will need some work to be able to load data into Oracle using the King.Oracle provider, which is strange as FDO is supposed to be an abstraction layer and FDO Toolbox is just simply working against the abstractions provided by the FDO API. If I have to code in Oracle-specific code paths and behaviour to get this functionality to work, then that is a sign of a leaky abstraction. I hope I don't have to do this!
3. Thankfully where FDO failed in this task, OGR was able to pick up the slack. FDO and OGR have this nice duality where one's strengths covers the other's weakness.
4. Always create a new Oracle user after install and load the spatial data into this user's schema, so you don't have to ask yourself why the default SYS/SYSTEM login/schema takes an eternity querying SDO_GEOM_METADATA tables.
5. Be sure to set that OracleSchema connection property in your FDO connection, even if it's a non-SYSTEM user.

Many thanks to Gordon Luckett, for the various pointers on Oracle.

^ Windows 10 and Server 2016 now have native docker hosts which can run both Linux and Windows containers. No more intermediate virtualization layer required. Being able to containerize windows applications is game changer! Older versions of Windows will still have to go the virtualization route via VirtualBox for the Windows Docker client to work.

React-ing to the need for a modern MapGuide viewer (Part 7): Laying out the blueprints

With the latest release of mapguide-react-layout out of the way, it's time to continue the journey of porting across the remaining fusion templates across.

One of the "problems" we have right now with this new viewer is that although we have got most of the foundational stuff right, the UI lacks stylistic cohesion. This is due to the viewer being a mish-mash of various react components, each with their own unique styling quirks. I really wanted a react-based UI toolkit that had a good enough baseline set of widgets/components with a unified look for building desktop-centric web applications (with an option to go mobile down the road).

Most of these UI toolkits however, take the reverse direction. They are mobile/tablet first, trying to emulate Material Design, iOS or Bootstrap and when trying to adapt such toolkits for a desktop-centric web application design, just look horribly out of place. Most of the UI toolkits that would've passed my desired criteria (like kendo, ExtJS, jQuery UI, etc) suffered from being imperative JavaScript APIs (meaning it would be a painstaking effort to interop with React and/or TypeScript), or some had incompatible/undesirable licenses to boot.

Well, I think my wish for such a toolkit has now been fulfilled, and its name is Blueprint.

What sold me on this particular UI toolkit was:

• It is React-based
• It has a good wide range of components that covers most of what I need to port across the remaining Fusion templates
• It comes with a diverse set of icon fonts
• It has a friendly license (BSD)
• It is written in TypeScript and the library comes bundled with TypeScript definitions
• It looks good!

With so many good selling points, I've decided to adopt Blueprint as the UI foundation for all my viewer templates. This means.

We have better looking modal dialogs (as seen in our updated Aqua template)

We have better styled UI for certain tools

 

And we have key components needed to start bringing across the other remaining Fusion templates, like the TurquoiseYellow template.

Compare the original Fusion template

With our blueprint-powered version

Looks good enough doesn't it?

Now there is one executive decision I'm making with the Fusion templates I'm porting over. The overview map will always be present as a toggle-able button on the main map viewer component and not outside of the map viewport.

Part of the problem is having the OpenLayers OverviewMap control render its content outside of the map viewport doesn't really play nice with React component updates in my attempts thus far, so I've taken the creative decision to not bother trying to get everything 1:1 when porting these templates over. As long as the main elements and styles are there, it's good enough for me.

Now sadly, despite having made several existing libraries and React components redundant (and they have been removed as a result), taking on Blueprint has added a lot of extra weight to our final bundle

Fortunately, this is still significantly under the current Fusion production bundle size, and as we are still in the process of reaching functional parity with our existing AJAX/Fusion viewers, bundle optimization is not a priority at the moment. When that time comes, we can look at things like custom OpenLayers build profiles and moving to Webpack 2 for its tree shaking feature, which should make some in-roads in cutting down our production bundle size to more acceptable levels.