Thursday 25 September 2014

Docker-izing MapGuide

There was a motivation behind this public service announcement.

It was part of my investigations into being able to deploy MapGuide as a Docker container.

What is Docker you may ask? Have a read of their introduction page.

TL;DR? Docker is a tool that allows you to run and deploy software inside virtualized software containers. This is not a heavyweight VM like VirtualBox and their ilk. This is something that leverages existing features in the Linux Kernel to provide "lightweight" VMs with very low overhead.

A docker-ized application gives us various benefits:
  • Your application and its dependencies are all self-contained and will not interfere with anything outside of its container and vice versa.
  • Deployment is dead simple. No figuring out what pre-requisites that have to be installed. They'll all be part of the docker image that you can pull down with a single command.
  • Your application can run in host environments that the application was not originally compiled/tested for. Build once, run anywhere (where docker is installed :))
There's many more benefits than what's listed here. So you can probably see where a docker-ized MapGuide would really be useful from a developer and administrator perspective.

But there is a catch to all this goodness. Firstly you need a Linux distribution that uses Linux Kernel v3.8 or newer as that contains the required OS virtualization features needed for Docker to work.

Secondly, you need to run a 64-bit Linux distribution as that is what Docker only supports. In order to run MapGuide within a Docker container, we need a way to run a 32-bit MapGuide within a 64-bit Linux environment as Docker can only be run from within a 64-bit Linux host and as previously mentioned, we still don't have a functional 64-bit Linux build of MapGuide yet :( So the workaround is to install the required 32-bit packages, which you can find in my previous post.

So after applying all of this newly acquired knowledge, allow me to introduce my first Docker image for MapGuide.

This is a CentOS 6 base image that has the following software and packages pre-installed:
From this base image, you can build your own docker image that installs your own MapGuide data, applications and configurations and deploy/run that image as a docker container.

Here's a basic example of getting a docker-ized MapGuide up and running with mapguide-rest and some sample data packages using 64-bit Ubuntu 14.04 using the above docker base image:

Firstly, we install the docker package like so

sudo apt-get install docker.io

Then make a new directory and create a file named Dockerfile


Put the following text into the Dockerfile. The comments should be self-explanatory. 


# This image is based from the MapGuide docker base image
FROM jumpinjackie/mapguide-base
# Install additional packages, load your MapGuide applications and data, etc.
# Download Sheboygan dataset
RUN wget -P /usr/local/mapguideopensource-2.6.0/server/Packages http://download.osgeo.org/mapguide/releases/2.6.0/Release/Sheboygan.mgp
# Download Melbourne dataset
RUN wget -P /usr/local/mapguideopensource-2.6.0/server/Packages https://github.com/jumpinjackie/mapguide-sample-melbourne/releases/download/v0.2/Melbourne.mgp
# Download mapguide-rest 0.10
RUN wget https://github.com/jumpinjackie/mapguide-rest/releases/download/v0.10-pre/mapguide-rest-0.10.zip
# Set up install location for mapguide-rest
RUN mkdir -p /usr/local/mapguideopensource-2.6.0/webserverextensions/www/rest
# Extract mapguide-rest
RUN unzip mapguide-rest-0.10.zip -d /usr/local/mapguideopensource-2.6.0/webserverextensions/www/rest
# Fix up permissions of the cache directory (for tiles and smarty templates)
RUN chown daemon:daemon /usr/local/mapguideopensource-2.6.0/webserverextensions/www/rest/cache
# Expose the web server port to the world outside the container. The default port is 8008
EXPOSE 8008
# Run supervisor, that will start the MapGuide Server and Apache httpd server
CMD ["/usr/bin/supervisord"]


Now save the Dockerfile, and run the following command to build the docker image from that file

sudo docker build -t my-mapguide-app .

This command will download the mapguide-base docker image which will take a few moments depending on your download speed. This is only downloaded once and will remain until you explicitly remove this base image.

my-mapguide-app will be the name of the docker image, which you'll need to reference when you will run a container from it, which is what we will do next. But before we do that, we should list our docker images and see if our new image is there.

sudo docker images



Now that we have confirmation that our image has been created, we can create a container from it like so.

sudo docker -d --name mapguide -t my-mapguide-app

The -d switch indicates that this container will run in the background. The --name switch assigns the name mapguide to this container so we don't have to remember a long UUID for referencing this container in future operations, which is what is outputted when the command succeeds.


Finally the -t switch indicates that we want to create a container from the my-mapguide-app image that we just created.

Now that we have started a container, we now have:

  • MapGuide Server running
  • The Sheboygan and Melbourne sample data packages downloaded to the Packages directory of the MapGuide Server installation directory
  • The mapguide-rest extension installed
  • Apache HTTP Server running on port 8008 which is exposed
We now just need to know what IP address has been assigned to this container. To do that, we can run the following command

sudo docker inspect mapguide | grep IPAddress


Now that we have an IP address, we can see if things are in order by firing up the Site Administrator


If we login and go to the Manage Packages page, we can see the packages we've downloaded as part of building our my-mapguide-app Docker image are there.


Now load these packages, and go to the mapguide-rest sample apps landing page just to see that the extension was installed.


If the samples on this landing page work, you have just verified that your docker-ized MapGuide container is now fully operational.

So that's a little run-through of how to get MapGuide running in a Docker container. Now the other thing about Docker that is really awesome is that the big players in Cloud Computing already (or starting to) support Docker containers as a PaaS deployment option.

Can you say MapGuide on the cloud?


Now just to clarify, MapGuide on the cloud via IaaS is already relatively easy. I, the AWS noob easily got a MapGuide demo site on Amazon EC2 set up over the GovHack weekend, which stayed up long enough to impress the judges to win some awards.

However the IaaS approach to deployment puts a burden on you to maintain the actual infrastructure (VMs, etc). As a case in point, our GovHack demo site has been taken down because I butchered my Amazon EC2 instance due to a combination of failure to act on an important email alert on this issue from Amazon and my general AWS noob-ery.

The PaaS approach lets you focus solely on deploying the applications, without having to worry about the underlying infrastructure. MapGuide as a Docker container enables the possibility of using this cloud deployment option.

Hopefully this post has sold you on the power and potential of Docker and how having MapGuide as a docker container may open up deployment and development scenarios that were previously not possible or took a lot of manual effort.

As for that Docker base image I linked to, that is just the tip of the iceberg. I'm just getting my feet wet with Docker and foresee many future blog posts on this topic as I learn more about what Docker can and can't do.

I can see the possibility of having different base images in the future:
  • Individual Server and Web Tier Docker images. Imagine how easy it would be to set up a load-balanced cluster via Docker containers? I'd have to grok how these containers communicate first, but the possibilities are very interesting.
  • Web Tier images tailored for the PHP or Java installation profiles.
  • Many others?
I also wouldn't comfortably say that the above docker image is currently production-ready. There's questions that need to be answered:
  • How do we handle the case of a mgserver or httpd process falling over on a background docker container?
  • How can we easily access and manage log files in a docker container?
  • How can we perform repository backup/restoration operations within a docker container?
  • How do we perform $MAPGUIDE_SERVER_ADMIN_TASK within the docker container?
If you have an idea on how these questions can be answered, I invite you to help improve the design of this docker image or to enlighten me on the comments below.

MapGuide tidbits: Strange fonts on Linux

Does your MapGuide Linux installation render labels that look like this?


Instead of this?


This is due to MapGuide not being able to correctly locate the required font in question (Verdana). To fix this, you need to install the appropriate font package that provides the missing font. In the case of verdana, that package is msttcorefonts.

On Ubuntu: Simply install the msttcorefonts package

On CentOS: Install the following packages:

  • curl
  • xorg-x11-font-utils
  • fontconfig
And then install the following RPM packages with the rpm command
  • rpm -i http://pkgs.repoforge.org/cabextract/cabextract-1.4-1.el6.rf.i686.rpm
  • rpm -i https://downloads.sourceforge.net/project/mscorefonts2/rpms/msttcore-fonts-installer-2.6-1.noarch.rpm
Now you can verify your font in question is install by running the fc-list command and looking for the name of your font in the output

fc-list | grep Verdana

Now even after this, MapGuide may still not properly locate this font even if it is installed.

If this is still the case, copy the physical font files to the directory of the mgserver executable (eg. /usr/local/mapguideopensource-2.6.0/server/bin) and restart the MapGuide Server.

In the case of msttcorefonts, the physical font files are found in:
  • On Ubuntu: /usr/share/fonts/truetype/msttcorefonts
  • On CentOS: /usr/share/fonts/msttcore

Tuesday 23 September 2014

MapGuide tidbits: Running 32-bit MapGuide on 64-bit Linux

We still don't have our mythical 64-bit build of MapGuide on Linux yet. So in the meantime, should you want to run the 32-bit CentOS or Ubuntu builds of MapGuide in their respective 64-bit versions, here's the packages you will need to have installed beforehand.

On 64-bit Ubuntu: Just install the ia32-libs package

On 64-bit CentOS: You will need to install the following packages:
  • glibc.i686 
  • libstdc++.i686 
  • expat.i686 
  • libcurl.i686 
  • pcre.i686 
  • libxslt.i686 
  • libpng.i686
This will satisfy the dependencies required by your 32-bit MapGuide, its bundled Apache HTTPD Server and PHP.

The bundled Tomcat and Java wrapper API has not been tested under this environment at the time this post was published, so to take a logical shot in the dark, you probably need to just install the respective 32-bit JVM package for the Tomcat and Java wrapper API to work. If it turns out I'm wrong about that, please do correct me in the comments below.

Thursday 18 September 2014

350 posts

This train ain't stopping yet!


Tuesday 16 September 2014

Announcing: mapguide-rest 0.10

Nope, this isn't a 1.0 release. Not only are we not using decimal release numbers, but there's still plenty of things to explore and refine before we can put the 1.0 stamp on this thing. Here's what's new and changed in this release.

(Experimental) Cesium CZML support

We now have support for outputting feature data as CZML for consumption inside the Cesium 3D web viewer. Support for CZML is made available as a representation of a given Layer Definition.

For example, the following route will return data from the trees layer as CZML:

http://localhost/mapguide/rest/library/Samples/Sheboygan/Layers/Trees.LayerDefinition/features.czml

We've included a CZML example in this release that demonstrates the level of support that has been implemented for this release.


When selecting an object, its tooltip data will be shown in Cesium's information window if available.


Note the object itself has no selection indicator unless the object is a point. Still trying to figure out if we can apply a different style for selected lines and polygons.

If your Layer Definition has elevation settings applied, they be extruded if they're polygons


Now if the (experimental) tag didn't stand out, here's the current limitations with this implementation:
  • The following properties are preserved when converted to CZML, anything not listed can be assumed lost in translation:
    • Point styles: Point color is preserved. The point size in CZML is the average of the height/width of the point as defined in the Layer Definition.
    • Line styles: Line color
    • Area/Polygon styles: Fill color. Outline color
  • If a Layer Definition has multiple scale ranges defined, mapguide-rest will only consider the first scale range when outputting CZML
  • If a given point/area/line style is themed, the default rule (the one without a filter) is ignored
Data publishing improvements

The restcfg.json now lets you specify a Layer Definition as the data source instead of a Feature Source and Feature Class name.

Also, when using a Layer Definition as a data source, you'll get tooltip, hyperlink and elevation FDO expression pre-evaluated, allowing you to use such computed properties within your templates.

You can find a new example that uses the building footprints from the Melbourne dataset.

File download support

Most GET routes can now prompt for downloads by appending download=1 to the query string of the URL.

XYZ tile improvements

Vector tiles can now be for a base layer group or for a layer within the base layer group.

Just to recap, this URL route fetches a vector tile for a given base layer group in a Map Definition

http://servername/mapguide/rest/library/{resourcePath}.MapDefinition/xyz/{groupName}/{z}/{x}/{y}/tile.{format}

If you want vector tiles for a specific layer within that group, you can use this new URL route

http://servername/mapguide/rest/library/{resourcePath}.MapDefinition/xyz/{groupName}/{layerName}/{z}/{x}/{y}/tile.{format}

You can find a new Leaflet example that uses the single-layer vector tiles.


Other changes

  • Fixed bad download links in the resource data list HTML representation
  • Fixed invalid chunked file transfer behavior under certain conditions
  • Samples updated to use OpenLayers 3 final and Cesium 1.1

Saturday 13 September 2014

300,000 views!

In the race between 300,000 page views and 350 blog posts, the page views crossed the finish line first. For the record, this is my 348th post.

Thank you all for your continued viewership!



Wednesday 10 September 2014

Make that 3/3

Thanks to this thread, I was finally able to crack the missing piece of the CZML output puzzle for mapguide-rest, and we can now output point, line and polygon features as CZML to Cesium.


CZML is a representation of a Layer Definition in mapguide-rest, this means we will get pre-evaluated properties to play with such as:

We currently use this information as follows

Tooltips get written as the description property of each CZML packet, allowing such information to be displayed in the Cesium information window when the object is selected.


Now as the above screenshot shows, there's no real visual way to know what object you selected. I'm still figuring out if there's a way in CZML or the Cesium APIs to specify how a selected object looks so it can stand out.

If the Layer Definition has elevation/extrusion settings applied, that's when the fun stuff happens. We apply the extruded value in the CZML packet for each feature, giving us 2.5D features. This extrusion only applies for polygon features at the moment.


Oh, and did you know Cesium works pretty well on any WebGL capable mobile browser?


Pretty cool stuff. Now that we've nailed down the fundamentals, it's time to figure out how much visual fidelity we can preserve from MapGuide to CZML:

  • Can we transfer thematics?
  • Can we transfer labels?
  • Can we transfer patterns?
  • Others?
Fun times ahead!

Friday 5 September 2014

2/3

After lots of trial and error, I am finally able to output features from MapGuide as CZML to Cesium via mapguide-rest


However, only 2 out of the 3 geometry types are working (points and polygons). I'm still trying to figure out:
  • How to properly output CZML for line features
  • How best to apply Z extrusion where it is defined in the Layer Definition
  • What parts of a Layer Definition are transferable/translatable to CZML? Doesn't have to be 1:1. KML-level visual/information fidelity would suffice here.
If only there were more comprehensive CZML examples I could refer to that would make implementing this stuff much easier! All the examples I could find try to demonstrate every bell and whistle when all I want to know is where do I stick the lat/lon/elevation coordinates from my source geometries? Their CZML documentation could do with some improvement, a single-page document is terribly hard to navigate and without reference examples it's amazing I was even able to get this far!

But in the end, all this pain and struggle will be worth it because Cesium is just plain awesome! I truly believe that Cesium will be to 3D maps what OpenLayers is to 2D maps: A powerful web-based map viewing platform that is rich in features and support for many different vector and raster data sources.