How we make our builds of MapGuide

For this post, I'm going to shed a little insight (for those interested) into how we produce our builds of MapGuide Open Source for Windows and Linux.

Windows

We use Jenkins to coordinate our Windows builds of MapGuide. Jenkins is a Java-based continuous integration tool, though in our case we generally emphasise the continuous aspect of CI more than the integration aspect due to the nature of C++ code. Jenkins was chosen because of its ease of use and its massive array of available plugins that enhance the functionality of Jenkins. In addition to MapGuide and FDO, Maestro is also built with Jenkins.

MapGuide and FDO are built with completely free tools. For 2.5 and older releases, we used Visual C++ 2010 Express edition and some installation gymnastics with the Windows 7.1 SDK to get access to the 64-bit Visual C++ compiler. Thanks to the migration to Visual Studio 2012 as the de-facto windows compiler for MapGuide and FDO, building MapGuide/FDO with free tools is now a totally painless experience as the Windows Desktop SKU of VS2012 Express satisfies all our requirements, including both x86 and x64 C++ compilers needed for us to produce 32-bit and 64-bit builds.

Sadly, since the majority of the code we're building is C++, that does not lend itself to fast iteration cycles where build times are generally expected to be several minutes. Our build times for MapGuide and FDO are several hours which generally hampers the quick feedback cycles that are expected of normal CI processes.

So our MapGuide and FDO build jobs are manually triggered instead of triggered off of polling a remote SCM repository. Maestro however, builds a lot faster (being .net code) so it is configured to trigger off of new commits meaning we can get fresh builds of Maestro several minutes after a svn commit.

So when it comes to put out a new build of MapGuide, we simply fire up Jenkins, launch the FDO build jobs (that will trigger the downstream MapGuide build job when completed), and check back in several hours later. If a build fails, we pore through the various log files to find out why and see how it can be rectified. This process is not perfect, but it's good enough given what we have to work with.

Linux

For 2.5 and older releases, producing Linux builds of MapGuide was a somewhat manual process of making a CentOS or Ubuntu VM in VirtualBox, setting up any required tools and libraries and running a set of build scripts to compile MapGuide and FDO for that particular distro. This is why the Linux builds always lagged several days behind the Windows builds.

For 2.6, I wanted to vastly streamline this process. Like the Windows build process, the Linux build process should be something we can trigger with a press of a button or running a script and we can check back several hours later to collect the various build artifacts for upload or additional testing.

To this end, I caught wind of this wonderful tool called vagrant, which made the process of setting up and configuring VMs (a process they call provisioning) much simpler. Some leg work was required to set up the CentOS and Ubuntu base boxes for vagrant but once that was done, producing a CentOS or Ubuntu build of MapGuide was a simple "vagrant up" away.

The use of vagrant means VMs are now truly disposable entities. Our Linux VMs are created as part of the build process and are disposed of after building MapGuide to ensure a clean slate VM is spun up every time for subsequent builds. This means MapGuide/FDO will always be built from a clean environment and means we won't encounter problems due to tainted build environments.

Like the Windows build process, this one is not perfect. Some of the MapGuide unit tests involving threads have a very high chance of locking up the shell provisioner (I don't know why this happens), so these tests have to be disabled and run elsewhere in a manually supervised VM.

Also editing Vagrantfiles requires careful review and scrutiny as a typo or syntax error in the shell script code in a Vagrantfile means several hours lost as you wonder and investigate why "vagrant up" didn't produce your set of build artifacts after leaving it for several hours.

Still, the process as a whole is a lot better than what we had before which is what ultimately counts.

In closing

If you're interested in setting up your own CI environment for MapGuide, you can find the configurations I've used:

• For Jenkins here
• For Vagrant here

Now when I'm ready to put out a build of MapGuide, it's just a case of:

• Triggering the FDO build jobs in Jenkins
• Running "vagrant up" on my CentOS and Ubuntu Vagrantfiles

I don't think it can get much simpler than that!