MSBuild gotcha in .net Framework 4.5

So I was originally going to put out the first point release of MapGuide Maestro 5.0 earlier this week, but I hit a snag in the build system that completely baffled me.

Since Google failed to reveal anything of substance on this particular issue, I figured I'd do the right and responsible thing and tell you what this problem is and how I fixed it so it can be indexed by Google so anyone else having this problem can just find this post :)

The problem

To illustrate, this is the set of build artifacts for Maestro 5.0

And here's the set of build artifacts of Maestro trunk

How on earth did the release zip file balloon out by 120MB? Well if we look at the build output directory that we then make our installer/zip files from, we see how it got to this size.

The FDO and CS-Map dictionaries for our Local Connection mode addin have been copied to the root of the output directory in addition to the addin's output directory. Our Maestro VS solution file has 48 projects. Some of these projects have unmanaged dlls and content/data files that we need to also copy across into the correct subdirectories of our main output directory.

Our build system wipes out the Local Connection mode addin directory before zipping (the zip release is our multi-platform release, where Local Connection mode is windows-only so we delete this directory), but because the FDO and CS-Map dictionary directories have been duplicated somewhere they're not supposed to be due to this particular issue, they've been inadvertently included in the zip file as a result

So that's the problem, but why did this happen?

The cause

In a nutshell, something changed with the MSBuild that is installed by .net Framework 4.5.

If you built the Maestro solution in the Visual Studio IDE (2010 or 2012) or with the MSBuild that comes with .net Framework 4.0. We got the expected file/directory structure that our build system then makes the installer/zip files from. But when we build Maestro with MSBuild 4.5, we get this strange file/directory structure.

A full verbose file-logged build with MSBuild 4.0 and MSBuild 4.5 and comparing the log files revealed the culprit.

It was these settings I had in my solution file.

I don't remember why the highlighted projects were ticked, but the highlighted items were indirect dependencies of Maestro.exe that I ticked some time in the past on the belief that building Maestro will then "pull in" these indirect project dependencies and cause them to be built as well. I did this because when I want to debug Maestro from Visual Studio for the first time, I want to make sure the addin projects were being built as well but because Maestro.exe does not reference these projects directly (it loads these addins at runtime), I had to use the project dependency mechanism in the solution settings to make sure these projects were being built when I hit F5 in Visual Studio.

It turns out with MSBuild 4.5, that these settings produced an undesirable (or is it desirable?) side-effect of any files marked Copy to output directory = Always from these projects also being copied over to the output directory of Maestro.exe in addition to their project's respective output directory.

The solution (for me)

The solution for me was to simply un-tick these dependent projects. I've conditioned myself to do a full solution build in Visual Studio anyway before I start debugging so when I hit F5, the addins will have already been built and ready to be loaded by the Maestro.exe I'm debugging, so having these projects ticked was not necessary.

Now if this is not an option. There were other options I explored that also worked for me with varying levels of feasibility

• Use post-build events to copy the relevant content/data files instead of using the Copy to output directory item property. Even if triggered as an indirect dependency project by MSBuild, the post-build events will ensure any content/data files will be copied to their intended location. If you have lots of projects with extra files that need to be copied this may be a painstaking process.
• Replace the Microsoft.Common.tasks file under %SystemRoot%\Microsoft.NET\Framework\v4.0.30319 with the .net Framework 4.0 copy. Because .net Framework 4.5 is an in-place upgrade over 4.0, the 4.0 copy of Microsoft.Common.tasks is replaced. This is not recommended as a viable solution. I just noted this because this worked for me. I guess the proper way to do this would be to use whatever APIs provided by MSBuild to override the various v4.5 MSBuild tasks/targets with their v4.0 equivalents to get the old behaviour. I'm not an MSBuild expert, so I can't really further elaborate on this or confirm what I just said about using MSBuild APIs is actually possible.

As for whether this behaviour in MSBuild 4.5 a bug or a feature, the jury's still out on that one. At first glance I thought it was a bug (because it sure didn't behave like this in MSBuild 3.5 and 4.0), but on second thought it does seem like a feature. If project A has manually marked dependencies B and C in the solution then logically speaking you would expect any files in projects B or C marked Copy to output directory = Always to be copied to project A's output directory in addition to their project's respective output directory.

I guess this depends on what your definition of "output directory" is. Whether it's the output directory of the referenced project or the output directory of the project doing the referencing. Either way for me, my main release roadblock was removed.

Something to keep in mind should you be moving to VS2012/.net Framework 4.5.

In praise of Visual Studio 2012 Express

Visual Studio 2012 gets a lot of flak for its questionable color scheme, the shouty all caps menus and Microsoft's initial decision to snub desktop-centric developers like me with their initial set of Express editions.

But this one single button makes it all worth it.

I don't think I can ever go back to 2008/2010. That one single button makes sifting through a 48-project solution (which Maestro currently is) an absolute breeze! Navigating through 48 projects worth of source code in previous versions of Visual Studio was a complete chore. In VS 2012 it's dead simple.

As an aside, the Windows Desktop Express edition of VS 2012 is quite an impressive piece of free (as in beer) software. I've been hacking on Maestro trunk with VS 2012 Express ever since it was released and I've had some of the most productive coding sessions with it.

I was expecting a bare-bones front-end to csc/vbc/cl.exe like previous Express editions with all sorts of SKU-related hurdles to overcome, but this is something out of the ordinary:

• Support for heterogeneous solutions with assorted project types
• .net Framework targeting
• Support for solution folders
• Integrated NuGet package manager
• Attach to process debugging
• Basic static code analysis
• Project/Solution roundtripping with VS2010*
• 64-bit C++ compilers included

This was stuff I was expecting from a Professional level SKU in previous versions of Visual Studio! I don't really care about MSTest (I use NUnit/xUnit externally), nor do I care about TFS integration (I'm use svn/git/mercurial externally). Actually I don't really care much about whatever integration Microsoft is marketing for their higher level SKUs as I am happy to get at such functionality through external tools. Refactoring support is very basic, but still workable (ReSharper is nice, but I don't need it like a crutch in order to crank out good code). 

So what Microsoft has actually offered in their 2012 Express editions actually meets most of my needs, which is especially surprising in light of their initial snub to desktop application developers (you know ... the people who make their platform). In their efforts to appease the many pissed off developers (like me) they punched well above their weight and did much more than I would've personally expected them to do.

So kudos to Microsoft for delivering a fine set of free developer tools. 

But if you think this is an olive branch manoeuvre that will get me to code for Windows 8? Dream on! :P

* Found out a little hitch in the round-tripping  The round-tripping breaks if you add a new project from VS 2012, as this upgrades the solution file to VS 2012 format. But the actual changes are minute and can be safely reverted (just need to revert the header portion of the solution file) to allow that solution file to be opened back in VS 2010.

Expanded .net development options for MapGuide Open Source 2.4

If you're a .net developer, you'll probably be interested in the expanded array of .net development options for MapGuide.

For the final release of MapGuide Open Source 2.4, we've taken the NuGet packaging work that we've done for mg-desktop and extended it to also include the official MapGuide API. As a result, we had to make some modifications to the existing nuget packages to accommodate the official MapGuide API, and to stay under the 30mb package limit in the NuGet gallery.

There are now 5 different nuget packages (in x86 and x64 flavors, suffixed by -x86 and -x64 respectively):

• mapguide-api-base
• mapguide-api-web
• mg-desktop-net40
• mg-desktop-viewer-net40
• cs-map-dictionaries (this package is CPU-agnostic and is not suffixed)

Whose dependency chain is like so

So based on the type of application you're trying to build, you have the following package configurations.

Building a normal MapGuide .net web application? Install the mapguide-api-web package, that will automatically install the mapguide-api-base pre-requisite. The full set of files in these two packages is the same set of files under the mapviewernet/bin directory that you've always been asked to copy over to your .net application in the past.

Building a desktop-based MapGuide application? Install the mg-desktop-net40 package, that will automatically install the mapguide-api-base pre-requisite. You will also need to have the CS-Map coordinate system dictionaries on hand in order to be able to use any of the MgCoordinateSystem classes in the MapGuide API, or you can install the optional cs-map-dictionaries package that contains a subset of the coordinate system dictionary files.

Building a desktop-based MapGuide windows application? Install the mg-desktop-viewer-net40 package, which will automatically install any upstream pre-requisites. Again, install the cs-map-dictionaries package if you require coordinate system dictionaries.

Building an application that only uses the shared MapGuide components (eg. MgCoordinateSystem)? Install the mapguide-api-base package, and optionally install the cs-map-dictionaries package.

The main benefit of the NuGet approach, is that you no longer fall to the rookie mistake of forgetting to copy over the unmanaged dlls to your application's output directory, as all these nuget packages will insert the appropriate post-build events into your project files to automatically do this for you!

Previously, setting up a skeleton MapGuide .net web application would've been something like this:

1. Create new project in Visual Studio 2010 or 2012
2. Copy over all files from mapviewernet/bin into a staging directory in your project
3. Reference the OSGeo.MapGuide assemblies from the staging directory
4. Set up a post-build event to copy over all unmanaged dlls from the staging directory to your application's output directory
5. Add a call to MapGuideApi.MgInitializeWebTier with the path to the webconfig.ini in the startup routine of your web application.

With the nuget version, this is now:

1. Create new project in Visual Studio 2010 or 2012
2. Install-Package mapguide-api-web-x86 or Install-Package mapguide-api-web-x64
3. Add a call to MapGuideApi.MgInitializeWebTier with the path to the webconfig.ini in the startup routine of your web application.

3 error prone steps (and probably the cause of most .net newbie questions) have been eliminated by nuget.

These nuget packages also include the relevant intellisense files to hopefully keep you away from the main API reference as much as possible :)

Now although NuGet greatly simplifies things, there may be some un-tested scenarios (eg. Continuous Integration) where this type of setup may not work perfectly. Another thing to note is that the .net assemblies in these NuGet packages are all signed whereas the equivalent assemblies that come with MapGuide Open Source 2.4 are not. If you cannot use signed MapGuide assemblies (I can't think of a reason why not), then nuget probably isn't the choice for you and you are better off sticking with the old fashioned way.

As I've mentioned when I first announced the nuget support. I'm still learning this nuget stuff. If I've done something wrong, or something could be done better. Do let me know.

MapGuide-related packages on nuget.org

WPF + Geospatial Platform API = A recipe for failure

If you use WPF in your AutoCAD Map applications and you are using the Geospatial Platform API you may encounter something like this when designing any WPF content:

The type initializer for '<Module>' threw an exception.
at System.Reflection.CustomAttribute._CreateCaObject(Void* pModule, Void* pCtor, Byte** ppBlob, Byte* pEndBlob, Int32* pcNamedArgs)
at System.Reflection.CustomAttribute.CreateCaObject(Module module, RuntimeMethodHandle ctor, IntPtr& blob, IntPtr blobEnd, Int32& namedArgs)
at System.Reflection.CustomAttribute.GetCustomAttributes(Module decoratedModule, Int32 decoratedMetadataToken, Int32 pcaCount, RuntimeType attributeFilterType, Boolean mustBeInheritable, IList derivedAttributes)
at System.Reflection.CustomAttribute.GetCustomAttributes(Assembly assembly, RuntimeType caType)
at System.Reflection.Assembly.GetCustomAttributes(Boolean inherit)
at MS.Internal.ReferenceAssemblyUtils.SafeGetCustomAttributes(Assembly assembly, Type filter, Boolean inherit)
at MS.Internal.Xaml.ReflectionProjectNode.BuildNamespaces()
at MS.Internal.Xaml.ReflectionProjectNode.Find(Identifier namespaceUri)
at MS.Internal.Xaml.PrefixScope.FindType(XamlName name)
at MS.Internal.Xaml.XmlElement.FindElementType(PrefixScope parentScope, IParseContext context)
at MS.Internal.DocumentTrees.Markup.XamlSourceDocument.get_RootType()
at Microsoft.Windows.Design.Documents.Trees.MarkupDocumentTreeManager.get_RootType()
at Microsoft.Windows.Design.Documents.MarkupDocumentManager.CalculateLoadErrorState()
at Microsoft.Windows.Design.Documents.MarkupDocumentManager.get_LoadState()
at MS.Internal.Host.PersistenceSubsystem.Load()
at MS.Internal.Host.Designer.Load()
at MS.Internal.Designer.VSDesigner.Load()
at MS.Internal.Designer.VSIsolatedDesigner.VSIsolatedView.Load()
at MS.Internal.Designer.VSIsolatedDesigner.VSIsolatedDesignerFactory.Load(IsolatedView view)
at MS.Internal.Host.Isolation.IsolatedDesigner.BootstrapProxy.LoadDesigner(IsolatedDesignerFactory factory, IsolatedView view)
at MS.Internal.Host.Isolation.IsolatedDesigner.BootstrapProxy.LoadDesigner(IsolatedDesignerFactory factory, IsolatedView view)
at MS.Internal.Host.Isolation.IsolatedDesigner.Load()
at MS.Internal.Designer.DesignerPane.LoadDesignerView()


A nested exception occurred after the primary exception that caused the C++ module to fail to load.

at <CrtImplementationDetails>.ThrowNestedModuleLoadException(Exception innerException, Exception nestedException)
at <CrtImplementationDetails>.ThrowNestedModuleLoadException(Exception , Exception )
at <CrtImplementationDetails>.LanguageSupport.Cleanup(LanguageSupport* , Exception innerException)
at <CrtImplementationDetails>.LanguageSupport.Initialize(LanguageSupport* )
at .cctor()


Type constructor threw an exception.
at System.Runtime.InteropServices.Marshal.ThrowExceptionForHRInternal(Int32 errorCode, IntPtr errorInfo)
at System.Runtime.InteropServices.Marshal.ThrowExceptionForHR(Int32 errorCode)
at <CrtImplementationDetails>.DoCallBackInDefaultDomain(IntPtr function, Void* cookie)
at <CrtImplementationDetails>.DoCallBackInDefaultDomain(IntPtr , Void* )
at <CrtImplementationDetails>.LanguageSupport.InitializeDefaultAppDomain(LanguageSupport* )
at <CrtImplementationDetails>.LanguageSupport._Initialize(LanguageSupport* )
at <CrtImplementationDetails>.LanguageSupport.Initialize(LanguageSupport* )

Though you can still build and run the project, the WPF designer is completely hosed, all intellisense is lost and as a result, the WPF designer becomes a bloated version of notepad.

The reason that happens is because the Autodesk.Map.Platform.dll is actually just a thin managed wrapper to the unmanaged AcMap* dlls in your AutoCAD Map installation, so the WPF designer is actually trying to locate these dlls from either:

• The project's output directory
• Or, some special working directory that's defined by Visual Studio (where this is, I have no idea)

After 5 hours of sleuthing around, I have found an acceptable workaround.

Simply put the AutoCAD Map installation directory into your PATH environment variable and restart Visual Studio. The WPF designer will correctly locate the AcMap* dlls and you'll regain full designer functionality.

This little tip will no doubt be useful to someone out there. I lost 5 work hours so you don't have to :-)

No more short coffee breaks in VS2010

As some of you may know, when you invoke the Add References dialog in Visual Studio, you might as well go for a coffee break, because the .NET and COM references take f o r e v e r to load. It is especially annoying when all you want to do is add a reference to another project from the same solution!

Well in Visual Studio 2010 (Beta 2), this is no longer the case. Invoking the Add References dialog is super fast! The reasons being:

1) .NET and COM references are now loaded asynchronously in the background.

2) The default tab is the Projects tab (which is the one I go for most of the time)

Truth be told, I would happily upgrade to VS2010 just on this one feature. But I'd probably lose out on the coffee breaks :-)

Simple FDO development tip.

When you develop any application in .net in Visual Studio using the FDO API you would normally reference the following dlls from your SDK bin directory.

OSGeo.FDO.dll

OSGeo.FDO.Common.dll

OSGeo.FDO.Geometry.dll

OSGeo.FDO.Spatial.dll

However, when debugging/running the application, you need to have *all* the files from the SDK bin directory in the application's output directory, otherwise things will break.

You could create a post-build event to copy these required files, or alternatively you could do this:

Right click your project and choose Add - Existing Item

Browse to your directory that contains your FDO dlls, select everything and choose Add As Link

Create a "com" directory in your project, and repeat this process for the .sql files in the "com" directory.

Now these linked files are in your project. They should have a shortcut overlay to indicate they are linked files. Select all these linked files, and set the Copy to Output Directory property to Copy Always

Now everytime you build your project (for debugging or deployment), your application and everything in your FDO bin directory are copied to the output directory. No post-build hackery required!