MapGuide tidbits: Log monitoring

When I talked about my MapGuide dev tools of choice a while back, I mentioned how valuable BareTail is for viewing the MapGuide log files as they get written.

Now if for some reason, you cannot use or install BareTail on the MapGuide Server box, the MapGuide Site Administrator comes with a built-in log file viewer.

Firstly, log into your MapGuide Site Administrator. Then click the Manage Logs link in the sidebar

In the log management page, click the log you want to view

This will spawn a new popup window containing the contents of the log file

You can configure this window to auto-refresh at a certain interval, which will approximate the functionality of BareTail. Since log files can grow really large, it's best to set the viewer to only view that last n records from the log file, which you can also specify.

MapGuide tidbits: FDO RDBMS provider trace logging

From the "I wished they told us about this in the documentation" department:

Ever wanted to see the SQL that your RDBMS-based FDO provider is executing behind the scenes?

Any provider built on top of the GenericRdbms core supports trace logging of SQL statements. For reference, the providers are:

• OSGeo.ODBC
• OSGeo.MySQL
• OSGeo.SQLServerSpatial
• OSGeo.PostgreSQL

Here's how you enable such tracing.

1. From the command-line, define a FDO_TRACE_FILE environment variable that points to a file where you want such statements logged. (eg. SET FDO_TRACE_FILE=C:\temp\fdo_trace.log)
2. Run your FDO client application from the same command line. Obviously for MapGuide, this means you have to run it interactively (ie. mgserver.exe run) instead of a service, which is the same thing we do if we want to inspect WMS/WFS traffic through fiddler.

Now when you interact with any Feature Source using any one of the above providers, you can see the full gory SQL that the provider is generating in the trace log file

For commerical providers like the Autodesk.Oracle provider, the environment variable is GVC_TRACE_FILE.

Finally! (part 2)

9 months ago, we finally brought MapGuide Open Source back into the Linux fray.

One thing I wanted to do before kicking the MapGuide Open Source 2.5 release cycle into high gear was to try and answer the other important question:

We can haz 64-bit MapGuide for Linux?

After a weekend of hacking around the current build system, we have the final answer:

This build is not perfect yet, FDO was a semi-dirty build and MapGuide was building against a minimal installation of FDO (core + few providers) as a result. Also I had to do some serious monkey-patching of the DWF Toolkit (and backport some bits from the latest 7.7 release) to support 64-bit compilation.

The true solution would be to fully replace the internal 7.1 copy of the DWF Toolkit with the latest 7.7 release, but that would also mean the breakage of eMap functionality and associated viewer and mapping service APIs in MapGuide. Sure this stuff is deprecated already, but such a change is too destructive at this point. So we're better off shelving such work for a later release.

Now to nuke the VM and start again because we didn't want to get to this point through blind luck. We want to get to this point because our build system has the necessary fixes in place!

MapGuide Open Source 2.5: What's new

So for those who downloaded and installed the preview release, here's what's new in MapGuide Open Source 2.5

FDO 3.8

MapGuide Open Source 2.5 uses FDO 3.8 which contains some key improvements and fixes:

• A fixed SQL Server FDO provider that no longer flips coordinates and will work properly when doing CRUD operations through FDO.
• The GDAL raster provider now supports raster re-sampling.
• The ODBC provider now supports proper geometric intersection testing and not BBOX approximation.

New Google Street View Fusion widget

The Fusion that comes with MapGuide Open Source 2.5 now includes a new Google Street View widget, allowing Google Street View to be used within a Fusion Flexible Layout

Using the latest beta of Maestro, here's how you can use this widget in your Flexible Layouts.

Firstly, create or open a Flexible Layout. Set up your Map Definition (which must be in WGS84.PseudoMercator), add one or more Google commercial layers (this is a TOS requirement for using the Google Street View service)

Then in the widgets section of the Fusion editor, click the widget management button (indicated by the cog icon)

Then in the widget management dialog, click Add to add a new widget to the widget set

Pick the new GoogleStreetViewer widget type and give it a name

Optionally, you can then edit the XML of this widget to change any widget parameters

Close the dialog. Then in the widgets section of the Fusion editor select a root container from the tree view and then add a new widget.

Pick the widget that we just registered and it will be added to your selected root container

Now if you preview the layout, you'll see the new Google Street View widget

Click it, and you'll see a camera icon indicating the current street view position, with the actual street view viewport situated in the Task Pane.

Now as you can see from the warning bar, the street view camera currently sees nothing because most likely it is not situated over a road or street, so you can drag and rotate the camera icon on to an actual road to then be able to see the street view from the Task Pane

To ensure TOS compliance, this widget will only work in the presence of an active Google base layer. Turning off the active Google layer (or switching to a non-Google base layer), will throw back an alert like this:

Make sure you understand the Google Maps TOS before using this widget. With great power comes great (legal) responsibility!

QuickPlot PDF output

The QuickPlot widget in Fusion now outputs to PDF for better printing control

Note that QuickPlot for the AJAX viewer is still image-based, and you will have to a manual customization job to backport such functionality to the AJAX viewer.

AJAX Viewer improvements

That's not to say that the AJAX Viewer has been neglected for this release. Thanks to contributions by Bruno Scott, the AJAX Viewer has some new features as well.

The legend now has a context menu option to toggle legend visibility of all layers

The layers that are not currently visible at the current scale will be greyed out

For tiled maps, the scale field on the status bar converts to a convenient drop down of the map's finite scales allowing for easy zooming

Streamed HTTP feature/data/SQL query results

If you've ever used the SELECTFEATURES, SELECTAGGREGATES or EXECUTESQLQUERY mapagent operations (if you ever previewed data using the Local Feature Source Preview in Maestro, it uses these operations), you might notice that for large query results, the memory usage is absolutely woeful. Just to give an example, a full query result of the Sheboygan parcels in 2.4 or older served over HTTP consumes the following amount of memory during the request:

• XML: ~50MB
• JSON: ~1.5GB (!!!)

This is because the XML of the full 17k parcel features is internally buffered up-front before being written out to the HTTP response. The JSON result is even worse because we not only maintain a full internal XML string buffer, but also an internal DOM for the purposes of converting to JSON.

And this is just one request! Obviously, MapGuide 2.4 or older is not suitable as a platform for serving data with this woefully inefficient query response processing.

So we've fixed that for this release. Through a combination of HTTP chunked encoding and outputting the response as we iterate through the query results, the same query in 2.5 now consumes the following amount of memory during the request:

• XML: ~500KB
• JSON: ~2MB

Which for those watching at home is 100 times improved memory efficiency for XML output and 560 times improved memory efficiency for JSON output. Of course this is just one test case, your mileage may vary depending on query, but the memory efficiency improvements will be most apparent for really large query results.

New Enhanced Java API

For 2.5, we're including a new enhanced Java wrapper API that provides the following benefits and enhancements over the current Java API:

• Method names adhering to common Java naming conventions (lowerCamelCase instead of UpperCamelCase)
• MgException and its derivatives are no longer checked exceptions
• MapGuide collection classes implement java.util.Collection and can be iterated using the enhanced for loop

This enhanced Java API is made available in 2.5 as a "developer preview" option. The current Java API is still there (the Java AJAX viewer still uses it), and will continue to exist for a few more releases before we decide if/when to pull the plug on the current crufty Java API. If you are building a new Java MapGuide application, consider giving the enhanced Java API a go. It will be a much more pleasant development experience.

Integrated API documentation

The Java API enhancement work also yielded additional improvements in API documentation. The Java and .net wrapper APIs now include the translated doxygen API documentation fragments to javadoc and .net XML doc respectively, allowing for such documentation to be visible from within an IDE

This enhancement work renders obsolete the previous .net-only method of transplanting API documentation as this can work with both Java and .net.

For PHP, you're out of luck and will still have to use the web-based API documentation as there is no official standard for documenting PHP code that can be consumed in a PHP IDE. If there is a standard, I haven't heard of it.

Support for user-defined coordinate system dictionaries

Probably not that important for regular users (CS-Map supports thousands of coordinate systems out of the box), but MapGuide now supports user-defined coordinate systems dictionary files allowing for the sharing and consumption of custom user-defined coordinate systems. Check the CS-Map RFC for all the details.

MapGuide tidbits: MgFeatureService *is* a data access layer too!

A question we see a lot of on the mapguide-users mailing list is how can we access our external databases in .net/PHP/Java for reporting and things like that (generally in response to a selected feature), which then generally devolves into questions about language-specific data access technologies like PDO/ADO.net/JDBC and it's ilk.

I can't help but think such approaches are unnecessary because MapGuide already has a data access API: MgFeatureService, and it's consistent across .net/PHP/Java.

I think users get the impression that because MgFeatureService is only for spatial data access that it can't be used for regular non-spatial data access, which is untrue.

The only time whether data is spatially-enabled comes into play is if you want to build Layer Definitions off of such class definitions. You'll notice that the Layer editor UIs in Maestro and Infrastructure Studio skip over non-spatial class definitions for this very reason (the lack of a geometry property).

Spatial data is just an extra optional dimension on top of regular data and MgFeatureService does not mandate that the data you're working with must be spatially-enabled. Such class definitions when viewed in a FDO logical schema will simply not have a geometry property, but can still do CRUD operations on such classes via MgFeatureService. Notice how you can still view attribute data for non-spatial class definitions in the feature source preview? That's the MgFeatureService APIs at work.

So the next time you need to access external database in your MapGuide application, ask yourself this question: Am I able to just set up a MapGuide Feature Source to my external database and access this data through MgFeatureService? Because if you can, it will make things much more simpler and make your data access code more portable as being part of the MapGuide API, it will work the same for .net/PHP/Java.

Admittedly, not all external data stores can be accessed cleanly via MgFeatureService. There are some corner cases where an external data store can't be accessed via MgFeatureService due to the physical data store structure not cleanly translating to its equivalent FDO logical schema. For such cases, you can generally workaround the problem with the use of database views and/or FDO schema overrides to either re-configure the problematic tables or weed out the incompatible tables from your FDO logical schema.

So give MgFeatureService some though when you need to access external databases for reporting/etc. It's not just for spatial data access!

MapGuide tidbits: MgInitializeWebTier

This is the first of a series of posts that focuses on various aspects of MapGuide and its related technologies. There may have been various posts in the past with a similar focus, but now we'll be using an official umbrella title for such posts.

So the topic of this post is the function you've probably seen many times: MgInitializeWebTier

In any MapGuide code examples, you'll probably see this function called first before anything else in the MapGuide API is used, so what does it do?

There are various classes in the MapGuide API that need to be set up with a whole bunch of configuration values defined in webconfig.ini. This function sets up all these configuration values in one shot so all the classes in the MapGuide API have access to your configured values in webconfig.ini. This is why you generally call this function first before using any classes from the MapGuide API.

What happens if you call this function multiple times? Nothing. Subsequent calls will return immediately.

At what scope does MgInitializeWebTier apply to? The configuration values apply at the process level, this means:

• Your whole application (if you're building a console/windows MapGuide application)
• Your application pool (if you're building a asp.net MapGuide application in IIS)
• Your php-cgi.exe process (if you're building a PHP MapGuide application is IIS)
• Your Apache httpd process (if you're building a PHP MapGuide application served by Apache)
• Your tomcat process (if you're building a Java MapGuide application)

Meaning if you change some values in webconfig.ini, you generally have to do the following for the new values to take effect:

• Restart the application (if you're building a console/windows MapGuide application)
• Recycle the IIS application pool (if you're building a asp.net MapGuide application in IIS)
• Kill the php-cgi.exe process (if you're building a PHP MapGuide application is IIS). IIS will spawn a new php-cgi process the next time a PHP script is run.
• Restart Apache (if you're building a PHP MapGuide application served by Apache)
• Restart tomcat (if you're building a Java MapGuide application)

Are there cases where we can get away with not calling this function? Yes. If you are writing code that only operates against the Foundation, Geometry and PlatformBase libraries, then there is no actual MgInitializeWebTier to call into. Such initialization concerns are generally handled by the PlatformBase-derived library (MapGuideApi.MgInitializeWebTier for MapGuide, MgdPlatform.Initialize for mg-desktop).

If you are building an application that uses the MgCoordinateSystem series of APIs (from MgGeometry), then there is some small initialization you need to take care of. These classes need to know where the coordinate system dictionaries are located. Generally this is defined by the MENTOR_DICTIONARY_PATH environment variable. So if you are building such an application (and you don't have access to the MgInitializeWebTier function), you need to set this environment variable to your coordinate system dictionary path first before you can use any MgCoordinateSystem API.

So what's the minimum thing can you take out of all of this? For cleanliness, you only need to call MgInitializeWebTier once when the application starts up. Classic ASP.net provides an Application_Start entry point in Global.asax that is an ideal spot to insert such a function call. For other web frameworks, you need to consider whether:

• Such a similar process-wide entry point is provided by said framework
• The process life-cycle of said framework

When in doubt, you can just call MgInitializeWebTier first before using any class from the MapGuide API.

Hopefully this post sheds some light on when and why you need to call this function.

Announcing: MapGuide Open Source 2.5 Preview

In the spirit of releasing early and releasing often, I've put up a "preview" release of MapGuide Open Source 2.5.

The purpose of this preview release is to check and verify that our Windows installers are packing the correct files and that we haven't left out any files from the installer.

So give the windows installers a spin, and let us know if everything's in order or something's left out.

Assuming no installer problems, we'll start putting the 2.5 release cycle into high gear.

We'll cover what's new in 2.5 in a future post.