Friday, 22 April 2022

MapGuide dev diary: Another (tangential) breakthrough!

Since I got the basic sanity checking PHP script running without crashing, there was one problem remaining around MgByteReader contents being read incorrectly. This turned out to be another out-of-date SWIG typemap around byte arrays. Once that got fixed we are able to read out the contents of the MgByteReader without content corruption. This is an important thing to test because a lot of results from the MapGuide API (XML results, rendered map images, map plots, etc) come in the form of MgByteReaders. If we can't read content out of these readers without data corruption we have major problems, but that is fortunately not the case (so far!).

With that out of the way, it was onto to the current PHP test suite for the MapGuide API and getting it fixed up and passing.

But before we go there, I wanted to check if our new PHP binding was leaking memory in the most obvious cases. To that effect, I built our PHP binding with reference counting diagnostics enabled and wrote this PHP script.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
<?php

function exceptionTest() {
	/*
	Expected release book-keeping:
		MgException - 1
	 */
	try {
		$rid = new MgResourceIdentifier("iamnotvalid");
	} catch (MgException $ex) {

	}
}

function throwingFunction() {
	$agfRw = new MgAgfReaderWriter();
	$wktRw = new MgWktReaderWriter();

	$rid = new MgResourceIdentifier("iamnotvalid");
}

function exceptionTest2() {
	/*
	Expected release book-keeping:
		MgAgfReaderWriter - 1
		MgWktReaderWriter - 1
		MgException - 1
	 */
	try {
		throwingFunction();
	} catch (MgException $ex) {

	}
}

function exceptionTest3() {
	/*
	Expected release book-keeping:
		MgAgfReaderWriter - 1
		MgWktReaderWriter - 1
		MgException - 2
	 */
	try {
		$rid = new MgResourceIdentifier("iamnotvalid");
	} catch (MgException $ex) {

	}

	try {
		throwingFunction();
	} catch (MgException $ex) {
		//Previous iterations of the binding would leak the previous $ex when
		//reusing the variable in this catch block
		//If the release book-keeping for MgException is 2, then this is no
		//longer the case
	}
}

function geometryXformTest() {
	/*
	Expected release book-keeping:
		MgCoordinateSystemFactory - 1
		MgCoordinateSystem - 2
		MgTransform - 1
		MgWktReaderWriter - 1
		MgPoint - 2
		MgCoordinateXY - 2
	 */
	$csFactory = new MgCoordinateSystemFactory();
	$cs1 = $csFactory->CreateFromCode("LL84");
	$cs2 = $csFactory->CreateFromCode("WGS84.PseudoMercator");
	$xform = $csFactory->GetTransform($cs1, $cs2);
	$wktRw = new MgWktReaderWriter();
	$pt = $wktRw->Read("POINT (1 2)");
	$coord = $pt->GetCoordinate();
	echo "Point (".$coord->GetX().", ".$coord->GetY().")\n";
	$pt = $pt->Transform($xform);
	$coord = $pt->GetCoordinate();
	echo "XPoint (".$coord->GetX().", ".$coord->GetY().")\n";
}

function connectionServiceTest() {
	/*
	Expected release book-keeping:
		MgUserInformation - 1
		MgSiteConnection - 1
		MgProxyResourceService - 1
		MgProxyFeatureService - 1
		MgProxyRenderingService - 1
		MgProxyMappingService - 1
		MgProxyKmlService - 1
		MgProxyDrawingService - 1
		MgProxyTileService - 1
		MgProxyProfilingService - 1
	 */

	$userInfo = new MgUserInformation("Anonymous", "");
	$site = new MgSiteConnection();
	$site->Open($userInfo);

	$service1 = $site->CreateService(MgServiceType::ResourceService);
	$service2 = $site->CreateService(MgServiceType::FeatureService);
	$service3 = $site->CreateService(MgServiceType::RenderingService);
	$service4 = $site->CreateService(MgServiceType::MappingService);
	$service5 = $site->CreateService(MgServiceType::KmlService);
	$service6 = $site->CreateService(MgServiceType::DrawingService);
	$service7 = $site->CreateService(MgServiceType::TileService);
	$service8 = $site->CreateService(MgServiceType::ProfilingService);
}

function functionWithParam($userInfo) {
	$userInfo->SetLocale("en");
}

function parameterPassingTest() {
	// We want to check parameters of Mg* objects passed do not encounter
	// refcounting shenanigans, but if they do, we want to make sure that
	// proper increment/decrement is happening so that the MgUserInformation
	// is released to 0 refcount (and thus deleted) when this function returns

	/*
	Expected release book-keeping:
		MgUserInformation - 1
	 */
	$userInfo = new MgUserInformation();
	functionWithParam($userInfo);
	echo "Locale: " . $userInfo->GetLocale() . "\n";
}

MgInitializeWebTier("C:\\mg-4.0-install\\Web\\www\\webconfig_dev.ini");

echo "=========== BEGIN - exceptionTest() ===========\n";
exceptionTest();
echo "=========== END - exceptionTest() ===========\n";

echo "=========== BEGIN - exceptionTest2() ===========\n";
exceptionTest2();
echo "=========== END - exceptionTest2() ===========\n";

echo "=========== BEGIN - exceptionTest3() ===========\n";
exceptionTest3();
echo "=========== END - exceptionTest3() ===========\n";

echo "=========== BEGIN - connectionServiceTest() ===========\n";
connectionServiceTest();
echo "=========== END - connectionServiceTest() ===========\n";

echo "=========== BEGIN - parameterPassingTest() ===========\n";
parameterPassingTest();
echo "=========== END - parameterPassingTest() ===========\n";

echo "=========== BEGIN - geometryXformTest() ===========\n";
geometryXformTest();
echo "=========== END - geometryXformTest() ===========\n";

?>

The basic tenets behind this test script are:
  • If we new any Mg* class, we expect it to be released (reference count dropped by 1) once the respective variable goes out of scope. We consider the script to not be leaking if every "new" statement has a corresponding release as reported by our refcounting diagnostics.
  • Any non-primitive value returned by any class in the MapGuide API is also released once its respective captured variable goes out of scope
  • If an MgException is thrown and we catch it, it is released once we exit the respective catch block
  • If we have multiple try/catch blocks on MgException and we use the same variable name for the caught exception, we're not leaking on subsequent catch blocks (this used to happen when we were generating the PHP bindings on the older SWIG/PHP version)
  • Finally any MapGuide API object passed as parameters to other functions does not get touched by reference counting. But if it does, we expect that the reference count for that object is the same before and after the function call.
Running this script produces the following output



The key things to notice in the script output is that every variable assignment is being properly released upon exiting the function and anything that we new'd explicitly was released with a 1 -> 0 reference count transition. When an object's reference count drops to 0, it will be de-allocated on the C++ side. The script output is proof that we aren't leaking objects we explicitly new'd up from the PHP script, they are being properly de-allocated. Some objects (like the MgCoordinateSystem) release and do not drop to 0 reference count, but that is okay because I know that the MapGuide API caches MgCoordinateSystem instances for performances reasons, so they aren't leaking instances. They can live beyond the scope of the function call where I initially requested for such objects.

So now knowing that we aren't leaking for the general cases, we can move onto the main PHP test suite.

Running the main PHP test suite it for the first time (in a long time) showed some PHP errors around ambiguous overloaded methods. All of these errors are because PHP 7/8 must've tightened up some of the type checking and passing null as a parameter to a method that has many overloaded signatures may cause confusion. Either that or our heavily modified version of SWIG that we previously used to generate the PHP binding had custom codegen to specifically handle overloaded method resolution.

Regardless, the fix was simple enough. Where passing null would cause ambiguity, we just needed to add the necessary cast (in our case, it was casting to string) to make sure the correct method signature is invoked. Once that fix was applied, the test suite ran to completion ...


... with a metric crapton of test failures :(

Now to figure out why these test cases are failing, which is where we come to the subject of this post.

Anytime in the past, when I need to debug a PHP script, it was a case of sprinkling var_dump and print_r statements everywhere, running it and try to comprehend the now extra verbose output, rinsing and repeating until I found the problem. Basically, printf debugging.

I had heard about a thing called xdebug, but I had no idea how to pair it with an IDE (and what IDE to even use?) to get the nice integrated debugging experience I take for granted with .net, Java or C++. Debugging PHP script felt so primitive because I simply didn't know better and getting proper debugging set up just sounded too hard.

Now, times have changed. We have Visual Studio Code. It has a nice ecosystem of extensions for working with PHP that supports the version of PHP that we are now targeting. If debugging C++ code inside docker containers turned out to be such a piece of cake, then surely the same can be said for using VSCode to debug PHP scripts with breakpoints, variable inspection and the whole works!

So I searched for a VSCode extension that could help me debug PHP, and the first result was this PHP debug adapter. I followed its instructions to the letter.

I went to the xdebug install wizard and pasted by dumped phpinfo details


Once submitted, the wizard figured out what version of PHP I had and gave me the download link for the xdebug extension to download and further instructions on what php.ini changes I needed to make


Then back to the VSCode extension instructions, I also updated php.ini to activate extra xdebug settings

1
2
xdebug.mode = debug
xdebug.start_with_request = yes

Then in VSCode, I opened the PHP script I intend to run and debug, and created a new launch.json file for my active workspace.


The newly installed VSCode PHP debug extension must've kicked in at this point because it created a launch.json with 3 xdebug launch presets ready to go. I modify the "Launch currently open script" preset to add in the extra command-line arguments needed by my PHP script.


Once I saved the launch.json, I open the PHP script again, stuck some breakpoints, switched to the debug tab, making sure to use the "Launch currently open script" preset.


I press the play button to start debugging and lo and behold! We're now breaking and stepping through PHP code and inspecting variables!


Now I can focus on figuring out why there are so many test failures, at a much faster pace than what I would've normally done in the past!

No doubt when MapGuide Open Source 4.0 is finally out the door, I'll have to write up a variation of this post on how to set up VSCode with PHP debugging for your PHP-based MapGuide applications. Because nobody should be sprinking var_dumps and print_r statements to debug PHP code like I have done in the past when this is many orders of magnitude faster and more productive!
Posted by Jackie Ng at 02:21
Labels: , ,

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)