The original FdoInfo/FdoUtil tools were implemented with the goal of getting a basic command-line experience for FDO functionality without much thought into proper best practices for how CLI tools should behave and work in a shell scripting context.
In the many years since, having accumulated experience in using various CLI tools (in particular, the git command-line client) and .net libraries coming into existence like CommandLine, which makes creating .net CLI applications in the behavioral template of git and friends dead-simple, I saw the re-activation of FDO Toolbox development as an opportunity to do away with FdoInfo/FdoUtil and build a proper CLI tool that can act as a true command-line companion for FDO Toolbox.
And thus FdoCmd.exe was born.
To understand why FdoCmd.exe came to be, it probably helps to explain what my high-level goals were for FdoCmd.exe
- It has to verb-based.
- It has to be mostly self-documenting and its functionality easily discover-able.
- Its output has to be in a form that allows for it to be easily capture/parsing in shell scripts. Since FDO Toolbox is a windows-only tool, we are primarily referring for FdoCmd.exe to be easily driven by Powershell script, powershell being the predominant shell scripting environment for Windows.
- It has to have functional parity with the core features provided by its GUI counterpart. The normal things you do in FDO Toolbox (the windows application) should be also doable in FdoCmd.exe.
Each goal I'll be explaining in greater detail below.
Verb-based terminology
FdoCmd.exe is a verb-based CLI. What we mean by this is that the first argument you pass to FdoCmd.exe is the verb that you want to run followed by arguments specific to that particular verb. The most famous example of CLI tools that follow a verb-based pattern is the git command line client.
When using the git command-line client, every command always starts with "git":
- git add somefile.txt
- git commit -m "some commit message"
- git reset --hard
- git merge origin/master
FdoCmd.exe follows this same pattern for all the functionality it provides.
Self-documenting and discoverability
The verb-based design contributes to the discover-ability of FdoCmd.exe. What do we mean by this? If you run FdoCmd.exe without arguments, you effectively get a "directory listing" of all supported verbs.
From here you know what verbs to use, and you can drill down further. Want to know what arguments you need for the list-schemas verb? Just run FdoCmd.exe list-schemas and the tool will tell you.
And for some verbs, we even provide some examples.
It is this discoverability that lends to the self-documenting nature of the tool. There's really no need to write a dedicated set of documentation for this tool when the tool can tell you itself how its meant to be used.
Shell-scripting friendliness
FdoCmd.exe was designed with the requirement that it should be easy to write shell scripts around the tool. On windows, the predominant shell scripting environment is Powershell, so the outputs of FdoCmd.exe is such that it can be easily parsed and processed in powershell code.
For example, listing the classes of a data store prints each class name line by line.
Which is easily captured as a string array when invoked within a powershell script
1 2 3 4 | $classNames = & .\FdoCmd.exe list-classes --provider OSGeo.SHP --connect-params DefaultFileLocation D:\fdo-trunk\Providers\SHP\TestData\Sheboygan foreach ($className in $className) { Write-Host "Doing something with: $className" } |
Where applicable, FdoCmd.exe provides CSV output for its verb, which when combined with the ConvertFrom-CSV cmdlet allows powershell to easily consume FdoCmd.exe output for particular verbs as an array of objects. For example, you can query out features as CSV (along with the usual querying features like filtering and selective output of properties)
Which easily allows for powershell scripts to read features as arrays of objects.
1 2 3 4 5 6 7 8 9 | $records = & .\FdoCmd.exe query-features --class Parcels --from-file D:\fdo-trunk\Providers\SHP\TestData\Sheboygan\Parcels.shp --schema Default --filter "RNAME LIKE 'SCHMITT%' AND RYEAR > 1950" --properties FeatId RNAME RYEAR RBILAD RCITY RZIP --format CSV | ConvertFrom-CSV foreach ($record in $records) { $fid = $item.FeatId $name = $item.RNAME $addr = $item.RBILAD $city = $item.RCITY $zip = $item.ZIP # Do stuff with these values } |
For convenience, querying features through FdoCmd also supports GeoJSON output for easy conversion of feature data to GeoJSON.
If you recall my previous post on splitting out a SHP file to individual GeoJSON files per feature, FdoCmd.exe has all the tools (pun intended) to now do this in a single powershell script!
1 2 3 4 5 6 7 8 9 | $srcPath="C:\TestData\national-esri-fe2019\COM_ELB_region.shp" $featIds = & .\FdoCmd.exe query-features --from-file $srcPath --class COM_ELB_region --properties FeatId --computed-properties FileName "concat(lower(Elect_div), '_new_0_', lower(State))" --format CSV | ConvertFrom-CSV foreach ($item in $featIds) { $fid = $item.FeatId $fn = $item.FileName $outPath = "C:\TestData\electorates_geojson\$fn.geojson" & .\FdoCmd.exe query-features --from-file $srcPath --class COM_ELB_region --filter "FeatId = $fid" --format GeoJSON | Out-File -FilePath $outPath Write-Host "Saved: $outPath" } |
Functional parity with FDO Toolbox
Anything you normally do in FDO Toolbox, you should be able to do in FdoCmd.exe as well.
- Creating/Destroying data stores
- Introspecting required provider parameters to connect, create, destroy data stores.
- FDO provider registration/unregistration
- Walk the structure (schemas/classes) of any data store
- Listing/creating/destroying spatial contexts
- Exporting/Applying FDO schemas
- Querying features in a feature class
- Run SQL commands on data stores where the provider supports SQL commands
- Easily bulk copy features from a feature class to a target data store
- Executing existing bulk copy / join definition files
Along with some convenience functionality that is exclusive to FdoCmd.exe:
- Get a total count of features in any feature class
- Get the extent of any feature class
- CSV or GeoJSON output for feature/SQL query commands
Where to from here?
As far as I am concerned, when FDO Toolbox 1.4 is released (coming real soon!). The FdoCmd.exe tool that ships with it will be functionally complete.
The only thing that can be improved on at that point would be to port it over to C++ and built against the native FDO APIs (instead of the windows-only .net wrapper APIs), so it can be part of the FDO native binary set, whether it's on Windows or Linux. That way, even if there won't be FDO Toolbox for Linux, at least you'll have access to its functionally equivalent CLI tool.
Posts
