Last week OSGeo held its annual code sprint, an event where developers of the world working on open source geospatial software come together to hit the commit and merge buttons as hard as possible. This year’s code sprint was held in Paris in the beautiful offices of the Mozilla foundation. It was the first participation from the orfeo-toolbox team and a great step in the right direction for us.
Thank you to OSGeo for organizing this sprint and making it such a success, to Mozilla for hosting in this wonderful venue in the heart of Paris and to all the sponsors who made it possible.
Here are some technical notes about what we achieved during this sprint.
OSGeo Live Documentation update
The documentation of OTB in the OSGeo Live needed a refresh. The description and quickstart pages have been updated :
- Description enhanced
- Quicklooks updated
- The examples in the quickstart now use images already on the Live DVD.
QGIS integration
OTB applications are accessible from QGIS by means of the processing plugin. The code responsible for this lives in QGIS main repository under
python/plugins/processing/algs/otb. The older otb-qgis-plugin is obsolete and not maintained anymore.
- We updated the relevant documentation on OTB’s website, cookbook and wiki as well as OSGeo quickstart.
- The maintenance task of updating the xml application descriptors has been added to the OTB developers’s wiki. We tried to do it for 5.2.1 but could not get the python script to work. This is a further todo item. Clarification is also needed between the roles of
maintenance/OTBHelper.pyandhelper/generate_application_descriptors.py - Using a custom OTB is not possible on installations of QGIS where an OTB is available on system paths (/usr/bin on linux, program files on windows, etc.). This is hard coded in
OTBAlgorithmProvider.py. With the email help of Victor Olaya, the plugin’s original author, we pushed a fix to QGIS. It still needs to be tested a bit (on OSGeo4W in particular).
OGR wrapper
There are currently two families of data structure and filters to process vector data :
- The original OTB implementation using otb::VectorData
- The more recent wrapping around OGRDataSource
This situation is ambiguous and may lead to duplicated filters (doing the same thing on different data structures). A unique data structure would be more convenient, and the OGR one seems the right candidate because it has been used for recent improvements on large scale segmentation (VectorData is not adapted for large volumes of vectors). The next question is then : how to migrate existing VectorData filters into their “OGR” equivalent ?
After a brief review of VectorData usage in OTB, it appears that almost all VectorData filters process geometries independently. It means that those filters could process one OGRFeature at a time. No development was made but the idea needs to be refined.
OTB & GDAL bash completion
Starting from a Request for Comments, the bash completion for OTB command line tools actually means :
- Proposing completion for the application parameter keys (-in, -out, -filter, …)
- Proposing completion for ChoiceParameters (for instance the smoothing filter : mean/gaussian/anidif)
A tool has been written to create the bash completion script, which is later installed on the system. The tool will load available applications and search the parameter list. This development is available in a public branch completion_generator.
Since the result was a nice addition for all people using command line and completion, we chose to do the same for GDAL utilities (gdal_translate, gdaladdo, ogr2ogr, …). This time, no common application framework to reproduce the same method as in OTB. The chosen approach was to use a Python script and parse the “Usage” message returned by GDAL tools. It end-up with a completion script proposing :
- parameter keys for each application
- available pixel types (Byte, UInt16,…) for tools using the key -ot
- list of raster drivers for GDAL tools using the -of or –format option (drivers list is generated dynamically)
- list of vector drivers for OGR tools using the -f option (drivers list is generated dynamically)
This feature has been contributed to GDAL : https://trac.osgeo.org/gdal/changeset/33585
ITK pipeline optimization
Some experiments have been made around the ITK “Diamond Problem”. It refers to cases when there are multiple routes between a filter and an upstream image reader. This can generate duplicated (or more) calls to the reader’s GenerateData() for (almost) the same region. Typical example is when branch A asks for the same output requested region and branch B asks for a requested region enlarged by 1 pixel because it computes a local gradient.
A candidate solution for this problem requires to analyse the pipeline before Update(), and modify otb::Image and otb::VectorImage in order to catch the propagation when necessary and concatenate requested region coming from the same “downstream” request. Some developments have been made, but not released yet.