OTB in QGIS: a brief report of FOSS4G 2016 codesprint

Posted on by Manuel Grizonnet

Last week, I had the chance to attend to the FOSS4G 2016 conference in Bonn and I participated with Rashad from the OTB team to the first code sprint of the conference which was located in the BaseCamp where quite a few of us were lucky enough to stay during the conference. The place is an indoor caravan park with plenty of space to support all the development teams, a perfect place to organize such event.

Check some photos and video.

Our goal during the sprint was to work on the improvement of the integration of OTB applications in QGIS processing. QGIS processing framework is a geoprocessing environment that can be used to call native and third-party algorithms from QGIS. It makes available a wide variety of dedicated tools for data analysis (GDAL, GRASS, SAGA…and also OTB)

OTB is available for a long time now in QGIS processing. It was initially contributed by Victor Olaya with then improvements done by Alexia Mondot, Julien Malik and Oscar Picas which allows to wrap most of otb applications in QGIS. Even if otb applications engine comes with an API which allows to retrieve applications parameters and setting this wrapping is not fully automatic. There are several reasons for that:

  • Not all the applications are wrapped (for some of them it does not really make sense to make them available in QGIS). There is therefore a list of applications wrapped which has to be updated “by hand” for now (white_list.xml and black_list.xml).
  • Some complex applications have been splitted in several applications to facilitate the integration in processing (to handle for instance the notion of group of parameters available in application engine for instance). For example the TrainImagesClassifier is split in several applications (one for each machine learning methods). There is no automatic procedure for now to determine which applications has to be splitted but there is some tips available in the maintenance procedure.

What was done during the sprint:

  • Update OTB descriptors in QGIS to support last OTB versions. Alexia had started to update the procedure in July and starting from her PR we’re able to update OTB in QGIS for versions 5.4 and 5.6. Improvements have been made also to the procedure and the documentation has also been updated. It is still not fully automatic but it requires a minimum number of manual operations and useful debug information are generated by the maintenance script.
  • Prototype the integration of simple tests for OTB integration following QGIS test procedure. The integration of OTB in QGIS is based on XML descriptors which means that there is for now no automatic way to test this integration (the procedure last step consists in testing otb application in QGIS manually :).
    We’ve started to work on am ore automatic way to test this integration. What we’ve done for now:

    • Set up a “simple” test for one applications following QGIS processing procedure following what is already existing for GDAL and GRASS GIS (based on YAML descriptors).
    • Prototype a way to have an automatic bindings of all OTB tests. QGIS test descriptors is based on YAML, the idea is to extend CMake macros in OTB (OTB_TEST_APPLICATION) to generate automatically this descriptor during the configuration. It could allows to have a external project which use QGIS API and OTB-Data repo to test OTB integration.
  • See finally how OTB can benefit of future improvements in QGIS processing regarding dynamic. There are ongoing discussions in a QGIS Enhancement proposals here if you’re interested.

What can be done in the future:

    • There is still room for improvements I think in OTB helper classes in processing with some specific code to handle some applications parameters and specific behavior.
    • We didn’t have time to finish the integration of the testing procedure. There was some remaining issue regarding test execution and validation. Ongoing work is available in a branch on my QGIS fork.
    • QGIS has started the development of a new major release (version 3). It is not clear for now if there will be in the meantime a new 2.X release. In this case we would be cool to backport OTB upgrade to branch master_2.

My caravan called “the Drag Queen”

So we’ve got a procedure to update OTB version in QGIS which is pretty straightforward. Even if there is still some part which can’t be automated, a pull request can be easily submitted to QGIS at each OTB releases in the future.

We really want to thank here all QGIS people for their help during the codesprint. Special thanks to Alex, Matthias and Arnaud for their support and kindness.Hope to see you next year in Paris for FOSS4G-EU and/or Boston for FOSS4G!

Manuel