The road to OTB 7

Posted on by Victor Poughon

The lastest OTB release, version 6.6.1, was released more than six months ago! We usually aim for a release every 3 months, so what happened? Let’s take a look.

OTB is a large project with many components, modules and dependencies. After more than 10 years of development, we have accumulated a lot of technical debt: it’s time for a cleanup! So on top of new features, we are preparing a new major release: OTB 7, with a lot of internal changes that will make maintenance easier. Most of theses changes are invisible to users. That’s why they have historically been low priority while we focused on more immediate gains. But the time has come for some major refactors! Here’s an overview of the changes coming to OTB sometime in 2019:

New applications

Zonal statistics: Compute zonal statistics from label image, or vector data. The application inputs one input multiband image, and another input for zones definition. Zones can be defined with a label image (inzone.labelimage.in) or a vector data layer (inzone.vector.in). The following statistics are computed over each zones: mean, min, max, and standard deviation. Statistics can be exported in a vector layer (if the input zone definition is a label image, it will be vectorized) or in a XML file.

LocalRxDetection: Perform local Rx score computation on an input hyperspectral image. For each hyperspectral pixel, the Rx score is computed using statistics computed on a dual neighborhood. The dual neighborhood is composed of all pixel that are in between two radiuses around the center pixel.

NumberOfEndmember: Estimate the number of endmembers in a hyperspectral image. First, compute statistics on the image and then apply an endmember number estimation algorithm using these statistics. Two algorithms are available: Virtual Dimensionality (VD) and Eigenvalue Likelihood Maximization (ELM).

Application enhancements

FunctorImageFilter

In the C++ API, we added a new generic filter, FunctorImageFilter. Using modern C++14 features (lambdas, SNIFAE, metaprogramming etc.), we are able to simplify and remove a lot of code that was spread over many classes. Details here.

Removal of old code

Spring cleaning! After some analysis, we realized that many OTB modules should not be part of the core project, because no one is using them, or there is a much better alternative and is has been deprecated for a while. There are many examples of this in the current develop branch, but notable instances are: TileMapImageIO, VectorDataRendering, or the Convert application (replaced by DynamicConvert).

Dependencies

Software dependencies are a major source of technical debt. Developers have to worry about forward compatibility, backwards compatibility, availability on target platforms, compatibility with compilers, licensing issues, and more! OTB has 26 external dependencies (though only 7 are mandatory). You can imagine the complexity grows rapidly!

Historically, bugs related to dependencies have been a very significant part of the cost of OTB maintenance. We want to improve this, and the starting point is reducing the complexity of the problem space. In software parlance, this means dropping support for some versions. So, starting with the next OTB version (7.0.0):

  • GDAL 1.x: no longer supported
  • OpenCV 2.x: no longer explicitly supported
  • Python 2.x: no longer supported
  • Mapnik: no longer needed!

If you look at our day-to-day development (which happens completely in the open on gitlab!) you’ll see a big topic is OSSIM. A long term goal for us is removing the dependency to OSSIM completely. It’s a lot of work and we don’t know if it will be ready for the next release yet, but we are working on it!

Semantic versioning and deprecation policy

Starting at OTB 7, we are officially switching to semantic versioning. We are also improving our deprecation and removal policy so that the impact on users and existing code is minimal. This means several things (read the entire specification, it’s interesting!), but in short starting with OTB 7:

  • The meaning of version numbers is precisely defined
  • The public API is backwards compatible between major versions

Note that this does not apply to OTB 7.0.0 itself. In a sense OTB 7.0.0 will be the “1.0.0” of semver.

Better tests

Orfeo Toolbox is heavily tested. We have more than 2000 individual tests! However running the entire test suite requires:

  • A complex build setup
  • A lot of time

These two points really reduce the possible pace of development. And bug fixes are a kind of development, so ultimately the quality of the final product is worse. We are spending a lot of effort to improve tests so that they are faster, easier to run, and simpler to maintain.

Better documentation

Another area where OTB needs improvement is documentation. We have completely rewritten the application documentation system in the CookBook. Applications are now better documented (including the Python API). You can already see it here.

We are also working on merging the two main OTB documentations: the CookBook and the Software Guide. Historically those two have been separate. But today we see the need to simplify as much as possible and have a single source of documentation for OTB.

Better QGIS integration

Integration with QGIS is a priority for the OTB team now. It’s a complicated story, but we are working with QGIS to improve the plugin so that QGIS users have a simple and reliable access to the latest OTB applications. We are also improving the documentation of the plugin to clarify the version compatibility issues.

If you have feedback or requests for the next version of OTB, feel free to open an issue on gitlab.

Happy remote sensing!