Chapter 20
Object-based Image Analysis

Object-Based Image Analysis (OBIA) focusses on analyzing images at the object level instead of working at the pixel level. This approach is particularly well adapted for high resolution images and leads to more robust and less noisy results.

OTB allows to implement OBIA by using ITK’s Label Object framework (http://www.insight-_journal.org/browse/publication/176). This allows to represent a segmented image as a set of regions and not anymore as a set of pixels. Added to the compression rate achieved by this kind of description, the main advantage of this approach is the possibility to operate at the segment (or object level).

A classical OBIA pipeline will use the following steps:

1. Image segmentation (the whole or only parts of it);
2. Image to LabelObjectMap (a kind of std::map<LabelObject>) transformation;
3. Eventual relabeling;
4. Attribute computation for the regions using the image before segmentation:
   1. Shape attributes;
   2. Statistics attributes;
   3. Attributes for radiometry, textures, etc.
5. Object filtering
   1. Remove/select objects under a condition (area less than X, NDVI higher than X, etc.)
   2. Keep N objects;
   3. etc.
6. LabelObjectMap to image transformation.

20.1 From Images to Objects

The source code for this example can be found in the file
Examples/OBIA/ImageToLabelToImage.cxx.

This example shows the basic approach for the transformation of a segmented (labeled) image into a LabelObjectMap and then back to an image. For this matter we will need the following header files which contain the basic classes.

The image types are defined using pixel types and dimension. The input image is defined as an otb::Image .

As usual, the reader is instantiated and the input image is set.

Then the binary image is transformed to a collection of label objects. Arguments are:

• FullyConnected: Set whether the connected components are defined strictly by face connectivity or by face+edge+vertex connectivity. Default is FullyConnectedOff.
• InputForegroundValue/OutputBackgroundValue: Specify the pixel value of input/output of the foreground/background.

Then the inverse process is used to recreate a image of labels. The itk::LabelMapToLabelImageFilter converts a LabelMap to a labeled image.

The output can be passed to a writer. The invocation of the Update() method on the writer triggers the execution of the pipeline.

Figure 20.1 shows the effect of transforming an image into a label object map and back to an image

20.2 Object Attributes

The source code for this example can be found in the file
Examples/OBIA/ShapeAttributeComputation.cxx.

This basic example shows how compute shape attributes at the object level. The input image is firstly converted into a set of regions ( itk::ShapeLabelObject ), some attribute values of each object are computed and then saved to an ASCII file.

The image types are defined using pixel types and dimensions. The input image is defined as an otb::Image .

Firstly, the image reader is instantiated.

Here the itk::ShapeLabelObject type is chosen in order to read some attributes related to the shape of the objects, by opposition to the content of the object, with the itk::StatisticsLabelObject .

The input image is converted in a collection of objects

Update the shape filter, so its output will be up to date.

Then, we can read the attribute values we’re interested in. The itk::BinaryImageToShapeLabelMapFilter produces consecutive labels, so we can use a for loop and GetLabelObject() method to retrieve the label objects. If the labels are not consecutive, the GetNthLabelObject() method must be use instead of GetLabelObject(), or an iterator on the label object container of the label map. In this example, we write 2 shape attributes of each object to a text file (the size and the centroid coordinates).

20.3 Object Filtering based on radiometric and statistics attributes

The source code for this example can be found in the file
Examples/OBIA/RadiometricAttributesLabelMapFilterExample.cxx.

This example shows the basic approach to perform object based analysis on a image. The input image is firstly segmented using the otb::MeanShiftSegmentationFilter Then each segmented region is converted to a Map of labeled objects. Afterwards the otb::otbMultiChannelRAndNIRIndexImageFilter computes radiometric attributes for each object. In this example the NDVI is computed. The computed feature is passed to the otb::BandsStatisticsAttributesLabelMapFilter which computes statistics over the resulting band. Therefore, region’s statistics over each band can be access by concatening STATS, the band number and the statistical attribute separated by colons. In this example the mean of the first band (which contains the NDVI) is access over all the regions with the attribute: ’STATS::Band1::Mean’.

Firstly, segment the input image by using the Mean Shift algorithm (see 8.7.2.3 for deeper explanations).

The otb::MeanShiftSegmentationFilter type is instantiated using the image types.

The itk::LabelImageToLabelMapFilter type is instantiated using the output of the otb::MeanShiftSegmentationFilter . This filter produces a labeled image where each segmented region has a unique label.

Instantiate the otb::RadiometricLabelMapFilterType to compute statistics of the feature image on each label object.

Feature image could be one of the following image:

• GEMI
• NDVI
• IR
• IC
• IB
• NDWI2
• Intensity

Input image must be convert to the desired coefficient. In our case, statistics are computed on the NDVI coefficient on each label object.

The otb::AttributesMapOpeningLabelMapFilter will perform the selection. There are three parameters. AttributeName specifies the radiometric attribute, Lambda controls the thresholding of the input and ReverseOrdering make this filter to remove the object with an attribute value greater than Lambda instead.

Then, Label objects selected are transform in a Label Image using the itk::LabelMapToLabelImageFilter .

And finally, we declare the writer and call its Update() method to trigger the full pipeline execution.

Figure 20.2 shows the result of applying the object selection based on radiometric attributes.

20.4 Hoover metrics to compare segmentations

The source code for this example can be found in the file
Examples/OBIA/HooverMetricsEstimation.cxx.

The following example shows how to compare two segmentations, using Hoover metrics. For instance, it can be used to compare a segmentation produced by your algorithm against a partial ground truth segmentation. In this example, the ground truth segmentation will be referred by the letters GT whereas the machine segmentation will be referred by MS.

The estimation of Hoover metrics is done with two filters : otb::HooverMatrixFilter and otb::HooverInstanceFilter . The first one produces a matrix containing the number of overlapping pixels between MS regions and GT regions. The second one classifies each region among four types (called Hoover instances):

• Correct detection : a region is matched with an other one in the opposite segmentation, because they cover nearly the same area.
• Over-segmentation : a GT region is matched with a group of MS regions because they cover nearly the same area.
• Under-segmentation : a MS region is matched with a group of GT regions because they cover nearly the same area.
• Missed detection (for GT regions) or Noise (for MS region) : un-matched regions.

Note that a region can be tagged with two types. When the Hoover instance have been found, the instance filter computes overall scores for each category : they are the Hoover metrics ¹.

The filters otb::HooverMatrixFilter and otb::HooverInstanceFilter are designed to handle itk::LabelMap images, made with otb::AttributesMapLabelObject . This type of label object allows storing generic attributes. Each region can store a set of attributes: in this case, Hoover instances and metrics will be stored.

The first step is to convert the images to label maps : we use itk::LabelImageToLabelMapFilter . The background value sets the label value of regions considered as background: there is no label object for the background region.

The Hoover matrix filter has to be updated here. This matrix must be computed before being given to the instance filter.

The instance filter computes the Hoover metrics for each region. These metrics are stored as attributes in each label object. The threshold parameter corresponds to the overlapping ratio above which two regions can be matched. The extended attributes can be used if the user wants to keep a trace of the associations between MS and GT regions : i.e. if a GT region has been matched as a correct detection, it will carry an attribute containing the label value of the associated MS region (the same principle goes for other types of instance).

The otb::LabelMapToAttributeImageFilter is designed to extract attributes values from a label map and output them in the channels of a vector image. We set the attribute to plot in each channel.

The output image contains for each GT region its correct detection score (”RC”, band 1), its over-segmentation score (”RF”, band 2), its under-segmentation score (”RA”, band 3) and its missed detection score (”RM”, band 4).

The Hoover scores are also computed for the whole segmentations. Here is some explanation about the score names : C = correct, F = fragmentation, A = aggregation, M = missed, N = noise.