BlockMatching - Pixel-wise Block-Matching¶

Performs block-matching to estimate pixel-wise disparities between two images.

Detailed description¶

This application allows one to performs block-matching to estimate pixel-wise disparities for a pair of images in epipolar geometry.

This application is part of the stereovision pipeline. It can be used after having computed epipolar grids (with StereoRectificationGridGenerator) and resampled each input image into epipolar geometry (with GridBasedImageResampling).

The application searches locally for the displacement between a reference image and a secondary image. The correspondence is evaluated for each pixel, based on a pair of local neighborhood windows. The displacement evaluated can be 1D (along lines) or 2D. Parameters allow setting the minimum and maximum disparities to search (both for horizontal and vertical directions). A winner-take-all approach is used to select the best match. There are different metrics implemented to evaluate the match between two local windows:

SSD : Sum of Squared Distances
NCC : Normalized Cross-Correlation
Lp : Lp pseudo norm

Once the best integer disparity is found, an optional step of sub-pixel disparity estimation can be performed, with various algorithms (triangular interpolation, parabollic interpolation, dichotimic search). As post-processing, there is an optional step of median filtering on the disparities. One can chose input masks (related to the left and right input image) of pixels for which the disparity should be investigated. Additionally, two criteria can be optionally used to disable disparity investigation for some pixel: a no-data value, and a threshold on the local variance. This allows one to speed-up computation by avoiding to investigate disparities that will not be reliable anyway. For efficiency reasons, if the image of optimal metric values is desired, it will be concatenated to the output image (which will then have three bands : horizontal disparity, vertical disparity and metric value). One can split these images afterward.

Parameters¶

This section describes in details the parameters available for this application. Table [1] presents a summary of these parameters and the parameters keys to be used in command-line and programming languages. Application key is BlockMatching .

[1]	Table: Parameters table for Pixel-wise Block-Matching.

Parameter Key	Parameter Name	Parameter Type
io	Input and output data	Group
io.inleft	Left input image	Input image
io.inright	Right input image	Input image
io.out	The output disparity map	Output image
io.outmask	The output mask corresponding to all criterions	Output image
io.outmetric	Flag to output optimal metric values as well	Boolean
mask	Image masking parameters	Group
mask.inleft	Mask to discard left pixels	Input image
mask.inright	Mask to discard right pixels	Input image
mask.nodata	Discard pixels with no-data value	Float
mask.variancet	Discard pixels with low local variance	Float
bm	Block matching parameters	Group
bm.metric	Block-matching metric	Choices
bm.metric ssd	Sum of Squared Distances	Choice
bm.metric ncc	Normalized Cross-Correlation	Choice
bm.metric lp	Lp pseudo-norm	Choice
bm.metric.lp.p	p value	Float
bm.radius	Radius of blocks	Int
bm.minhd	Minimum horizontal disparity	Int
bm.maxhd	Maximum horizontal disparity	Int
bm.minvd	Minimum vertical disparity	Int
bm.maxvd	Maximum vertical disparity	Int
bm.subpixel	Sub-pixel interpolation	Choices
bm.subpixel none	None	Choice
bm.subpixel parabolic	Parabolic fit	Choice
bm.subpixel triangular	Triangular fit	Choice
bm.subpixel dichotomy	Dichotomy search	Choice
bm.step	Computation step	Int
bm.startx	X start index	Int
bm.starty	Y start index	Int
bm.medianfilter	Median filtering of disparity map	Group
bm.medianfilter.radius	Radius	Int
bm.medianfilter.incoherence	Incoherence threshold	Float
bm.initdisp	Initial disparities	Choices
bm.initdisp none	None	Choice
bm.initdisp uniform	Uniform initial disparity	Choice
bm.initdisp maps	Initial disparity maps	Choice
bm.initdisp.uniform.hdisp	Horizontal initial disparity	Int
bm.initdisp.uniform.vdisp	Vertical initial disparity	Int
bm.initdisp.uniform.hrad	Horizontal exploration radius	Int
bm.initdisp.uniform.vrad	Vertical exploration radius	Int
bm.initdisp.maps.hmap	Horizontal initial disparity map	Input image
bm.initdisp.maps.vmap	Vertical initial disparity map	Input image
bm.initdisp.maps.hrad	Horizontal exploration radius	Int
bm.initdisp.maps.vrad	Vertical exploration radius	Int
ram	Available RAM (Mb)	Int
inxml	Load otb application from xml file	XML input parameters file
outxml	Save otb application to xml file	XML output parameters file

[Input and output data]: This group of parameters allows setting the input and output images.

Left input image: The left input image (reference). It should have the same size and same physical space as the right input. This image can be generated by GridBasedImageResampling.
Right input image: The right input (secondary). It should have the same size and same physical space as the left input. This image can be generated by GridBasedImageResampling.
The output disparity map: An image containing the estimated disparities as well as the metric values if the option is used. If no metric is output and no sub-pixel interpolation is done, pixel type canbe a signed integer. In the other cases, floating point pixel is advised.
The output mask corresponding to all criterions: An output mask image corresponding to all citerions (see masking parameters). Only required if variance threshold or nodata criterions are set. Output pixel type is unsigned 8bit by default.
Flag to output optimal metric values as well: If enabled, the output image will have a third component with metric optimal values.

[Image masking parameters]: This group of parameters allows determining the masking parameters to prevent disparities estimation for some pixels of the left image.

Mask to discard left pixels: This parameter allows providing a custom mask for the left image. Block matching will be only perform on pixels inside the mask (non-zero values).
Mask to discard right pixels: This parameter allows providing a custom mask for the right image. Block matching will be perform only on pixels inside the mask (non-zero values).
Discard pixels with no-data value: This parameter allows discarding pixels whose value is equal to the user-defined no-data value.
Discard pixels with low local variance: This parameter allows discarding pixels whose local variance is too small (the size of the neighborhood is given by the radius parameter).

[Block matching parameters]: This group of parameters allow tuning the block-matching behaviour.

Block-matching metric: Metric to evaluate matching between two local windows. Available choices are:

Sum of Squared Distances: Sum of squared distances between pixels value in the metric window.

Normalized Cross-Correlation: Normalized Cross-Correlation between the left and right windows.

Lp pseudo-norm: Lp pseudo-norm between the left and right windows.

p value: Value of the p parameter in Lp pseudo-norm (must be positive).

Radius of blocks: The radius (in pixels) of blocks in Block-Matching.
Minimum horizontal disparity: Minimum horizontal disparity to explore (can be negative).
Maximum horizontal disparity: Maximum horizontal disparity to explore (can be negative).
Minimum vertical disparity: Minimum vertical disparity to explore (can be negative).
Maximum vertical disparity: Maximum vertical disparity to explore (can be negative).
Sub-pixel interpolation: Estimate disparities with sub-pixel precision. Available choices are:

None: No sub-pixel search.

Parabolic fit: The metric values closest to the best match are used in order to fit a parabola to the local extremum of the metric surface. The peak position of this parabola is output.

Triangular fit: The metric values closest to the best match are used in order to fit a triangular peak to the local extremum of the metric surface. The peak position of this triangle is output.

Dichotomy search: An iterative dichotomic search is performed to find the best sub-pixel position. The window in the right image is resampled at sub-pixel positions to estimate the match.

Computation step: Location step between computed disparities. Disparities will be computed every ‘step’ pixels in the left image (step for both rows and columns). For instance, a value of 1 corresponds to the classic dense disparity map.
X start index: X start index of the subsampled grid (wrt the input image grid). See parameter bm.step.
Y start index: Y start index of the subsampled grid (wrt the input image grid). See parameter bm.step.
Median filtering of disparity map: Use a median filter to get a smooth disparity map.

Radius: Radius (in pixels) for median filter.

Incoherence threshold: Incoherence threshold between original and filtered disparity.

Initial disparities Available choices are:

None: No initial disparity used.

Uniform initial disparity: Use an uniform initial disparity estimate.

Horizontal initial disparity: Value of the uniform horizontal disparity initial estimate (in pixels).

Vertical initial disparity: Value of the uniform vertical disparity initial estimate (in pixels).

Horizontal exploration radius: Horizontal exploration radius around the initial disparity estimate (in pixels).

Vertical exploration radius: Vertical exploration radius around the initial disparity estimate (in pixels).

Initial disparity maps: Use initial disparity maps to define the exploration area. This area in the right image is centered on the current position shifted by the initial disparity estimate, and has a given exploration radius in horizontal and vertical directions.

Horizontal initial disparity map: Map of the initial horizontal disparities.

Vertical initial disparity map: Map of the initial vertical disparities.

Horizontal exploration radius: Horizontal exploration radius around the initial disparity estimate (in pixels).

Vertical exploration radius: Vertical exploration radius around the initial disparity estimate (in pixels).

Available RAM (Mb): Available memory for processing (in MB).

Load otb application from xml file: Load otb application from xml file.

Save otb application to xml file: Save otb application to xml file.

Example¶

To run this example in command-line, use the following:

otbcli_BlockMatching -io.inleft StereoFixed.png -io.inright StereoMoving.png -bm.minhd -10 -bm.maxhd 10 -mask.variancet 10 -io.out MyDisparity.tif

To run this example from Python, use the following code snippet:

#!/usr/bin/python

# Import the otb applications package
import otbApplication

# The following line creates an instance of the BlockMatching application
BlockMatching = otbApplication.Registry.CreateApplication("BlockMatching")

# The following lines set all the application parameters:
BlockMatching.SetParameterString("io.inleft", "StereoFixed.png")

BlockMatching.SetParameterString("io.inright", "StereoMoving.png")

BlockMatching.SetParameterInt("bm.minhd", -10)

BlockMatching.SetParameterInt("bm.maxhd", 10)

BlockMatching.SetParameterFloat("mask.variancet", 10)

BlockMatching.SetParameterString("io.out", "MyDisparity.tif")

# The following line execute the application
BlockMatching.ExecuteAndWriteOutput()

Limitations¶

None

Authors¶

This application has been written by OTB-Team.