PVT Online Documentation

Computing the fundamental matrix and the trilinear tensor in an image sequence requires several steps; finding features, finding corresponding features between images, removing or filtering of incorrect correspondences and using a random sampling algorithm to compute the fundamental matrix. From the correspondences that support the fundamental matrix we compute potential corresponding corners in three images, and from these compute the tensor, again using random sampling.

PVT offers separate utilities to perform each of the steps listed above. The idea is that each of the steps in this process is done by a separate program, that reads and writes ascii files. These programs are chained together using simple shell scripts where the output of one program in the chain is subsequently used as input to the next program. In this way any of the programs in the chain can be easily replaced with any program that performs those same operations. The only restriction in doing this replacement, is that the critical output file (that is used as input to the next program in the chain) of the substituting program must exactly mimic that of the replaced toolkit program.

In this documentation, italicized names refer to shell scripts, while bold name refer to actual compiled programs.

The two basic operations in the toolkit are to compute the fundamental matrix between two images, and to compute the trilinear tensor between three images. Each of these operations is invoked by a shell script; the one to compute the fundamental matrix is called compFM. The format of these scripts is as follows:

compFM <filename 1> <filename 2> <optional arg1> … <optional arg n>

The compFM script invokes a number of programs to compute the fundamental matrix. For various reasons it actually goes through this sequence twice, first computing a fundamental matrix with a relatively small number of corners, and then computing a more accurate fundamental matrix using a large number of corners, but guided by the previously computed fundamental matrix.

The programs invoked by the compFM script are:

Each of these programs is documented in a separate file with that name. After the compFM script is executed the result is a set of files, including the fundamental matrix for that image pair. Each command in the script produces a number of output files, some of which are used as input to the next command, and some which are purely for diagnostic purposes. Each of the above commands can also be invoked directly, but the compFM script makes sure that the arguments are correct, and that the commands are executed in the proper sequence. The basic naming rule when computing a fundamental matrix is that the all the output files are named by first concatenating the names of the two input files (i.e. p000101.jpg-p000102.jpg) and then appending extensions such as ".match" or ".FM", etc. The actual files and extensions for a particular command are described in the documentation associated with that command.

The compFM script has a number of optional arguments to control the computational process. These arguments (which default to preset values) are the ones we have found that are most important for experimentation. To see the settings of the default arguments simply type compFM. There are currently 5 optional arguments for compFM. They are:

The script to compute the trilinear tensor from three images is called compTR, and is similar to the script for computing the fundamental matrix. However, it assumes that the fundamental matrix has already been computed for the image pairs for which the tensor is about to be computed. The format of this command is as follows:

compTR <filename 1> <filename 2> <filename 3> <optional arg1>…<optional argn>

The compTR script invokes only two programs;

The correspondences already computed for the fundamental matrices between the three images (filename1 and filename2, along with filename2 and filename3) are used to compute a set of potential correspondence triples across three images. These triples are then used to compute the trilinear tensor, again using a random sampling algorithm. The files produced by this tensor computation process have a naming convention which is again similar to that used for the fundamental matrix. The names of the three input files are concatenated (i.e. p000101.jpg-p000102.jpg-p000102.jpg) and then appropriate extensions such as ".TM", etc are appended. The actual files and extensions for a particular command are described in the documentation associated with that command.

The compTR script has a number of optional arguments, which are used to control the process. These arguments (which default to preset values) are the ones we have found to be the most important for experimentation. To see the settings of the default arguments simply type compTR. If there are no optional arguments then only the defaults are used. There are currently 3 optional arguments for compTR. They are:

These two script files, compFM and compTR are usually invoked on a numbered set of images in a sequence. Typically the numbering of the images files follows a simple system; a string preamble (i.e. "p000") followed by a particular image number (i.e. 606) to make a complete file name (i.e. "p000606"). The goal is to run compFM on all the pairs of images in a sequence, and then to run compTR for all the triples in a sequence. To make this easier we have created two more shell scripts, compallFM and compallTR. These shell scripts go through a series of numbered images, and compute the fundamental matrix (compallFM) and the trilinear tensor (compallTR) for that image sequence. One of the arguments of compallFM and compallTR is the name of the shell script which actually computes the fundamental matrix or trilinear tensor. Normally, we just use the standard compFM, and compTR scripts.

The syntax of the compallFM script is as follows:

compallFM <preamble> <start number> <end number> compFM <optional args>

For example, the command

compallFM p000 101 105 compFM –d 2.0

simply executes the following commands:

compFM p000101 p000102 –d 2.0

compFM p000102 p000103 –d 2.0

compFM p000103 p000104 –d 2.0

compFM p000104 p000105 –d 2.0

In the same way the computation of the trilinear tensor between image triples is simplified by the compallTR command. The syntax of this command is:

compallTR <preamble> <start number> <end number> compTR <optional args>

For example, the command

compallTR p000 101 105 compTR –d 2.0

simply executes the following shell commands:

compTR p000101 p000102 p000103 –d 2.0

compTR p000102 p000103 p000104 –d 2.0

compTR p000103 p000104 p000105 –d 2.0

As we have stated, before one executes compallTR, one must have executed compallFM for the same image sequence. The final result of the process is a series of files containing the fundamental matrix, and trilinear tensor, along with files containing other diagnostic information. We note that while we can process various types of input files (the extension can be GIF, BMP, PGM, PPM, JPG, PNG) all the diagnostic files are in jpeg format.

To view the final result one should use the program FMTviewer. This program graphically displays all the results (both fundamental matrix and trilinear tensor) in an image sequence. The syntax of the command is:

FMTViewer <current directory> <preamble> <start number> <end number> [extension]

The first three arguments are required, and the last is optional and defaults to jpg. The viewer has a simple interactive GUI interface for displaying the results for each image triple and double.

There is a directory called examples, which has a number of test sequences, and a number of script files already created to process the images in these sequences. The script file with the name doall in each directory computes the appropriate compallFM an compallTR command. The script file with the name displayit invokes the appropriate FMTviewer command for that image sequence.

There are a number of examples, but they have all been processed with the same parameters. They use the default script options, and produced reasonable results. In each directory we also place a VRML file, which contains a graphical view of the reconstruction of the feature points and camera positions.

The quickest way to start is to simply go the examples directory, and display the current results using the displayit script. Then try and reproduce these results by running the doall script. Before, doing this of course you must install the software (look in the install document).