This free code is an ImageJ plugin that allows the simultaneous registration of two images based on elastic deformations represented by B-splines. The invertibility of the deformations is forced through a consistency restriction. bUnwarpJ is also integrated in Fiji. Contact author: Ignacio Arganda-Carreras.
is an algorithm for
elastic and consistent
image registration developed as an ImageJ
It performs a
simultaneous registration of two images, A and B. Image A is
elastically deformed in order to
look as similar as possible to image B, and, at the same time,
the "inverse" transformation (from B to A) is also calculated
so a pseudo-invertibility of the final deformation could be guaranteed.
Two images are given as a result: the deformed versions of A and B
The latest stable distribution of bUnwarpJ can always be found included in the Fiji toolkit or at the git
repository. If you detect any bug, please feel free to contact the author.
Any feedback will be very appreciated. The
files can be downloaded with the following JAR file:
To execute bUnwarpJ
macro or from the command line, see the description in the User Manual.
The algorithm implemented on bUnwarpJ and its technical explanations are detailed on the paper:
Ignacio Arganda-Carreras, Carlos O. S. Sorzano, Roberto
Jose M. Carazo, Carlos Ortiz de Solorzano,
Elastic Registration of Histological Sections using
Vector-Spline Regularization”, Lecture Notes in
Springer Berlin / Heidelberg, volume 4241/2006, CVAMIA:
Vision Approaches to Medical Image Analysis, pages 85-95,
The previous work (UnwarpJ) is reachable here, and its related paper is:
"Elastic Registration of Biological Images Using Vector-Spline Regularization", C.Ó. Sánchez Sorzano, P. Thévenaz, M. Unser, IEEE Transactions on Biomedical Engineering, vol. 52, no. 4, pp. 652-663, April 2005.
This image registration algorithm is based on the minimization of an energy functional that includes the dissimilarity between the source and target images -in both directions- Eimg, an optional landmark constraint Eµ, a regularization term (Ediv + Erot), and an energy term Econs that accounts for the geometrical consistency between the elastic deformation in both directions. Namely, the energy function is given by
E = wiEimg + wµEµ + (wdEdiv + wrErot) + wcEcons
Where the weights of every term are set by the user in the main window of the plugin. The optimization process is a Levenberg-Marquardt minimization enhanced by a Broyden-Fletcher-Goldfarb-Shanno (BFGS) estimate of the local Hessian of the goal function, and both, images and deformations are represented by cubic B-splines.
First of all, the plugin must be installed as described on Installation (for further information go to ImageJ). Once it is properly installed, the plugin should appear in the main menu under Plugins > Registration > bUnwarpJ. Two images (8, 16, 32-bit grayscale or RGB Color) need to be opened in order to be able to use the plugin. If so, the following window will open:
Figure 2: Main plugin window (v2.6)
Both selected images will work simultaneously as source and target, their tags are there only for the sake of clarification. The registration mode can be "Accurate", "Fast" and "Mono". The registration mode "Mono" (included since version 2.5) makes the program to perform only unidirectional registration, i.e. from source to target. The two registration modes "Accurate" and "Fast" involve performing bidirectional registration and affect the stopping criteria internally used by the program. More internal options can be modified in the "Advanced Options" panel. This panel gives you access to most of the internal parameters of the algorithm. The "Initial" and "Final" deformation lists allow you to select the coarsest and finest scale of the spline deformation field. "Very coarse" corresponds to 4 splines (one in each corner of the image). As you increase the deformation level, the number of splines is doubled in each direction (horizontal and vertical).
Since bUnwarpJ 2.5 there is a new parameter on the main window to allow subsampling the input images. The registration will be then calculated using the subsampled versions of the images but the results will be applied to the original ones. The image subsampling parameter can be chosen between 0 and 7, i.e. the image dimensions can be reduced by a factor of 20 = 1 to 27 = 128. This is very useful when registering large images.
The different weights of the goal function control the relative weight of each one of the terms. These weights are not restricted to be between 0 and 1, and they may take any value as long as it is non-negative=2E. You can see a description of the different function weights in the presentation "bUnwarpJ: Consistent and Elastic Registration in ImageJ. Methods and Applications.", given at the Second ImageJ User & Developer Conference (2008) or have a look at the FAQ page.
The stop threshold is used by the algorithm to stop the
process at each multiresolution level when the error relative change is
not larger than this threshold.
RGB Color images will be converted to grayscale during the registration process but the resulting transformations will be applied to the original color images.
If you want to exit the plugin, press "Cancel". When you want the
plugin to perform the registration press "OK". After running
the plugin (on "Accurate" or "Fast" mode), the
results are two stacks with the following three
1. One image (warping image) registered as to fit the other image (fixed image);
2. The fixed image;
3. The warping mask with the same deformation as the warping image.
The final registration values appear in a separate ("Results") window.
The verbose mode produces more information:
1. The elastic
deformation as a vector field. Each point in the fixed image must be
deformed according to this field to fit into the warping image;
2. The grid obtained after deforming the fixed image with the vector field described above;
3. The step values of the optimization process in a separate ("Results") window.
Figure 4: Additional result images (verbose mode), deformation field and grid
Figure 5: Results window with values of the goal function f during the optimization process and final optimal values
Since both, source and target images work as warping and fixed images, there are two sets (stacks) of results: from source to target and from target to source.
The "Mono" mode produces only results from the source to the target image.
During the registration process, the current difference images and a mapping of the grid from the fixed images onto the moving images are shown.
Figure 6: Examples of output images during the registration process (current target-source difference image and original source image with current grid)
During the registration process the toolbar will be changed to
Figure 7. Appearance of the window while registration is in progress
Click on the stop button to stop the process. The output at the current state of the optimization will be returned in the normal way.
When the plugin is called and before pressing "OK" in the main window, the toolbar changes its appearance and it is possible to manually add landmarks to the selected images:
Figure 8: Adding landmarks activated
The depressed button indicates that you may ADD A LANDMARK now. Landmarks are added in either image. The landmark will be automatically placed in the same position on both images. The new landmark becomes the "current landmark" (indicated by a thicker [+] sign in the current image and a [×] sign in the other image, while all the rest are represented by [+] signs). To move any landmark, press on the MOVE LANDMARK button.
Figure 9: Moving landmarks activated
Click and drag on any landmark to make it correspond to the same position in both images. Here goes an example of the two Lena images with corresponding landmarks:
Figure 10: Corresponding landmarks
Landmarks can be removed through the REMOVE LANDMARK (crosses) button.
Figure 11: Removing landmarks activated
This is the way of manually adding landmarks to the registration process. However, since bUnwarpJ v2.0 there is the option as well of using automatic landmarks as explained in the section SIFT and MOPS support, or manually adding point selections in both images before calling the plugin. If the number of point selections is the same in both images, they will be transformed into landmarks.
When exiting bUnwarpJ all the images are restored to their previous state, i.e. the original regions of interest and point selections are restored.
This program allows you using masks in two mutually exclusive ways. In the first way, masks are introduced together with the input images. In this mode, input images must be a stack of images (first slice: the image itself, second slice: the mask). In this way, the mask can have any shape. In the second way, the input images must not be stacks and simple polygonal masks can be used. These masks are defined using the two buttons (INNER MASK and OUTER MASK) shown below:
Figure 13: Outer mask activated
The inner mask keeps the information in the interior of the polygon, while the outer mask keeps the information in the exterior of the polygon. The thrown-out information is grayed. Here goes an example of an inner mask:
Figure 14: Inner mask example
Masks can be used for one of the images, both, or none. You can put a mask in one of the images and not in the other, you can put a mask (with different shapes) in both images, or you may not use masks at all. After calling the plugin, the masks are erased and the initial images are restored.
When using the "I/O Menu" from the toolbar, we have the possibility of processing different input and output files that will affect the registration.
Figure 15: ImageJ toolbar when using bUnwarpJ plugin. I/O menu option is selected.
In the last release, the plugin presents the following Input/Output options:
- Load Landmarks: it allows loading landmarks files.
- Save Landmarks As: it allows saving the image landmarks into a file.
- Show Landmarks: it forces the landmarks to be displayed in a separate table.
- Load Elastic Transformation: it loads an elastic transformation from a file and applies it to the so-called source image. The transformation file must be in the format of the plugin B-spline transformations, i.e. the same format as the files created with the "Save Transformation" option.
- Load Raw Transformation: it loads a raw transformation from a file and applies it to the so-called source image. The transformation file must be in "raw" format, i.e. the same format as the files created with the "Convert Transformation To Raw" option. See raw_transformation_template.txt for a format description.
- Compare Opposite Elastic Transformations: it calculates the warping index of two elastic transformations, i.e. the average of the geometrical distance between every pixel and its version after applying both transformations (direct and inverse). This value is given as a result. The transformation files must be in the format of the plugin B-spline transformations (same format as the files created with the "Save Transformation" option).
- Compare Raw Transformations: it calculates the warping index of two raw transformations (same direction). The transformation file must be in "raw" format, i.e. the same format as the files created with the "Convert Transformation To Raw" option.
- Convert Transformation To Raw: it converts a B-spline transformation file into a raw transformation file.
- Compose Elastic Transformations: it composes two elastic transformations into a raw transformation. The input files must be in the B-spline transformation format, and the output file will be in "raw" format.
- Compose Raw Transformations: it composes two raw transformations into another raw transformation. Both input and output files will be in "raw" format.
- Compose Raw and Elastic Transformations: it composes one raw transformation and one elastic transformation into a raw transformation.
- Evaluate Image Similarity: it calculates de current similarity error between the source and target images. The results is displayed in the "Results" window.
- Adapt Coefficients: it transforms the coefficients of an specific elastic transformation according to a real image factor. Very useful for example when we have very large images. We can register subsampled versions of our images (let us say 4 times smaller) and then adapt the result transformations (image factor = 4) so we can apply them to the high resolution images.
- Load Source Mask: it loads a mask from file (binary image) in the source image.
- Load Source Initial Affine Matrix: it loads an affine matrix as starting point for the source-target transformation from a file. See affine_transformation_matrix.txt for a description of the format.
These new options (since version 1.1) allow the user comparing the results of our program with any other registration method.
You can also test bUnwarpJ with SplineDeformationGenerator, an ImageJ plugin that allows the user to generate five different image deformations: elastic, fisheye, perspective, barrel/pincushion and smile effect. SplineDeformationGenerator uses the same raw transformation file format as bUnwarpJ, so they are completely compatible.
Since bUnwarpJ v2.0 the plugin allows using ImageJ macro language to be called. Therefore, the user can launch the plugin from a macro setting all the parameters of the plugin main window, for instance:
When calling the plugin like this, if the "save_transformations" parameter is set to true, the transformations files will be saved in the following predefined format (not save dialog is shown):source image name + "_direct_transf.txt" and target image name + "_inverse_transf.txt".
Only the main registration method can be called like this. To use the main Input/Output options from a macro, the call function must be used. There is a corresponding static method defined in the plugin for any of this options: elasticTransformImageMacro, rawTransformImageMacro, composeRawElasticTransformationsMacro, composeRawTransformationsMacro, composeElasticTransformationsMacro, compareRawElasticTransformationsMacro, compareElasticTransformationsMacro, compareRawTransformationsMacro, convertToRawTransformationMacro, adaptCoefficientsMacro.
Notice here that the input and output file names must include the path, since they are not taken from the list of images. For instance, if we want to apply an elastic deformation stored in the file A_direct_transf.txt to the source image A.jpg with target image B.jpg and save the result in output.tif, we call:
call("bunwarpj.bUnwarpJ_.elasticTransformImageMacro", "My_path/A.jpg", "My_path/B.jpg", "My_path/A_direct_transf.txt", "My_path/output.tif");
bUnwarpJ might be called as well as an ImageJ from the command line. In the command line, the program offers the following options:
- -help: shows the syntax of the program
- -align: launches the registration of two input images
- -elastic_transform: transforms the source image with a given elastic deformation (previously calculated)
- -raw_transform: transforms the source image with a given raw deformation (previously calculated)
- -compare_elastic: compares two previously calculated opposite elastic deformations through the warping index
- -compare_elastic_raw: compares an elastic deformation with a raw deformation (both direct transformations) through the warping index
- -compare_raw: compares two previously calculated and direct raw deformations through the warping index
- -convert_to_raw: converts an elastic transformation into raw format
- -compose_elastic: composes two elastic deformations, the result will be in raw format
- -compose_raw: composes two raw deformations, the result will be too in raw format
- -compose_raw_elastic: composes a raw deformation and an elastic one. Result in raw format
- -adapt_transform: adapts an specific elastic transformation given a resolution image factor
For instance, to see the program help we can call the program
from the command line (where $IJDIR is the directory where
is installed) like this:
For the rest of the options, follow the help instructions.
The main reason to create bUnwarpJ was the
idea of enforcing deformations' consistency through the consistency
weight. This number forces the
algorithm to move into solutions that ensure the invertibility of the
resulting deformations. Therefore, higher is this number, more strictly
is one deformation the inverse of the other. Due to the different
units, there is no rule for selecting the right parameters. For
instance, to perform the registration of the tumor images above, we
recommend consistency weight values around 20.0.
One important advantage of bUnwarpJ over the previous method lies in the fact that many registration problems can be solved without using the landmarks and regularization terms of the energy function (that means setting their corresponding parameters to 0.0). Therefore, no user interaction is needed and the computational complexity of the algorithm is reduced.
The last release of bUnwarpJ has compatibility with the Stephan Saalfeld's plugin for automatic feature extraction: SIFT and MOPS algorithms. You only need to download the following files to your ImageJ plugins folder:
- Stephan Saalfeld's plugin: mpicbg_.jar (browse source here)
- imagescience.jar from Erik Meijering
After applying SIFT or MOPS methods, you will get two sets of corresponding points in both images. If you call then bUnwarpJ, then the corresponding points will appear as source and target landmarks.
Figure 17: SIFT/MOPS - bUnwarpJ compatibility. Top row: MOPS results. Bottom row: landmarks after calling bUnwarpJ.
This program is free
software; you can redistribute it and/or modify it under
the terms of the GNU
General Public License as published by the Free Software
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
We just expect you to include a citation or acknowledgment whenever you present or publish results that are based on it. Enjoy it!
bUnwarpJ has been developed during several years already and many people need to be acknowledged for:
It started as an extension of UnwarpJ from Carlos O. S. Sorzano and thanks to him and Jan Kybic at the Center for Machine Perception in Prague, bUnwarpJ was born in the summer 2005 and published online in 2006.
Many of the plugin updates and improvements would have never been possible without the hackathons that took place in Janelia Farm Research Campus (Virginia, summer 2008) and the Institute of Neuroinformatics (Zürich, winter 2008).
Marta Rivera-Alba helped with the macro support and code debugging.