Image registration
The purpose of this repository is to provide an overview of github repositories on non-parametric image registration.
All our code is written in python using PyTorch except for our original deep-learning registration work [Yang16] which is written in lua using torch.
Mermaid: iMagE Registration via autoMAtIc Differentiation
_ _
(_) | |
_ __ ___ ___ _ __ _ __ ___ __ _ _ __| |
| '_ ` _ \ / _ \ '__| '_ ` _ \ / _` | |/ _` |
| | | | | | __/ | | | | | | | (_| | | (_| |
|_| |_| |_|\___|_| |_| |_| |_|\__,_|_|\__,_|
Mermaid is a registration toolkit making use of automatic differentiation for rapid prototyping. It includes various image registration models. In particular, stationary velocity field models (both based on velocity fields and momentum fields), scalar vector momentum Large Displacement Diffeomorphic Metric Mapping (LDDMM) models as well as the more generalized Region-specific Diffeomorphic Metric Mapping model (RDMM).
The Mermaid repository can be found here: https://github.com/uncbiag/mermaid
Supported transformation models:
affine_map: map-based affine registration
diffusion_map: displacement-based diffusion registration
curvature_map: displacement-based curvature registration
total_variation_map: displacement-based total variation registration
svf_map: map-based stationary velocity field
svf_image: image-based stationary velocity field
svf_scalar_momentum_image: image-based stationary velocity field using the scalar momentum
svf_scalar_momentum_map: map-based stationary velocity field using the scalar momentum
svf_vector_momentum_image: image-based stationary velocity field using the vector momentum
svf_vector_momentum_map: map-based stationary velocity field using the vector momentum
lddmm_shooting_map: map-based shooting-based LDDMM using the vector momentum
lddmm_shooting_image: image-based shooting-based LDDMM using the vector momentum
lddmm_shooting_scalar_momentum_map: map-based shooting-based LDDMM using the scalar momentum
lddmm_shooting_scalar_momentum_image: image-based shooting-based LDDMM using the scalar momentum
lddmm_adapt_smoother_map: map-based shooting-based Region specific diffemorphic mapping, with a spatio-temporal regularizer
svf_adapt_smoother_map: map-based shooting-based vSVF, with a spatio regularizer
Supported similarity measures:
ssd: sum of squared differences
ncc: normalize cross correlation
ncc_positive: positive normalized cross-correlation
ncc_negative: negative normalized cross-correlation
lncc: localized normalized cross correlation (multi-scale)
Supported solvers:
embedded RK4
torchdiffeq: explicit_adams, fixed_adams, tsit5, dopri5, euler, midpoint, rk4
Optimizer:
support single/multi-scale optimizer
support SGD, l-BFGS and some limited support for adam
EasyReg
EasyReg is an extension that builds on Mermaid, providing a simple interface to Mermaid and other popluar registration packages.
The currently supported methods include Mermaid-optimization (i.e., optimization-based registration) and Mermaid-network (i.e., deep network-based registration methods using the mermaid deformation models, like SVF, LDDMM, RDMM...). We also added some supports on ANTsPy, NiftyReg and Demons(embedded in SimpleITK).
The EasyReg repository can be found here: https://github.com/uncbiag/easyreg
Gallery
Here are some examples:
Vector Momentum-parameterized Stationary Velocity Field(vSVF): with a temporal-invariant velocity field and a constant regularizer.
Large Displacement Diffeo-morphic Metric Mapping (LDDMM): with a spatial-temporal velocity field and a constant regularizer.
Region-specific Diffeomorphic Metric Mapping (RDMM): with a spatial-temporal velocity field and a spatial-temporal regularizer.
RDMM with an optimized regularizer
RDMM with a pre-defined regularizer
Related work
The first work on deep-learning for non-parametric medical image registration [Yang16] predicted the initial momentum of an LDDMM solution, hence was already able to guarantee diffeomorphic transformations. Because it is based on patch-wise predictions, it can be applied to very large images if needed. This work was extended and extensively analyzed in [Yang17], showing competitive performance with state-of-the-art optimization-based deformable image registration approaches; it also included deep-network models to predict displacement and velocity fields. Follow-up work focused on learning the registration metric [Niethammer19]; jointly learning to predict affine transformations and a nonparametric transformation [Shen19a] (including multi-step approaches and training to encourage symmetric models); as well as extensions of the LDDMM approach to spatio-temporal regularizers (i.e., that move with the deforming images) also within a deep-network approach [Shen19b].
Bibtex entries for this related work can be found here [bibtex] and the citations are listed in the next section.
Bibliography
[Yang16] Fast Predictive Image Registration [pdf] [code]
Xiao Yang, Roland Kwitt, Marc Niethammer. DLMIA 2016.
[Yang17] Quicksilver: Fast predictive image registration--a deep learning approach [pdf] [code]
Xiao Yang, Roland Kwitt, Martin Styner, Marc Niethammer, NeuroImage 2017.
[Niethammer19] Metric Learning for Image Registration [pdf] [code]
Marc Niethammer, Roland Kwitt, Francois-Xavier Vialard. CVPR 2019.
[Shen19a] Networks for Joint Affine and Non-parametric Image Registration [pdf] [code]
Zhengyang Shen, Xu Han, Zhenlin Xu, Marc Niethammer. CVPR 2019.
[Shen19b] Region-specific Diffeomorphic Metric Mapping [pdf] [code]
Zhengyang Shen, François-Xavier Vialard, Marc Niethammer. NeurIPS 2019.