VDD-Reg

[ paper ]

This repository serves as the official placeholder for the code and models accompanying our paper *MEMO: Dataset and Methods for Robust Multimodal Retinal Image Registration with Large or Small Vessel Density Differences (BOE 2024)

Overview

VDD-Reg is a multimodal retinal image registration framework designed for robust registration across different retinal imaging modalities. It is particularly effective for multimodal image pairs with large vessel density differences, such as OCTA and EMA, but can also be applied to other retinal imaging modalities.

The framework consists of two major components:

1. LVD-Seg (Segmentation Module) LVD-Seg is a two-stage semi-supervised vessel segmentation framework:

• Stage 1: Supervised vessel segmentation training on EMA images.
• Stage 2: Unsupervised vessel distribution adaptation between EMA and OCTA images.

2. Registration Modeul The registration pipeline includes:

• Vessel segmentation using LVD-Seg
• Feature extraction using SuperPoint
• Mutual nearest-neighbor matching
• RANSAC-based affine registration
• Registration evaluation using:
  • Six-point landmark error (RMSE, MEE, MAE)
  • Frangi-based Soft Dice score

For more details, please refer to our paper.

Getting Started

Installation

1. Clone the Repository and Install Dependencies

git clone https://github.com/chiaoyiwang0424/VDD-Reg.git
cd VDD-Reg
conda create -n vddreg -f environment.yml
conda activate vddreg

2. Install SuperPoint This project uses the official SuperPoint implementation: https://github.com/magicleap/SuperPointPretrainedNetwork

Clone the repository into:

VDD-Reg/
└── external/
    └── SuperPointPretrainedNetwork/

Dataset Sources

The MEMO dataset used in this project is available at: https://chiaoyiwang0424.github.io/MEMO/

Project Structure

Please organize the repository as follows:

VDD-Reg/
├── external/
│   └── SuperPointPretrainedNetwork/
├── src/
│   ├── Dataset_gt.py
│   ├── loss.py
│   ├── LVDSeg_eval.py
│   ├── LVFSeg_stage1.py
│   ├── LVFSeg_stage2.py
│   ├── NEtwork.py
│   ├── RANSAC.py
│   ├── reg_eval_rmse.py
│   ├── reg_eval_softdice.py
│   ├── run_superpoint.py
│   ├── utils_image.py
│   └── utils_pytorch.py
├── dataset/
├── run_main.py
└── run.sh

Run

We provide example scripts (run.sh) which demonstrate a variety of use cases, including training, inference, and registration.

Below is an example that runs the complete VDD-Reg pipeline:

python run_main.py \
  --mode stage1_stage2 \
  --data-folder ./dataet/MEMO/ \
  --experiment-dir ./Model/Exp1/ \
  --stage1-list ./dataet/MEMO/stage1_list.csv \
  --stage2-list ./dataet/MEMO/stage2_list.csv \
  --test-list ./dataet/MEMO/test.csv

What this example does:

• Trains Stage 1 of LVD-Seg
• Trains Stage 2 of LVD-Seg
• Performs segmentation inference on the test set
• Runs SuperPoint feature extraction
• Runs RANSAC-based registration
• Computes registration evaluation metrics (RMSE, MEE, MAE, and Soft Dice)

Available Modes

This implementation contains 5 modes:

1. stage1_stage2 Runs the complete pipeline:

• Stage 1 training
• Stage 2 training
• Segmentation inference
• Registration
• Registration evaluation

2. stage2 Runs:

• Stage 2 training using a pretrained Stage 1 model
• Segmentation inference
• Registration
• Registration evaluation

3. inference Runs segmentation inference using a pretrained Stage 2 model.

4. registration Runs only the registration pipeline:

• Segmentation inference
• RANSAC registration
• Registration evaluation

5. inference_registration Runs:

• Segmentation inference using a pretrained Stage 2 model
• Registration
• Registration evaluation

Acknowledgements

This project builds upon several excellent open-source projects. We thank the authors for making their implementation publicly available.

1. RetinalSegReg: Some components of the segmentation framework (including DRIU network architecture implementations, style loss and self-comparison loss) were adapted from: https://github.com/JunkangZhang/RetinalSegReg

2. SuperPoint: Feature extraction is performed using the official SuperPoint implementation: https://github.com/magicleap/SuperPointPretrainedNetwork

Citation

If you use this code for your research, please cite our papers.

@article{wang2024memo,
  title={MEMO: dataset and methods for robust multimodal retinal image registration with large or small vessel density differences},
  author={Wang, Chiao-Yi and Sadrieh, Faranguisse Kakhi and Shen, Yi-Ting and Chen, Shih-En and Kim, Sarah and Chen, Victoria and Raghavendra, Achyut and Wang, Dongyi and Saeedi, Osamah and Tao, Yang},
  journal={Biomedical Optics Express},
  volume={15},
  number={5},
  pages={3457--3479},
  year={2024},
  publisher={Optica Publishing Group}
}