Roadmap: path to a minimal viable dMRIPrep pipeline

[oesteban]

Context

After restructuring dMRIPrep to follow fMRIPrep's fit/transform architecture, this issue captures a comprehensive assessment of the current state and lays out a prioritized roadmap toward a functioning pipeline.

The CLI, config system, and argument parser are essentially complete. The workflow scaffolding (base.py, dwi/fit.py, dwi/base.py) is well-structured and fMRIPrep-aligned. However, several critical components are placeholder or disconnected, preventing an end-to-end run.

P0 — Blockers for a minimal working pipeline

1. SDC never applied during resampling

ResampleDWISeries (workflows/dwi/resampling.py) has a pass placeholder where fieldmap correction should be composed into the transform chain. The entire SDC estimation pipeline produces results that are never consumed during resampling.

Action: Use SDCFlows utilities to compose the fieldmap warp into the per-volume transform chain (motion × fieldmap × coreg), then resample in a single interpolation step. Implement Jacobian modulation.

2. NiFreeze interface API unvalidated

The interfaces in interfaces/nifreeze.py (NiFreezeEstimate, ExtractTransforms) assume an API (DWI.from_filename, Estimator, .motion_affines, .eddy_xfms) that was drafted speculatively.

Additionally, transforms3d is used in _write_motion_params but is not in the project dependencies.

Action: Validate against the actual nifreeze package and adapt. Wire FLIRT fallback (init_dwi_hmc_flirt_wf) as an interim alternative. Related: #181.

3. Workflow graph smoke test

No test verifies that init_dwi_fit_wf or init_dwi_wf can build a graph without errors. Also missing: __init__.py in workflows/dwi/tests/.

Action: Add a minimal test using mock_config() that constructs the workflow graph. Related: #182.

4. BIDSSourceFile references bold instead of dwi

interfaces/bids.py (line ~108): self.inputs.bids_info['bold'] should be 'dwi'.

P1 — Feature completeness

5. Standard-space resampling loop

dwi/base.py receives anat2std_xfm, std_t1w, std_mask from the template iterator but never creates a standard-space resampling workflow. --level full only produces T1w-space outputs, not volumetric standard-space outputs.

6. Transform format consistency

The non-BBR coregistration path outputs LTA-format transforms, but downstream consumers (nitransforms, ResampleDWISeries) expect ITK format. Need LTA→ITK conversion or consistent format throughout.

7. Disconnected reportlet wiring

• reportlets_wf.inputnode.summary_report and validation_report are not connected in dwi/base.py.
• Coregistration out_report from init_dwi_reg_wf is generated but never datasunk.

P2 — Quality and interpretability

8. T2w-based coregistration

init_dwi_t2w_reg_wf exists in registration.py, and the config supports dwi2anat_init = 't2w', but the fit workflow never receives T2w data. The intent is set in base.py but never propagated.

9. Motion/quality confounds

Generate _desc-confounds_timeseries.tsv with framewise displacement (adapted for DWI), motion parameters, mean signal, and outlier flags.

10. QC reportlets

• SDC before/after comparison
• Coregistration overlay
• Motion parameter time-series plots
• DiffusionSummary exists in reports.py but is never instantiated

11. HMC model auto-selection

Currently hardcodes model='DTI'. Should auto-select based on shell structure: 'average' for single-shell with few directions, 'DTI' for single-shell, 'DKI'/'GP' for multi-shell.

12. BIDS-URI provenance

BIDSURI interface exists in interfaces/bids.py but is never used. No Sources fields in derivative sidecar JSONs.

13. FLIRT fallback wiring

init_dwi_hmc_flirt_wf exists in hmc.py but has no selection logic to use it when NiFreeze is unavailable or --ignore eddy is set.

P3 — Longer-term vision

• Multi-run DWI merging — combine opposed-PE runs into a single corrected dataset (mentioned in code comments, not implemented)
• CIFTI DWI outputs — project to grayordinates for surface-based connectivity; anatomical CIFTI machinery is already wired from SMRIPrep
• Slice-level outlier detection — detect/replace corrupted slices, a common DWI artifact
• MP-PCA denoising — optional Marchenko-Pastur denoising before any interpolation
• Gibbs ringing correction — optional via DIPY or MRtrix3 (related: Gibbs ringing - estimation and correction workflow #179)
• Signal drift correction — RescaleB0 interface exists but isn't integrated
• Integration tests & CI — end-to-end test with THP data; update stale Dockerfile (related: Build comprehensive integration and stress testing into dMRIPrep #182)

Suggested attack order

1. Validate NiFreeze API (or wire FLIRT fallback) → HMC can execute
2. Implement SDC in ResampleDWISeries → fieldmaps actually get applied
3. Workflow graph smoke test → safety net for everything else
4. Fix trivial bugs (bold→dwi, missing __init__.py, reportlet wiring)
5. Standard-space resampling loop → --level full works properly
6. Confounds + QC reportlets → interpretable outputs

This gets to a minimal viable pipeline that can run --level minimal (steps 1–4) and produce corrected outputs in standard spaces (steps 5–6).

Priority matrix

Priority	Item	Effort	Impact
P0	ResampleDWISeries SDC placeholder	Medium	Pipeline correctness
P0	Validate NiFreeze interface API	Medium	Pipeline can execute
P0	Workflow graph smoke test	Low	Prevents regressions
P0	BIDSSourceFile bold→dwi	Trivial	Correctness
P1	Standard-space resampling loop	Medium	Feature completeness
P1	Transform format consistency (LTA→ITK)	Low	Correctness
P1	Disconnected reportlet wiring	Low	Clean execution
P2	T2w coregistration wiring	Medium	Feature completeness
P2	Confounds TSV generation	Medium	Downstream usability
P2	QC reportlets	Medium	Interpretability
P2	BIDS-URI provenance	Medium	Reproducibility
P3	Multi-run merging	High	Advanced datasets
P3	CIFTI DWI outputs	High	Connectivity analysis
P3	Denoising / Gibbs	Medium	Signal quality

[oesteban]

Roadmap Re-assessment: NiPreps Guru Analysis vis-à-vis fMRIPrep

A principled analysis of this roadmap against fMRIPrep's mature architecture, following the NiPreps standardization stance. BOLD-specific concerns (STC, multi-echo, DVARS, CompCor, tedana) are excluded.

Priority Re-adjustments

The roadmap is largely sound, but three items need re-prioritization:

Item	Current	Proposed	Rationale
LTA→ITK format (#6)	P1	P0	Silent correctness bug: non-BBR path outputs LTA but ResampleDWISeries reads ITK
BIDS-URI provenance (#12)	P2	P1	Standards compliance, not polish — outputs are not BIDS-Derivatives compliant without Sources
Workflow smoke test (#3)	P0	P0-first	Must be the very first item — without it, all other work is flying blind

Fit Stage: Structurally Excellent

The fit stage closely mirrors fMRIPrep's proven pattern — buffer nodes, 5-stage structure, level gating, precomputed derivative ingestion all present. Three architectural gaps noted:

1. Brain extraction before HMC — dMRIPrep extracts the brain before HMC (stage 2), while fMRIPrep does it after fieldmap registration (stage 4). The mask may be worse with strong susceptibility distortion.
2. No precomputed check for dwiref2fmap — the io_spec.json query exists but init_dwi_fit_wf never checks for it.
3. No regref_buffer — the SDC-corrected reference has no precomputed path; the SDC stage runs even when someone provides a precomputed coreg_dwiref.

Critical Gap Not in Roadmap: B-vector Rotation Through Full Transform Chain

This should be P0. Currently, b-vectors are only rotated for motion transforms. When outputting to T1w or standard space, b-vectors must additionally be rotated by the rotation component of coreg + normalization transforms. Without this, any downstream tractography or tensor fitting in non-native spaces will produce wrong results. This is a DWI-specific correctness requirement with no BOLD analog.

Resampling Architecture (the biggest gap)

The resampling stage is non-functional. Drawing from fMRIPrep's ResampleSeries + ReconstructFieldmap pattern, the recommended architecture:

init_dwi_native_wf:
  transforms = motion_xfm (per-volume)
  fieldmap = reconstruct(fmap_coeff, dwiref2fmap_xfm, target=dwiref)
  b-vectors: already rotated for motion (from fit stage)

init_dwi_anat_wf:
  transforms = Merge: motion_xfm + dwiref2anat_xfm
  fieldmap = reconstruct(..., target=T1w)
  b-vectors: rotate for motion + coreg rotation

init_dwi_std_wf (per template space):
  transforms = Merge: motion_xfm + dwiref2anat_xfm + anat2std_xfm
  fieldmap = reconstruct(..., target=template)
  b-vectors: rotate for motion + coreg + normalization rotation

DWI-specific considerations:

• Jacobian modulation is essential — signal intensity encodes diffusion attenuation, so incorrect modulation biases diffusion model fits.
• Fieldmap must be reconstructed per-volume in the motion-corrected position.

NiFreeze Risk

• The interfaces assume an API (DWI.from_filename, .motion_affines, .eddy_xfms) drafted speculatively against a 0.0.1 package.
• transforms3d is used but missing from dependencies — should be replaced with scipy.spatial.transform.Rotation (already a dependency).
• FLIRT fallback is sound but only does rigid-body (no eddy correction) and may have FSL matrix convention issues.

Testing: Critically Insufficient

Only 5 tests exist, none building a workflow graph. Recommended pyramid:

• Tier 1 (immediate): mock_config() + graph smoke tests for init_dwi_fit_wf and init_dwi_wf at each level
• Tier 2: Interface unit tests (RotateBVecs, ResampleDWISeries with identity transforms, transform format round-trips)
• Tier 3: Integration tests with real data (THP/Sherbrooke)

Additional Gaps Not in Roadmap

• B-vector rotation through full chain (discussed above, P0)
• dwi_mask not resampled in native-space workflow
• No DWI carpet plot equivalent (volume-by-voxel heatmap sorted by b-value)
• Missing workflow.__desc__ on init_dwi_wf
• Gradient table validation edge cases (b=0 threshold, unit vector check, transposition detection)

Recommended Attack Order

Phase	Items	Effort
1. Safety net	mock_config() + graph tests, trivial bug fixes (bold→dwi, transforms3d)	1–2 days
2. Correctness	LTA→ITK fix, NiFreeze validation + FLIRT fallback, SDC in resampling	3–5 days
3. Feature completeness	B-vector rotation through chain, standard-space loop, reportlet wiring, BIDS-URI	1–2 weeks
4. Quality	T2w coreg, confounds TSV, QC reportlets, HMC model auto-selection	1–2 weeks
5. Vision	Multi-run merging, denoising, CIFTI, integration tests	Ongoing

Bottom Line

The codebase is structurally sound but operationally incomplete. The three most critical gaps: (1) resampling is non-functional, (2) no safety-net tests exist, and (3) transform format inconsistency creates silent correctness bugs. Once these are closed, dMRIPrep has a minimal viable pipeline for --level minimal and a working resampling pipeline for --level resampling/full.

---

Analysis generated with NiPreps Guru agent, cross-referencing fMRIPrep's fit/transform architecture.