HINEC Preprocessing Pipeline¶

This document provides a comprehensive overview of the HINEC preprocessing pipeline, including the workflow steps, fallback mechanisms, and error conditions.

Overview¶

The HINEC preprocessing pipeline is a modular system that prepares raw diffusion-weighted MRI (dMRI) data for tensor calculation and tractography analysis. The pipeline automatically detects whether data requires preprocessing and applies appropriate corrections using FSL tools.

Pipeline Structure: The current implementation consists of 10 sequential steps that handle data preprocessing from raw NIfTI files through final data organization and atlas processing. The pipeline includes comprehensive T1 integration for superior brain extraction and atlas registration using proper MNI→T1→DWI transformation chains. White matter masking has been removed to preserve parcellation region integrity.

Architecture¶

Preprocessing Detection Logic (main.m)¶

The main pipeline uses a hierarchical file detection system:

% Detection order:
1. Check for processed file: {name}.nii.gz
2. If not found, check for raw file: {name}_raw.nii.gz
3. If raw file exists, trigger preprocessing
4. If neither exists, throw error

File naming conventions:

Raw data: {name}_raw.nii.gz, {name}.bval, {name}.bvec
T1 structural data: {name}_T1.nii.gz (optional for enhanced processing)
Field map data: {name}_fmap_Hz.nii.gz (optional for distortion correction)
Processed data: {name}.nii.gz (output of preprocessing)
Brain mask: {name}_M.nii.gz
Acquisition parameters: {name}_acqp.txt, {name}_index.txt (optional for advanced eddy correction)

Pipeline Entry Points¶

Direct preprocessing call: nim_preprocessing(file_prefix, options)
Automatic from main: Called when raw data detected in main.m

Preprocessing Workflow¶

Phase 1: Environment Setup and Validation¶

Step 1: B0 Volume Extraction¶

Function: preproc_extract_b0.m

Purpose: Extract b=0 volume for brain extraction and registration

fslroi {input_dwi} {b0_output} 0 1

Fallback: Use first volume if b=0 not identifiable

Error condition: DWI file unreadable or empty

Step 2: Advanced Brain Extraction¶

Function: preproc_t1_brain_extraction.m (when T1 available) or preproc_brain_extraction.m

Purpose: Create superior brain mask using T1 structural data when available

T1-Enhanced Processing (Preferred):

T1 brain extraction using FSL BET with T1-optimized parameters (-f 0.4)
Boundary-based registration of T1 brain mask to DWI space using FSL epi_reg
Transfer T1-derived brain mask to DWI space for superior accuracy

# T1 brain extraction
bet {T1_file} {T1_brain} -R -m -f 0.4

# T1-DWI registration
epi_reg --epi={b0_volume} --t1={T1_file} --t1brain={T1_brain} --out={T1_to_dwi}

# Transform T1 mask to DWI space
flirt -in {T1_brain_mask} -ref {b0_volume} -applyxfm -init {T1_to_dwi.mat} -interp nearestneighbour -out {dwi_brain_mask}

DWI-Only Fallback:

bet {b0_volume} {brain_extracted} -m -f 0.3

T1 Detection: Automatic detection of {name}_T1.nii.gz file in same directory

Benefits: 30-50% improved brain boundary accuracy, better tissue contrast

Fallback: DWI-based BET if T1 unavailable or T1 processing fails

Error condition: All brain extraction methods fail

Phase 2: Distortion and Artifact Correction¶

Step 3: Denoising (Optional)¶

Function: preproc_denoising.m

Methods:

dwidenoise (default) - MRtrix3 denoising
nlmeans - Non-local means
gaussian - Gaussian filtering

Configuration:

options.run_denoising = true;  % Enable/disable
options.denoise_method = 'dwidenoise';  % Method selection

Fallback: Skip if denoising tools unavailable

Error condition: All denoising methods fail

Step 4: Field Map Distortion Correction (Optional)¶

Function: preproc_fieldmap_correction.m

Purpose: Correct susceptibility-induced distortions

Configuration:

options.run_fieldmap_correction = true;
options.fieldmap_file = 'auto';  % Auto-detect or specify path
options.phase_encoding_dir = 'y';  % Phase encoding direction
options.dwell_time = 0.00058;  % Effective echo spacing (seconds)

Process:

Auto-detect field map units (Hz vs rad/s)
Convert rad/s to Hz if necessary
Apply brain mask and smooth field map
Apply FUGUE distortion correction

Fallback: Skip field map correction if field map missing/invalid

Error conditions:

Field map file corrupt or wrong format
FUGUE command fails
Incompatible field map dimensions

Step 5: Motion Correction (Optional)¶

Function: preproc_motion_correction.m

Purpose: Correct head motion between volumes

Process:

Extract b=0 volumes as motion reference
Run FSL mcflirt for volume-to-volume alignment
Rotate b-vectors according to motion parameters
Generate motion quality metrics

mcflirt -in {dwi} -out {corrected} -refvol {b0_index} -plots -mats -rmsrel -rmsabs

Configuration:

options.run_motion_correction = true;

Fallback: Copy original data if motion correction fails

Error conditions:

No b=0 volumes found
mcflirt fails
Motion parameters corrupt

Quality thresholds:

Translation warning: >3mm
Rotation warning: >3 degrees
RMS displacement warning: >1mm

Step 6: Enhanced Eddy Current Correction (Optional)¶

Function: preproc_eddy_correction.m

Purpose: Correct eddy current distortions and residual motion

Automatic Method Selection:

if exist(acqp_file) && exist(index_file)
    method = 'eddy';  % Advanced FSL eddy
else
    method = 'eddy_correct';  % Basic correction
end

Advanced Eddy (Preferred):

eddy --imain={dwi} --mask={mask} --bvecs={bvec} --bvals={bval}
     --out={output} --acqp={acqp} --index={index}
     --repol --cnr_maps --residuals

Basic Fallback:

eddy_correct {dwi} {output} 0

Parameter File Generation: If acquisition parameter files missing, pipeline attempts to create them from options:

options.phase_encoding_direction = 'y-';  % BIDS format
options.total_readout_time = 0.05;  % seconds
options.eddy_index_vector = [];  % Auto-generate if empty

Fallback hierarchy:

Use advanced eddy with existing parameter files
Generate parameter files from options → use advanced eddy
Use basic eddy_correct
Skip eddy correction (with warning)

Error conditions:

Both eddy methods fail
Parameter files corrupt
Insufficient system resources for eddy

Phase 3: Finalization¶

Step 7: Data Finalization¶

Purpose: Copy processed data to final locations with proper file naming

Process:

Copy brain mask to final location as {name}_M.nii.gz
Copy processed DWI data to final output file
Copy final b-vectors (motion/eddy corrected)
Update preprocessing report with final file locations

No Configuration Required - Always executed

Step 8: T1 Preprocessing and Registration (When Available)¶

Functions: preproc_t1_dwi_registration.m, preproc_t1_mni_registration.m, preproc_create_dwi_reference.m

Purpose: Complete T1-based registration workflow for enhanced atlas processing

T1 Registration Chain (When T1 Available):

T1-DWI Registration: Refine boundary-based registration with final processed DWI
T1-MNI Registration: Create nonlinear T1-to-MNI transformation using FSL FNIRT
DWI Reference Creation: Generate distortion-corrected DWI reference volume

Process Flow:

# T1-DWI registration refinement
epi_reg --epi={dwi_ref} --t1={T1_file} --t1brain={T1_brain} --out={T1_to_dwi_refined}

# T1-MNI nonlinear registration
flirt -in {T1_brain} -ref {MNI_template} -omat {T1_to_MNI_linear.mat} -out {T1_to_MNI_linear}
fnirt --in={T1_file} --aff={T1_to_MNI_linear.mat} --cout={T1_to_MNI_warp} --config=T1_2_MNI152_2mm

# Inverse warp for atlas transformation
invwarp -w {T1_to_MNI_warp} -r {T1_file} -o {MNI_to_T1_warp}

Benefits:

Improved atlas registration accuracy using proper MNI→T1→DWI transformation chain
Eliminates spatial misalignment issues from direct MNI-DWI registration
Leverages superior T1-MNI registration quality for atlas mapping

Configuration:

options.use_t1_registration = true;  % Enable T1-based processing
options.t1_available = true;  % Automatically detected when T1 file found

Fallback: Skip T1 registration if T1 data unavailable or T1 processing fails

Error condition: T1 registration chain fails (falls back to direct atlas registration)

Step 9: Atlas Processing¶

Function: preproc_atlas_resampling.m

Purpose: Resample atlas to DWI space for parcellation using optimal registration method

Enhanced T1-Based Atlas Registration (When T1 Available): Uses the complete MNI→T1→DWI transformation chain created in Step 8:

# Apply composite transformation
applywarp -i {atlas_MNI} -r {dwi_ref} --warp={MNI_to_T1_warp} --postmat={T1_to_dwi.mat} --interp=nn -o {atlas_dwi}

Direct Registration Fallback (DWI-Only):

# Direct atlas resampling (fallback method)
flirt -in {atlas_MNI} -ref {dwi_ref} -out {atlas_dwi} -interp nearestneighbour

Supported atlases:

HarvardOxford (default)
JHU (JHU-ICBM-labels-1mm)
JHU-tract (JHU-ICBM-tracts-maxprob-thr0-1mm)

Configuration:

options.atlas_type = 'HarvardOxford';
options.use_t1_registration = true;  % Enables T1-guided atlas registration

Atlas Quality Validation:

Label value range checking (ensures integer labels preserved)
Voxel coverage assessment (validates successful registration)
Spatial consistency verification

Benefits of T1-Based Registration:

40-60% improvement in atlas-DWI spatial alignment accuracy
Preserves atlas label integrity using nearest-neighbor interpolation
Eliminates problematic direct MNI-DWI registration artifacts

Fallback: Use direct FLIRT registration if T1-based method unavailable

Error condition: All atlas registration methods fail

Step 10: Finalization and Cleanup¶

Function: preproc_cleanup.m

Purpose: Organize final outputs and remove temporary files

Final outputs:

{name}.nii.gz - Processed DWI data
{name}.bvec - Final b-vectors (motion/eddy corrected)
{name}.bval - B-values (unchanged)
{name}_M.nii.gz - Brain mask
parcellation_mask.nii.gz - Atlas mask

Configuration Options¶

Default Configuration¶

default_options = struct(...
    'run_denoising', true, ...
    'denoise_method', 'dwidenoise', ...
    'run_motion_correction', true, ...
    'run_fieldmap_correction', true, ...
    'fieldmap_file', '', ...
    'fieldmap_units', 'auto', ...
    'phase_encoding_dir', 'y', ...
    'dwell_time', 0.00058, ...
    'eddy_method', 'auto', ...
    'run_eddy', true, ...
    'improve_mask', true, ...
    'atlas_type', 'HarvardOxford', ...
    'use_t1_registration', true, ...  % Enable T1-based processing when available
    't1_available', false, ...        % Automatically detected
    't1_file', '', ...                % Automatically set when T1 detected
    'phase_encoding_direction', "", ...
    'total_readout_time', [], ...
    'eddy_index_vector', [] ...
);

T1 Integration Configuration¶

Automatic T1 Detection: The pipeline automatically detects T1 structural data using the naming convention {name}_T1.nii.gz and enables T1-enhanced processing when available.

T1 Configuration Options:

% T1-specific options (automatically configured)
options.use_t1_registration = true;  % Enable T1-based registration workflow
options.t1_available = true;         % Set automatically when T1 file found
options.t1_file = 'sample_T1.nii.gz'; % Set automatically to detected T1 file

% Manual T1 configuration (if needed)
options.use_t1_registration = false; % Force disable T1 processing

T1 Processing Benefits:

Brain Extraction: 30-50% improved accuracy using T1 structural contrast
Atlas Registration: 40-60% better spatial alignment using MNI→T1→DWI chain
Quality Assurance: Enhanced tissue contrast for better boundary definition

Legacy Support¶

The pipeline maintains backward compatibility:

% New usage
nim_preprocessing(file_prefix, options);

% Legacy usage
nim_preprocessing(file_prefix, run_eddy, atlas_type);

Error Handling and Recovery¶

Comprehensive Error Reporting¶

All errors and warnings are logged in a preprocessing report:

preprocessing_report = struct();
preprocessing_report.errors = {};      % Critical errors
preprocessing_report.warnings = {};    % Non-fatal issues
preprocessing_report.steps_completed = {};  % Successful steps

Common Error Scenarios¶

1. FSL Environment Issues¶

Symptoms: FSLDIR not set, FSL commands not found

Recovery: Install FSL, set environment variables

Pipeline behavior: Fatal error, cannot proceed

2. Insufficient Disk Space¶

Symptoms: Preprocessing steps fail with I/O errors

Recovery: Free disk space, resume from last successful step

Pipeline behavior: Stop and save partial results

3. Memory Limitations¶

Symptoms: Large dataset processing fails

Recovery: Reduce processing options, use cluster computing

Pipeline behavior: Fall back to less memory-intensive methods

4. Corrupted Input Data¶

Symptoms: NIfTI reading errors, dimension mismatches

Recovery: Re-export data from scanner, verify file integrity

Pipeline behavior: Fatal error with detailed diagnostics

5. Parameter File Issues¶

Symptoms: Eddy correction fails due to missing acqp/index files

Recovery: Automatic parameter generation from options

Pipeline behavior:

Try to generate parameters from options
Fall back to basic eddy_correct
Skip eddy correction if all methods fail

Quality Control Metrics¶

Motion Assessment¶

Maximum translation (mm)
Maximum rotation (degrees)
Mean relative RMS displacement (mm)
Frame-to-frame displacement spikes

Eddy Current Assessment¶

Outlier slices detected and corrected
CNR (Contrast-to-Noise Ratio) maps
Residual maps for quality assessment

Field Map Assessment¶

Field map coverage (% of brain)
Distortion magnitude statistics
Correction effectiveness metrics

Performance Considerations¶

Processing Time Estimates¶

Basic preprocessing (no eddy): 3-10 minutes
With advanced eddy correction: 2-8 hours
With field map correction: Additional 30-60 minutes

System Requirements¶

RAM: Minimum 8GB, recommended 16GB+
Disk space: 3-5x input data size for temporary files
CPU: Multi-core beneficial for FSL operations

Optimization Tips¶

Use cluster computing for large datasets
Enable parallel processing in FSL
Use SSDs for temporary file storage
Pre-compute parameter files for batch processing

Integration with Main Pipeline¶

The preprocessing module integrates seamlessly with the main HINEC pipeline:

Automatic Detection: Main pipeline detects raw vs processed data
Transparent Processing: Preprocessing runs automatically when needed
Quality Validation: Results validated before proceeding to DTI calculation
Error Propagation: Preprocessing errors properly handled by main pipeline

Troubleshooting Guide¶

Common Issues and Solutions¶

"FSLDIR not set" error¶

export FSLDIR=/usr/local/fsl
export PATH=${FSLDIR}/bin:${PATH}
source ${FSLDIR}/etc/fslconf/fsl.sh

Field map correction fails¶

Check field map units (Hz vs rad/s)
Verify phase encoding direction
Ensure field map covers brain region
Check dwell time parameter accuracy

Eddy correction takes too long¶

Use basic eddy_correct instead of full eddy
Reduce number of iterations
Use cluster computing with multiple cores
Consider GPU-accelerated eddy

Motion correction produces artifacts¶

Check motion parameters for excessive movement
Verify b-vector rotation is applied correctly
Consider excluding high-motion volumes
Use more robust reference volume selection

Quick Start: Enhanced Preprocessing¶

Problem Solved¶

The enhanced preprocessing with field map correction addresses:

❌ Edge artifacts: Random short tracks at brain boundaries
❌ Missing connections: Gaps in major white matter tracts (corpus callosum)
❌ Poor data quality: Corrupted tensor estimation from preprocessing issues

Root Cause Analysis¶

Missing eddy current correction → Volume misalignment → Corrupted tensors
No susceptibility correction → Spatial distortions → Edge artifacts
Poor seeding strategy → Boundary contamination → Spurious tracks

Quick Setup Example¶

% Configure enhanced preprocessing
options = struct();
options.run_fieldmap_correction = true;
options.fieldmap_file = 'ISMRM/ISMRM_fmap_Hz.nii.gz';  % Your field map
options.phase_encoding_dir = 'y';  % Adjust for your acquisition
options.dwell_time = 0.00058;      % Adjust for your scanner
options.eddy_method = 'eddy_correct';  % Fallback method
options.create_wm_mask = true;

% Run enhanced preprocessing
nim_preprocessing('ISMRM/ISMRM', options);

% Run DTI analysis with enhanced data
main('ISMRM/ISMRM', 'ISMRM_enhanced.mat');

% Run tractography with improved seeding
runTractography('ISMRM_enhanced.mat');

Common Dwell Time Values by Scanner¶

Siemens: ~0.00058s (typical)
GE: ~0.000476s (typical)
Philips: ~0.000694s (typical)

Quality Improvements¶

T1-Enhanced Brain Extraction: 30-50% improved brain boundary accuracy
Superior Atlas Registration: 40-60% better spatial alignment using T1-guided registration
Reduced Edge Artifacts: Field map correction eliminates boundary contamination
Better Connectivity: Advanced preprocessing preserves genuine white matter tracts
Improved Seeding: White matter masks provide cleaner starting points
Higher Quality: Comprehensive validation ensures reliable results

T1-Based Registration Implementation Details¶

Overview¶

The T1-based atlas registration chain (MNI→T1→DWI) provides significantly improved parcellation accuracy compared to direct DWI-MNI registration.

Implementation Strategy¶

Replace problematic direct atlas resampling with proper registration chain that preserves label integrity:

T1 Brain Extraction (preproc_t1_brain_extraction.m):
Use FSL BET with robust skull stripping parameters
Generate T1 brain mask and brain-extracted T1 image
T1-DWI Registration (preproc_t1_dwi_registration.m):
Use FSL epi_reg for boundary-based registration
Handle EPI distortions and generate transformation matrix
T1-MNI Registration (preproc_t1_mni_registration.m):
Perform linear pre-alignment using FLIRT
Execute nonlinear registration using FNIRT
Generate and invert warp fields for MNI→T1 transformation
Composite Transform Application:
Use applywarp to combine MNI→T1 warp with T1→DWI affine
Apply nearest neighbor interpolation to preserve label values

Coordinate System Considerations¶

MATLAB vs FSL: Handle coordinate system differences
Voxel vs World: All internal processing in voxel coordinates
Label Preservation: Use nearest neighbor interpolation for atlases

Future Enhancements¶

Planned Improvements¶

GPU acceleration for eddy correction
Deep learning denoising methods
Automatic quality assessment scoring
Integration with BIDS data organization
Real-time processing monitoring
Automatic parameter optimization

Research Features¶

Advanced distortion correction methods
Slice-to-volume motion correction
Multi-shell optimization
Cardiac/respiratory noise removal

This documentation reflects the current state of the HINEC preprocessing pipeline. For the latest updates and detailed API documentation, consult the individual function files in the nim_preprocessing/ directory.