oir_flatmount_segmentation - Hartnett Lab - A MONAI bundle for automated rat and mouse oxygen-induced retinopathy flatmount segmentation (total retina/avascular area/intravitreal neovascularization) #778
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| MIT License | ||
|
|
||
| Copyright (c) 2026 Hartnett Lab | ||
|
|
||
| Permission is hereby granted, free of charge, to any person obtaining a copy | ||
| of this software and associated documentation files (the "Software"), to deal | ||
| in the Software without restriction, including without limitation the rights | ||
| to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||
| copies of the Software, and to permit persons to whom the Software is | ||
| furnished to do so, subject to the following conditions: | ||
|
|
||
| The above copyright notice and this permission notice shall be included in all | ||
| copies or substantial portions of the Software. | ||
|
|
||
| THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
| IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
| FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||
| AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | ||
| LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | ||
| OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||
| SOFTWARE. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,51 @@ | ||
| # OIR Flatmount Segmentation (Hartnett Lab) | ||
|
|
||
| ### **Authors** | ||
| Neal S. Shah*, Aniket Ramshekar*, Bright Asare-Bediako, Morgan P. Tankersley, Heng-Chiao Huang, Shreya Beri, Eric Kunz, Aaron Y. Lee, M. Elizabeth Hartnett | ||
|
|
||
| Byers Eye Institute Department of Ophthalmology, Stanford University School of Medicine, Stanford, CA, USA | ||
|
|
||
| ### **Tags** | ||
| Segmentation, Retinal Flatmount, Oxygen-Induced Retinopathy, OIR, Mouse, Rat, Intravitreal Neovascularization, Avascular Area | ||
|
|
||
| ## **Model Description** | ||
| This model performs automated segmentation of oxygen-induced retinopathy (OIR) retinal flatmount images into three regions: total retina (TR), intravitreal neovascularization (IVNV), and avascular area (AVA). | ||
| The architecture is a multi-task Attention U-Net with a ConvNeXt-Tiny encoder [1] and deep supervision (~8.7M trainable parameters). | ||
| For inference, the final release uses an ensemble of 5 cross-validation models, with test-time augmentation and per-class thresholding, to improve robustness across mouse and rat OIR images. | ||
|
|
||
| ## **Data** | ||
| Model development used three datasets: | ||
|
|
||
| 1. **Rat IVNV pretraining dataset:** 72 rat OIR flatmount images with IVNV-only annotations (used in intermediate Stage 2 training). | ||
| 2. **Final development dataset:** 345 annotated images total (267 mouse, 78 rat), including: | ||
| - 127 expert human-annotated images (49 mouse, 78 rat) | ||
| - 218 curated open-source mouse images [2] with reviewed masks generated from a prior published model [3] | ||
| 3. **Independent test dataset:** 37 images (18 mouse OIR, 19 rat OIR), held out from training/validation/model selection. | ||
|
|
||
| For final model development, a modified 5-fold cross-validation strategy was used, with expert-annotated images serving as fold-level validation references and curated open-source mouse images used in training only. | ||
|
|
||
| #### **Preprocessing** | ||
| Input retinal flatmount images were converted to grayscale, resized to 512×512, and intensity-normalized. | ||
| During training, joint image-mask augmentation was applied using random horizontal/vertical flips, random rotations (up to 180 degrees), brightness/contrast perturbation, CLAHE, Gaussian noise, elastic/grid/optical distortions, coarse dropout, motion blur, and random gamma adjustments. | ||
| These augmentations are implemented in the project training pipeline (`retrain_kfold_v2.py` / `train_with_split.py` using `dataset.py`). The MONAI `configs/train.json` file in this bundle is a compatibility template and keeps transform lists minimal. | ||
|
|
||
| ## **Performance** | ||
|
|
||
| Dice agreement between model masks and human consensus masks was high for total retina (TR) and AVA, and moderate for IVNV in both species: | ||
| - Rat: TR Dice=0.983, AVA Dice=0.924, IVNV Dice=0.612 | ||
| - Mouse: TR Dice=0.975, AVA Dice=0.912, IVNV Dice=0.601 | ||
|
|
||
| At the metric level, the deep learning model showed strong correlation with the mean of three graders for rat percent AVA (r=0.979) and rat percent IVNV (r=0.943). In mouse OIR, correlation was strong for percent AVA (r=0.957) but weak for percent IVNV (r=0.265), likely due to high inter-grader variability for mouse IVNV scoring. | ||
|
|
||
| (For full analysis please refer to the manuscript.) | ||
|
|
||
| ## **System Configuration** | ||
| This model was trained on an Apple M2 pro 16GB Macbook. 5-fold cross-validation was run sequentially with batch size 4 and a maximum of 120 epochs per fold. The folds ran for 86, 88, 120, 61, and 80 epochs, with total training time of approximately 24 hours. | ||
|
|
||
| ## **Additional Usage Steps** | ||
| Model checkpoints are hosted externally and linked through `large_files.yml` (not committed directly in the repo due to file size limits). | ||
|
|
||
| ## **References** | ||
| 1. Liu Z, Mao H, Wu CY, Feichtenhofer C, Darrell T, Xie S. A ConvNet for the 2020s. 2022:11966-11976. | ||
| 2. Marra KV, Chen JS, Robles-Holmes HK, et al. Development of an Open-Source Dataset of Flat-Mounted Images for the Murine Oxygen-Induced Retinopathy Model of Ischemic Retinopathy. Transl Vis Sci Technol. Dec 2 2024;13(12):4. | ||
| 3. Xiao S, Bucher F, Wu Y, et al. Fully automated, deep learning segmentation of oxygen-induced retinopathy images. JCI Insight. Dec 21 2017;2(24)doi:10.1172/jci.insight.97585 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,71 @@ | ||
| { | ||
| "bundle_root": ".", | ||
| "device": "$torch.device('cuda' if torch.cuda.is_available() else 'cpu')", | ||
| "output_dir": "$@bundle_root + '/infer_output'", | ||
| "input_dir": "$@bundle_root + '/inputs'", | ||
| "pred_threshold": 0.5, | ||
| "network_def": { | ||
| "_target_": "model_transformer.AttentionUNetTransformer", | ||
| "in_ch": 1, | ||
| "out_ch": 3, | ||
| "backbone": "convnext_tiny", | ||
| "pretrained": false | ||
| }, | ||
| "inferer": { | ||
| "_target_": "monai.inferers.SimpleInferer" | ||
| }, | ||
| "preprocessing": { | ||
| "_target_": "monai.transforms.Compose", | ||
| "transforms": [ | ||
| { | ||
| "_target_": "monai.transforms.LoadImaged", | ||
| "keys": [ | ||
| "image" | ||
| ], | ||
| "ensure_channel_first": true | ||
| }, | ||
| { | ||
| "_target_": "monai.transforms.ScaleIntensityRanged", | ||
| "keys": [ | ||
| "image" | ||
| ], | ||
| "a_min": 0, | ||
| "a_max": 255, | ||
| "b_min": -1.0, | ||
| "b_max": 1.0, | ||
| "clip": true | ||
| }, | ||
| { | ||
| "_target_": "monai.transforms.Resized", | ||
| "keys": [ | ||
| "image" | ||
| ], | ||
| "spatial_size": [ | ||
| 512, | ||
| 512 | ||
| ], | ||
| "mode": "area" | ||
| } | ||
| ] | ||
| }, | ||
| "postprocessing": { | ||
| "_target_": "monai.transforms.Compose", | ||
| "transforms": [ | ||
| { | ||
| "_target_": "monai.transforms.Activationsd", | ||
| "keys": [ | ||
| "pred" | ||
| ], | ||
| "sigmoid": true | ||
| }, | ||
| { | ||
| "_target_": "monai.transforms.AsDiscreted", | ||
| "keys": [ | ||
| "pred" | ||
| ], | ||
| "threshold": "$@pred_threshold" | ||
| } | ||
| ] | ||
| }, | ||
| "notes": "This config includes minimal standalone postprocessing (sigmoid + threshold) for bundle inference. For production results, use infer.py for 5-fold ensemble, D4 TTA, and calibrated per-class thresholds." | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,53 @@ | ||
| { | ||
| "version": "0.1.0", | ||
| "changelog": { | ||
| "0.1.0": "Initial MONAI bundle packaging for OIR flatmount segmentation." | ||
| }, | ||
| "monai_version": "1.4.0", | ||
| "pytorch_version": "2.3.0", | ||
| "numpy_version": "1.26.4", | ||
| "name": "OIR Flatmount Segmentation (Hartnett Lab)", | ||
| "task": "pathology_segmentation", | ||
| "description": "Multi-task segmentation of total retina, intravitreal neovascularization, and avascular area in OIR flatmount images.", | ||
| "authors": "Neal Shah, Aniket Ramshekar, Bright Asare-Bediako, Morgan Tankersley, Heng-Chiao Huang, Shreya Beri, Eric Kunz, Aaron Y. Lee, M. Elizabeth Hartnett", | ||
| "copyright": "Hartnett Lab", | ||
| "data_source": "Hartnett Lab OIR flatmount datasets", | ||
| "data_type": "image", | ||
| "image_classes": "retinal_flatmount", | ||
| "intended_use": "Research", | ||
| "network_data_format": { | ||
| "inputs": { | ||
| "image": { | ||
| "type": "image", | ||
| "format": "png/jpg/tif/bmp", | ||
| "dtype": "float32", | ||
| "num_channels": 1, | ||
| "spatial_shape": [ | ||
| 512, | ||
| 512 | ||
| ] | ||
| } | ||
| }, | ||
| "outputs": { | ||
| "pred": { | ||
| "type": "image", | ||
| "dtype": "float32", | ||
| "num_channels": 3, | ||
| "channels": [ | ||
| "tr", | ||
| "ivnv", | ||
| "ava" | ||
| ] | ||
|
coderabbitai[bot] marked this conversation as resolved.
|
||
| } | ||
| } | ||
| }, | ||
| "schema": "https://github.com/Project-MONAI/MONAI-extra-test-data/releases/download/0.8.1/meta_schema_20240725.json", | ||
| "optional_packages_version": { | ||
| "timm": "", | ||
|
Member
There was a problem hiding this comment. You have to specify actual versions here, so look at what you're using now and use those versions here as minimums. We should perhaps update the bundle spec to allow unversioned dependencies like this. |
||
| "albumentations": "", | ||
| "opencv-python": "", | ||
| "pandas": "", | ||
| "matplotlib": "", | ||
| "openpyxl": "" | ||
| } | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,70 @@ | ||
| { | ||
| "bundle_root": ".", | ||
| "dataset_dir": "$@bundle_root + '/data'", | ||
| "output_dir": "$@bundle_root + '/train_output'", | ||
| "device": "$torch.device('cuda' if torch.cuda.is_available() else 'cpu')", | ||
| "network_def": { | ||
| "_target_": "model_transformer.AttentionUNetTransformer", | ||
| "in_ch": 1, | ||
| "out_ch": 3, | ||
| "backbone": "convnext_tiny", | ||
| "pretrained": true | ||
| }, | ||
| "train": { | ||
| "trainer": { | ||
| "_target_": "monai.engines.SupervisedTrainer" | ||
| }, | ||
| "trainer#max_epochs": 120, | ||
| "dataset": { | ||
| "_target_": "dataset.OIRSegmentationDataset" | ||
| }, | ||
| "dataset#data": "$@dataset_dir", | ||
| "preprocessing": { | ||
| "_target_": "monai.transforms.Compose", | ||
| "transforms": [] | ||
| }, | ||
| "postprocessing": { | ||
| "_target_": "monai.transforms.Compose", | ||
| "transforms": [] | ||
| }, | ||
|
coderabbitai[bot] marked this conversation as resolved.
|
||
| "inferer": { | ||
| "_target_": "monai.inferers.SimpleInferer" | ||
| }, | ||
| "key_metric": { | ||
| "mean_dice": { | ||
| "_target_": "monai.metrics.DiceMetric", | ||
| "include_background": true | ||
| } | ||
| }, | ||
| "handlers": [] | ||
| }, | ||
| "validate": { | ||
| "evaluator": { | ||
| "_target_": "monai.engines.SupervisedEvaluator" | ||
| }, | ||
| "dataset": { | ||
| "_target_": "dataset.OIRSegmentationDataset" | ||
| }, | ||
| "dataset#data": "$@dataset_dir", | ||
| "preprocessing": { | ||
| "_target_": "monai.transforms.Compose", | ||
| "transforms": [] | ||
| }, | ||
| "postprocessing": { | ||
| "_target_": "monai.transforms.Compose", | ||
| "transforms": [] | ||
| }, | ||
| "inferer": { | ||
| "_target_": "monai.inferers.SimpleInferer" | ||
| }, | ||
| "key_metric": { | ||
| "mean_dice": { | ||
| "_target_": "monai.metrics.DiceMetric", | ||
| "include_background": true | ||
| } | ||
| }, | ||
| "handlers": [] | ||
| }, | ||
| "val_interval": 1, | ||
| "notes": "Primary training is executed via retrain_kfold_v2.py and train_with_split.py in the parent project. This file exists to satisfy MONAI preferred train-config keys. Override dataset_dir to your local dataset path when running training. Augmentation and dataset-specific preprocessing are implemented in the external training pipeline, so transform lists in this template config are intentionally minimal." | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| fold,epochs_ran,best_val_dice,last_val_dice_retina,last_val_dice_nv,last_val_dice_vo,thr_retina,thr_nv,thr_vo | ||
| 0,86,0.803034,0.954282,0.455693,0.872625,0.3,0.95,0.85 | ||
| 1,88,0.830743,0.974396,0.569426,0.907811,0.8,0.6,0.65 | ||
| 2,120,0.825497,0.966939,0.591886,0.916625,0.75,0.65,0.85 | ||
| 3,61,0.852035,0.980751,0.545412,0.932191,0.35,0.95,0.5 | ||
| 4,80,0.815581,0.977167,0.534101,0.897727,0.9,0.75,0.4 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,93 @@ | ||
| # OIR Flatmount Segmentation (Hartnett Lab) | ||
|
|
||
| ## Overview | ||
|
|
||
| This bundle provides automated segmentation of oxygen-induced retinopathy (OIR) flatmount images for: | ||
|
|
||
| - Total Retina (TR) | ||
| - Intravitreal Neovascularization (IVNV) | ||
| - Avascular Area (AVA) | ||
|
|
||
| The model is a multi-task Attention U-Net with a ConvNeXt-Tiny encoder and deep supervision, trained with fold-wise threshold calibration and ensemble inference. | ||
|
|
||
| ## Intended Use | ||
|
|
||
| - Research use in preclinical retinal OIR image analysis. | ||
| - Automated quantification support for TR, IVNV, and AVA area measurements. | ||
| - Batch processing workflows for publication and reproducible analytics. | ||
|
|
||
| ## Not Intended For | ||
|
|
||
| - Clinical diagnosis, triage, or autonomous treatment decisions. | ||
| - Use outside domain conditions (non-flatmount, unrelated species, unrelated disease context) without additional validation. | ||
|
|
||
| ## Input and Output | ||
|
|
||
| - **Input**: Single-channel retinal flatmount image. RGB inputs are converted to grayscale. | ||
| - **Output**: | ||
| - Binary masks for TR, IVNV, AVA | ||
| - Overlay visualizations | ||
| - Per-image quantitative metrics (areas and percentages) | ||
|
|
||
| ## Training Summary | ||
|
|
||
| - Data split strategy: | ||
| - Deduplicated by basename to avoid `.jpg/.tif` leakage. | ||
| - HQ expert annotations reserved for validation folds. | ||
| - Auto-generated images included only in training. | ||
| - Loss: | ||
| - BCE + Dice (all channels) | ||
| - Focal Tversky (TR/IVNV/AVA channel-weighted) | ||
| - Boundary Dice with Sobel gradients | ||
| - Deep supervision loss at decoder intermediates | ||
| - Optimization: | ||
| - AdamW + warmup cosine LR | ||
| - Discriminative learning rates (backbone vs decoder/heads) | ||
| - EMA weights | ||
| - Gradient clipping | ||
| - Early stopping | ||
| - Strong augmentation | ||
|
|
||
| ## Inference Summary | ||
|
|
||
| - Ensemble over fold checkpoints (`fold_*/best.pth`) | ||
| - D4 test-time augmentation (rotations and flips) | ||
| - Threshold calibration and post-processing: | ||
| - mask binarization by per-class threshold | ||
| - retina-constrained IVNV/AVA | ||
| - optional component filtering and morphological closing | ||
|
|
||
| ## Reproducibility Artifacts | ||
|
|
||
| Each training fold is expected to emit: | ||
|
|
||
| - `training_history.csv` | ||
| - `learning_curves.png` | ||
| - `dataset_log.xlsx` | ||
| - `run_manifest.json` | ||
| - `best.pth`, `final.pth`, `thresholds.json` | ||
|
|
||
| ## Authors | ||
|
|
||
| Neal Shah1*, Aniket Ramshekar1*, Bright Asare-Bediako1, Morgan Tankersley1, Heng-Chiao Huang1,2, Shreya Beri1, Eric Kunz3, Aaron Y. Lee4, M. Elizabeth Hartnett1,# | ||
|
|
||
| 1 Byers Eye Institute Department of Ophthalmology, Stanford University School of Medicine, Stanford, California, USA | ||
| 2 Department of Ophthalmology, Chang Gung Memorial Hospital, Chiayi, Taiwan | ||
| 3 John A. Moran Eye Center, University of Utah, Salt Lake City, Utah, USA | ||
| 4 John F. Hardesty Department of Ophthalmology and Visual Sciences, Washington University in St. Louis, St. Louis, Missouri, USA | ||
|
|
||
| ## Contacts | ||
|
|
||
| - Neal Shah: neals1@stanford.edu | ||
| - Aniket Ramshekar: aniket.ramshekar@stanford.edu | ||
| - M. Elizabeth Hartnett: me.hartnett@stanford.edu | ||
|
|
||
| ## Citation | ||
|
|
||
| If you use this model, please cite the associated TVST publication (to be updated after acceptance) and acknowledge the Hartnett Lab. | ||
|
|
||
| ## Known Limitations | ||
|
|
||
| - Performance may degrade for out-of-distribution scanners/prep protocols. | ||
| - Small IVNV lesions are sensitive to threshold and component filtering settings. | ||
| - Cross-species domain shift (mouse vs rat) should be evaluated explicitly per cohort. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| Data licensing and access notes for `oir_flatmount_segmentation`. | ||
|
|
||
| 1. Data distribution: | ||
| - No training or evaluation images are distributed in this bundle. | ||
| - This bundle contains model code/configuration and references to externally hosted model weights only. | ||
|
|
||
| 2. Source datasets: | ||
| - Model development used retinal flatmount datasets from Hartnett Lab studies and curated open-source data. | ||
| - Each source dataset remains subject to its original license/terms of use. | ||
|
|
||
| 3. User responsibilities: | ||
| - Users are responsible for obtaining lawful access to any data used with this model. | ||
| - Users must comply with all applicable data use agreements, institutional policies, and local regulations. | ||
|
|
||
| 4. Privacy and ethics: | ||
| - Do not use data in ways that violate privacy, consent, or ethics approvals. | ||
|
|
||
| 5. Attribution: | ||
| - Please cite the associated manuscript and acknowledge the Hartnett Lab when using this model. | ||
|
|
||
| 6. Contact: | ||
| - neals1@stanford.edu | ||
| - aniket.ramshekar@stanford.edu | ||
| - me.hartnett@stanford.edu |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This field is meant to indicate what format the content of the image is, in this case I think it's just some magnitude value from 0 (rather than values in Hunsfield units or RGB).