𝗡𝗲𝘄 𝗿𝗲𝘀𝗲𝗮𝗿𝗰𝗵 𝗽𝘂𝗯𝗹𝗶𝘀𝗵𝗲𝗱! Radiology reporting is one of the most time‑consuming parts of clinical practice, and spinal MRI is especially challenging: 3D anatomy, multiple sequences, and subtle findings that really matter for patient outcomes.
In our latest work, we introduce 𝐒𝐏𝐈𝐍𝐄 – 𝐒𝐞𝐠𝐦𝐞𝐧𝐭𝐚𝐭𝐢𝐨𝐧-𝐠𝐮𝐢𝐝𝐞𝐝 𝐏𝐫𝐨𝐜𝐞𝐬𝐬𝐢𝐧𝐠 𝐚𝐧𝐝 𝐈𝐧𝐭𝐞𝐠𝐫𝐚𝐭𝐢𝐨𝐧 𝐨𝐟 𝐌𝐮𝐥𝐭𝐢𝐦𝐨𝐝𝐚𝐥 𝐒𝐩𝐢𝐧𝐚𝐥 𝐌𝐑𝐈 𝐟𝐨𝐫 𝐍𝐚𝐭𝐮𝐫𝐚𝐥-𝐋𝐚𝐧𝐠𝐮𝐚𝐠𝐞 𝐄𝐧𝐡𝐚𝐧𝐜𝐞𝐝 𝐑𝐞𝐩𝐨𝐫𝐭 𝐆𝐞𝐧𝐞𝐫𝐚𝐭𝐢𝐨𝐧, a 3D vision–language framework that fuses T1, T2, and segmentation maps to generate anatomically aware spine MRI reports.
What we built
• A 3D-aware VLM pipeline that ingests full lumbar spine volumes (axial and sagittal) instead of isolated 2D slices, better reflecting how radiologists read studies.
• Segmentation-guided inputs where vertebrae, discs, canal, and key regions are encoded as additional channels (T1 + Seg, T2 + Seg, and T1 + T2 + Seg).
• A report-generation stage that pairs volumetric MRI with LLM-standardized radiology reports, ensuring consistent, structured language for training.
Key findings
• Adding segmentation consistently improved report quality across BLEU, ROUGE, METEOR, and BERTScore compared to using MRI alone.
• The T1 + T2 + Seg configuration delivered the most balanced and robust performance overall, with higher lexical precision and strong semantic alignment.
• Models trained on LLM-standardized reports outperformed those trained on raw human-written text, highlighting how linguistic consistency boosts image–text learning.
• For synthetic report generation from structured gradings, GPT‑4o produced markedly more fluent and clinically detailed narratives than Grok‑3 across all language metrics.
• Expert review of generated reports showed high scores for clarity and terminology, with clinically acceptable accuracy.
Why this matters
SPINE shows that segmentation-aware, multimodal 3D VLMs can move automated reporting beyond generic image captioning toward anatomically grounded, clinically useful narratives.
This opens the door to AI systems that support radiologists with more consistent, interpretable, and scalable spine MRI reporting—particularly valuable in settings with high workload and limited subspecialty expertise.
This work was led by Hoda Helmy, with AI Innovation Lab team members Abdullah Hosseini and Ahmed Ibrahim, in collaboration with Mr. Ahmed-Ramadan Sadek and Asfand Baig Mirza from Barking, Havering and Redbridge University Hospitals NHS Trust.
📄 Article: https://lnkd.in/dgxgtnU6
👩💻 Code: https://lnkd.in/du8Xqd4g
#AI #MedicalImaging #Radiology #SpineMRI #VisionLanguageModels #Segmentation #ReportGeneration #GenerativeAI #HealthcareAI #AppliedAI #WeillCornellMedicine #Qatar #NHS #Neurosurgery