MedTech · Medical imaging and surgical planning
Anatomical segmentation that took hours now takes seconds
Before every operation that relies on MRI or CT imaging, someone outlines the relevant anatomical structures by hand: bones, vessels, brain ventricles, slice by slice. theBlue.ai built AI models that do this automatically for apoQlar’s VSI HoloMedicine® platform, turning hours of manual work into seconds.
Key results
Client: apoQlar
A MedTech company in Hamburg and the creator of VSI HoloMedicine®, a platform that puts patient anatomy into extended reality headsets for 3D surgical planning and patient education.
- Industry
- MedTech
- Use case
- Anatomical segmentation for surgical planning
- AI approach
- U-Net variants and deep learning
- Imaging
- MRI and CT scans
- Platform
- Azure ML with Microsoft InnerEye
- Integration
- VSI HoloMedicine® on XR devices
In short
The step that held up every surgical model
- VSI HoloMedicine® lets surgeons see a patient’s anatomy in 3D through an XR headset, built from that patient’s own MRI and CT data.
- That only works once the anatomical structures are segmented, and radiologists and technicians traced them by hand, slice by slice, for hours per patient.
- We built U-Net models that segment the structures automatically from MRI and CT, one architecture variant per target structure.
- The work now takes seconds instead of hours, comes out consistent, and the whole pipeline is documented well enough to be reproduced on Azure.
The starting point
The challenge
The 3D surgical model could not be built until the anatomical structures were segmented, and that segmentation was hours of manual tracing per patient.
VSI HoloMedicine® puts a patient’s anatomy in front of a surgeon in 3D, through an extended reality headset, using that patient’s actual MRI and CT data. Everything the surgeon sees depends on segmented anatomical structures.
Radiologists and technicians produced those structures by tracing them slice by slice. One patient took hours.
That made segmentation the bottleneck. Throughput was limited, results varied between the people doing the tracing, and a surgical model could not be ready in time when it was needed quickly.
The build
What we built
Together with apoQlar and medical specialists, we developed AI models that segment anatomical structures from MRI and CT scans automatically.
One U-Net variant per structure
Bones, vessels and brain ventricles do not look alike in a scan, and one general model handles none of them well. We adapted the U-Net architecture separately for each target structure, so each model is optimised for the shape and contrast it has to find.
Both imaging modalities
MRI and CT differ in what they show and how they show it. The models cover both modalities across multiple structures, which means the platform is not tied to one kind of scan.
Documented well enough to reproduce
Datasets, training runs and evaluation are fully documented, and the pipelines run reproducibly on Microsoft Azure ML with Microsoft InnerEye. In a medical context that documentation is not paperwork: it is what lets someone check later how a model reached its result.
Straight into the headset
The segmentation feeds directly into VSI HoloMedicine®, so a surgeon works with the 3D model through an XR headset, Meta Quest devices among them, without a manual step in between.
What changed
The results
Before
Hours of manual slice-by-slice tracing per patient. Limited throughput, results that varied with whoever did the work, and a bottleneck in front of every surgical model.
After
Seconds of automated processing, consistent quality, pipelines documented well enough to reproduce, and the result going straight into 3D surgical planning.
Segmentation work dropped from hours to seconds, which removes the step that had been holding up surgical model generation.
The output is consistent. A model applies the same criteria to every scan, so the result no longer depends on who was available to trace it.
Because datasets, training and evaluation are documented and the pipelines are reproducible on Azure, the models can be checked, retrained and extended rather than trusted blindly.
Questions about this project
Anatomical segmentation for surgical planning. The 3D surgical model could not be built until the anatomical structures were segmented, and that segmentation was hours of manual tracing per patient.
Manual segmentation work, from hours to seconds: Hrs → sec. Models for both imaging modalities: MRI + CT. Datasets, training and evaluation fully documented: Traceable. Reproducible pipelines on Microsoft Azure: Azure ML.
U-Net variants and deep learning. Technology used: Deep learning, U-Net architecture, Medical image segmentation, MRI and CT analysis, Microsoft Azure ML, Microsoft InnerEye, Reproducible ML pipelines, ShareMedix video anonymisation.
Technology used
Talk to the people who built this
theBlue.ai comes back within one business day.
