Publication of the NeuroLens paper in IJCARS
We are pleased to announce the publication of our latest paper, “NeuroLens: Organ Localization Using Natural Language Commands for Anatomical Recognition in Surgical Training,” in the International Journal of Computer Assisted Radiology and Surgery.
In this work, we present NeuroLens, a system that supports surgical training, specifically anatomical understanding, through organ localization in surgical videos using natural language. The system makes use of a multimodal transformer to process surgical videos alongside spoken or written language descriptions, enabling it to accurately detect and localize anatomical structures. For speech processing, we utilize large language models.
The system was evaluated in a preliminary study with clinicians, who highlighted its intuitiveness, responsiveness, and overall usability. Future work will expand on this by incorporating 3D visualization and conducting a broader evaluation with a larger student cohort to assess learning outcomes.
🔗 Paper: https://link.springer.com/article/10.1007/s11548-025-03463-5
Authors: Nevin Matasyoh, Daniel Delev, Waseem Masalha, Franziska Mathis-Ullrich, Ramy A. Zeineldin
Affiliations: FAU Erlangen-Nürnberg, University Hospital Erlangen