AR in Surgery - Hololens 2|
Despite advancements in technology, surgery still heavily relies on the expertise of the surgeon. Conventional navigation systems in neurosurgery are constrained by their significant space and time requirements. However, leveraging intuitive and space-saving solutions through AR technology could potentially address this issue.
The objective of this project is to create a modular navigation software tailored for use at the University Hospital Leipzig. This development process involves direct collaboration with doctors and medical technicians to ensure alignment with their needs and expertise.
Technologies
- Unity
- C#
- MRTK
- NodeJS
Challenges
Because the Microsoft Hololens 2 being a rather unpopular and expensive platform, the resources for developing complex software are limited. Together with complicated documentation and an unusual setup, this lead to a challenging development experience.
In terms of content, the biggest challenges were an accurate relationship between real world structures and virtual objects, a real-time tracking of instruments and a master-follower-mode to visualize the same experience on multiple devices. All this needed to be implemented while keeping the design as simple as possible to allow untrained users to find their way around on an unfamiliar platform.
Communication with medical professionals turned out to be challenging as well, because of different technical terms and understandings of complexity.
Results
The development process resulted in an intuitive and modular AR navigation system for the University Hospital Leipzig. The system supports localization of real world structures in the patient's body through a landmark based registration process. Any surgical instrument can be tracked in real-time by simply attaching markers and setting them up in the system. After importing standardized DICOM data, for example from MRI scans, the instrument position inside the patient's body can be visualized.
Various extensions like a spatial drawing module to draw on body structures can be simply added to the system.
Multiple precision studies confirmed the feasibility of surgical navigation with the Hololens 2 and further test the accuracy.
Screenshots
