From 3D Model Intraoperative Navigation to Augmented-Reality Aided Surgery
by : S.C.Hiridis MD
Volumetric CT data is like a patient in the ultrasound suite, ever ready to be investigated with a 3D visualization ‘probe’. All that was needed was the evolution of computer hardware and software, to allow the imager to approach the virtual patient with real-time volumetric interactivity (computer mouse acts as the sonographic probe).
What is augmented reality?
In many ways augmented reality is analogous to virtual reality. It is a combination of virtual and graphic 3D images transmitted to a user. However, augmented reality differs from virtual reality in that it involves overlaying these graphic images onto real life, engaging the user in a semi-immersive, interactive, three dimensional environment. By superimposing virtual images, videos, or text onto real life, an experience can be heightened or even modified. Augmented reality also has applications in surgery, including in surgical training, preoperative planning,
intraoperative imaging, and enhanced visualisation. Previously, surgeons were limited to having constantly to alter their field of vision by looking away from the operating site and comparing this to a static graphic scan or representation. Augmented reality allows them to maintain a fixed field of vision on the surgical site while having graphic scans augmented over the top for reference or guidance.
Registration is fundamental to surgical navigation. it involves taking a virtual image, such as a three dimensional image of a patient’s tumour, and transferring it so that it displays the route in which to resect the tumour. The result is a registered image that combines a graphic image overlaid on a real patient. Registration must be continually monitored and adjusted in relation to subtle movements throughout a procedure. Electronic markers or infrared light emitting diodes can be used to accurately map the patient’s orientation and anatomy and send this information back to the user.1
Intraoperative Benefits of Navigated surgery:
- Patient-specific anatomy becomes visible!
- Intraoperative guidance
- Visible landmarks of the operation
- Visible vital structures to be protected
- Visible anatomical variations !!!
- Visible neovascularization and or shape-structure of tumors
It is applied in neurosurgical stereotactic operations for years. In abdominal surgery there are only a few papers published. In Strasbourg, France Prof.Marescaux and L.Soller has developed an excellent Augmented Reality Research Program. Their work has been published in JAMA in 2004 and was presented in MIRA 2008 in Rome as well. They managed to build an augmented-reality patient simulator overlayed on same patient’s anatomy during laparoscopic adrenalectomy. Augmented reality technology is in its infancy. There are still many risks, difficulties, and limitations to be assessed. Safety is of course paramount: a minor miscalculation could result in patient’s death. Augmented reality relies upon an exact registration of virtual to real life, and misalignment cannot be allowed. It is important that the surgeon retains tight control and can monitor, evaluate, and adjust factors throughout a procedure that depends on augmented reality. Ergonomics and ease of use must also be considered. Augmented reality uses a variety of monitors, cameras, and complex computing equipment, which must be of a high quality but also affordable. The relatively high costs are likely to drop with increased demand and availability.
Future developments for augmented reality may encompass the application of augmentation to other senses as well. In particular, adding and removing sound might be useful. Auditory signals could warn if a surgeon begins to stray from the augmented resection line, and this could also help in preoperative planning and the training of surgeons.
In ATHENS MEDICAL CENTER we have used such reconstructions for preoperative planning as well as intraoperative navigation after my personal proposal to Dr.K.M.Konstantinidis pioneer of robotic surgery in Greece. We do have the actual 3D models inside the operating theater and this gives us the ability to turn the models in the correct plan to aid the surgeon during anticipation of anatomical variations. This is one of the first applications in the world using augmented reality inside an operating theater for real-time guidance in abdominal robotic surgery.
REFERENCES :