Augmented reality tools for microsurgery

P.Milgram, M. Kim, J. Drake,
University of Toronto, Toronto, Canada


As discussed by other participants, operating microscopes and
endoscopes are effective visualisation tools for minimally invasive
surgery. In addition to making otherwise inaccessible sites accessible,
these instruments provide useful magnification beyond that afforded by
direct viewing. Whereas the loss of 3D viewing at one time presented an
obstacle to efficient task execution, this shortcoming has been
overcome through the introduction of stereoscopic (3D) displays - that
is, stereoscopic microscopes and endoscopes. In spite of vastly
improved ability to estimate relative distances
between objects with stereo displays, however, the problem of
estimating absolute sizes and distances remains, due
to problems of variable instrument magnification and absence of known
fixed frames of reference. Research will be presented on the
application to microsurgery of the ARGOS (Augmented Reality through
Graphic Overlays on Stereovideo) Virtual Tape Measure,
for making measurements of object locations within an unstructured
visual field. Using this system the surgeon is able to grab a frame of
stereo video and align a calibrated stereographic pointer with
anatomical features whose absolute locations are to be measured. In
addition, the ARGOS system can be used to generate virtual
landmarks, to provide a reference frame for direct depth
scaling, as well as  virtual navigational displays to overcome
practical axis transformation problems associated with manual