Augmented Reality System Lets Doctors See under Patients' Skin without the Scalpel

New technology is bringing the power of augmented reality into clinical practice. The system, called ProjectDR, allows medical images such as CT scans and MRI data to be displayed directly on a patient's body in a way that moves as the patient does.

"We wanted to create a system that would show clinicians a patient's internal anatomy within the context of the body," explained Ian Watts, a computing science graduate student and the developer of ProjectDR.

The technology includes a motion-tracking system using infrared cameras and markers on the patient's body, as well as a projector to display the images. But the really difficult part, Watts explained, is having the image track properly on the patient's body even as they shift and move. The solution: custom software written by Watts that gets all of the components working together.

Vast applications

"There are lots of applications for this technology, including in teaching, physiotherapy, laparoscopic surgery and even surgical planning," said Watts, who developed the technology with fellow graduate student Michael Fiest.

ProjectDR also has the capacity to present segmented images - for example, only the lungs or only the blood vessels - depending on what a clinician is interested in seeing.

For now, Watts is working on refining ProjectDR to improve the system's automatic calibration and to add components such as depth sensors. The next steps are testing the program's viability in a clinical setting, explained Pierre Boulanger, professor in the Department of Computing Science.

Next steps

"Soon, we'll deploy ProjectDR in an operating room in a surgical simulation laboratory to test the pros and cons in real-life surgical applications," said Boulanger. "We are also doing pilot studies to test the usability of the system for teaching chiropractic and physical therapy procedures." added Greg Kawchuk, a co-supervisor on the project from the Faculty of Rehabilitation Medicine. Once these pilot studies are complete, the research team expects the deployment of the system in real surgical pilot studies will quickly follow.

Watts is co-supervised by Boulanger, Cisco Chair in Healthcare Solutions and professor in the Faculty of Science, and by Kawchuk, professor in the Faculty of Rehabilitation Medicine.

ProjectDR was presented last November at the Virtual Reality Software and Technology Symposium in Gothenburg, Sweden.

Most Popular Now

Stanford Medicine Study Suggests Physici…

Artificial intelligence-powered chatbots are getting pretty good at diagnosing some diseases, even when they are complex. But how do chatbots do when guiding treatment and care after the diagnosis? For...

OmicsFootPrint: Mayo Clinic's AI To…

Mayo Clinic researchers have pioneered an artificial intelligence (AI) tool, called OmicsFootPrint, that helps convert vast amounts of complex biological data into two-dimensional circular images. The details of the tool...

Testing AI with AI: Ensuring Effective A…

Using a pioneering artificial intelligence platform, Flinders University researchers have assessed whether a cardiac AI tool recently trialled in South Australian hospitals actually has the potential to assist doctors and...

AI Accelerates the Search for New Tuberc…

Tuberculosis is a serious global health threat that infected more than 10 million people in 2022. Spread through the air and into the lungs, the pathogen that causes "TB" can...

Students Around the World Find ChatGPT U…

An international survey study involving more than 23,000 higher education students reveals trends in how they use and experience ChatGPT, highlighting both positive perceptions and awareness of the AI chatbot’s...

Adults don't Trust Health Care to U…

A study finds that 65.8% of adults surveyed had low trust in their health care system to use artificial intelligence responsibly and 57.7% had low trust in their health care...

How AI Bias Shapes Everything from Hirin…

Generative AI tools like ChatGPT, DeepSeek, Google's Gemini and Microsoft’s Copilot are transforming industries at a rapid pace. However, as these large language models become less expensive and more widely...

AI Unlocks Genetic Clues to Personalize …

A groundbreaking study led by USC Assistant Professor of Computer Science Ruishan Liu has uncovered how specific genetic mutations influence cancer treatment outcomes - insights that could help doctors tailor...

The 10 Year Health Plan: What do We Need…

Opinion Article by Piyush Mahapatra, Consultant Orthopaedic Surgeon and Chief Innovation Officer at Open Medical. There is a new ten-year plan for the NHS. It will "focus efforts on preventing, as...

Deep Learning to Increase Accessibility…

Coronary artery disease is the leading cause of death globally. One of the most common tools used to diagnose and monitor heart disease, myocardial perfusion imaging (MPI) by single photon...

People's Trust in AI Systems to Mak…

Psychologists warn that AI's perceived lack of human experience and genuine understanding may limit its acceptance to make higher-stakes moral decisions. Artificial moral advisors (AMAs) are systems based on artificial...

DMEA 2025 - Innovations, Insights and Ne…

8 - 10 April 2025, Berlin, Germany. Less than 50 days to go before DMEA 2025 opens its doors: Europe's leading event for digital health will once again bring together experts...