Augmented Reality System Lets Doctors See under Patients' Skin without the Scalpel

New technology is bringing the power of augmented reality into clinical practice. The system, called ProjectDR, allows medical images such as CT scans and MRI data to be displayed directly on a patient's body in a way that moves as the patient does.

"We wanted to create a system that would show clinicians a patient's internal anatomy within the context of the body," explained Ian Watts, a computing science graduate student and the developer of ProjectDR.

The technology includes a motion-tracking system using infrared cameras and markers on the patient's body, as well as a projector to display the images. But the really difficult part, Watts explained, is having the image track properly on the patient's body even as they shift and move. The solution: custom software written by Watts that gets all of the components working together.

Vast applications

"There are lots of applications for this technology, including in teaching, physiotherapy, laparoscopic surgery and even surgical planning," said Watts, who developed the technology with fellow graduate student Michael Fiest.

ProjectDR also has the capacity to present segmented images - for example, only the lungs or only the blood vessels - depending on what a clinician is interested in seeing.

For now, Watts is working on refining ProjectDR to improve the system's automatic calibration and to add components such as depth sensors. The next steps are testing the program's viability in a clinical setting, explained Pierre Boulanger, professor in the Department of Computing Science.

Next steps

"Soon, we'll deploy ProjectDR in an operating room in a surgical simulation laboratory to test the pros and cons in real-life surgical applications," said Boulanger. "We are also doing pilot studies to test the usability of the system for teaching chiropractic and physical therapy procedures." added Greg Kawchuk, a co-supervisor on the project from the Faculty of Rehabilitation Medicine. Once these pilot studies are complete, the research team expects the deployment of the system in real surgical pilot studies will quickly follow.

Watts is co-supervised by Boulanger, Cisco Chair in Healthcare Solutions and professor in the Faculty of Science, and by Kawchuk, professor in the Faculty of Rehabilitation Medicine.

ProjectDR was presented last November at the Virtual Reality Software and Technology Symposium in Gothenburg, Sweden.

Most Popular Now

Welcome Evo, Generative AI for the Genom…

Brian Hie runs the Laboratory of Evolutionary Design at Stanford, where he works at the crossroads of artificial intelligence and biology. Not long ago, Hie pondered a provocative question: If...

Research Study Shows the Cost-Effectiven…

Earlier research showed that primary care clinicians using AI-ECG tools identified more unknown cases of a weak heart pump, also called low ejection fraction, than without AI. New study findings...

Remote Telemedicine Tool Found Highly Ac…

Collecting images of suspicious-looking skin growths and sending them off-site for specialists to analyze is as accurate in identifying skin cancers as having a dermatologist examine them in person, a...

New Study Reveals Why Organisations are …

The slow adoption of blockchain technology is partly driven by overhyped promises that often obscure the complex technological, organisational, and environmental challenges, according to research from the University of Surrey...

Almost All Leading AI Chatbots Show Sign…

Almost all leading large language models or "chatbots" show signs of mild cognitive impairment in tests widely used to spot early signs of dementia, finds a study in the Christmas...

Philips Aims to Advance Cardiac MRI Tech…

Royal Philips (NYSE: PHG, AEX: PHIA) and Mayo Clinic announced a research collaboration aimed at advancing MRI for cardiac applications. Through this investigation, Philips and Mayo Clinic will look to...

Deep Learning Model Accurately Diagnoses…

Using just one inhalation lung CT scan, a deep learning model can accurately diagnose and stage chronic obstructive pulmonary disease (COPD), according to a study published today in Radiology: Cardiothoracic...

Shape-Changing Device Helps Visually Imp…

Researchers from Imperial College London, working with the company MakeSense Technology and the charity Bravo Victor, have developed a shape-changing device called Shape that helps people with visual impairment navigate...

Bayer Acquires HiDoc Technologies and Ca…

Bayer is today announcing that it plans to acquire HiDoc Technologies GmbH in the first quarter of 2025 and to start commercialization of the digital health application, Cara Care®. Cara...

AI-Based Chatbot Created for Bioimage An…

Scientists from Universidad Carlos III de Madrid (UC3M), together with a research team from Ericsson and the KTH Royal Institute of Technology in Sweden, have developed an artificial intelligence-based software...

Analyzing Multiple Mammograms Improves B…

A new study from Washington University School of Medicine in St. Louis describes an innovative method of analyzing mammograms that significantly improves the accuracy of predicting the risk of breast...

Emotional Cognition Analysis Enables Nea…

A joint research team from the University of Canberra and Kuwait College of Science and Technology has achieved groundbreaking detection of Parkinson's disease with near-perfect accuracy, simply by analyzing brain...