New Tool Overcomes Major Hurdle in Clinical AI Design

Harvard Medical School scientists and colleagues at Stanford University have developed an artificial intelligence (AI) diagnostic tool that can detect diseases on chest X-rays directly from natural-language descriptions contained in accompanying clinical reports.

The step is deemed a major advance in clinical AI design because most current AI models require laborious human annotation of vast reams of data before the labeled data are fed into the model to train it.

A report on the work, published Sept. 15 in Nature Biomedical Engineering, shows that the model, called CheXzero, performed on par with human radiologists in its ability to detect pathologies on chest X-rays.

The team has made the code for the model publicly available for other researchers.

Most AI models require labeled datasets during their "training" so they can learn to correctly identify pathologies. This process is especially burdensome for medical image-interpretation tasks since it involves large-scale annotation by human clinicians, which is often expensive and time-consuming. For instance, to label a chest X-ray dataset, expert radiologists would have to look at hundreds of thousands of X-ray images one by one and explicitly annotate each one with the conditions detected. While more recent AI models have tried to address this labeling bottlenck by learning from unlabeled data in a "pre-training" stage, they eventually require fine-tuning on labeled data to achieve high performance.

By contrast, the new model is self-supervised, in the sense that it learns more independently, without the need for hand-labeled data before or after training. The model relies solely on chest X-rays and the English-language notes found in accompanying X-ray reports.

"We’re living the early days of the next-generation medical AI models that are able to perform flexible tasks by directly learning from text," said study lead investigator Pranav Rajpurkar, assistant professor of biomedical informatics in the Blavatnik Institute at HMS. "Up until now, most AI models have relied on manual annotation of huge amounts of data - to the tune of 100,000 images - to achieve a high performance. Our method needs no such disease-specific annotations.

"With CheXzero, one can simply feed the model a chest X-ray and corresponding radiology report, and it will learn that the image and the text in the report should be considered as similar - in other words, it learns to match chest X-rays with their accompanying report," Rajpurkar added. "The model is able to eventually learn how concepts in the unstructured text correspond to visual patterns in the image."

The model was "trained" on a publicly available dataset containing more than 377,000 chest X-rays and more than 227,000 corresponding clinical notes. Its performance was then tested on two separate datasets of chest X-rays and corresponding notes collected from two different institutions, one of which was in a different country. This diversity of datasets was meant to ensure that the model performed equally well when exposed to clinical notes that may use different terminology to describe the same finding.

Upon testing, CheXzero successfully identified pathologies that were not explicitly annotated by human clinicians. It outperformed other self-supervised AI tools and performed with accuracy similar to that of human radiologists.

The approach, the researchers said, could eventually be applied to imaging modalities well beyond X-rays, including CT scans, MRIs, and echocardiograms.

"CheXzero shows that accuracy of complex medical image interpretation no longer needs to remain at the mercy of large labeled datasets," said study co-first author Ekin Tiu, an undergraduate student at Stanford and a visiting researcher at HMS. "We use chest X-rays as a driving example, but in reality CheXzero's capability is generalizable to a vast array of medical settings where unstructured data is the norm, and precisely embodies the promise of bypassing the large-scale labeling bottleneck that has plagued the field of medical machine learning."

Tiu E, Talius E, Patel P et al.
Expert-level detection of pathologies from unannotated chest X-ray images via self-supervised learning.
Nat. Biomed. Eng, 2022. doi: 10.1038/s41551-022-00936-9

Most Popular Now

MEDICA 2024 + COMPAMED 2024: Adapted Hal…

11 - 14 November 2024, Düsseldorf, Germany. The final preparations for MEDICA 2024 and COMPAMED 2024 in Düsseldorf have begun. A total of more than 5,500 exhibitors from approximately 70 countries...

AI does Not Necessarily Lead to more Eff…

The use of artificial intelligence (AI) in hospitals and patient care is steadily increasing. Especially in specialist areas with a high proportion of imaging, such as radiology, AI has long...

Commission Joins Forces with Venture Cap…

The Commission has launched a Trusted Investors Network bringing together a group of investors ready to co-invest in innovative deep-tech companies in Europe together with the EU. The Union's investment...

Why the NHS is Seeking to Make Media Ser…

Opinion Article by Dean Moody, Healthcare Services Director, Airwave Healthcare. Tim Kelsey and Martha Lane Fox called for WiFi to be made available free of charge throughout the NHS back in...

An AI-Powered Pipeline for Personalized …

Ludwig Cancer Research scientists have developed a full, start-to-finish computational pipeline that integrates multiple molecular and genetic analyses of tumors and the specific molecular targets of T cells and harnesses...

Wearable Cameras Allow AI to Detect Medi…

A team of researchers says it has developed the first wearable camera system that, with the help of artificial intelligence (AI), detects potential errors in medication delivery. In a test whose...

Philips and Medtronic Advocacy Partnersh…

Royal Philips (NYSE: PHG, AEX: PHIA), a global leader in health technology, and Medtronic Neurovascular, a leading innovator in neurovascular therapies, today announced a strategic advocacy partnership. Delivering timely stroke...

AI could Transform How Hospitals Produce…

A pilot study led by researchers at University of California San Diego School of Medicine found that advanced artificial intelligence (AI) could potentially lead to easier, faster and more efficient...

New AI Tool Predicts Protein-Protein Int…

Scientists from Cleveland Clinic and Cornell University have designed a publicly-available software and web database to break down barriers to identifying key protein-protein interactions to treat with medication. The computational tool...

Great Start for Ideas and Innovations: D…

8 - 10 April 2025, Berlin, Germany. From 15 October to 15 November 2024, the DMEA invites experts from business, science, politics and practice to actively participate in shaping the congress...

Start-Ups will Once Again Have a Starrin…

11 - 14 November 2024, Düsseldorf, Germany. The finalists in the 16th Healthcare Innovation World Cup and the 13th MEDICA START-UP COMPETITION have advanced from around 550 candidates based in 62...

AI for Real-Rime, Patient-Focused Insigh…

A picture may be worth a thousand words, but still... they both have a lot of work to do to catch up to BiomedGPT. Covered recently in the prestigious journal Nature...