Ravi B. Parikh, Director of Human-Algorithm Collaboration Lab (HACLab) and Associate Professor at Winship Cancer Institute of Emory University, shared a post on LinkedIn:
“Latest Human Algorithm Collaboration Lab study in Nature Portfolio Nature Communications: Fewer than 10% of patients with cancer enroll in clinical trials, even though most want to. One underappreciated reason: the exhausting, error-prone process of manually combing through medical records to find eligible patients – ‘prescreening.’
Thrilled to share our new study, the first randomized trial to evaluate both the accuracy and time savings of a human-AI collaboration for trial prescreening.
We found that Human+AI teams were significantly more accurate than humans alone (76% vs. 71%), especially for biomarker identification and tumor staging, which are among the most complex and consequential criteria in trial eligibility.
But performance varied a lot by criterion. Human+AI was dramatically better for biomarker interpretation (91% vs. 81%), but for ECOG performance status, AI assistance actually hurt accuracy. The AI’s value, and its risks, are not uniform.
AI didn’t save time! review time was nearly identical in both groups (~37 min/chart). Our interpretation: staff shifted from finding information to carefully checking the AI’s work. That tradeoff may actually help prevent over-reliance on AI.
Two important lessons about human-AI collaboration: In areas where the AI was badly wrong, staff using it performed worse (automation bias). In areas where the AI was right but staff were skeptical, human+AI still underperformed AI alone (confirmation bias). Real-world human-AI teams are genuinely complex – and we need to study them that way.
The bottom line: AI doesn’t replace clinical research staff. It makes them better, in some areas more than others. But this needs to be tested, not assumed. As AI tools proliferate across healthcare, randomized evidence of what works – and where AI can quietly make things worse – is essential.
Huge thanks to co-first author Likhitha Kolla, Zeke Emanuel, + the incredible teams Penn Medicine, University of Pennsylvania Health System, Winship Cancer Institute of Emory University, and Mendel.ai for their support.”
Title: Human-AI teaming to improve accuracy and efficiency of eligibility criteria prescreening for oncology trials: a randomized evaluation trial using retrospective electronic health records
Authors: Ravi B. Parikh, Likhitha Kolla, Elizabeth A. Beothy, William J. Ferrell, Brenda Laventure, Matthew Guido, Anthony Girard, Yang Li, Khaled Essam Mahmoud Dosoky, Karim Tarabishy, Parth S. Patel, Ayana Andalcio, Kristin Maloney, Jose Ulises Mena, Wael Salloum, Jinbo Chen, Ezekiel J. Emanuel
Read The Full Article
Other articles about AI on OncoDaily.