
Perry Wilson: AI in Medicine is suffering from what I would like to call “The Cassandra Problem”
Quoting Perry Wilson, Director, Clinical and Translational Research Accelerator at Yale University, on X/Twitter:
“AI in Medicine is suffering from what I would like to call ‘The Cassandra Problem’.
I am thinking about this issue this week thanks to this study, appearing in JAMA Network Open, which examines the utility of a model that predicts blood clots in hospitalized kids.
The model takes in a bunch of data from the electronic health record and makes what turns out to be a pretty accurate prediction about the risk of blood clots (AUC~0.90).
But accuracy is just table stakes for machine learning models. It’s not enough. Enter Cassandra.
Cassandra was a priestess of Troy and daughter of King Priam. Apollo had given her the gift of perfect prophecy – everything she foresaw would come to pass. But she was cursed that no one would ever believe her.
She told her brother Paris, for example, not to go to Sparta to abduct Helen. Doing so, she said, would lead to the fall of Troy. He ignored her, of course, and you know the rest of the story.
The point is – even a perfect prediction is useless if no one does anything about it. Accuracy is table stakes for ML in Medicine. It is not enough. The JAMA Network Open study does the right thing, randomizing the use of the model vs. usual care.
Kids were randomized to use the model or not. In the intervention group, model-predicted high risk kids would lead to a hematology consult to discuss anticoagulation with the primary team.
And the model was good! Of 135 kids who developed blood clots, the model had flagged 121 in advance. So, clearly, the use of the model would reduce blood clots, right? Wrong. Blood clot rate did not significantly differ between the usual care and intervention group. Why?
Well, let’s look at the 77 kids in the intervention group that developed blood clots, because they tell us the story of why accurate models fail in clinical practice. Of those 77, the model failed to flag 6. That’s issue #1: accuracy. Table stakes.
Of the remaining 71, a bunch got flagged on the weekend when the hematology team wasn’t around to provide recs to the treatment team. Issue #2: good predictions don’t work if you don’t tell people about it.
A bunch of the kids who were predicted to be at risk of clot (and got a clot!) had contraindications to anticoagulation. Issue #3: good predictions don’t work if you can’t do anything about it.
Of the 16 remaining kids for whom heme recommended anticoagulation, only 7 GOT anticoagulation. Issue #4: good predictions don’t work if people CHOOSE not to do anything about it.
This is the gulf that AI models need to bridge now. The arms race of higher accuracy is an illusion. We need to move to implementation.
So the next time an AI company with a slick slide deck shows you how accurate their model is, ask them if accuracy is the thing that matters most. If they say “yes, of course!”. Then tell them about Cassandra.
More on this theme in my Medscape column this week.”
Source: Perry Wilson/Twitter
-
17:28 29/11JeeSuk Chang: A multi-center trial - great efforts from the KROG team
-
17:00 29/11Maite Bourlon: Access to novel drugs for metastatic renal cell carcinoma is an unmet need in the Mexican population
-
23:17 28/11Manni Mohyuddin: Current trials must have observation/surveillance as control arm
-
23:04 28/11Juan P. Alderuccio: Our approach to secondary CNS involvement by aggressive lymphomas
-
17:17 27/11Vivek Subbiah: Treatment patterns and overall survival (OS) in patients with advanced NSCLC in three countries between 2011 and 2020
-
17:43 28/11Michele Carbone: The world is full of good, honest, nice people
-
17:24 28/11Jessica Clerc: Celebrating 90 Years of Excellence and Dedication in Cancer Care!
-
17:44 25/11Carlos Pedraz: Congratulations, Dr. Rafael Rosell for the very well deserved 2023 ISLB Lifetime Achievement Award
-
17:10 25/11100 Influential Celebrities in Oncology: The 2023 Edition – Part 4
-
13:34 24/11"Bald for a Cause" initiative - UWC Dilijan College
-
17:34 28/11The International Association of Cancer Registries annual conference - IARC
-
18:05 26/11Submit your abstract by 13 February for ESMO-Breast-24 - ESMO
-
17:58 26/11Lisa A. Lacasse: My sincere gratitude to the ACS CAN team, our incredible volunteers, and supporters for another outstanding year
-
16:47 16/11How social determinants can influence a patient’s ability to stay in treatment - ASCO
-
13:22 16/11SIOP Europe Student Summer School for Paediatric Oncology
-
17:25 29/11NIH Director’s Early Independence Award supports junior scientists to launch independent careers - National Cancer Institute
-
17:07 29/11Santhosh Kumar Devadas: I’m starting a new position as a Professor of Medical Oncology and Chief of Bone Marrow Transplant at Ramaiah Memorial Hospital and Ramaiah Medical College
-
16:48 27/11CRCHD seeks motivated scientists to serve as Program Directors - NCI Disparities
-
16:25 27/11Luisa Basset completed a fruitful nine-year tenure on the CCI board of trustees - Childhood Cancer International
-
18:14 26/11Tristan Knight: Attention hematology / oncology and pediatric hematology / oncology fellows