Roupen Odabashian: I Think LLMs Skipped 5th Grade Math
Roupen Odabashian, Hematology/Oncology Fellow at the Karmanos Cancer Institute, shared a post on LinkedIn:
“Large language models really suck with numbers!
I’m a big fan of tools like ChatGPT and OpenEvidence for clinical reasoning and rapid research synthesis. But when it comes to interpreting numbers? It’s like watching someone try to do math with oven mitts on.
Recently, I tested a case involving testicular cancer. AFP level? 650 ng/mL. OpenEvidence confidently responded with staging criteria for levels greater than 10,000, leading to the wrong conclusion. ChatGPT made the same mistake.
Another time, I asked about the prevalence of ovarian cancer in BRCA2 carriers. It confidently told me that 25 percent is not between 20 and 40 percent.
I’m starting to think LLMs skipped 5th grade math.
Has anyone else experienced numerical errors like this?
How are you dealing with them in clinical workflows or research outputs?
Do you see this improving with newer models?”
More Posts Featuring Roupen Odabashian.
-
Challenging the Status Quo in Colorectal Cancer 2024
December 6-8, 2024
-
ESMO 2024 Congress
September 13-17, 2024
-
ASCO Annual Meeting
May 30 - June 4, 2024
-
Yvonne Award 2024
May 31, 2024
-
OncoThon 2024, Online
Feb. 15, 2024
-
Global Summit on War & Cancer 2023, Online
Dec. 14-16, 2023