Iyad Sultan: New study in JAMA Oncology reveals that Large Language Models (LLMs) can be misleading when it comes to cancer treatment info.

Iyad Sultan, Pediatric Oncologist at King Hussein Cancer Center, recently tweeted:

“New study in JAMA Oncol reveals that Large Language Models (LLMs) can be misleading when it comes to cancer treatment info. Researchers tested chatGPT3.5 against 2021 NCCN guidelines for breast, prostate, and lung cancer. While the chatbot made some accurate recommendations, 1/3 were at least partially non concordant with guidelines. Even board-certified oncologists had disagreements interpreting chatbot output. Bottom line: While LLMs have promise in healthcare, they’re not yet a reliable source for treatment recommendations.

1- The study’s use of zero-shot training is noteworthy; achieving a two-thirds accuracy rate for complex cancer treatment decisions right out of the box is impressive.

2- I wonder why the study didn’t employ ChatGPT-4 for the testing. A comparison with more advanced models could provide richer insights.

3- For intricate scenarios like cancer treatment recommendations, a chain-of-thought prompting method may yield more accurate responses than single prompts.

4- The study relied on text-based analysis, which led to a 40% disagreement among board-certified oncologists. Employing an MCQ format could offer a more reliable metric for assessing the model’s performance.

5- Oncologists should be aware of potential uses of LLMs by patients and provide them with enough guidance to make decisions (e.g. patient versions of the NCCN guidelines).”

Source: Iyad Sultan/Twitter