Iyad Sultan: Language models struggle with the spatial understanding of human anatomy
Iyad Sultan, Head of the Solid Tumors Service in the Department of Pediatric Oncology at the King Hussein Cancer Center, shared a post on LinkedIn:
“Looking for collaborators & ideas.
Over the past month, I’ve been working intensively with our multi-agent language model to stage cancer cases. In doing so, I’ve run into a very specific but critical limitation: language models—including ChatGPT—struggle with the spatial understanding of human anatomy.
One recurring issue is confusion around what lies above or below the diaphragm. Despite their impressive language capabilities, these models often misclassify or mislabel organs and structures based on their anatomical location. This reflects a deeper gap in spatial reasoning when it comes to the human body.
I’m proposing an idea: to train a model—maybe GPT-4 or even a smaller, specialized one—on spatial relationships between human organs. This would involve curating a dataset that maps organs relative to one another and to key anatomical landmarks like the diaphragm.
Has anyone tackled a problem like this before? Or is anyone interested in collaborating to explore this idea further? I’d love to hear your thoughts, suggestions, or just connect with others who find this intersection of anatomy and AI compelling.
Thank you!”
More posts featuring Iyad Sultan.
-
Challenging the Status Quo in Colorectal Cancer 2024
December 6-8, 2024
-
ESMO 2024 Congress
September 13-17, 2024
-
ASCO Annual Meeting
May 30 - June 4, 2024
-
Yvonne Award 2024
May 31, 2024
-
OncoThon 2024, Online
Feb. 15, 2024
-
Global Summit on War & Cancer 2023, Online
Dec. 14-16, 2023