Physicians using AI for their job is becoming more and more common. A new survey from the American Medical Association shows, that around 81% of physicians use AI in their job, an increase of over 250% in comparison to 2023. Unfortunately LLMs sound extremely persuasive and empathetic, but medicine doesn't require feeling for the language, but facts.
Why LLMs fail at diagnosingLLMs like GPT or Claude are basically just "stochastic parrots". The only thing they are able to do is calculate the probability of the next word, based on the data the model has been trained with. Exactly that is where the problem lies: LLMs are solely based on probability and don't have any logical thinking.
Differential Diagnosis are based on systematic exclusion of conditions. For DDx the significance of symptoms has to be assessed, for example through the classification between general symptoms and pathognomonic symptoms. Moreover life-threatening courses have to be excluded first and patterns have to be recognized, apart fro only statistical frequency.
This is where the main problem lies: AIs only return the most probable words. DDx are however also often used to diagnose rare diseases, and especially such are often underrepresented or not even present in the dataset. If these pieces of information are not available, the AI model just chooses the next most probable word. Especially in a medical context this can have fatal consequences: symptom-relationships that don't exist, trials, that were never done, or treatments that are not adequate.
A study from January 2026 shows this phenomenon: 22.2% of all the AI models decisions were wrong, they were however still accepted without thinking critically. In a clinical context this can result in the wrong treatment of the patient, resulting in permanent damage or even death.
What are medical ontologies and how can they solve this problem?A ontology or rather knowledge graph is a digital, structured, network, in this case of medical facts like symptoms, conditions, treatment etc. and their relationship to eachother.
An example for such a knowledge graph is for example the ICD-10 or SNOMED. The data is structured: there are node points, which are then further connected to other nodes. The diagnosis "Acute Myocardial Infarction, unspecified" (I21.9) is a good example for this arrangement, I21.9 is a diagnosis that is connected to I21 ("Acute Myocardial Infarct"), which in turn is connected to I20-25 ("Ischaemic Heart Diseases"). In this example there is no margin for probability, a connection either exists or it doesn't.
While LLMs are based on correlation, ontologies are based on causality and logic.
Symbiosis of LLMs and ontologiesThe question whether AI should be used in the medical field is long gone, the new question to ask is, how can we ensure the safe usage of AI in a clinical context.
That is where Retrieval-Augmented Generation (RAG) comes into play. RAG is based on this concept: connect an LLM to a medical Database, the ontology. In this case we don't let the AI diagnose with probability, but using our digital "medical library". A physician enters the symptoms, and the system matches them to the ontology to filter logical DDx. Now the LLM is only used to format these facts into readable language.
The advantage of such a system is, that the eloquence stays, but the risk for hallucination decreases massively, because the model doesn't generate the facts it self anymore, it just retrieves them from the database.
Conclusion and outlookIt is fair to say, that LLMs should absolutely not be used in Differential diagnosis and that they can even have lethal consequences. Only the combination of such an language model and a knowledge graph with logically linked medical information provides the basis for a real clinical assistance.
Current AIs won't replace physicians in DDx. But the physician, who in the future uses such AI applications backed by real data, will replace these who refuse to use AI.