Large Language Models (LLMs) like ChatGPT can handle two aspects of communication very well: plausibility and fluency. Given an input context they determine what are the most probable sequence of words and string them in a way that is superbly eloquent. That makes the output very convincing.
But it’s no secret that LLMs can provide entirely false outputs – they can confabulate. Not hallucinate (as it’s gotten popular to say in mainstream descriptions), confabulate. I first heard this distinction in Geoffrey Hinton’s 2023 lecture at MIT (19:02 mark). The difference is that hallucination involves a sensory perception, implying that the person hallucinating is consciously aware of something that is not there in reality. Confabulation, on the other hand, is a false reconstruction of information. It doesn’t imply a subjective experience.
Recently I came across three simple examples of LLM confabulations that seemed simple at first but startling after a moment of thought.
- Fake book: Prof. Melanie Mitchell’s Feb 2024 talk (43:40 mark) had the example of asking ChatGPT to list four books written by her. The LLM’s answer had four books, one of which doesn’t exist in the real world. The LLM made up the fictional book because it sounded like something she could have written.
- Fake degree: In a December 2023 lecture Prof. Michael Wooldridge of Oxford explains (33:03 mark) what happened when he asked ChatGPT to describe who Michael Wooldridge is. The LLM came back with a convincing biography that had most things right but made up the fact that he got his undergraduate degree at Cambridge (he didn’t). The statistical probability that an Oxford professor got their undergraduate degree at Cambridge is high enough that the LLM produced that, irrespective of the facts.
- Fake URLs: Stanford’s linguistics Professor Christopher Potts in his Spring 2023 lecture (1:03:50 mark) describes what happened when he asked the LLMs to “provide some links to the evidence” for an answer. The LLM produced real-sounding https:// links that didn’t exist in the real world.
All three stand out in my mind because they are the kind usual information retrieval that all of us are used to. Intuition allows us to maybe expect some missing information in such search results (like perhaps one or two of the books from an author can be missing in a search result). But would you expect complete fabrications? I wouldn’t.
This feels alarming because it goes against the online instinct that is ingrained in web users today: plenty of us think we can spot misinformation because we are tuned to the implicit cues (sketchy website, incomplete content, bad grammar, etc.). But what if no aberration like that exists and the information looks authoritative… yet has a concocted aspect that is perfectly intertwined with truth?
Well-placed false information, congruently embedded in a much bigger accurate content will be near-impossible to spot, specially for complex topics (like health!). Premature use of LLM superpower that can wreak havoc in health and wellness domains. I’m curious how healthcare-focused companies like Hippocartic, Abridge, GenHealthAI are mitigating the risks involved.