OpenAI just recently took a systematic look into why models hallucinate [0][1].
The article you shared raises an interesting point by comparing human memory with LLMs, but I think the analogy can only go so far. They're too distinct to explain hallucinations simply as a lack of meta-cognition or meta-memory. These systems are more like alien minds, and allegories risk introducing imprecision when we're trying to debug and understand their behavior.
OpenAI's paper instead identifies hallucinations as a bug in training objectives and benchmarks, and is grounding the explanation in first principles and the mechanics of ML.
Metaphors are useful for creativity, but less so when it comes to debugging and understanding, especially now that the systematic view is this advanced.
Hallucinations are actually not a malfunction or any other process outside of the normal functioning of the model. They are merely an output that we find unuseful, but in all other ways is optimal based on the training data, context, and model precision and parameters being used.
I honestly have no idea why OAI felt that they needed to publish a “paper” about this, since it is blazingly obvious to anyone who understands the fundamentals of transformer inference, but here we are.
The confusion on this topic comes from calling these suboptimal outputs “hallucinations” which drags anthropomorphic fallacies into the room by their neck even though they were peacefully minding their own business down the corridor on the left.
“Hallucination” implies a fundamentally fixable error in inference, a malfunction of thought caused by a pathology or broken algorithm.
LLMs “Hallucinating” are working precisely as implemented, only we don’t feel like the output usefully matches the parameters from a human perspective. It’s just unhelpful results from the algorithm, like any other failure of training, compression, alignment, or optimisation.
Did you read TFA? It gives concrete advice on how to change the training and eval of these models in order to decrease the error rate. Sure, these being stochastic models, the rate will never reach zero, but given that they are useful today, decreasing the error rate is worthy cause. All this complaining on semantics is just noise to me. It stems from being fixated on some airy-fairy ideas of AGI/ASI, as if anything else doesn't matter. Does saying that a model "replied" to a query mean we are unwittingly anthropomorphizing them? It's all just words, we can extend their use as we see fit. I think "confabulation" would be a more fitting term, but beyond that, I'm not seeing the problem.
We can call it whatever, and yes, the answer is training- just like all things regarding the quality of LLM output per parameter count. The problem is that many people understand“hallucination” as a gross malfunction of an otherwise correctly functioning system, I.e. a defect that can/must be categorically “fixed”, not understanding that it is merely a function of trained weights, inference parameters, and prompt context that they can:
A: probably work around by prompting and properly structuring tasks
B: never completely rule out
C: not avoid at all in certain classes of data transformations where it will creep in in subtle ways and corrupt the data
D: not intrinsically detect, since it lacks the human characteristic of “woah, this is trippy, I feel like maybe I’m hallucinating “
These misconceptions stem from the fact that in LLM parlance, “hallucination” is often conflated with a same-named, relatable human condition that is largely considered completely discrete from normal conscious thought and workflows.
Words and their meanings matter, and the failure to properly label things often is at the root of significant wastes of time and effort. Semantics are the point of language.
I haven't read the full paper yet, but my intuition is that hallucinations are a byproduct of models having too much information that needs to be compressed for generalizing.
We already know that larger models hallucinate less since they can store more information, are there any smaller models which hallucinate less
I'd recommend checking out the full conclusions section. What I can tell you is that with LLMs, it's never a linear correlation. There's always some balance you have to strike, as they really do operate on a changing-anything-changes-everything basis.
excerpt:
Claim: Avoiding hallucinations requires a degree of intelligence which is exclusively achievable with larger models.
Finding: It can be easier for a small model to know its limits. For example, when asked to answer a Māori question, a small model which knows no Māori can simply say “I don’t know” whereas a model that knows some Māori has to determine its confidence. As discussed in the paper, being “calibrated” requires much less computation than being accurate.
Hallucinations are actually not a malfunction or any other process outside of the normal functioning of the model.
They are merely an output that we find unuseful, but in all other ways is optimal based on the training data, context, and model precision and parameters being used.
I have temporal lobe epilepsy. The premise of this article seems quite wrong to me.
From the article:
"I think it’s because I don’t only know things: I remember learning them. My knowledge is sedimentary, and I can “feel” the position and solidity of different facts and ideas in that mass. I can feel, too, the airy disconnect of a guess."
I think this can be rephrased as a statement that episodic memory (i.e. recalling the act of learning) is associated with semantic memory (i.e. recalling the fact itself). And for people with more normal brains, it seems it often is.
For people with temporal lobe epilepsy, in many cases the episodic memory isn't there. TLE frequently damages the hippocampus. Often, immediately after taking my meds, I can't recall doing so. I only know I did because I mark it in a notebook when I do it. Things like that are common. However, I can absolutely learn things like structure of systems, API calls, and the like. I believe that I perceive when I'm not sure of something, or that I'm guessing.
Of course, this kind of perception is difficult to verify outside of a study. A lot of memory is very unreliable, even for normal people. Us lucky people with TLE are often more aware of this, because in my case, I have very vivid memories of events that certainly did not occur. These memories are some of the most vivid I have; they seem more real than reality. I have memories of events that I can't verify by comparing them to the current state of the world; I have no idea if they happened or not. Even with all that, I believe my semantic memory is reasonably good; I can do my job as a software engineer.
Yesterday I asked one LLM about certain provisions of Ukrainian law, where severity threshold for a financial crime was specified indirectly through a certain well known constant. The machine got it wrong and when asked to give sources cited the respective law referencing a similary sounding unit. Amazingly it gave the correct English tranlation but gave me the wrong original in Ukrainian.
I guess it merged two tokens why learning the text.
Amazingly it also knows about difference between two constants, but referrs to the wrong one in both calculations and in hallucinating the quote.
It's tedious to always check for stuff like this.
Then I asked a different LLM and it turned out that actually the constant is monkey patched for specific contexts and both me and the first lying machine were wrong
I'm not convinced the brain stores memories, or that memory storage is required for human intelligence. And we "hallucinate" all the time. See: eye witness testimony being wrong regularly, "paranormal" experiences etc.
It's a statement that /feels/ true, because we can all look "inside" our heads and "see" memories and facts. But we may as well be re-constructing facts on the fly, just as re-construct reality itself to sense it.
What do you mean, you're not convinced that the brain stores memories? What is happening in the brain when you have an experience and later recall that same experience? It might not be perfect recall but something is being stored.
I mean an LLM (bad example, but good enough for what I'm trying to convey) doesn't need any sort of "memory" to be able to reconstruct something that looks like intelligence. It stores weights, and can re-assemble "facts" from those weights, independent of the meaning or significance of those facts. It's possible the brain is similar, on a much more refined scale. My brain certainly doesn't store 35,000 instances of my mum's image to help me identify her, just an averaged image to help me know when I'm looking at my mum.
The brain definitely stores things, and retrieval and processing are key to the behaviour that comes out the other end, but whether it's "memory" like what this article tries to define, I'm not sure. The article makes it a point to talk about instances where /lack/ of a memory is a sign of the brain doing something different from an LLM, but the brain is pretty happy to "make up" a "memory", from all of my reading and understanding.
Addressing the second paragraph here - while conflation and reconsolidation are real phenomena, it's also quite clear that most humans form episodic memories. Some quite clearly have incredible abilities in this regard [2].
A distinction between semantic (facts/concepts) and episodic (specific experiences) declarative memories are fairly well established since at least the 1970s. That the latter is required to construct the former is also long posited, with reasonable evidence [1]. Similarly, there's a slightly more recent distinction between "recollecting" (i.e., similar to the author's "I can remember the event of learning this") and "knowing" (i.e., "I know this but don't remember why"), with differences in hypothesized recall mechanisms [3].
"That the latter is required to construct the former is also long posited, with reasonable evidence [1]."
This is very interesting to me. I have temporal lobe epilepsy. My episodic memory is quite poor. However, I believe I'm fairly good at learning new facts (i.e. semantic memory). Perhaps my belief is an illusion, or I'm really only learning facts when my episodic memory is less impaired (which happens; it varies from hour to hour). It's difficult for me to tell of course.
Is the idea cars start a fact or a concept? Is the certainty I remember how this particular car starts a fact?
Once we begin to disengage from the arbitrariness inherent in arbitrary metaphors, and rely on what actually generates memories (action-neural-spatial-syntax), we can study what's really happening in the allocortex's distribution of cues between sense/emotion into memory.
Until then we will simply be trapped in falsely segregated ideas of episodic/semantic.
You've prompted some interesting memories out of my teenage years and my first truck.
I can recall the experience of getting in on a cold morning, pumping the throttle pedal three times to activate the semi-automatic choke, starting it, and getting out to clear frost off the window while it warmed up a little. The tactile feeling and squeak of the throttle linkage, the sound of the starter motor, the hollow sound of the door closing, and the noxious exhaust from the cold start (which I haven't smelled in 30 years). I remember how my little plastic window scraper sounded when scraping the glass, and even how the defrost vents made two regions which were always easier to scrape. But, I cannot really remember a specific episode of this on a certain date or leading to a particular trip.
On the other hand, I do have an episodic memory on my final trip in this truck. It was sliding off an icy road, rolling over, and sledding down a steep slope. I remember the ruptured, snow-filled windshield, and the sound of the engine idling upside-down. I remember the slow motion way the whole crash unfolded, and the clothes I was wearing as I crawled back to the roadway.
Ironically, I have more emotional context with the generic cold-start memory. It brings with it some vignette of teenage eagerness and pride in having this vehicle and the freedom it represented. The crash is more dissociated, a movie I was watching from within myself. I can meta-remember that I was very distressed afterward, but those feelings are not connected with the memory during recall.
No problem. Let me also clarify, my language was a bit ambiguous in the prior post. I am not dissociated from the memory. I recall the way the original experience was dissociative.
It was all slow motion and detached while it the car was in motion, then went back to normal after I got my bearings, detached the safety belt to drop out of my seat, and crawled out the window into the snow...
I had a minor "mortality crisis" in the hours and days after, as I processed the facts of the event. But, the real-time experience was numb, almost passive observer. And my recall of it also recalls that meta context.
The article isn't about LLMs storing things - it's about why they hallucinate, which is in large part due to the fact that they just deal in word statistics not facts, but also (the point of the article) that they have no episodic memories, or any personal experience of any sort for that matter.
Humans can generally differentiate between when they know something or not, and I'd agree with the article that this is because we tend to remember how we know things, and also have different levels of confidence according to source. Personal experience trumps watching someone else, which trumps hearing or being taught it from a reliable source, which trumps having read something on Twitter or some grafitti on a bathroom stall. To the LLM all text is just statistics, and it has no personal experience to lean to to self-check and say "hmm, I can't recall ever learning that - I'm drawing blanks".
Frankly it's silly to compare LLMs (Transformers) and brains. An LLM was only every meant to be a linguistics model, not a brain or cognitive architecture. I think people get confused because if spits out human text and so people anthropomorphize it and start thinking it's got some human-like capabilities under the hood when it is in fact - surprise surprise - just a pass-thru stack of Transformer layers. A language model.
Hey, I know what the article wanted to say, see the last two-ish sentences of my previous response. My point, is that the article might be mis-interpreting what the causes and solutions for the problems it sees. Relying on the brain as an example of how to improve might be a mistaken premise, because maybe the brain isn't doing what the article thinks it's doing. So we're in agreement there, that the brain and LLMs are incomparable, but maybe the parts where they're comparable are more informative on the nature of hallucinations than the author may think.
But the thing is that humans don't hallucinate as much as LLMs, so it's the differences not similarities (such as they are) that are important to understand why that is.
It's pretty obvious that an LLM not knowing what it does or does not know is a major part of it hallucinating, while humans do generally know the limits of their own knowledge.
Answering questions in the Turing test (What are roses?) seems to require the same type of real-world knowledge that people use in predicting characters in a stream of natural language text (Roses are ___?), or equivalently, estimating L(x) [the probability of x when written by a human] for compression.
Perhaps in 1999 it seemed reasonable to think that passing the Turing Test, or maximally compressing/predicting human text makes for a good AI/AGI test, but I'd say we now know better, and more to the point that does not appear to have been the motivation for designing the Transformer, or the other language models that preceded it.
The recent history leading to the Transformer was the development of first RNN then LSTM-based language models, then the addition of attention, with the primary practical application being for machine translation (but more generally for any sequence-to-sequence mapping task). The motivation for the Transformer was to build a more efficient and scalable language model by using parallel processing, not sequential (RNN/LSTM), to take advantage of GPU/TPU acceleration.
The conceptual design of what would become the Transformer came from Google employee Jakob Uzkoreit who has been interviewed about this - we don't need to guess the motivation. There were two key ideas, originating from the way linguists use syntax trees to represent the hierarchical/grammatical structure of a sentence.
1) Language is as much parallel as sequential, as can be seen by multiple independent branches of the syntax tree, which only join together at the next level up the tree
2) Language is hierarchical, as indicated by the multiple levels of a branching sytntax tree
Put together these two considerations suggests processing the entire sentence in parallel, taking advantage of GPU parallelism (not sequentially like an LSTM), and having multiple layers of such parallel processing to hierarchically process the sentence. This eventually lead to the stack of parallel-processing Transformer layers design, which did retain the successful idea of attention (thus the paper name "Attention is all you need [not RNNs/LSTMs]").
As far as the functional capability of this new architecture, the initial goal was just to be as good as the LSTM + attention language models it aimed to replace (but be more efficient to train & scale). The first realization of the "parallel + hierarchical" ideas by Uzkoreit was actually less capable than its predecesssors, but then another Google employee, Noam Shazeer, got involved and eventually (after a process of experimentation and ablation) arrived at the Transformer design which did perform well on the language modelling task.
Even at this stage, nobody was saying "if we scale this up it'll be AGI-like". It took multiple steps of scaling, from early Google's early Muppet-themed BERT (following their LSTM-based ELMo), to OpenAI's GPT-1, GPT-2 and GPT-3 for there to be a growing realization of how good a next-word predictor, with corresponding capabilities, this architecture was when scaled up. You can read the early GPT papers and see the growing level of realization - they were not expecting it to be this capable.
Note also that when Shazeer left Google, disappointed that they were not making better use of his Transformer baby, he did not go off and form an AGI company - he went and created Character.ai making fantasy-themed ChatBots (similar to Google having experimented with ChatBot use, then abandoning it, since without OpenAI's innovation of RLHF Transformer-based ChatBots were unpredictable and a corporate liability).
> An LLM was only every meant to be a linguistics model, not a brain or cognitive architecture.
Plenty of people did in fact see a language model as a potential path towards intelligence, whatever might be said about the beliefs of Mr. Uszkoreit specifically.
There's some ambiguity as to whether you're talking about the transformer specifically, or language models generally. The "recent history" of RNNs and LSTMs you refer to dates back to before the paper I linked. I won't speak to the motivations or views of the specific authors of Vaswani et al, but there's a long history, both distant and recent, of drawing connections between information theory, compression, prediction, and intelligence, including in the context of language modeling.
I was really talking about the Transformer specifically.
Maybe there was an implicit hope of a better/larger language model leading to new intelligent capabilities, but I've never seen the Transformer designers say they were targeting this or expecting any significant new capabilities even (to their credit) after it was already apparent how capable it was. Neither Google's initial fumbling of the tech or Shazeer's entertainment chatbot foray seem to indicate that they had been targeting, and/or realized they had achieved, a more significant advance than the more efficient seq-2-seq model which had been their proximate goal.
To me it seems that the Transformer is really one of industry/science's great accidental discoveries. I don't think it's just the ability to scale that made it so powerful, but more the specifics of the architecture, including the emergent ability to learn "induction heads" which seem core to a lot of what they can do.
The Transformer precursors I had in mind were recent ones, in particular Sutskever et als "Sequence to Sequence learning with Neural Networks [LSTM]" from 2014, and Bahdanau et als "Jointly learning to align & translate" from 2016, then followed by the "Attention is all you need" Transformer paper in 2017.
Circling back to the original topic: at the end of the day, whether it makes sense to expect more brain-like behavior out of transformers than "mere" token prediction does not depend much on what the transformer's original creators thought, but rather on the strength of the collective arguments and evidence that have been brought to bear on the question, regardless of who from.
I think there has been a strong case that the "stochastic parrot" model sells language models short, but to what extent still seems to me an open question.
I'd say that whether to expect more brain-like capabilities out of Transformers is more an objective matter of architecture - what's missing - and learning algorithms, not "collective arguments". If a Transformer simply can't do something - has no mechanism to support it (e.g. learn at run time), then it can't do it, regardless of whether Sam Altman tells you it can, or tries to spin it as unimportant!
A Transformer is just a fixed size stack of transformer layers, with one-way data flow through this stack. It has no internal looping, no internal memory, no way to incrementally learn at runtime, no autonomy/curiosity/etc to cause it to explore and actively expose itself to learning situations (assuming it could learn, which it anyways can't), etc!
These are just some of the most obvious major gaps between the Transformer architecture and even the most stripped down cognitive architecture (vs language model) one might design, let alone an actual human brain which has a lot more moving parts and complexity to it.
The whole Transformer journey has been fascinating to watch, and highly informative as to how far language and auto-regressive prediction can take you, but without things like incremental learning and the drive to learn, all you have is a huge, but fixed, repository of "knowledge" (language stats), so you are in effect building a giant expert system. It may be highly capable and sufficient for some tasks, but this is not AGI - it's not something that could replace an intern and learn on the job, or make independent discoveries outside of what is already deducible from what is in the training data.
One of the really major gaps between an LLM and something capable of learning about the world isn't even the architecture with all it's limitations, but just the way they are trained. A human (and other intelligent animals) also learns by prediction, but the feedback loop when the prediction is wrong is essential - this is how you learn, and WHAT you can learn from incorrect predictions is limited by the feedback you receive. In the case of a human/animal the feedback comes from the real world, so what you are able to learn critically includes things like how your own actions affect the world - you learn how to be able to DO things.
An LLM also learns by prediction, but what it is predicting isn't real world responses to it's own actions, but instead just input continuations. It is being trained to be a passive observer of other people's "actions" (limited to the word sequences they generate) - to predict what they will do (say) next, as opposed to being an active entity that learns not to predict someone else's actions, but to predict it's own actions and real-world responses - how to DO things itself (learn on the job, etc, etc).
A payload plus a transformation process is still storage, just compressed storage. There's conceptually little difference between storing data and the process that gets called to create that data.
Yes, I agree. I'm not against the idea that the brain can "store" things. Just whether our concept of how a "memory" "feels" is useful to us further understanding the brain's function.
There is a theory known as "morphic resonance" which suggests that events and objects exist in the world as something analogous to radio waves. So when you "recall" a memory it is not because it is stored in the brain anywhere but rather you are picking up signals from the universe like a radio receiver.
That theory sounds woo-woo. So something I remember that happened a few days ago, or my phone number exists somewhere out in the universe and our brain is able to access it?
The fundamental limitation of LLMs is that they represent knowledge as parametric curves, and their generalization is merely interpolation of those curves. This can only ever produce results that correlate with facts (training data), not ones that are causally derived from them, which makes hallucinations inevitable. Same as with human memory.
> I’ll remind you that biologists do not, in the year 2025, know memory’s physical substrate in the brain! Plenty of hypotheses — no agreement. Is there any more central mystery in human biology, maybe even human existence?
Did they not recently transfer memory of how to solve a maze from one mouse to another, giving credibility to what can store information?
Searching, I only find the RNA transfers done in 60s, which ran into some problems. I thought a recent study did transfer proteins.
That work has been pretty thoroughly debunked. What's getting transferred is stress hormones, and you can "duplicate" the results by using animals that have been stressed (but have never seen the maze) as the donors. They don't know anything about the maze, so can't transfer any knowledge. The recipients do better in the maze because they are more alert, due to the transfer of stress hormones.
I believe this is why the importance of written (human) knowledge is only increasing, especially internally at companies. Written knowledge (i.e documentation) has always served as a knowledge cache and a way to transfer knowledge between people.
Without fundamental changes to the LLMs or the way we think about agentic systems, high quality, comprehensive written knowledge is the best path to helping agents "learn" over time.
As a memory neuroscientist, I enjoyed the shoutout here to episodic memory. It strikes me, however, that a feature that I've noticed when observing "reasoning" models is that they may explicitly search for evidence for intermediate pieces of their responses. If we're following the "remember"/"know" distinction developed by Squire and others, perhaps the more apt analogy might be that a singly pass through an LLM is similar to a "I know this" result, prone to hallucination, conflation, etc., and the multipass reasoning model might be more akin to the "I remember this" result, where primary evidence serves as a substrate for the response?
The terms are arbitrary and don't relate to human memory. Using the term "I know this" is a post-hoc retrofit to a process exclusively accessed wordlessly.
Just as "I remember this" does not access that process, but rather comments on the aftermath pretty unscientifically, like a sportscaster trying to read a pitcher's mind.
“We refute (based on empirical evidence) claims that humans use linguistic representations to think.”
Ev Fedorenko Language Lab MIT 2024
You know, whatever memory is or where it’s at and however the mind works, I’m grateful I’ve got mine in tact right now and I appreciate science’s inability to zero in on these things.
It’s nice to know that this sort of appreciation is becoming more common. Somewhere between tech accelerationism and protestant resistance are those willing to re-interrogate human nature in anticipation of what lies ahead.
A model is capable of learning the "calibration" during the reinforcement learning phase, in this "old" post from OpenAI: https://openai.com/index/introducing-simpleqa/ you can see the positive correlation between stated confidence and accuracy.
"I’ll remind you that biologists do not, in the year 2025, know memory’s physical substrate in the brain! Plenty of hypotheses — no agreement. Is there any more central mystery in human biology, maybe even human existence?"
A hypothesis is very distinct from theoretical knowledge. A hypothesis lacks empirical evidence. A theory uses empirical information. That CS personnel are lacking both the scientific method and the ability to discern the current state of the art empirical research to disprove such wildly unsupported statements speaks to the field's total failure to develop present-day relevant tools. I would direct the author to two critical books
If the author wrote the same sentence, but with "Plenty of theories -- no agreement", would you have an issue? I think the point works just as well.
"Theory" and "hypothesis" are pretty interchangeable in colloquial usage, and the post's tone is very colloquial. On top of that, there are things referred to as hypotheses that have empirical evidence, like the amyloid beta hypothesis. There is empirical evidence that supports it, but there is also evidence that doesn't, and it's an open question (depending on who you ask).
I don't think it shows that the author lacks the ability to discern state of the art research or is making wildly unsupported statements, I think they were using plain-English terms to describe a state where there's a lot of uncertainty about the physical mechanism as opposed to say, how a car engine works (which at a certain point relies on theories of physics, but they're theories that are almost universally agreed upon).
Theory and hypothesis are not interchangeable. Hypothesis precedes theory. Always. Had he said "theories" he would be saying something completely different. It's another stage of scientific process altogether. In order: Hypothesis, empirical observation, theory, demonstration, new theoretical stage or field.
The bizarre CS idea of using "plain English" as a propagandic thrust to infantilize complex scientific theory is suspect. Let's eschew the idea. "Plain English" is Orwellian dumbing down.
There is a general agreement which links ontogeny and phylogeny, details working memory, its relationship to the allocortex, and subsequent episodic memory. So the neural correlates have been worked out.
The working memory and episodic memory papers in the last few years have isolated the correlates, we a have a fairly empirical neurobiological description of memory function and process.
What we're lacking now are the molecular mechanisms.
> The working memory and episodic memory papers in the last few years have isolated the correlates, we a have a fairly empirical neurobiological description of memory function and process.
Would you kindly provide some references? I'm very interested in this research as an armchair enthusiast, but in my own reading I've yet to find anything this confident.
Thank you very much, this is exactly the starting point I needed. I'll keep following the citations trail into more recent years, but your second link from 2007 reinforces my feeling that we're still very far from understanding the mechanisms:
> Second, within LIPC, we found a gradient in which a more dorsal-posterior region was involved in SR, a mid region was involved in both SR and EE, and a more ventral-anterior region was involved in EE, but only when SR was high.
To me, these are merely clues about how the high-level pieces fit together, and there's a long road to actually understanding the neural correlates of memory.
I agree with the folks who call these screwups rather than hallucinations because the point of LLMs isn't to be right, it's to be statistically highly likely. If making something up fits that model, then that's what it will do. That's literally how it works.
Sorry for harping on it, but I think this clearly reflects the difference between 2 approaches to storing knowledge, lossy but humongous, and lossless but limited.
LLMs - Lossy highly compressed knowledge which when prompted "hallucinates" facts. LLMs hallucinations are simply how the stored information is retrieved.
Memory (human in this case) - Extremely limited, but almost always correct.
honestly humans are nowhere near as lossless as you think, look up any study on eye-witness acounts of crimes and you will see how fallible to hallucination the human mind is as well .... at least when it comes to one-shot learning.
I feel from my own experience teaching, that it's repetition and pruning of information that really makes human memory and learning much more effective and not the act of storing the information the first time.
OpenAI just recently took a systematic look into why models hallucinate [0][1].
The article you shared raises an interesting point by comparing human memory with LLMs, but I think the analogy can only go so far. They're too distinct to explain hallucinations simply as a lack of meta-cognition or meta-memory. These systems are more like alien minds, and allegories risk introducing imprecision when we're trying to debug and understand their behavior.
OpenAI's paper instead identifies hallucinations as a bug in training objectives and benchmarks, and is grounding the explanation in first principles and the mechanics of ML.
Metaphors are useful for creativity, but less so when it comes to debugging and understanding, especially now that the systematic view is this advanced.
[0] https://openai.com/index/why-language-models-hallucinate/?ut... [1] https://cdn.openai.com/pdf/d04913be-3f6f-4d2b-b283-ff432ef4a...
Hallucinations are actually not a malfunction or any other process outside of the normal functioning of the model. They are merely an output that we find unuseful, but in all other ways is optimal based on the training data, context, and model precision and parameters being used.
I honestly have no idea why OAI felt that they needed to publish a “paper” about this, since it is blazingly obvious to anyone who understands the fundamentals of transformer inference, but here we are.
The confusion on this topic comes from calling these suboptimal outputs “hallucinations” which drags anthropomorphic fallacies into the room by their neck even though they were peacefully minding their own business down the corridor on the left.
“Hallucination” implies a fundamentally fixable error in inference, a malfunction of thought caused by a pathology or broken algorithm.
LLMs “Hallucinating” are working precisely as implemented, only we don’t feel like the output usefully matches the parameters from a human perspective. It’s just unhelpful results from the algorithm, like any other failure of training, compression, alignment, or optimisation.
Did you read TFA? It gives concrete advice on how to change the training and eval of these models in order to decrease the error rate. Sure, these being stochastic models, the rate will never reach zero, but given that they are useful today, decreasing the error rate is worthy cause. All this complaining on semantics is just noise to me. It stems from being fixated on some airy-fairy ideas of AGI/ASI, as if anything else doesn't matter. Does saying that a model "replied" to a query mean we are unwittingly anthropomorphizing them? It's all just words, we can extend their use as we see fit. I think "confabulation" would be a more fitting term, but beyond that, I'm not seeing the problem.
We can call it whatever, and yes, the answer is training- just like all things regarding the quality of LLM output per parameter count. The problem is that many people understand“hallucination” as a gross malfunction of an otherwise correctly functioning system, I.e. a defect that can/must be categorically “fixed”, not understanding that it is merely a function of trained weights, inference parameters, and prompt context that they can:
A: probably work around by prompting and properly structuring tasks
B: never completely rule out
C: not avoid at all in certain classes of data transformations where it will creep in in subtle ways and corrupt the data
D: not intrinsically detect, since it lacks the human characteristic of “woah, this is trippy, I feel like maybe I’m hallucinating “
These misconceptions stem from the fact that in LLM parlance, “hallucination” is often conflated with a same-named, relatable human condition that is largely considered completely discrete from normal conscious thought and workflows.
Words and their meanings matter, and the failure to properly label things often is at the root of significant wastes of time and effort. Semantics are the point of language.
I haven't read the full paper yet, but my intuition is that hallucinations are a byproduct of models having too much information that needs to be compressed for generalizing.
We already know that larger models hallucinate less since they can store more information, are there any smaller models which hallucinate less
I'd recommend checking out the full conclusions section. What I can tell you is that with LLMs, it's never a linear correlation. There's always some balance you have to strike, as they really do operate on a changing-anything-changes-everything basis.
excerpt: Claim: Avoiding hallucinations requires a degree of intelligence which is exclusively achievable with larger models. Finding: It can be easier for a small model to know its limits. For example, when asked to answer a Māori question, a small model which knows no Māori can simply say “I don’t know” whereas a model that knows some Māori has to determine its confidence. As discussed in the paper, being “calibrated” requires much less computation than being accurate.
Hallucinations are actually not a malfunction or any other process outside of the normal functioning of the model.
They are merely an output that we find unuseful, but in all other ways is optimal based on the training data, context, and model precision and parameters being used.
One robot's "hallucination" is another robot's "connecting the dots" or "closing the circle".
I have temporal lobe epilepsy. The premise of this article seems quite wrong to me.
From the article: "I think it’s because I don’t only know things: I remember learning them. My knowledge is sedimentary, and I can “feel” the position and solidity of different facts and ideas in that mass. I can feel, too, the airy disconnect of a guess."
I think this can be rephrased as a statement that episodic memory (i.e. recalling the act of learning) is associated with semantic memory (i.e. recalling the fact itself). And for people with more normal brains, it seems it often is.
For people with temporal lobe epilepsy, in many cases the episodic memory isn't there. TLE frequently damages the hippocampus. Often, immediately after taking my meds, I can't recall doing so. I only know I did because I mark it in a notebook when I do it. Things like that are common. However, I can absolutely learn things like structure of systems, API calls, and the like. I believe that I perceive when I'm not sure of something, or that I'm guessing.
Of course, this kind of perception is difficult to verify outside of a study. A lot of memory is very unreliable, even for normal people. Us lucky people with TLE are often more aware of this, because in my case, I have very vivid memories of events that certainly did not occur. These memories are some of the most vivid I have; they seem more real than reality. I have memories of events that I can't verify by comparing them to the current state of the world; I have no idea if they happened or not. Even with all that, I believe my semantic memory is reasonably good; I can do my job as a software engineer.
Yesterday I asked one LLM about certain provisions of Ukrainian law, where severity threshold for a financial crime was specified indirectly through a certain well known constant. The machine got it wrong and when asked to give sources cited the respective law referencing a similary sounding unit. Amazingly it gave the correct English tranlation but gave me the wrong original in Ukrainian.
I guess it merged two tokens why learning the text.
Amazingly it also knows about difference between two constants, but referrs to the wrong one in both calculations and in hallucinating the quote.
It's tedious to always check for stuff like this.
Then I asked a different LLM and it turned out that actually the constant is monkey patched for specific contexts and both me and the first lying machine were wrong
I'm not convinced the brain stores memories, or that memory storage is required for human intelligence. And we "hallucinate" all the time. See: eye witness testimony being wrong regularly, "paranormal" experiences etc.
It's a statement that /feels/ true, because we can all look "inside" our heads and "see" memories and facts. But we may as well be re-constructing facts on the fly, just as re-construct reality itself to sense it.
What do you mean, you're not convinced that the brain stores memories? What is happening in the brain when you have an experience and later recall that same experience? It might not be perfect recall but something is being stored.
I mean an LLM (bad example, but good enough for what I'm trying to convey) doesn't need any sort of "memory" to be able to reconstruct something that looks like intelligence. It stores weights, and can re-assemble "facts" from those weights, independent of the meaning or significance of those facts. It's possible the brain is similar, on a much more refined scale. My brain certainly doesn't store 35,000 instances of my mum's image to help me identify her, just an averaged image to help me know when I'm looking at my mum.
The brain definitely stores things, and retrieval and processing are key to the behaviour that comes out the other end, but whether it's "memory" like what this article tries to define, I'm not sure. The article makes it a point to talk about instances where /lack/ of a memory is a sign of the brain doing something different from an LLM, but the brain is pretty happy to "make up" a "memory", from all of my reading and understanding.
Addressing the second paragraph here - while conflation and reconsolidation are real phenomena, it's also quite clear that most humans form episodic memories. Some quite clearly have incredible abilities in this regard [2].
A distinction between semantic (facts/concepts) and episodic (specific experiences) declarative memories are fairly well established since at least the 1970s. That the latter is required to construct the former is also long posited, with reasonable evidence [1]. Similarly, there's a slightly more recent distinction between "recollecting" (i.e., similar to the author's "I can remember the event of learning this") and "knowing" (i.e., "I know this but don't remember why"), with differences in hypothesized recall mechanisms [3].
[1] https://www.science.org/doi/full/10.1126/science.277.5324.33... or many other reviews by Eichenbaum, Squire, Milner, etc
[2] https://youtu.be/hpTCZ-hO6iI?si=FeFv8MGmHTzkLd8p
[3] https://psycnet.apa.org/record/1995-42814-001
"That the latter is required to construct the former is also long posited, with reasonable evidence [1]."
This is very interesting to me. I have temporal lobe epilepsy. My episodic memory is quite poor. However, I believe I'm fairly good at learning new facts (i.e. semantic memory). Perhaps my belief is an illusion, or I'm really only learning facts when my episodic memory is less impaired (which happens; it varies from hour to hour). It's difficult for me to tell of course.
Is the idea cars start a fact or a concept? Is the certainty I remember how this particular car starts a fact?
Once we begin to disengage from the arbitrariness inherent in arbitrary metaphors, and rely on what actually generates memories (action-neural-spatial-syntax), we can study what's really happening in the allocortex's distribution of cues between sense/emotion into memory.
Until then we will simply be trapped in falsely segregated ideas of episodic/semantic.
You've prompted some interesting memories out of my teenage years and my first truck.
I can recall the experience of getting in on a cold morning, pumping the throttle pedal three times to activate the semi-automatic choke, starting it, and getting out to clear frost off the window while it warmed up a little. The tactile feeling and squeak of the throttle linkage, the sound of the starter motor, the hollow sound of the door closing, and the noxious exhaust from the cold start (which I haven't smelled in 30 years). I remember how my little plastic window scraper sounded when scraping the glass, and even how the defrost vents made two regions which were always easier to scrape. But, I cannot really remember a specific episode of this on a certain date or leading to a particular trip.
On the other hand, I do have an episodic memory on my final trip in this truck. It was sliding off an icy road, rolling over, and sledding down a steep slope. I remember the ruptured, snow-filled windshield, and the sound of the engine idling upside-down. I remember the slow motion way the whole crash unfolded, and the clothes I was wearing as I crawled back to the roadway.
Ironically, I have more emotional context with the generic cold-start memory. It brings with it some vignette of teenage eagerness and pride in having this vehicle and the freedom it represented. The crash is more dissociated, a movie I was watching from within myself. I can meta-remember that I was very distressed afterward, but those feelings are not connected with the memory during recall.
Fantastic to read this. Thank you for it, that you can interpret these associations/dissociations still is incredible.
No problem. Let me also clarify, my language was a bit ambiguous in the prior post. I am not dissociated from the memory. I recall the way the original experience was dissociative.
It was all slow motion and detached while it the car was in motion, then went back to normal after I got my bearings, detached the safety belt to drop out of my seat, and crawled out the window into the snow...
I had a minor "mortality crisis" in the hours and days after, as I processed the facts of the event. But, the real-time experience was numb, almost passive observer. And my recall of it also recalls that meta context.
The article isn't about LLMs storing things - it's about why they hallucinate, which is in large part due to the fact that they just deal in word statistics not facts, but also (the point of the article) that they have no episodic memories, or any personal experience of any sort for that matter.
Humans can generally differentiate between when they know something or not, and I'd agree with the article that this is because we tend to remember how we know things, and also have different levels of confidence according to source. Personal experience trumps watching someone else, which trumps hearing or being taught it from a reliable source, which trumps having read something on Twitter or some grafitti on a bathroom stall. To the LLM all text is just statistics, and it has no personal experience to lean to to self-check and say "hmm, I can't recall ever learning that - I'm drawing blanks".
Frankly it's silly to compare LLMs (Transformers) and brains. An LLM was only every meant to be a linguistics model, not a brain or cognitive architecture. I think people get confused because if spits out human text and so people anthropomorphize it and start thinking it's got some human-like capabilities under the hood when it is in fact - surprise surprise - just a pass-thru stack of Transformer layers. A language model.
Hey, I know what the article wanted to say, see the last two-ish sentences of my previous response. My point, is that the article might be mis-interpreting what the causes and solutions for the problems it sees. Relying on the brain as an example of how to improve might be a mistaken premise, because maybe the brain isn't doing what the article thinks it's doing. So we're in agreement there, that the brain and LLMs are incomparable, but maybe the parts where they're comparable are more informative on the nature of hallucinations than the author may think.
I think you can confidently say that brains do the following and LLMs don't:
* Continuously updates its state based on sensory data
* Retrieves/gathers information that correlates strongly with historic sensory input
* Is able to associate propositions with specific instances of historic sensory input
* Uses the above two points to verify/validate its belief in said propositions
Describing how memories "feel" may confuse the matter, I agree. But I don't think we should be quick to dismiss the argument.
But the thing is that humans don't hallucinate as much as LLMs, so it's the differences not similarities (such as they are) that are important to understand why that is.
It's pretty obvious that an LLM not knowing what it does or does not know is a major part of it hallucinating, while humans do generally know the limits of their own knowledge.
> An LLM was only every meant to be a linguistics model, not a brain or cognitive architecture.
See https://gwern.net/doc/cs/algorithm/information/compression/1... from 1999.
Answering questions in the Turing test (What are roses?) seems to require the same type of real-world knowledge that people use in predicting characters in a stream of natural language text (Roses are ___?), or equivalently, estimating L(x) [the probability of x when written by a human] for compression.
I'm not sure what your point is?
Perhaps in 1999 it seemed reasonable to think that passing the Turing Test, or maximally compressing/predicting human text makes for a good AI/AGI test, but I'd say we now know better, and more to the point that does not appear to have been the motivation for designing the Transformer, or the other language models that preceded it.
The recent history leading to the Transformer was the development of first RNN then LSTM-based language models, then the addition of attention, with the primary practical application being for machine translation (but more generally for any sequence-to-sequence mapping task). The motivation for the Transformer was to build a more efficient and scalable language model by using parallel processing, not sequential (RNN/LSTM), to take advantage of GPU/TPU acceleration.
The conceptual design of what would become the Transformer came from Google employee Jakob Uzkoreit who has been interviewed about this - we don't need to guess the motivation. There were two key ideas, originating from the way linguists use syntax trees to represent the hierarchical/grammatical structure of a sentence.
1) Language is as much parallel as sequential, as can be seen by multiple independent branches of the syntax tree, which only join together at the next level up the tree
2) Language is hierarchical, as indicated by the multiple levels of a branching sytntax tree
Put together these two considerations suggests processing the entire sentence in parallel, taking advantage of GPU parallelism (not sequentially like an LSTM), and having multiple layers of such parallel processing to hierarchically process the sentence. This eventually lead to the stack of parallel-processing Transformer layers design, which did retain the successful idea of attention (thus the paper name "Attention is all you need [not RNNs/LSTMs]").
As far as the functional capability of this new architecture, the initial goal was just to be as good as the LSTM + attention language models it aimed to replace (but be more efficient to train & scale). The first realization of the "parallel + hierarchical" ideas by Uzkoreit was actually less capable than its predecesssors, but then another Google employee, Noam Shazeer, got involved and eventually (after a process of experimentation and ablation) arrived at the Transformer design which did perform well on the language modelling task.
Even at this stage, nobody was saying "if we scale this up it'll be AGI-like". It took multiple steps of scaling, from early Google's early Muppet-themed BERT (following their LSTM-based ELMo), to OpenAI's GPT-1, GPT-2 and GPT-3 for there to be a growing realization of how good a next-word predictor, with corresponding capabilities, this architecture was when scaled up. You can read the early GPT papers and see the growing level of realization - they were not expecting it to be this capable.
Note also that when Shazeer left Google, disappointed that they were not making better use of his Transformer baby, he did not go off and form an AGI company - he went and created Character.ai making fantasy-themed ChatBots (similar to Google having experimented with ChatBot use, then abandoning it, since without OpenAI's innovation of RLHF Transformer-based ChatBots were unpredictable and a corporate liability).
> I'm not sure what your point is?
I was just responding to this claim:
> An LLM was only every meant to be a linguistics model, not a brain or cognitive architecture.
Plenty of people did in fact see a language model as a potential path towards intelligence, whatever might be said about the beliefs of Mr. Uszkoreit specifically.
There's some ambiguity as to whether you're talking about the transformer specifically, or language models generally. The "recent history" of RNNs and LSTMs you refer to dates back to before the paper I linked. I won't speak to the motivations or views of the specific authors of Vaswani et al, but there's a long history, both distant and recent, of drawing connections between information theory, compression, prediction, and intelligence, including in the context of language modeling.
I was really talking about the Transformer specifically.
Maybe there was an implicit hope of a better/larger language model leading to new intelligent capabilities, but I've never seen the Transformer designers say they were targeting this or expecting any significant new capabilities even (to their credit) after it was already apparent how capable it was. Neither Google's initial fumbling of the tech or Shazeer's entertainment chatbot foray seem to indicate that they had been targeting, and/or realized they had achieved, a more significant advance than the more efficient seq-2-seq model which had been their proximate goal.
To me it seems that the Transformer is really one of industry/science's great accidental discoveries. I don't think it's just the ability to scale that made it so powerful, but more the specifics of the architecture, including the emergent ability to learn "induction heads" which seem core to a lot of what they can do.
The Transformer precursors I had in mind were recent ones, in particular Sutskever et als "Sequence to Sequence learning with Neural Networks [LSTM]" from 2014, and Bahdanau et als "Jointly learning to align & translate" from 2016, then followed by the "Attention is all you need" Transformer paper in 2017.
Circling back to the original topic: at the end of the day, whether it makes sense to expect more brain-like behavior out of transformers than "mere" token prediction does not depend much on what the transformer's original creators thought, but rather on the strength of the collective arguments and evidence that have been brought to bear on the question, regardless of who from.
I think there has been a strong case that the "stochastic parrot" model sells language models short, but to what extent still seems to me an open question.
I'd say that whether to expect more brain-like capabilities out of Transformers is more an objective matter of architecture - what's missing - and learning algorithms, not "collective arguments". If a Transformer simply can't do something - has no mechanism to support it (e.g. learn at run time), then it can't do it, regardless of whether Sam Altman tells you it can, or tries to spin it as unimportant!
A Transformer is just a fixed size stack of transformer layers, with one-way data flow through this stack. It has no internal looping, no internal memory, no way to incrementally learn at runtime, no autonomy/curiosity/etc to cause it to explore and actively expose itself to learning situations (assuming it could learn, which it anyways can't), etc!
These are just some of the most obvious major gaps between the Transformer architecture and even the most stripped down cognitive architecture (vs language model) one might design, let alone an actual human brain which has a lot more moving parts and complexity to it.
The whole Transformer journey has been fascinating to watch, and highly informative as to how far language and auto-regressive prediction can take you, but without things like incremental learning and the drive to learn, all you have is a huge, but fixed, repository of "knowledge" (language stats), so you are in effect building a giant expert system. It may be highly capable and sufficient for some tasks, but this is not AGI - it's not something that could replace an intern and learn on the job, or make independent discoveries outside of what is already deducible from what is in the training data.
One of the really major gaps between an LLM and something capable of learning about the world isn't even the architecture with all it's limitations, but just the way they are trained. A human (and other intelligent animals) also learns by prediction, but the feedback loop when the prediction is wrong is essential - this is how you learn, and WHAT you can learn from incorrect predictions is limited by the feedback you receive. In the case of a human/animal the feedback comes from the real world, so what you are able to learn critically includes things like how your own actions affect the world - you learn how to be able to DO things.
An LLM also learns by prediction, but what it is predicting isn't real world responses to it's own actions, but instead just input continuations. It is being trained to be a passive observer of other people's "actions" (limited to the word sequences they generate) - to predict what they will do (say) next, as opposed to being an active entity that learns not to predict someone else's actions, but to predict it's own actions and real-world responses - how to DO things itself (learn on the job, etc, etc).
[dead]
I don’t know if this aligns with your thinking but there is a theory that memory is largely reconstructed every time it is remembered: https://en.wikipedia.org/wiki/Reconstructive_memory
Thank you for that! I think I read about this a long time ago, internalized it, and forgot it. Pretty on the nose in this conversation... haha.
A payload plus a transformation process is still storage, just compressed storage. There's conceptually little difference between storing data and the process that gets called to create that data.
its not reliable, but we can also recall things accurately.
it does not store things in the way records of any sort do, but it does have a some store and recall mechanism that works.
To be fair, LLMs do this too - I just got ChatGPT to recite Ode to Autumn.
Yes, I agree. I'm not against the idea that the brain can "store" things. Just whether our concept of how a "memory" "feels" is useful to us further understanding the brain's function.
> I'm not convinced the brain stores memories
By what mechanism do you feel I "remember" last week?
There is a theory known as "morphic resonance" which suggests that events and objects exist in the world as something analogous to radio waves. So when you "recall" a memory it is not because it is stored in the brain anywhere but rather you are picking up signals from the universe like a radio receiver.
Ahhh, ok. Magic, gotcha. :D
Always amazes me when people want to replace a simple explanation with magic.
That theory sounds woo-woo. So something I remember that happened a few days ago, or my phone number exists somewhere out in the universe and our brain is able to access it?
Yes sort of like Platonic Forms and ideas. Where do your ideas exist? Can you isolate it to a part of the brain?
Reconstructing from what?
The fundamental limitation of LLMs is that they represent knowledge as parametric curves, and their generalization is merely interpolation of those curves. This can only ever produce results that correlate with facts (training data), not ones that are causally derived from them, which makes hallucinations inevitable. Same as with human memory.
> I’ll remind you that biologists do not, in the year 2025, know memory’s physical substrate in the brain! Plenty of hypotheses — no agreement. Is there any more central mystery in human biology, maybe even human existence?
Did they not recently transfer memory of how to solve a maze from one mouse to another, giving credibility to what can store information?
Searching, I only find the RNA transfers done in 60s, which ran into some problems. I thought a recent study did transfer proteins.
That work has been pretty thoroughly debunked. What's getting transferred is stress hormones, and you can "duplicate" the results by using animals that have been stressed (but have never seen the maze) as the donors. They don't know anything about the maze, so can't transfer any knowledge. The recipients do better in the maze because they are more alert, due to the transfer of stress hormones.
I would love to see some sources for this. Sounds super interesting. When do we get Matrix-style learning modules?
I believe this is why the importance of written (human) knowledge is only increasing, especially internally at companies. Written knowledge (i.e documentation) has always served as a knowledge cache and a way to transfer knowledge between people.
Without fundamental changes to the LLMs or the way we think about agentic systems, high quality, comprehensive written knowledge is the best path to helping agents "learn" over time.
As a memory neuroscientist, I enjoyed the shoutout here to episodic memory. It strikes me, however, that a feature that I've noticed when observing "reasoning" models is that they may explicitly search for evidence for intermediate pieces of their responses. If we're following the "remember"/"know" distinction developed by Squire and others, perhaps the more apt analogy might be that a singly pass through an LLM is similar to a "I know this" result, prone to hallucination, conflation, etc., and the multipass reasoning model might be more akin to the "I remember this" result, where primary evidence serves as a substrate for the response?
The terms are arbitrary and don't relate to human memory. Using the term "I know this" is a post-hoc retrofit to a process exclusively accessed wordlessly. Just as "I remember this" does not access that process, but rather comments on the aftermath pretty unscientifically, like a sportscaster trying to read a pitcher's mind.
“We refute (based on empirical evidence) claims that humans use linguistic representations to think.” Ev Fedorenko Language Lab MIT 2024
You know, whatever memory is or where it’s at and however the mind works, I’m grateful I’ve got mine in tact right now and I appreciate science’s inability to zero in on these things.
It’s nice to know that this sort of appreciation is becoming more common. Somewhere between tech accelerationism and protestant resistance are those willing to re-interrogate human nature in anticipation of what lies ahead.
A different blog post from this month detailing an experience with ChatGPT that netted a similar reflection: https://zettelkasten.de/posts/the-scam-called-you-dont-have-...
A model is capable of learning the "calibration" during the reinforcement learning phase, in this "old" post from OpenAI: https://openai.com/index/introducing-simpleqa/ you can see the positive correlation between stated confidence and accuracy.
This is patently false.
"I’ll remind you that biologists do not, in the year 2025, know memory’s physical substrate in the brain! Plenty of hypotheses — no agreement. Is there any more central mystery in human biology, maybe even human existence?"
A hypothesis is very distinct from theoretical knowledge. A hypothesis lacks empirical evidence. A theory uses empirical information. That CS personnel are lacking both the scientific method and the ability to discern the current state of the art empirical research to disprove such wildly unsupported statements speaks to the field's total failure to develop present-day relevant tools. I would direct the author to two critical books
Evolution of Memory Systems
https://academic.oup.com/book/26033
How we remember: brain mechanisms of episodic memory
https://direct.mit.edu/books/monograph/2909/How-We-RememberB...
If the author wrote the same sentence, but with "Plenty of theories -- no agreement", would you have an issue? I think the point works just as well.
"Theory" and "hypothesis" are pretty interchangeable in colloquial usage, and the post's tone is very colloquial. On top of that, there are things referred to as hypotheses that have empirical evidence, like the amyloid beta hypothesis. There is empirical evidence that supports it, but there is also evidence that doesn't, and it's an open question (depending on who you ask).
I don't think it shows that the author lacks the ability to discern state of the art research or is making wildly unsupported statements, I think they were using plain-English terms to describe a state where there's a lot of uncertainty about the physical mechanism as opposed to say, how a car engine works (which at a certain point relies on theories of physics, but they're theories that are almost universally agreed upon).
Theory and hypothesis are not interchangeable. Hypothesis precedes theory. Always. Had he said "theories" he would be saying something completely different. It's another stage of scientific process altogether. In order: Hypothesis, empirical observation, theory, demonstration, new theoretical stage or field.
The bizarre CS idea of using "plain English" as a propagandic thrust to infantilize complex scientific theory is suspect. Let's eschew the idea. "Plain English" is Orwellian dumbing down.
There is a general agreement which links ontogeny and phylogeny, details working memory, its relationship to the allocortex, and subsequent episodic memory. So the neural correlates have been worked out.
The working memory and episodic memory papers in the last few years have isolated the correlates, we a have a fairly empirical neurobiological description of memory function and process.
What we're lacking now are the molecular mechanisms.
> The working memory and episodic memory papers in the last few years have isolated the correlates, we a have a fairly empirical neurobiological description of memory function and process.
Would you kindly provide some references? I'm very interested in this research as an armchair enthusiast, but in my own reading I've yet to find anything this confident.
My favorite molecular mechanisms paper
https://bmcbiol.biomedcentral.com/articles/10.1186/s12915-02...
In terms of NCC of memory, the papers beginning around the mid 2000s are seismic. The encoding and retrieval are subtly varied depending on the lab.
https://pubmed.ncbi.nlm.nih.gov/17425535/
Also the cohort studies are immensely helpful, dementia, Down's syndrome let us see the impairment.
Thank you very much, this is exactly the starting point I needed. I'll keep following the citations trail into more recent years, but your second link from 2007 reinforces my feeling that we're still very far from understanding the mechanisms:
> Second, within LIPC, we found a gradient in which a more dorsal-posterior region was involved in SR, a mid region was involved in both SR and EE, and a more ventral-anterior region was involved in EE, but only when SR was high.
To me, these are merely clues about how the high-level pieces fit together, and there's a long road to actually understanding the neural correlates of memory.
I agree with the folks who call these screwups rather than hallucinations because the point of LLMs isn't to be right, it's to be statistically highly likely. If making something up fits that model, then that's what it will do. That's literally how it works.
Sorry for harping on it, but I think this clearly reflects the difference between 2 approaches to storing knowledge, lossy but humongous, and lossless but limited.
LLMs - Lossy highly compressed knowledge which when prompted "hallucinates" facts. LLMs hallucinations are simply how the stored information is retrieved.
Memory (human in this case) - Extremely limited, but almost always correct.
Just an observation. No morals.
honestly humans are nowhere near as lossless as you think, look up any study on eye-witness acounts of crimes and you will see how fallible to hallucination the human mind is as well .... at least when it comes to one-shot learning.
I feel from my own experience teaching, that it's repetition and pruning of information that really makes human memory and learning much more effective and not the act of storing the information the first time.
LLMs behaviour is a lot closer to primary orality than literacy.
this is such drivel and you don't realize it because the guy writes good ... and you don't realize it until you see the "are llms in hell" article ...
let's stop taking opinions on ai from randoms. please. they haven't a fkin clue.