I plan to write periodically about “LLM psychology”: a project of finding the right high-level abstractions to describe the way that LLMs accomplish cognitive tasks like making decisions, drawing inferences from evidence, generalizing to new settings, and so on. I believe that these questions are both scientifically fascinating, and important to build systems that are legible to and steerable by people.
An example question to set the stage: do LLMs have beliefs? Many of you reading this will have very different, immediate opinions about this question! Some people take it as almost obvious that we can ascribe beliefs, preferences, and potentially even more (emotions?) to models. Others see these as overreaching anthropomorphisms, perhaps even a category error. Over a series of posts, I am going to try and work my way through what would count as evidence about such questions. I’ve recently started doing some research related to these questions and some of that material will work its way in – based on my own experiences and work so far, I do think it is reasonable to describe LLMs as having many such cognitive characteristics.
First though: why does this matter?
Why should we care whether LLMs have beliefs (or preferences, or emotions, or…)? I think an analogy to how we reason about humans is useful here. We often explain other people using “folk psychology”, common intuitions that people have for making sense of the behavior of others. This typically involves reasoning about what others are likely to believe, what information they have access to, what they are likely to want, and so on.
While we all start with folk psychology, various layers of science try to empirically test what kinds of abstractions are actually good descriptions of people’s behavior, from experimental psychology looking at people’s observable behaviors down to neuroscience attempting to identify underlying mechanisms. Understanding and predicting people’s behavior using folk psychology works well most of the time, but this scientific endeavor generates both understanding of the mechanisms behind intuitively describable behavior as well as cases where less intuitively-apparent processes substantively impact what people do.
I believe that we will want descriptors across levels of analysis to make sense of LLMs. Clearly, mechanistic understanding is scientifically important. But in day-to-day interactions, abstraction is very helpful – if reasoning about quantities like beliefs, desires, and intentions can help us predict what models will do, then we will be much better off in terms of clearly communicating with models, ensuring that their decision-making is legible and aligned with our interests, and so on. Alternatively, if these categories simply can’t be applied usefully to LLMs, we also need to know that, because in that case many of our intuitive strategies for reasoning about them simply won’t work, and many questions, e.g. about “alignment”, will be ill-posed.
Many intriguing phenomena observed about LLMs are also situated exactly at the interface between abstract or idealized explanations of behavior and lower-level mechanisms. For example, “emergent misalignment” hints that LLMs role-play a persona, such that training LLMs for one behavior causes them to adopt a persona which “rationalizes” the target behavior and impacts traits that weren’t trained for. Bundles of traits are correlated in ways that are human-like, but patterns of switching between bundles are not.
I think it will be very productive to push at how much of such phenomena have good explanations at an abstract level, and which need to be explained using lower-level mechanisms. For example, attempting to interpret “irrational” behavior exhibited by humans through an idealized lens has given rise to fruitful abstractions like satisficing or rational inattention which add to our conceptual toolbox for understanding rational-but-bounded agents. One could ask such questions for LLMs, e.g., could emergent misalignment be conceptualized in a Bayesian model where LLM infers its “type” from cues in training and in-context data? This would tell us how literally to take the metaphor. A significant part of a science of “LLM psychology” would be to develop better measurement tools for the constructs underlying decision making, so that we could ask about such phenomena in more detail.
A starting point about LLM beliefs
Do LLMs have beliefs? What does that mean, and how could we tell? If they do have beliefs, how do we know what those beliefs are?
My perspective (which is certainly not original to me) is that attributing such abstractions to a system is productive if that system displays enough behavioral coherence: I should be able to draw inferences about what an agent will try to do next if I know something about what beliefs they hold, what goals they are trying to pursue, what information they have access to, and so on. At one extreme end of the spectrum is an agent who is completely disordered; their responses to one situation tell you nothing at all about their responses to another. That agent can’t really be described as holding beliefs at all. At the other extreme end is an agent who acts like a perfect Bayesian expected utility maximizer, where their behavior is completely predictable from some very structured quantities (e.g., a description of their prior or utility function).
Raw predictability isn’t enough; systems can be perfectly predictable without having “beliefs”. Rather, these qualities require that an agent’s behavior be predictable via a particular kind of interpretable process. For example, in Daniel Dennett’s view, an entity has beliefs if their behavior is well-predicted by modeling them as a rational agent (or at least somewhat rational) who believes the sort of things that they ought to, given their purposes, information, capabilities, and so on. See Goldstein & Lederman for a great exposition of how related ideas in the philosophical literature might be applied to LLMs.
Starting from the right normative picture also allows you to give a mathematically precise account of “beliefs” within the process of decision making. For example, classical representation theorems in statistical decision theory – most prominently Savage’s theorem – show that agents with “sufficiently coherent” preferences must act as if they are maximizing the expectation of some utility function with respect to some probability measure, and we can call that probability measure their belief. Similar ideas are present in the recent world models literature in ML, where various theoretical results that agents who behave sufficiently capably across diverse environments, goals, etc, must internally track specific features of the dynamics.
I do not see any way around some specification of a normative standard – in this case measuring against the beliefs that a rational agent ought to form under the circumstances. Some idealization is needed to specify how different beliefs, actions, expressions, and so on are supposed to relate to one another. Any behavior is rationalizable by attributing sufficiently bizarre or disordered beliefs and preferences to an agent, so the description stops being useful without anchoring relative to some normative account. On the one hand, this is frustrating. Normative standards are never fully met. Certainly, people do not live up to them; why should we expect this to be a good way to think about LLMs? On the other hand, it gives us some tools to work with in order to be very specific about what qualities beliefs should have.
This is a viewpoint that my PhD student Khurram Yamin started exploring in a recent paper. There, we restrict ourselves to simple tasks that have flavor of statistical decision theory – an agent is confronted with a single unknown state of the world, selects an action from a finite menu, and so on. For such tasks, we can be mathematically exact about what the input/output relationship induced by belief should look like. For example, beliefs should summarize all information that an agent has about unknowns – their actions should become independent of the true state conditional on their belief. LLMs, especially strong ones, are pretty close (though not 100%) to satisfying such conditions! At least for such tasks, I think it is therefore useful to describe LLMs as having beliefs. Clearly though, there is much to be done to extend these ideas to more complicated settings that put more “stress test” on the model.
There is also plenty of theorizing to be done about what the object of analysis is – what is the entity holding the beliefs? Is it the LLM in the sense of the base model, the LLM plus a particular system prompt, the LLM plus an entire conversation history? This becomes particularly acute when LLMs produce different outputs in different conditions (i.e., given different prompts) that are not mutually rationalizable. For example, if I induce different personas in a LLM and these personas act in very different ways, is it better to think of the LLM as a platform which can simulate multiple different agents who have different beliefs? Or is there a sense in which the LLM is an agent that could hold beliefs under the surface, apart from the different personas? In that case, some kinds of inconsistencies in outputs might reveal incoherence in the beliefs of the underlying agent (or a sense in which that agent isn’t usefully described as having beliefs at all).
Interesting references
Daniel A. Herrmann, Benjamin A. Levinstein. Standards for Belief Representations in LLMs. https://arxiv.org/abs/2405.21030
Simon Goldstein, Harvey Lederman. What Does ChatGPT Want? An Interpretationist Guide. https://philpapers.org/rec/GOLWDC-2
Jessica Hullman. When does it make sense to talk about LLMs having beliefs? https://statmodeling.stat.columbia.edu/2025/08/15/when-does-it-make-sense-to-talk-about-llms-having-beliefs/
Pierre Beckmann and Patrick Butlin. Where is the Mind? Persona Vectors and LLM Individuation. https://arxiv.org/pdf/2604.17031
Jonathan Richens, Tom Everitt. Robust agents learn causal world models. https://arxiv.org/abs/2402.10877
Aran Nayebi. What Capable Agents Must Know: Selection Theorems for Robust Decision-Making under Uncertainty. https://arxiv.org/abs/2603.02491