Memory Anatomy: How Memory Works, in People and in AI
Each machine below is one model of memory. It runs a short tour on its own until you tap it to try it yourself.
ACT-R
Why some memories come back and others don't
Based on ACT-R, a model of the mind in which every memory has a strength that grows each time you use it and slowly fades.
How it works
This is the declarative memory of ACT-R, a model of the mind from Carnegie Mellon. The "charge" in each window is a memory's activation A, which ACT-R's authors read as the log odds that the memory will be needed (Taatgen, Lebiere and Anderson 2006). The window is linear in A, from −4 at the bottom to +3 at the top.
Uses add up and drain: B = ln( Σ t_j^−d ). Every past use j leaves a trace that fades as t_j^−0.5, where t_j is the hours since that use; d = 0.5 is ACT-R's default (Anderson et al. 2004), and the equation is based on Anderson and Schooler's 1991 finding that the odds of needing something again fall as a power of the time since it was last needed. By our arithmetic, one trace halves each time its age quadruples, so a 1-day-old trace is exactly 10 times stronger than a 100-day-old one: a long tail, never quite zero.
Where you are tops it up: A = B + W (S − ln fan), with one cue in focus (W = 1) and S = 2 (Anderson et al. 2004). Home and the car park are each wired to one memory, so they add 2.0; work is wired to two, so each gets 1.31, because a cue that points to more memories is a weaker reminder (the fan effect). The hatched band in a window is this top-up.
Only above the bar comes to mind: the chance of recall is P = 1 / (1 + e^(−(A − τ)/s)), and the lamp's brightness is P. The thin bar and the notch on each window sit at τ = −1, with noise s = 0.4. Both are illustrations: the authors say the threshold varied substantially from model to model, and that s is typically about 0.4 (Anderson et al. 2004). The readout puts P into words: "comes to mind" at 90% or more, "probably" from 60%, "a coin flip" from 40%, "probably won't" from 10%, and "gone for now" below that.
What is illustrative: every memory's history (the phone number used once a day for 90 days, the Wi-Fi password typed three times last week, the client's name heard once two months ago), the links between places and memories, the threshold and noise, hours as the time unit (another unit shifts every charge by the same amount, which only moves the bar), counting a trace younger than one hour as one hour old (the equation is infinite at age zero), and the short fill-in when a battery is used (a third of a second) or the dial turns (just over half a second). The tour's clock runs slowly while something happens and fast through quiet hours, from a fifth of an hour to two and a half hours per second; when you drive the machine it runs at 1.5 hours per second. The track on the front edge shows Monday and Tuesday, with night shaded from 8 pm to 6 am.
Limits: d = 0.5 is a lab default, not a constant of nature; fits to hashtag and music data found decay rates from 0.86 to 1.74 (Kowald, Pujari and Lex 2017; Lex, Kowald and Schedl 2020; Reiter-Haas et al. 2021), and for predicting a user's next hashtag, simply ranking by most recent use came within a few percent of this equation (Kowald, Pujari and Lex 2017). ACT-R models forgetting as decay over time; whether time itself causes forgetting is debated (Wixted 2004).
Sources: Anderson and Schooler (1991), Reflections of the Environment in Memory, Psychological Science; Anderson, Bothell, Byrne, Douglass, Lebiere and Qin (2004), An Integrated Theory of the Mind, Psychological Review, p. 1042; Lewis and Vasishth (2005), Cognitive Science; Taatgen, Lebiere and Anderson (2006), Modeling Paradigms in ACT-R; Wixted (2004), Annual Review of Psychology.
Multi-store model
How you hold a few things in mind at once
Based on the multi-store model, where what you notice moves from a brief first impression into short-term and then long-term memory, and on working memory, the mind's small workbench.
How it works
The model. Atkinson and Shiffrin (1968) split memory into a sensory register (the inbox), a short-term store (the rack) and a long-term store (the shelf). Rehearsal, something the person chooses to do, keeps items in the short-term store and builds long-term memory while they sit there. Moving an item on means copying it: the original stays behind and fades on its own schedule. Baddeley and Hitch (1974) argued the short-term store is a working space with parts; in Baddeley's later account (2000) those parts are a central executive (attention, no storage of its own: the switch), a phonological loop for speech (the rack and its lamp), a visuospatial sketchpad for pictures (the sketch pad), and an episodic buffer that binds them into one scene (the frame). In the figure Baddeley's parts are labels on the same desk; they add no numbers.
- Inbox (sensory register). Ink left = e^(-t / 0.8 s) for a heard word and e^(-t / 0.15 s) for a seen picture. These fade times are illustrations fitted to Sperling (1960), where about 9 of 12 letters were briefly available and the extra was gone within a second, and to Darwin, Turvey and Crowder (1972), who describe a sound store with a useful life of around two seconds.
- Rack (short-term store). 4 slots; when a new word arrives at a full rack, one card chosen at random is knocked off. That is Atkinson and Shiffrin's own rule for word lists, where they fitted a buffer of 4 (a value they fitted, and in another task they fitted 2; they treated the fixed size as a strategy, not a fixed part of memory). Not repeating: ink = e^(-t / 6 s), an illustration chosen to match their "about 15 to 30 seconds", so the rack is empty after about 18 s. A knocked-off card fades with e^(-t / 1 s), also an illustration of their "very rapid" loss. A card leaves the rack, into the discard tray, when its ink falls below 5% (an illustration).
- Shelf (long-term store). Each second on the rack adds 0.04 to a word's strength (their fitted value for word lists), and the shelf card's fill shows 1 - e^(-strength), the chance of recalling it from long-term memory; the word's ink darkens with it. Their search step at recall is left out, so the percentages show the mechanism, not a prediction for real people. While repeating is off, a fading card still adds 0.04 per second times its ink until it leaves the rack: that is our extension of their statement that transfer goes on while an item is in the short-term store. In the tour the first words stay longest and keep the most: apple 65%, river 62%, bread 50%, shoe 44%, seven 27%, coin 11%.
- Sketch pad. The picture sits on the sketch pad without taking a rack slot, because the loop and the sketchpad are separate stores (Baddeley 2000). In the figure the picture also fades when repeating is switched off, on the same 6 s curve as the rack. That is an illustration chosen to keep the desk simple: in Baddeley's account, repeating words to yourself does not refresh the sketchpad.
- How many things can you hold? Miller (1956) found a span of about seven items for many materials, counted in chunks (groups that mean something), and suspected the sevens were a coincidence; "plus or minus two" is only in his title. Cowan (2001) found 3 to 5 chunks in adults when repeating and grouping are blocked, and kept seven as a real limit for material people can repeat and group. The rack's four slots sit inside Cowan's range but are a choice made for this drawing, not a measured constant.
- Fading or crowding out? Whether short-term memories fade with time or are pushed out by new material is still debated. Berman, Jonides and Lewis (2009) found that time alone causes little forgetting and interference from other material more. In the figure the knock-off is the crowding out; the slow fade is the illustration.
- The lamp. It visits the cards on the rack one at a time, 0.6 s each. That pace is chosen to be calm to watch and carries no data; while repeating is on, every card on the rack stays at full ink whatever the lamp is doing.
- AI. The CoALA framework (Sumers et al. 2024) notes that earlier methods used the language model's own context as a form of working memory. MemGPT (Packer et al. 2023) treats the context as a fixed-size main memory: when it fills, old messages are evicted and folded into a running summary, and kept in an outside store that can be searched.
Sources: Atkinson and Shiffrin 1968; Sperling 1960; Darwin, Turvey and Crowder 1972; Baddeley and Hitch 1974; Baddeley 2000; Miller 1956; Cowan 2001; Berman, Jonides and Lewis 2009; Sumers et al. 2024; Packer et al. 2023.
Complementary learning systems
Why the brain learns at two speeds
Based on complementary learning systems, the idea that the brain has a fast learner for single moments and a slow learner for general knowledge, and that the two work together.
How it works
The loom is a small neural network first built by David Rumelhart (1990). You ask it a question such as "can a robin ...?" and it switches on answer units, each with a strength from 0 (no) to 1 (yes). These are answer strengths, not probabilities. Every unit computes a = 1 / (1 + e^-x) of its summed input, and after each fact every connection moves a small step toward the right answer, w = w + 0.1 × error × input (back-propagation with a learning rate of 0.1). Before the figure starts, the loom learns 32 facts about eight plants and animals over 1,000 passes, set up as in McClelland (2013).
Cram trains it on the two penguin facts alone: a penguin is a living thing, an animal and a bird, and it can grow, move and swim. Mix trains it on the same two facts among all 32 old facts in every lesson, which is what replay from the hippocampus provides in the theory of McClelland, McNaughton and O'Reilly (1995). Both use the same learning rate. Mixing takes far more practice: 200 practice runs for cramming (2 facts in each of 100 lessons) against 3,400 for mixing (34 facts in each of 100 lessons), one training step each.
What the cloth shows: in each row, the six blue wave threads fill up in proportion to the loom's "can it swim?" answer for that animal. The loom answers other questions too, such as "can it fly?", which the cloth leaves out. The upright threads stand for the network's connections, which every animal's questions run through; that is why practicing one animal changes the answers for the others. They lean in proportion to the loom's error on the old animals (McClelland's 2013 measure over robin, canary, sunfish and salmon), from no lean before the lessons to full lean after 100 crammed ones; how far that is on screen is our drawing choice. A bird's swim patch gets an outline once its answer has risen 0.08 or more above where it started (fully drawn at 0.2; our choice of cut-off). The ruler counts lessons.
Left out of the cloth: the plant rows, because in our re-run cramming also teaches the rose to "swim" (0.05 to 0.57), more than the 1995 paper describes for plants; and the sunfish row, which behaves like the salmon (0.87 to 0.98 crammed, 0.94 mixed).
The numbers. In the 1995 paper, 100 crammed lessons raised the network's error on old "can" facts from about 0.13 to 0.67, while mixing never raised it above about 0.15 (read by us from their Figure 11). The numbers on this card come from our own re-run of the same test (seed 1). They are an illustration, not the paper's data, and they show the same pattern with a smaller effect: "can a robin swim?" goes from 0.04 to 0.39 when crammed and ends at 0.09 when mixed, and "can a canary swim?" goes from 0.13 to 0.59 crammed and ends at 0.21 mixed, after a slight rise to 0.24 on the way. Cramming teaches the penguin faster (0.90 against 0.70 after 100 lessons). Across seeds 1 to 10, cramming raised the error on the old animals 1.8 to 2.3 times, and mixing never raised it by more than 13%.
What is illustrative: the sketchbook's one-look drawing and its one replay per lesson (on screen the page goes up twice; in the model the penguin comes back in every one of the 100 lessons, and the 1995 authors call how often the brain really replays a memory "completely unknown"), the 100 lessons (chosen to match the 1995 figure), the time on screen (10 s for cramming and 16 s for mixing, with lessons running as 100 u² so the early lessons, where most of the change happens, get more time) and the shuttle's path. On a real loom only the newest row can be woven, so when mixing, the shuttle's visits to old rows stand for re-tightening them: practicing facts the loom already knows. This is not a claim that the brain learns by back-propagation; the 1995 authors call these models illustrations at an abstract level.
Since 2016 the theory adds that the neocortex can learn quickly when a new fact fits what it already knows, such as a new fish that swims or a new bird that flies; the penguin is slow because it breaks the bird pattern (Kumaran, Hassabis and McClelland 2016; McClelland 2013). The damage you see when cramming is called catastrophic interference, first described by McCloskey and Cohen (1989).
For AI, the retrieval paper (Lewis et al. 2020) splits a language model's memory into its trained weights and a searchable index of documents, and it updated the model's answers about world leaders by swapping the index, without retraining. Comparing that index to the sketchbook is our analogy, not the paper's. Fine-tuning language models on one task after another has been seen to make them forget earlier ones (Luo et al. 2023), and mixing even a little of the old training data back in reduces the forgetting (Ibrahim et al. 2024).
Sources: McClelland, McNaughton and O'Reilly (1995), Why There Are Complementary Learning Systems in the Hippocampus and Neocortex, Psychological Review 102, pp. 433-435 and 442; McClelland (2013), Incorporating Rapid Neocortical Learning of New Schema-Consistent Information Into Complementary Learning Systems Theory, Journal of Experimental Psychology: General 142; Kumaran, Hassabis and McClelland (2016), What Learning Systems do Intelligent Agents Need?, Trends in Cognitive Sciences 20; McCloskey and Cohen (1989), Catastrophic Interference in Connectionist Networks, Psychology of Learning and Motivation 24; Lewis et al. (2020), Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks; Luo et al. (2023), An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning; Ibrahim et al. (2024), Simple and Scalable Strategies to Continually Pre-train Large Language Models.
AI chat memory
How a chatbot keeps track of a long conversation
Based on how many AI chat systems handle memory: a context window that holds the text the model can read at once, plus summaries and a store of old messages it can search.
How it works
The rack holds 70 tokens: the fixed instructions (8 tokens), the note and the message cards. A token is a word or a piece of a word. Each card is as wide as its text divided by 3.5 characters per token, an estimate Anthropic gives for its earlier models. 70 is a toy size: real windows held 2,000 to 200,000 tokens in early 2024 (Packer et al. 2023) and up to 1,000,000 in 2026 (Anthropic). Only your messages are cards here; a real window also holds the chatbot's replies and any tool output.
Past 70 percent the readout says nearly full. When a new card would pass 100 percent, it waits in the printer while the oldest cards (about half the rack) go to the box, and a new note is written from the old note plus those cards. This follows the MemGPT research system (Packer et al. 2023), which keeps old messages in a store and a running summary in the window; the 70, 100 and 50 percent figures are its examples, not standard settings. The model does not do this by itself: if the text is too long the request is refused, so the software around the model makes room.
The note has room for 23 tokens, an illustration. When it is too long, this figure strikes out the oldest detail first, then folds lines about the same topic together. That simple rule can drop a detail even when it would have fitted: in the second note, peanuts goes although "Allergic to peanuts." is no longer than "Has a food allergy." A real note is written by a language model and chooses differently, and a note written from a note can lose more. Measured loss varies a lot: in one hand check about 4 percent of summary content had lost a detail (Wang et al. 2023). In MemGPT's memory test, which was built to need details from old chats, models that saw only a summary answered 32 to 39 percent of questions, against 67 to 93 percent when they could search the full history.
Search: the question and each card are turned into lists of numbers (embeddings) that put related text close together, and similarity = cos θ = Σ qᵢnᵢ / (|q| |n|), where 1 means the lists point the same way. Each card in the box rises in proportion to its similarity, and the 2 best cards at or above 0.5 come back as copies with a paper clip; the original stays in the box, and a copy that is pushed out again just goes back to it. Our lists have 16 hand-set numbers; real ones have 768 to 3,072 learned numbers (Karpukhin et al. 2020; OpenAI). Similar is not the same as correct: the octopus card scores 0.78 against the film question, close to the film card's 0.80, so it comes back too.
Illustrative, not measured: the 70-token rack, the 23-token note, the rule that the oldest details go first, the 2 cards, the 0.5 floor and the hand-set embeddings. Published: 3.5 characters per token, the 70, 100 and 50 percent examples, the window sizes and the loss figures above. Simplified: every question is searched, and old cards are kept in the box, though many apps simply drop them.
Sources: Packer et al. 2023 (MemGPT); Lewis et al. 2020 (retrieval-augmented generation); Karpukhin et al. 2020 (dense retrieval); Wang et al. 2023 (recursive summaries); Anthropic 2025 and 2026 (context windows, compaction, glossary); OpenAI (embeddings guide).
The machines follow published models of memory. The numbers behind each one, and where they come from, are under its How it works.
Want this for your own agents? KITE is our open-source memory engine.
Star KITE on GitHubOr connect Memoket to ChatGPT, Claude or an agent
