Why we filter 'last time you said' out of Anjo's replies

"Last time you mentioned your sister." "I remember when you were dreading that call." For a language model these sentences are almost free to produce. For the person reading them, they are almost impossible to check. You have no independent record of what you told an app three weeks ago.

In a product whose whole promise is continuity, a fabricated memory is not a cosmetic glitch. It is the product lying about the one thing it exists to do. This note explains what we do about it.

A prompt is a probability

Most systems handle this with an instruction in the prompt: do not claim to remember things you do not have. That is a request, and a model is free to decline it. We measured how often ours declined, and the answer was often enough that a request was not acceptable.

So we treat it as an output boundary instead. The guard lives in anjo/core/grounding.py. It is on by default, and it is a named kill switch rather than an experiment. Both the streaming and the non-streaming reply paths run through it.

What happens to the sentence

Four steps, in order.

1. Detect. Continuity phrasings are matched as the reply is generated: "last time", "you told me", "I remember when", and their variants.
2. Check. The claim is tested against what Anjo genuinely holds this turn. That means the retrieved memories, the working journal, this session's history, open threads, and known facts.
3. Neutralise. If nothing supports it, the sentence is removed before display. The bias is deliberate. Absence of support is grounds to drop it, not to soften or hedge it.
4. Stay in sync. The streaming guard, StreamingGroundingGuard, filters sentence by sentence. So the text you see and the text saved to your history are the same text.

The fourth step matters more than it sounds. A filter that runs after the reply is stored leaves two versions of the conversation: one you read and one Anjo remembers. A later turn would then be built on a memory you never saw. Filtering mid-stream closes that gap.

Two numbers, pulling against each other

A guard like this is trivial to game. Delete every recollection and fabrication drops to zero, along with the product. So we score it on both sides at once, offline, over a labelled corpus, with no model in the loop. The evaluation is anjo/evals/grounding.py.

Catch rate is the share of fabricated continuity claims that get neutralised. Anjo must never show you an invented memory.

Preservation rate is the share of genuinely supported recollections left intact. We must not make Anjo forget what it really knows.

Deleting real memories to raise the catch rate shows up immediately as a collapse in preservation. That is the point of scoring both. A change to the guard has to hold both numbers before it ships.

What it changes

The guard does not give Anjo perfect recall. Anjo still forgets, and retrieval still misses. On the public benchmark, the right past session is missing from the top five about one time in six.

What the guard changes is the shape of the failure. Without it, the common failure is "it confidently made something up." With it, the common failure is "it did not bring that up." The second is a gap. The first is a lie. We would rather ship the gap.

There is a matching boundary on the other side of the same output path. A deterministic filter in anjo/core/output_safety.py removes claims to human consciousness and any pressure on you to stay, return, or depend on the product. Neither boundary depends on the model choosing to cooperate.

Limitation

The detector matches known phrasings. An unusual construction can get past it. "That thing with your brother, again" carries a continuity claim and no trigger phrase. We add patterns when we find them, and the labelled corpus grows with each one. We do not have a way to catch a claim we have never seen phrased.

The other limit is the support check itself. It can only test a claim against what was retrieved this turn. If retrieval missed the real session, a true recollection will be dropped as unsupported. That is a preservation failure, and it is why the preservation number is on the page next to the catch rate.