I run Lovora, an AI companion service for adults in nine languages. The number one thing people ask an AI companion is "will she remember me?", and "memory" turns out to mean two different things. Here is the design we use, and why.
Two layers, two jobs
1. The recent conversation, word for word.
On every reply the model receives the last 40 messages of the chat, both sides, exactly as written. In voice calls it is the last 24, because spoken turns need to be answered faster and the prompt has to stay small.
This layer is what keeps a scene or an argument coherent. A summary would lose the exact wording, the jokes and the thing she promised two messages ago, which is exactly what people notice in a conversation.
2. Long-term facts about the user.
Every 5 messages the user sends, a separate pass reads the latest part of the conversation and updates a short list of lasting facts: at most 15 short facts, about 1,500 characters. That list goes into the prompt of every later reply, in chat and in calls.
That is the layer that lets her ask about the job interview you mentioned last week, hundreds of messages later.
What goes into the fact list (and what does not)
The distillation pass is instructed to keep facts about the user, never about the character, whose personality and backstory are fixed:
- name, age, job and where they live, if shared;
- interests, likes and dislikes;
- relationship, family and pet details;
- plans and events mentioned, and how they feel;
- stated preferences, including intimate ones;
- anything they explicitly asked her to remember.
Small talk is discarded. A fact that is contradicted later is updated or dropped, not appended: the list is rewritten, not grown. The character is told to use what she remembers naturally, by asking about it or referring to it, and never to recite it as a list. Reciting is the fastest way to make a companion feel like a database.
Why not embeddings and retrieval?
We chose not to retrieve past messages by similarity. For a companion it has three problems:
- Relevance is not similarity. The message that matters ("my mother is in hospital") is rarely the one most similar to the current message ("good morning").
- Retrieved fragments arrive without context, so the model quotes them oddly or at the wrong moment.
- It grows forever. A capped, rewritten list of 15 facts costs the same on message 10 and message 10,000, and it is easy to inspect, correct and delete.
A small fixed budget forces the distillation step to decide what matters, which is exactly the judgment retrieval skips.
Scope and privacy
- Per companion. Each companion has her own conversation and her own memory; what you tell one is not known to another.
- Deletable. Deleting the account erases the conversations and what she remembers (what deletion erases).
- Model-agnostic. Both layers are plain text in the prompt, so they survive a model change.
What it costs
The verbatim window is the expensive part: 40 messages is a real share of the context on every reply. The fact pass is cheap: it runs once every 5 user messages, as a separate call on a short excerpt.
If you are building anything conversational (support bots, tutors, companions), I would start here: a verbatim window sized to your latency budget, plus a small, rewritten list of facts about the user. Retrieval can come later if you ever need it.
The full, current description is in our docs: How memory works on Lovora.
