Consumer Chatbots
Conversations that continue where they left off
Give consumer assistants durable memory of preferences and past exchanges, so returning users are recognised months later without resending the transcript.
The problem
Transcript replay works within a session and fails between them. A user returns after a month and the assistant has forgotten everything, so they re-establish preferences they already stated. Keeping the transcript instead makes every message more expensive and slower than the last, and eventually exceeds the window anyway.
With MemorySync
MemorySync stores what each conversation established rather than the conversation itself. A returning user's preferences are retrieved in a small ranked set, so recall does not depend on transcript length and does not grow with it.
How it works
- 1
Extract
Durable preferences and facts are drawn from each exchange.
- 2
Scope
Memories bind to a stable opaque user identifier.
- 3
Retrieve
A returning user's relevant facts are fetched before the reply.
- 4
Update
Changed preferences supersede the ones they replace.
Why transcript replay runs out
Resending the conversation is the obvious first implementation and it works, until three things happen in order: cost grows with the square of the conversation, latency grows with the payload, and the session ends and takes everything with it.
A larger context window raises those ceilings without removing them, and introduces a relevance problem — one fact inside eighty thousand tokens is harder for the model to use than one fact inside two hundred.
Preferences are the durable part
Almost nothing in a consumer conversation is worth keeping verbatim. What matters is the small set of facts it established: dietary constraints, tone preference, the project they are working on, the thing they said not to suggest again.
Those stay true across sessions and are worth retrieving months later. The rest of the conversation is not.
Users do not expect an assistant to recite what they said. They expect it not to ask again.
Changed preferences must replace, not accumulate
Consumer preferences change more often than any other memory class. Someone who preferred concise answers in March may want detail in September.
If both facts are retrieved, the model picks inconsistently between requests, and users read that as the product being unreliable. MemorySync supersedes contradicted facts so the current preference governs.
Scope with an identifier you will not regret
Use a stable, opaque internal identifier as the memory key — never an email address or phone number. Those change, they are personal data, and once they are the key both facts become expensive to fix.
Retrieve a small set
When answers seem to lack context the instinct is to raise the retrieval limit. That usually makes things worse: precision falls and the model spends attention discriminating rather than answering. Five to ten memories is the working range for conversational applications.
Degrade rather than fail
Memory should improve a reply, not gate it. Wrap retrieval in a timeout and answer without it if it is slow — a slightly less personalised response is far better than an error. Writes can happen off the request path entirely.
What you get
- Recognition across sessions and devices
- Cost does not grow with conversation length
- Preferences persist for months
- Changed preferences replace old ones
- Small ranked sets keep prompts fast
- Opaque per-user scope
Use cases
Build consumer chatbots with durable memory
Start on the free tier and test retrieval, scoping, and deletion against your own workload.
Start free. No credit card required.