Short-Term vs Long-Term Memory in AI Agents

An agent planning Sam's training week calls a weather tool, reads Sam's calendar, drafts a plan and revises it twice. Most of that is useful for the next ten minutes and useless next month.

A few things from the same task are worth keeping: Sam prefers morning runs, and Sam's long run has moved to Sundays. Handling both needs two kinds of memory, one for the task in progress and one for what outlives it.

Two lifetimes

Short-term memory is the state of the current task or thread: the messages so far, tool calls and their results, and scratch work such as a draft plan. It exists so the agent can continue the task. When the task ends, most of it has done its job.

Long-term memory is the set of facts that outlive the thread: Sam lives in Lisbon, prefers morning runs and is training for a half marathon in March. It exists so the next task, weeks later and in a different thread, starts with what the agent already learned.

Where each one lives

In a typical tool-calling loop, short-term memory is the message list the agent appends to: the request, each tool call and its result, and the model's intermediate replies. The agent reads it in full, in order, on every step.

That state lives close to the agent runtime. For a single run, process memory is enough. When a task can pause and resume, the state goes in a database keyed by thread id.

LangGraph draws the same line. A checkpointer persists a thread's state so a conversation can resume where it stopped, and a store holds memories shared across threads.

Long-term memory lives in a store scoped to the user it describes and searchable by meaning, because each task needs a few relevant items rather than the whole archive.

Two common mistakes

Treating the transcript as long-term memory

The thread log is kept forever and loaded into every new task. Prompts grow with each session, old tool output crowds out current facts, and a preference Sam changed in April sits next to the one from January with nothing to say which is current.

Treating long-term memory as a log

Every message, tool result and draft is written to the long-term store. Searches then return "checking the weather for Saturday" next to "lives in Lisbon", and the facts that matter compete with records that were true for one afternoon.

The fix is the same in both cases. Keep short-term state with the task, and write to long-term memory only what should still be true after the task ends. For what replaying a long thread on every step costs, see Why a Bigger Context Window Is Not Memory.

Summaries sit in between

Long threads are often compressed into a running summary of the older turns. That summary is still short-term memory: it belongs to the thread and describes the task, not the user. Promote the individual facts in it, such as the new long-run day, and let the summary expire with the thread.

What to promote, and when

Promotion copies a durable fact from short-term state into long-term memory. Good candidates:

• Facts users state about themselves: "I live in Lisbon."
• Preferences, especially corrections: "Mornings work better than evenings for me."
• Decisions and outcomes the user confirmed: "Yes, keep long runs on Sundays."

Poor candidates include tool results that will soon be stale, such as Saturday's forecast, intermediate reasoning, drafts the user rejected, and anything sensitive such as credentials.

Two moments work well for writing:

• After a turn, when the user has just stated something durable. Writing then means the fact survives even if the session ends without warning.
• At the end of a task, when the outcome is known. Writing the final plan instead of each draft keeps rejected options out of memory.

For the planning task above, promotion means two writes: Sam prefers morning runs, and Sam's long run is on Sundays. The forecast, the calendar lookup and the two rejected drafts stay with the thread and expire with it.

Expiry, change and deletion

The two kinds also need different upkeep.

Short-term memory needs expiry. A thread paused three months ago is rarely worth resuming, so set a time to live on thread state or archive it when the task closes. Otherwise storage and load times grow with every task the agent has ever run.

Long-term memory needs change handling and deletion. When Sam moves to Porto, the Lisbon fact has to stop being returned, or the agent plans runs in the wrong city. When Sam asks the agent to forget something, it has to leave the store along with any copy derived from it.

A useful rule: thread state expires on a timer, while a long-term memory changes only when the fact it describes changes.

Where MemorySync fits

MemorySync is the long-term half. Your agent stores durable memories for each end user, and before a task it asks for the ones relevant to that task. We return them ranked by meaning, scoped to the end user named in the X-End-User-ID header, and reads, searches and deletes are filtered to that end user, project and environment.

In code, the long-term half is a read when a task starts and a few writes when it ends. Each call is a request over the network to our API, so one read at the start of a task usually covers it; there is no need to query again on every tool step:

from memorysync import MemorySyncClient

ms = MemorySyncClient(
    api_key="...",
    base_url="https://api.memorysync.io",
    end_user_id="sam",
)

# Start of the task: load what is relevant to it.
known = ms.query("Plan Sam's training week", k=5)

# ... the agent works; thread state stays in your runtime ...

# End of the task: promote only what should outlive it.
ms.add("Sam prefers morning runs.")
ms.add("Sam's long run is on Sundays.")

Short-term state belongs with your agent runtime, next to the thread it describes. The optional session_id on MemorySync calls groups related calls, but it does not change which end user owns a memory and is not an access boundary.

Next step

The agent memory guide lists what an agent should read before a task, what to write after it, and what to keep out of memory.