ElevenLabs + MemorySync

Overview

Install elevenlabs-memorysync and ElevenLabs agents remember callers across calls — on the web and on the phone. Three tiers in one package: a zero-code webhook tool the agent calls directly (dashboard only), session-start dynamic variables plus an HMAC-verified post-call transcript receiver, and the flagship Custom-LLM proxy that injects memories into every turn under a hard budget (default 1.2s) while capturing both sides of the conversation with idempotency seeds. The proxy works with any OpenAI-compatible upstream (competitors hardcode OpenAI), and the novel prompt-tag mechanism — one memorysync-user: {{system__caller_id}} line the proxy extracts and strips — gives phone-caller identity with zero client code. Running proxy and webhook together yields live memory plus an end-of-call sweep that converges at zero duplicates.

Setup status and requirements

Supported version

elevenlabs-memorysync 1.0.0 (PyPI)

Last setup review

2026-08-24

Permissions

A MemorySync API key with read and write scopes; an ElevenLabs workspace with Custom LLM and/or post-call webhooks.

Limits

Python 3.10+. The proxy must be reachable by ElevenLabs over HTTPS. Post-call webhooks are a workspace-level setting (they fire for all agents in the workspace).

Capabilities

• Hard recall budget (default 1.2s) — a reply is never late, tested against a 5s-slow backend
• Any OpenAI-compatible upstream LLM — OpenAI, Azure, Groq, Gemini compat, LiteLLM
• Phone-caller identity via prompt tags reading {{system__caller_id}} — stripped before the model sees them
• No identity → pure passthrough: unattributable words are never stored
• Byte-faithful SSE relay — tool-call deltas (end_call, …) pass through untouched
• Post-call webhook with signature verification pinned to the official SDK scheme
• Live capture + end-of-call sweep converge at zero duplicates via shared seeds
• Optional buffer words spoken only when the turn is already slow

Quick Start

# server.py — the agent's Custom LLM
from elevenlabs_memorysync import create_proxy_app

app = create_proxy_app(
    api_key="ms_...",              # MemorySync
    upstream_api_key="sk-...",     # your LLM provider
    proxy_api_key="a-long-random-secret",
)
# uvicorn server:app --port 8013
#
# Agent prompt gains one line for phone identity:
#   memorysync-user: {{system__caller_id}}

Use Cases

Explore More