Skip to content

Memory

Memory is stored in a Cloudflare Durable Object:

CHAT_MEMORY -> ChatMemorySQLite

Configured in wrangler.toml:

[[durable_objects.bindings]]
name = "CHAT_MEMORY"
class_name = "ChatMemorySQLite"

State Stored Per Chat

The Durable Object stores one state object per chat id:

Field Meaning
messages recent ring buffer
facts simple extracted facts
indexJson serialized MiniSearch index
totalMessages total indexed messages
routerUsage recent model usage by tier

Short-Term Memory

The Worker keeps the last 12 messages in the ring buffer.

Mode Injects up to
auto (normal) 4 recent messages
full 12 recent messages

Search Memory

The Worker uses MiniSearch BM25-style full-text search.

On each user message, it:

  1. Adds the user message to the index.
  2. Searches older messages using the new user text.
  3. Retrieves up to 3 older relevant messages.
  4. Injects those as a system memory prompt.

Memory Prompt

The injected prompt starts like:

Relevant chat memory. Use these exact prior words only when helpful.

Then it may include:

Recent messages
Retrieved older messages

Fact Extraction

The Worker also extracts simple facts from user messages, such as:

name
location
preferences

These facts are stored, but the current answer prompt mainly uses recent and recalled messages.

Chat Memory Compaction

When a conversation's message history grows beyond 25 messages, the Worker automatically triggers a background compaction:

  1. The full message history is sent to a lightweight summarization model.
  2. If a previous summary exists, it is merged with the new messages.
  3. The summary is saved to the Durable Object state.
  4. Only the 5 most recent messages are kept in the active history.
  5. The cycle repeats every time the history grows back to 25 messages.

This keeps the Durable Object SQLite storage efficient and prevents context windows from growing unbounded, while preserving long-term conversational context through the rolling summary.

Compaction prompt structure

The compaction prompt uses an anchored summary structure:

## Objective
## Important Details
## Work State (Completed / Active / Blocked)
## Next Move
## Relevant Files