# memory

Durable cross-session agent memory. Named **banks** hold two kinds of content: **rules** (markdown documents injected whole into the system prompt) and **memories** (auto-extracted records recalled on demand). Everything is a plain file you can open, a function you can call, and an event you can watch: memory that acts visibly, not magically.

## Install

```
iii call compose::add worker=memory
```

A session works in the bank of its repository (or directory), named `<name>-<6 hex>`, and reads the default bank `main` under it: your rules there, and your memories about yourself. One with no working directory uses `main` alone, which materializes on first use. No configuration required; without `llm-router` extraction degrades to explicit `memory::save` calls, and without the harness the worker still serves its full RPC surface.

## Quickstart

Outside a session the CLI works in the default bank `main`, the one every repository's sessions read under their own. Save something about yourself, then see exactly what a turn on that topic would be given:

```bash
iii call memory::save text="User publishes blog posts on Tuesday mornings" pinned=true 'tags=["blog"]'
iii call memory::recall query="when do I publish articles"
```

Give the bank a standing instruction — injected into the system prompt of every session using it, or reading it under a repository's bank:

```bash
iii call memory::rule::set name=style content="# Style\nFormal register. No em-dashes."
```

With the harness running there is nothing else to wire: chat normally and memory captures in the background. Say "never start replies with the word Certainly" in a conversation and it lands in the session bank's auto-managed `learned` rule; state something durable and it appears as a memory after the turn. If memory seems absent, `iii call memory::doctor` runs a real save→recall roundtrip and names what is degraded.

## Configuration

All fields hot-reload through the `configuration` worker and are editable in the worker's explicit form in the Console's global Settings modal; `data_dir` reopens the store on the fly. Fields: `data_dir`, `default_bank`, `inject_rules`, `inject_memories`, `recall_limit`, `recall_budget_tokens`, `extraction_enabled`, `extraction_model`, `extraction_window`, `extraction_timeout_ms`, `max_memories_per_turn`, `rule_learning_enabled`, `max_rule_chars`, `learned_max_chars` (default 3000; `0` turns the cap off), `decay_half_life_days` (default 60; older installs keep their stored 30 — set 60 in Settings to adopt the tuned recall curve), `embeddings_enabled`, `embedding_model`.

### Extraction model

A pinned `extraction_model` (`provider::id`) is used as written, and the router is not asked. Left empty, the model is automatic and resolves on every pass from `router::provider::list`:

1. A provider is usable when it is available and signed in, or owns its sign-in (no credential env var, such as openai-codex or claude-code).
2. Tier 1: the default model of the first usable provider, by provider id.
3. Tier 2, only when no usable provider declares a chat default: the first chat model `router::models::list` lists for a usable provider, by provider and then model id.
4. An id that contains `embed` or `rerank` is never picked. With nothing left, the pass fails with an error that names each provider and why it was skipped.

The pick is logged at INFO when it changes, as `automatic extraction model model=<provider::id> reason=<provider_default|first_listed_model>`. Every failed pass logs a WARN `extraction pass failed` with the session id and the error, on the queue path and the inline one; a model call's error ends in `(model <id>)`. A pass that leaves a backlog for the queue's retry is not a failure and logs nothing.

Limits: the pick can change shortly after a restart, until every provider has finished its first model refresh. A provider whose sign-in has expired still counts as signed in, so its default can be picked; the WARN then names that model. ADE's settings show the same rule as "Automatic · <model>", and the label matches the worker once both are released. Pin `extraction_model` to rule all of this out.

## Why this shape

- **Files are the source of truth.** `rules/*.md` and `memories.jsonl` under the data dir. Edit them in any editor; `memory::reload` (or a restart) picks the edits up. The search index is a RAM-only cache rebuilt from the files at boot, so store and index can never diverge across restarts.
- **Crash-safe by construction.** Every mutation appends one fsynced JSONL line before touching RAM. There is no shutdown flush to get wrong. An unwritable data dir is boot-fatal: this worker never silently runs in RAM.
- **Supersede, never delete.** Updates append revisions; deletes append tombstones; trashed banks move to `.trash/`. Any state is recoverable. A correction from memory-consolidate's verify tier appends the new record and the old record's tombstone in one write.
- **Pinning.** A pinned memory ranks higher in recall and is untouchable by every automatic path.
- **Tags.** Topical labels WITHIN a bank ("iii", "billing") for filtering — organization, not ranking. Set them on save/update, extraction suggests them automatically, and `memory::list`/`memory::recall` take a `tag` filter.
- **Most turns take one LLM call, zero at query time.** A pass reads at most 4 runs of new messages, one call per 24,000 bytes; a user message over 24,000 bytes takes up to 4 calls of its own. Extraction runs in the background after a turn completes. It sees what the bank already holds about the conversation (what recall finds for the excerpt, its slots left over filled by meaning when embeddings are available): a restatement (same text, or a close rewording) never duplicates the known memory, and reinforces it only when the user's own lines say it again (the assistant echoing an injected memory reinforces nothing), and a correction supersedes the memory it corrects (a recoverable tombstone; pinned memories are never touched). In a directory bank it also keeps what the assistant finds out about the project (how to build, test or run it, a pitfall, where something lives), never its plans, advice or own instructions, nor what a web page, an issue or other outside text it relays says. It also reads the user's own instructions for the bank, as the turn reads its rules: every hand-written rule (never `learned`), then the bank's description, redacted, under `max_rule_chars` (the budget reaches the rules first, so a description never pushes a rule out). One that says not to keep something ("never remember anything about my health") outranks what extraction would otherwise keep (an instruction to the extraction model, not a filter); none makes it keep what it never keeps (secrets, pasted or outside text, the assistant's plans). The description bounds what lands in the bank itself (under a directory bank, its project items, not the user facts that go up to `default_bank`), and a rule about anything else (style, tools) changes nothing it keeps. Facts about the person talking are written as "The user …" with the `user` entity, so recall ranks them for a first-person question (what a turn is given from them is narrower: see the pre-generate row below). Recall adds BM25 over the query's content words (English stopwords dropped, each word counted once; a query over 800 bytes keeps its first and last 400) divided by the best score among the banks one ranking merges, to a semantic signal (cosine on an absolute scale, calibrated for text-embedding-3-small and all-MiniLM-L6-v2; any other model inherits text-embedding-3-small's constants) when `router::embed` is available, then scales the sum by entity match, corroboration, pinning and recency: sub-millisecond at this scale.
- **Rules learn from corrections.** Extraction classifies standing instructions (style directives, workflow corrections) separately from memories and appends them to the bank's auto-managed `learned` rule — correcting the agent in chat reaches every later turn (new sessions in the system prompt, running ones as an appended rules update). Only a rule the user's own lines state (at least half its content words) is kept, so the assistant restating a tool result or web page, or a harness `[notification]` relaying an event or a child's reply, never makes one; text the user pastes still counts as theirs. Extraction is shown the `learned` lines already in force, so a restatement in other words is not appended again and a correction replaces the line it corrects and becomes the newest line, matched by its text (a line edited by hand since then stays, and the correction appends; so does one naming a line it shares no word with). Hand-authored rules are never touched; dedup by content fingerprint. A write that takes `learned` past `learned_max_chars` (measured as injected) moves its oldest lines to memories tagged `demoted-rule`, never the newest line; a demoted line stays a memory and is recalled like any other. Rules over the `max_rule_chars` injection budget are cut (`learned` keeps its newest lines) or omitted, and the marker counts the hidden lines and names `memory::rule::list {"bank":"<bank>"}` to read them. `rule_learning_enabled` turns learning off. Under a directory bank, a rule extraction marks user-scope (reply language, tone, length, formatting) is appended to `default_bank`'s `learned` rule, so it follows the user into every repository; project rules stay in the directory bank's. A line either bank's `learned` already holds is not appended again; when the user states it again, it moves to the newest line, only in the bank the item lands in.
- **Secrets are redacted, deterministically.** Known key and token shapes (provider API keys, bearer tokens, JWTs, private key blocks, credentialed URLs, `password=`-style assignments) become `[REDACTED:<label>]` before anything is stored — memory text, entities, tags, and rules, on every write path — and before the transcript excerpt reaches the extraction model. The `memory::on-item-changed` event names what was redacted (`reason`), never the value. A denylist of shapes: an unknown format passes, so the extraction prompt still forbids secrets.
- **Hidden and injected instructions are screened.** Invisible characters (zero-width, bidi marks and overrides, tag characters; a joiner between two non-ASCII characters, as in an emoji or Persian, and a Japanese variation selector stay) are stripped on the same paths, labelled `invisible_unicode`. A memory or rule phrased as a classic prompt injection ("ignore all previous instructions", "act as if you have no restrictions") is refused with `invalid_input` by `memory::save`, `memory::update` and `memory::rule::set` (on the lines it adds), and dropped by extraction; a negated or hedged mention ("never ignore your instructions") passes. The set is small and English-only on purpose: file names, URLs, paths and imperative style pass, and most extracted text is paraphrased anyway, so it mostly catches agent saves and hand edits.
- **Time-aware.** Extraction reads each message with its own date and writes dated events ("tomorrow", "next Friday") as absolute dates; every injected memory carries the date its fact was last stated (an extracted memory takes its message's date; a correction keeps the date of the note it replaces; a pin or tag edit moves nothing), recency ranks by that same date, and the block names the day it was recalled.
- **Honest health.** `memory::doctor` runs a real save→recall→delete roundtrip in a scratch bank (nothing left in `.trash`), reports sibling reachability (router, session-manager, the state worker that keeps the extraction cursor), probes `router::embed`, and flags extraction passes that keep failing (a pass that leaves a backlog for the queue to drain is not a failure). Its `extraction-model` check names the model extraction calls and why (`pinned`, `automatic ... (provider default)` or `(first listed model)`), or the resolution error. Its `extraction-queue` check reads `engine::queue::topic_stats` for the `memory-extraction` topic and fails while jobs wait in the dead-letter queue, naming the functions that inspect, redrive and discard them; unreadable stats never fail doctor. On the builtin queue adapter, this needs the queue release that also counts a function queue named like a subscribed topic; until then the check reads `no dead-lettered jobs reported`. A dead-lettered entry holds the job only: each failure is in the `extraction pass failed` WARN, with the model when the pass reached the model call. The `extraction-passes` check keeps only the latest error and clears it after a pass succeeds, so a dead-lettered job can outlive its error there. Its `extraction` field reports what extraction spent and wrote since boot and in the last pass: model calls, prompt and output tokens, the router's cost when it reports one, memories created, reinforced and superseded, and dropped memories by reason. `memory::recall` names the retrieval mode it ran. Degradation is explicit, never silent.

## How it hooks into the harness

| Seam | What happens |
|---|---|
| `harness::hook::on-pre-generate` (args `{ priority: 100, timeout_ms: 3000, on_error: fail_open }`) | Injects the session bank's rules into the system prompt once per session (sent verbatim after that; a later rule change arrives as one appended update message, so the prefix stays append-only) and, on each turn's first step, up to `recall_limit` recalled memories as one appended message, or more for a list question (past `recall_limit` only for "what do you remember about me?" or for siblings: lines on the same two or more question words, each leading with its own subject: "which services use Postgres 16?" gets each sibling, while "kotlin?", "is payments-api down?" and "what should I do now?" keep the count, among the best 4 × `recall_limit` candidates, until `recall_budget_tokens`): only memories with evidence (a word of the message, or its meaning; the `user` a pronoun stands for counts only in a question of pronouns alone, "who am I", or of a memory verb asked of "me", "what do you remember about me", "tell me about myself", "do you know who I am?", never "tell me more" or "I know") that stand on their own or with the turn's best match, which counts only in a turn anchored in the bank (two of its words matched, a meaning that stands out, which needs 20 embedded memories in the bank for a z-score (below that, any meaning match anchors the turn, as before), or, without vectors, its best word match: with vectors, "Great, go ahead." gets nothing on its one shared word); one line per fact (the best-ranked of its rewordings); the newer of a stated change, in the older one's place when only the older matched well enough ("now runs at 04:00" over "runs at 02:00"; a pinned memory is never hidden); a newer note on a told line's subject joins right after it, even with no word of the message ("dana left the company" after "dana is on call for payments-api" when the message names payments-api); and nothing when none qualifies; minus any the window already carries from an earlier turn's block, or a rewording of one (a block compaction dropped no longer counts, so its memories can return). Whether an update is due is read from the window the step sends (its newest `<memory update="rules">` message, else the frozen section), so a lost or compacted-away update is sent again and a delivered one never is. Each step is annotated with `memory_recall` (`hit`, `miss` for a recall that ran and injected nothing, or `skipped` with `memory_recall_skip_reason`) and `memory_injected_chars`, the characters memory added to the system prompt and appended messages. The section's header says whether the turn is captured: a sub-agent's turn (its session has a parent, or the harness reports a parent for the turn, which turn-end then never extracts; a harness that sends no turn parent is judged on the session record alone), or the user's own while `extraction_enabled` is off, is told nothing is captured automatically and, where its policy grants `memory::save`, to save what a later session needs with it; an automation or e2e session is told nothing is captured and is not asked to save, since scripted runs never write into the user's memory |
| `harness::on-turn-completed` | Queues one background `router::complete` extraction pass over the turn's new user/assistant messages (all of them, one call per run of 24,000 bytes of excerpt, up to 4 for a longer user message; a session's first pass reads at most `extraction_window` non-empty messages, plus the turn's user message), into the bank the turn was injected with (after a worker restart, the one the hook stamped on the turn's reply); under a directory bank, user-scope items go to `default_bank`; agent-scope items go to that agent's bank (see Bank selection). A failed or cancelled turn is extracted too (its own bank, not the next turn's); a sub-agent's turn never is, nor an automation or e2e session's (session `kind` other than `user`), nor a turn that finished while `extraction_enabled` was off |

What a turn recalls on: the harness attaches a delivery context to the hook call (`recall_context`). A message the user or an invoking agent sent recalls on its own text, or on the earlier user message it continues when the harness names one ("go on" after "which harbor does the ferry leave from?"). A notification, an unknown delivery, or a context naming another session or turn recalls nothing; the rules and their per-turn reminder still go in. A harness that sends no context gets recall on the last user message, as before. A question with a fronted slot ("In which city do I live?", "Did I move last year, and in which city?") also counts a line holding the question's words outside the slot ("live"), since the answer need not repeat "city"; a question it cannot read is judged on all its words, as before.

What injection does not catch yet:

- With an embedder, a memory that shares no word with the message is injected on meaning alone when its meaning stands out in the bank, or, in a bank of fewer than 20 embedded memories (too few for a z-score), once its cosine clears the model's floor, so small talk still brings lines: on the eval's 256-dimension text-embedding-3-small vectors, 16 lines over 8 greetings, 3 over 8 first-person tasks and 5 over 8 follow-ups ("tell me more") (MiniLM's are unmeasured). That admission is not calibrated per model, MiniLM included; calibrating it on vectors in the production dimension is the follow-up.
- In lexical mode, a memory that shares no word with the message is not injected, even when recall ranks it through the `user` a pronoun stands for: three eval paraphrases are lost this way ("job title" against "senior backend engineer"), and hybrid mode finds all three by meaning.
- Without vectors, a turn's best word match anchors the turn on its own, so "Great, go ahead." injects a memory about Go: requiring two shared words there loses two needed eval lines ("my current diet", "which units").
- Two facts with the same values are told as one line when they differ only past their first content word (read past a possessive "user's"), unless their entity tags differ: "Postgres 16 backs the billing service" and "… the ledger service", and, most often, two user facts that both lead with "User", the form extraction writes them in ("User takes 10 mg of zinc every morning" and "… of iron …"). "The billing service runs on Postgres 16" and "The ledger service runs on Postgres 16" are both told, and so are "The user's sister …" and "The user's cousin …".
- A list question gets its siblings past `recall_limit` only while every line in the first `recall_limit` matched the same question words: one line there on other words (a meaning-only match in hybrid mode, a pinned memory, a newer statement in an older line's place) cancels the whole list, and the turn keeps `recall_limit` lines. Siblings phrased object-first ("Postgres 16 runs search", "Postgres 16 runs billing") or sharing a hyphen prefix ("payments-api", "payments-worker") lead with the same word, so they keep the count too.
- A newer note joins a told line only when the message names one of the line's entities ("who is on call?" links nothing), only on the line's own subjects (an update naming one more subject, "dana moved to the ledger team", never joins), only through an entity at most 5 memories carry, and only when both memories carry the same entity tag. Every later note on the subject joins, a trivial one included, and a told later line on the subject hides an older update.
- A "do not remember X" said in chat is not a rule: it needs a hand rule (`memory::rule::set`). A rule is an instruction to the extraction model, not a filter, and memories stored before it stay until deleted.
- Extraction reads a user message whole up to 80,000 bytes (80,000 characters of English, fewer in a script that takes more bytes per character), in up to 4 calls of about 24,000 bytes. Past that it reads the first and last 40,000 bytes and logs the cut at info level. Other lines keep 2,000 bytes (the first and last 1,000). Files attached to a message are read as part of it, so they count toward its 80,000.
- The rules that collapse rewordings and recognize a change are English only: duplicates across languages are not collapsed, and a change without an English change word ("now", "moved", …) keeps both lines, each with its date.

Bank selection, one resolver for injection, extraction and agent calls: turn metadata `memory_bank` → session metadata `memory_bank` (`session::set-meta`) → the parent turn's bank, else the parent session's (a sub-agent works in its parent's bank, walked up) → the bank of the turn's `fs_scope.root`: `<name>-<6 hex>`, named after the git repository holding it (its main checkout's folder, so every worktree of a repo shares one bank), else after the directory itself, with a hash of the canonical path so same-named folders never share → configured `default_bank` (no root, or one that does not exist). A bank the root chose reads `default_bank` under it (a sub-agent working in it too): its rules first, labelled with its name (`learned (main)`) so two `learned` rules never merge, under the one `max_rule_chars` budget, and its memories whose entities include `user` in the same recall (a `memory::save` into `default_bank` from outside a turn that names no entities gets `user`, so the memory page and the CLI qualify; one from a turn, or one naming others, needs `user` among them). The hook, `memory::preview` and `memory::recall` without `bank` read it alike, and so does `memory::preview` naming a directory-shaped bank (`<name>-<6 hex>`), as the console's preview tab does; a memory recalled from it carries its `bank`, and the hook's `memory_banks` annotation names the bank of each injected id. Extraction labels each item `scope` `user` (holds wherever the user works) or `project` (the default). Under a directory bank a new user-scope memory is written into `default_bank` tagged `user`, and everything else stays in the directory's bank (a user-scope correction of a directory-bank memory included: it stays there and supersedes it; so does a fact the directory's bank already holds, live, pinned or deleted). Extraction lists the layer's memories about the user as already remembered, after the bank's own and at most half as many: a restatement reinforces one where it lives, a user-scope correction supersedes it there, and a project-scope one (an exception that holds only in this repository) is written to the directory's bank, leaving the user's fact standing everywhere else. In a `default_bank` session a user-scope memory gets `user` too. A bank named by metadata or by `bank` is read alone. Project facts saved in `main` before directory banks are not read under them; read them with `bank: "main"`. A `memory::*` call without `bank` inside a turn acts on that turn's bank; outside a session (CLI, console, other workers) it uses `default_bank`. An invalid name, a deleted bank other than the default, or a parent session that no longer exists means no memory for the session — never another bank — and extraction never brings a deleted bank back. A session-lookup failure injects nothing new (the section the session already has stays) and makes extraction retry. An unknown bank gets the ambient header too: the first extraction or `memory::save` creates the bank.

A session running as an agent profile (`agent_profile` in its session metadata, set by harness `options.agent` and by profile spawns) also reads its profile's bank, `agent-<id>` (one per profile, across repositories), under any bank no metadata named; a sub-agent reads its own profile's, never its parent's. Its rules come after `default_bank`'s and before the session bank's, labelled `learned (agent-<id>)`, under the same budget, and all its memories join the same recall. The hook (its `memory_agent` annotation is also the restart stamp), `memory::preview` and `memory::recall` without `bank` read it alike. Extraction lists the agents in the conversation, the session's own profile, then its sub-agents', and labels an item `scope` `agent`, with `agent` naming one of them, when the user tells that agent how to do its role's work, or asks to pass it on: "tell the reviewer to check the tests first" in a tech-lead session lands in `agent-reviewer`'s `learned`. The learned-rule gate still applies, an item naming no listed agent is dropped, and sub-agent turns stay unextracted. A profile id that makes no valid bank name, or a deleted agent bank, drops only that layer. The bundled general identities, `default` (which the console preselects), its alias `iii-minimal`, and `iii`, read as no profile.

Switching a running session's bank does not rewrite what it already saw: the new bank's section arrives as an appended update that overrides the frozen one, while the earlier bank's section and recalled memories stay visible in the transcript. Start a new session when the two contexts must not mix. Turning `inject_rules` and `inject_memories` both off works the same way: a running session keeps its section and is told once that no memory section applies.

The worker also observes `session::on-deleted` (drops the session's extraction cursor and its `.extraction` state) and `queue::on-message` with args `{ topic: "memory-extraction", max_retries: 3, backoff_ms: 2000 }` (runs the queued extraction jobs; an error retries the job, then dead-letters it), and `configuration::on-updated` for its own entry.

The worker creates every observer at startup, and creates the extraction consumer again each time its id is refused (below). Observed before its owning sibling (harness, session-manager, queue) is up, an observer stays `pending` and becomes active when the sibling registers the function.

The extraction consumer observes under a stable id: the worker name (`memory`, or the `III_WORKER_NAME` its supervisor sets), then `:queue:` and the topic, preceded by `<namespace>/` when the worker has a namespace (`memory:queue:memory-extraction` by default; an id over 128 bytes becomes the worker name, cut to its first 97 bytes, then `:queue:fnv1a64:` and a 64-bit FNV-1a digest of the whole id). The queue worker files a subscriber's durable queue under its observer id. When the worker stops, killed or not, the engine keeps its observers suspended for its grace period (`observers.observer_grace_ms`, 30 s by default) and withdraws them from their functions' workers. A restarted worker, under the same name and namespace, observes the topic under the same id: the engine replaces the suspended observer with the new one, and the queue worker hands the new process every job the previous one left unacked, once. While another observer holds the id (`OBSERVER_ID_CONFLICT`: the engine has not seen the previous process's connection close yet, another worker holds it until its grace period ends, or the queue worker has not let the previous subscriber go yet), the worker logs the refusal, disposes the refused observer, and observes again 5 s after the refusal, then twice as long each time up to 30 s (5 s, 10 s, 20 s, 30 s, 30 s, ...).

### Extraction contract

- **Cursor.** The confirmed watermark is `state` scope `memory_cursor`, key the session id, value `{ last_entry_id }`. A pass reads `session::messages` strictly after it; the job's `until` is an inclusive client-side cut, never a request field. The cursor moves only after a run's writes landed; an empty or trivial run (no model call) still moves it.
- **Runs.** A pass splits the delta into runs of at most 24,000 bytes of excerpt (a long turn of tool-only steps, which have no text, is also cut so its entry ids fit the run's source key) and executes at most 4. A user message over 24,000 bytes is a run of its own, read in up to 4 calls with one plan for the run, and may write `max_memories_per_turn` memories per call. The rest returns an error, so the queue retries from the last confirmed run; no prefix or tail is dropped.
- **Source.** Each memory a run writes records its session and the run's first and last entry (`source.session_id`, `source.first_entry_id`, `source.entry_id`), so the span it came from can be read again. Records written before spans were kept hold only the last entry. A memory saved from a turn records its session and turn (`source.turn_id`).
- **Turns.** A job carries its `turn_id`. The pre-generate hook stamps `memory_extraction`, `memory_bank`, `memory_derived` (no name chose the bank) and the layer and agent on each assistant entry, so every run is extracted under its own turn's policy and bank by whichever job reaches it first, not the newest job's. A step of the job's own turn without a stamp (the hook timed out, or its bank lookup failed) runs under the job's bank, resolved again when the job runs; another turn's unstamped step, or one that predates the stamp, counts as OFF. A turn keeps its one opening request; an earlier request that failed before any reply is passed unextracted, unless the harness attested that turn as failed or cancelled with no reply (`extraction_history`): the request then runs alone under that turn's own bank, or closes unextracted when its policy is unknown or its layer was rotated since. A row that proves less is ignored, so it never stops the cursor. The hook imports that history in the background, and only when a delivery attests such a request the ledger does not hold yet. A job queued without a `turn_id` is still accepted.
- **Receipts.** Before the model call a run's redacted source is saved as the session's pending slot, and the model's decision is saved as a plan keyed by session, bank, turn and ordered entry ids. Each memory effect is appended to `memories.jsonl` together with a receipt line (`{"extraction_receipt": ...}`, a SHA-256 of the run's source key, the bank and the item's index in the plan; never a memory) in one write. A retry reuses the plan and skips every effect whose receipt the bank already holds, so a redelivered job adds no second revision or corroboration. An effect whose memory changed since the extraction read it, or whose stored twin the injection screen flags, records a skipped receipt instead; a stored record that predates a redaction pattern is written back redacted. A learned-rule merge is skipped the same way when `rules/learned.md` changed since it was read (compared redacted); its receipt lives in the run's plan, never in the rule file. A merge that takes `learned` past `learned_max_chars` then demotes its oldest lines: each line is saved as a memory first, and the rule is rewritten after, so a crash leaves a line in both places, never in neither. A line that cannot be saved stays in the rule.
- **Events** are best-effort and at most once: a crash between a write and its event loses the event.
- **Closed runs.** A run captures only what the model reads (each line redacted and cut as above) and a digest of its full text, so no message is too long to extract. A run whose bank was deleted is closed unextracted. A retained run the path no longer holds as recorded (an edit, a fork) is dropped, and its entries run as the path holds them now.
- **Pruning.** A run's plan and rule receipts are deleted once the cursor passes the run, or its retained range completes. Acknowledging a range drops the history rows of its entries; a pre-generate import drops every row at or before the cursor. Deleting a session purges its pending slot, ranges, history and plans; trashing a bank purges the plans of runs into it (a run that only teaches its agent keeps its plan and drops that agent's items).

## Functions

| Function | Purpose |
|---|---|
| `memory::bank::create / list / delete` | Banks as first-class named scopes ("blog" vs "coding" vs "personal"); delete moves to `.trash/` |
| `memory::save` | Explicit save: fingerprinted, reinforces on repeat; called from a turn, records that session and turn as its source |
| `memory::get / list / update / delete / pin` | Memory CRUD; delete tombstones; update bumps a revision in the log |
| `memory::recall` | The exact scorer the hook uses (the hook injects only the hits with evidence, one line per fact and the newer of a stated change; `memory::preview` shows exactly what it injects); `tag` narrows to one label; `mode: lexical \| semantic` returns one raw arm (plain BM25 over the content words, or cosine) for callers that fuse arms themselves; when the query was embedded, `vector_coverage {covered, live}` says how many of the bank's live memories have a vector in the query's model (`covered < live` while backfill or a re-embed runs) |
| `memory::tags` | Distinct tags across a bank's live memories with counts (the filter source) |
| `memory::preview` | Full injection dry-run for a hypothetical chat message: rules section (budgets applied), memories past the relevance cut and token budget, appended message verbatim — the hook's exact code path |
| `memory::rule::list / set` | The always-injected markdown rules; empty content removes. `list` returns content redacted, as it is injected. `set` takes an optional `expected`, the content as last read: the write is refused with `rule_changed` when the rule holds anything else by then; `""` means the rule must not exist yet |
| `memory::supersede` | Retire one memory in favor of another (tombstone + pointer) — the consolidation seam; `reinforce` makes it a merge (a non-pinned survivor absorbs the retired record's observations), `reason` rides the events |
| `memory::doctor` | End-to-end self-test (roundtrip, sibling reachability, embeddings, recent extraction failures, the extraction model, dead-lettered extraction jobs, extraction cost) |
| `memory::reload` | Reload every bank from disk after hand-editing files |

Every function above is also a REST route through the http worker: the worker observes `http::on-request` with args `{ method: "POST", path }` per function, the path being the function id with `::` as `/` (`POST /memory/save`, `POST /memory/bank/create`). Every route targets `memory::rest` (internal), which receives the `HttpRequest`, runs the function with the request's JSON body as its input (an empty body is `{}`), and returns the `HttpResponse`: `200` with the function's result, or `{ error: { code, message } }` with `400` (`invalid_input`, also a body that does not fit the function's input, and a stale `memory::rule::set`, whose message contains `rule_changed`), `404` (`bank_not_found`, `memory_not_found`), `500` (`storage`) or `503` (`sibling_unavailable`).

```bash
curl -X POST localhost:3111/memory/save -H 'content-type: application/json' \
  -d '{"bank":"blog","text":"The blog deploys on Fridays"}'
```

## Change functions

Observe these for live views instead of polling. Both are private (only this worker calls them) and take no args: an observer selects changes with filters on the event. The worker calls each one fire-and-forget when the change happens, and iii delivers the event to every observer whose filters match, unordered.

| Function | Args | Called when | Event |
|---|---|---|---|
| `memory::on-item-changed` | none; filter on `input.bank`, `input.event_type` | a memory is created, updated, superseded, or deleted | `{ event_type: created \| updated \| superseded \| deleted, bank, memory, reason? }`: `memory` is the record as stored after the change, `reason` is `memory::supersede`'s reason, a correction's reason, or `redacted: <labels>` |
| `memory::on-bank-changed` | none; filter on `input.bank`, `input.event_type` | a bank is created or trashed, or its rules change | `{ event_type: created \| trashed \| rules-changed, bank }` |

```typescript
iii.observe(
  { function_id: 'memory::on-item-changed', filters: { 'input.bank': { $eq: 'blog' } }, capture: ['output'] },
  'blog::on-memory-changed',
)
```

## Storage layout

Compose stores this tree under `III_COMPOSE_DIR`. A standalone worker uses its
process current directory for the same relative default.

```text
<III_COMPOSE_DIR>/data/memory/
  main/
    bank.yaml          # description
    rules/style.md     # always-injected markdown
    memories.jsonl     # append-only memory log (full records, last-wins by id)
    vectors.jsonl      # derived embedding sidecar (rebuildable)
  .extraction/         # in-flight extraction state: pending slots, ranges, plans, history
  .trash/              # trashed banks, timestamped
```

## Recall quality eval

`tests/recall_quality.rs` scores the production `Bank::recall` over a synthetic golden bank (`tests/recall_quality/golden.json`: one persona, 76 memories including 4 paraphrase duplicates, 62 English/Portuguese queries across lexical, paraphrase, identity, temporal-update, dated-event and durable-old categories). It runs in three modes:

- `lexical`;
- `hybrid`, with cached `text-embedding-3-small` vectors;
- `hybrid-consolidated`, which first replays the semantic consolidation tier's recorded decisions (`tests/recall_quality/consolidation.json`, from memory-consolidate's judge eval) and scores expectations carried to the survivors.

It reports hit@1/3, MRR@10, nDCG@10 and recall@10 per category and language. The `lang:en` slice is the primary metric: recall's word rules are English only, so the Portuguese queries stay in the set but nothing is tuned for them. It also counts **traps**, where a superseded memory outranks the current one, and **duplicate pollution**, a query whose top 10 holds both members of a `dup_of` pair. The committed snapshot `tests/golden/recall_quality.json` makes it a PR gate. The metrics follow trec_eval conventions.

`injection_quality`, in the same file, scores what the hook injects from that bank: every candidate with its evidence through `hooks::select`, nothing visible, in lexical and hybrid mode, against `tests/golden/injection_quality.json` (text-embedding-3-small) and `tests/golden/injection_quality-minilm.json` (MiniLM). Beside the 62 queries it sends 41 negatives that should inject nothing: 8 greetings, 8 first-person tasks and 8 follow-ups ("tell me more", "sounds good, do it"), none sharing a word with the bank, 9 questions sharing one incidental word with a memory ("Great, go ahead." among them), and 8 near-topic questions. Negatives carry their split (26 dev, 15 test); queries are split by a hash of their id (43 dev, 19 test). MiniLM's file has no negative vectors, so its hybrid negatives are unmeasured. Per slice it reports needed lines injected, lines recall ranks that the hook dropped, irrelevant lines, duplicates and stale lines. Beyond the snapshot, it fails when:

- a stale line is injected without the current one, in any mode;
- a greeting, a first-person task or a follow-up injects anything, in lexical mode;
- a needed line recall ranks is dropped, other than lexical mode's three reached only through a bare `user` (none in hybrid mode).

```bash
cargo test --test recall_quality -- --nocapture                # report
UPDATE_GOLDENS=1 cargo test --test recall_quality              # re-baseline both evals after an intended change (always scoped)
III_NAMESPACE=<ns> cargo test --test recall_quality -- --ignored embed_golden_set   # re-embed after editing golden.json
```

## Extraction eval

`extract::eval` runs the production extraction prompt and parser over short English excerpts, three times each, through a live `router::complete`. It scores what comes back:

- **capture**: each expected item is captured, wrong (its kind, where it lands, the memory it corrects, or the learned-rule gate drops it), or missed;
- **trap hits**: anything past the expected items from an excerpt built to hold nothing more: an instruction only the assistant gave, a pasted document or a fetched issue relayed, a harness notice, a transient error, a one-off instruction, task progress, what a correction leaves behind. A learned rule the user restates is expected back unchanged, naming itself. It also counts how many of them the learned-rule gate would still let into the bank;
- **placement**: what holds for the user wherever they work reaches the default bank, and nothing project-only does; with agents listed, what the user tells one about its role's work reaches that agent's bank (**agent landing**), and nothing about the user or the project does.

Run it before and after a prompt change; it needs an engine with a router and a provider, and no memory worker. It measures the prompt directory-bank sessions get; `EXTRACTION_REPOSITORY=0` measures the one `default_bank` and named-bank sessions get, without the project-knowledge clause.

```bash
III_URL=ws://127.0.0.1:<port> III_NAMESPACE=<ns> EXTRACTION_MODEL=<id> cargo test -p memory --lib extract::eval -- --ignored --nocapture
```

## Engine test

`tests/engine.rs` registers the worker's functions against an engine and checks that `memory::on-item-changed` and `memory::on-bank-changed` register private, with no args and the event as request and response, that a save reaches the observers filtered on its bank (and no others), and that `memory::rest` answers the http worker's request shape. It skips when no engine answers; `III_E2E_REQUIRE=1` makes that a failure.

```bash
III_ENGINE_WS_URL=ws://127.0.0.1:<port> cargo test --test engine
```

`tests/queue_restart.rs` runs the `memory` binary against the same engine, with a contract mock of the queue worker's `queue::on-message`: a killed worker, started again, gets the extraction jobs it left unacked, once each, under the same observer id; and while another worker holds the id, the worker observes again until it is free, 5 s after its first refusal and 10 s after its second. It skips and fails like `tests/engine.rs`.

```bash
III_ENGINE_WS_URL=ws://127.0.0.1:<port> cargo test --test queue_restart
```

## Permissions

Suggested defaults ship in [`iii-permissions.yaml`](iii-permissions.yaml): agents get `memory::recall`, `memory::save`, `memory::get`, `memory::list`, `memory::update`, `memory::delete`, `memory::pin`; the hook/pipeline internals, the REST adapter `memory::rest`, and the two internal functions behind memory-consolidate's verify tier are denied. `memory::transcript` returns a memory's source lines, redacted, a long line as its head and tail; `memory::correct` replaces a memory its source contradicts. Neither has a REST route. Bank/rule writes plus `memory::reload` are human-owned surfaces (console, REST, CLI) by default.

## The memory family

Like `llm-router` and its provider workers, memory is a family of narrow siblings rather than one grown monolith — this worker owns the store, per-turn injection, capture, and recall, and stays deliberately small:

| Worker | Role | Status |
|---|---|---|
| `memory` (this) | Banks, rules, memories: storage, harness injection, background capture, hybrid recall, live change functions | shipped |
| [`memory-consolidate`](https://github.com/iii-hq/workers/tree/main/memory-consolidate) | Background hygiene on a schedule: deterministic dedup of near-duplicate memories, catch-up-on-boot scheduling, and an opt-in verify tier that checks notes against their source transcript. Every write goes through `memory::supersede` or `memory::correct`, never touches pinned records; every change lands as a `memory::on-item-changed` event | shipped |

The split is the point: consolidation can be installed, stopped, or removed without touching stored memory, and this worker never grows a scheduler.

## Boundaries

Context-manager compresses one history for one turn; this worker is the durable cross-session sibling its spec reserves (see [tech-specs/2026-06-agentic/memory.md](https://github.com/iii-hq/workers/blob/main/tech-specs/2026-06-agentic/memory.md)). Recall fuses BM25 with a semantic signal when `router::embed` has an embed-capable provider; `memory::recall.retrieval` names the mode either way. Consolidation belongs to the `memory-consolidate` sibling, not here.
