# context-manager

Turns a raw conversation history plus a target model into a **model-ready
context**: a system prompt and an ordered message list that fits inside the
model's usable token budget. It owns *what the model sees this turn* — token
counting, function-result pruning, and history compaction (summarisation) —
and nothing else. It is stateless with respect to conversation storage: pass
messages in, get results back, persist them yourself (or with
[`session-manager`](https://github.com/iii-hq/workers/tree/main/session-manager)). Summarisation and model-limit
lookups go through `llm-router` when installed; token counting and pruning
work standalone.

## Install

```bash
iii trigger compose::add worker=context-manager
```

`iii trigger compose::add` declares the worker in `worker-compose.yaml` and
starts it as part of the Compose project. Runtime configuration lives in the `configuration` worker
(enabled by default in the engine): at boot the worker registers its schema and
fetches the authoritative value, and it hot-reloads on change — every field,
with no restart (`summarizer_timeout_ms` is read per call; a `lease_dir` change
rebuilds and swaps the lease store). An optional `--config <file>.yaml` only
seeds the first registration.

## Quickstart

Build a budgeted context before every model call:

```rust
use iii_sdk::{register_worker, InitOptions, TriggerRequest};
use serde_json::json;

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let iii = register_worker("ws://localhost:49134", InitOptions::default());

    let result = iii.trigger(TriggerRequest {
        function_id: "context::assemble".into(),
        payload: json!({
            "messages": [
                { "role": "user", "content": [{ "type": "text", "text": "hi" }], "timestamp": 1 }
            ],
            // Inline limits keep the call standalone; pass only `id`/`provider`
            // to resolve limits through llm-router instead.
            "model": { "id": "claude-sonnet-4",
                       "limits": { "context_window": 200000, "max_output_tokens": 8000 } },
            "system_prompt": "You are a helpful assistant."
        }),
        action: None,
        timeout_ms: Some(30_000),
    }).await?;

    // -> { system_prompt, messages, token_count, usable, model_resolved,
    //      applied: { pruned, pruned_tokens, capped_parts, capped_tokens,
    //                 compacted, summary?, tail_start_index? } }
    println!("{result:#?}");
    Ok(())
}
```

The compaction round trip: when `applied.compacted` is true, **persist
`applied.summary`** and the boundary your storage maps `applied.tail_start_index`
to. On later calls pass only the post-compaction window as `messages` plus the
stored summary as `options.previous_summary` — the summary is rendered into the
system prompt under `# Conversation summary`, and any further compaction
*updates* it instead of starting over. Callers that skip persistence stay
correct at the cost of repeating the bounded summarisation pipeline on every
over-budget request.

Before token accounting, `context::assemble` normalizes images in its cloned
model-facing view. A known catalog model receives placeholders unless it
explicitly declares vision support. For a known vision model, tool images age
after the next assistant response and user images age when a later user turn
begins after a response. Inline limits and unresolved models keep images. Live
images cost a fixed 4,096-token heuristic budget; base64 bytes are never counted
as text. The caller's transcript is unchanged.

Before pruning or compaction runs, assemble unconditionally caps any message-level
or inline function result over `max_result_tokens` (default 20000; 0 disables the
pass) to a bounded `[…result capped: was ~N tokens; middle omitted; re-call
{function_id} with narrower arguments if the omitted middle is needed]` head+tail
view. Text uses provider-visible newline separators and each image costs the fixed
4,096-token allowance — see Configuration below.

Every successful `context::assemble` response satisfies `token_count <= usable`.
Callers should include the complete `tools` array and set
`options.request_overhead_tokens` for response-format and provider-specific request
fields. If ordinary pruning and compaction are insufficient, assembly replaces
oversized function results in its model-facing copy with bounded references to the
full result retained in the session transcript. Requests that still cannot fit fail
with `context/overflow`; callers must not issue a provider request in that case.

The other three functions: `context::count-tokens` (estimate messages + tools +
system prompt vs a model), `context::prune` (replace eligible old function outputs
with `[output of {function_id} pruned: was ~N tokens; re-call it if still
needed]` placeholders, no LLM involved), and `context::compact` (summarise the
head, keep a recent tail verbatim — returns `ok | busy | empty | overflow`).

## Configuration

```yaml
reserved_tokens_cap: 20000     # default reserve = min(cap, pct% of context window)
reserved_pct: 10
tail_turns: 2                  # user+assistant pairs kept verbatim by compaction
protect_recent_tokens: 40000   # newest function-output tokens never pruned
decay_user_turns: 0            # prune after this many subsequent user turns; 0 disables decay
protected_user_turns: 2        # recent user turns exempt from normal pruning; 0 disables this guard
min_free_tokens: 20000         # skip pruning when it would free less
max_output_chars: 2000         # larger outputs are immediately eligible; smaller ones may decay
max_result_tokens: 20000       # per-result ceiling for assemble's unconditional cap pass; 0 disables
lease_ttl_secs: 300            # compaction mutual-exclusion lease TTL
allow_fallback_limits: true    # conservative 8192/1024 when limits can't resolve
summarizer_timeout_ms: 320000  # outer budget for one router::chat summariser call
```

Other details (and the defaults' single source of truth) live in
[`src/config.rs`](src/config.rs).

## Local development & testing

```bash
cargo test                          # unit + BDD (engine scenarios soft-skip)
cargo test --test bdd -- --tags @pure    # no engine required
UPDATE_GOLDENS=1 cargo test --test schemas   # regenerate wire-schema goldens
```


### Switching to a smaller model (MOT-4937)

Automatic assembly and explicit compaction summarize on the selected model and
provider; neither silently routes history to a different provider. If the rendered
history exceeds the resolved usable input budget, it is consumed in consecutive,
UTF-8-safe fragments. Each request includes the summary of earlier fragments as an
anchor and is estimated with its system prompt, JSON-escaped user message and
request overhead. The final verbatim tail and its original indices are unchanged.

The pipeline allows at most 32 summary calls and shares one deadline across them,
limited by `summarizer_timeout_ms` and the compaction lease lifetime. A failed call,
oversized anchor, deadline or call-count limit does not return a partial summary:
assembly falls through to its existing emergency reduction and hard-limit check;
explicit compaction returns `overflow`. No user/assistant text is discarded merely
to fit a fragment. The estimator remains heuristic, so an upstream overflow can
still require a larger-budget model, shorter input, or a new conversation.


### Read-only model-switch preview

`context::assemble` accepts `options.preview_only: true` for a read-only
pre-compaction estimate. It uses the same resolved model budget, media
normalization, result caps and normal pruning, but never invokes the summarizer,
acquires a compaction lease, or performs emergency reduction. It returns
`token_count` and `usable` even when over budget. Callers must compare those
values: a preview response is not a model-ready request or a guarantee that a
future user input will fit. The option defaults to false; normal assembly keeps
its existing hard-limit behavior.

The Console checks the full persisted compaction window before a model-picker
switch. It asks for confirmation only when that preview is over budget. Cancel
keeps the current model; confirmation compacts with the destination model and
persists the summary before changing selection. Failed checks or compaction keep
the previous selection. The preview includes the session prompt and known tool
framing; a later user message, changed configuration, or dynamic hook can change
the actual next turn's budget needs.