# provider-deepseek

DeepSeek Chat Completions provider worker behind [llm-router](https://github.com/iii-hq/workers/tree/main/llm-router).
Implements the provider protocol from
`tech-specs/2026-06-agentic/llm-router.md`: `provider::deepseek::stream`
(SSE chunks → `AssistantMessageEvent` frames into a router-owned channel) and
`provider::deepseek::refresh_models` (upstream `GET /models`, enriched with
local metadata → `router::models::reconcile`), plus
`provider::deepseek::count_tokens` (prompt token counting behind
`router::count_tokens`, with DeepSeek's own published vocabulary rather than
a borrowed one; never runs the model and costs nothing).

Default upstream: `https://api.deepseek.com/chat/completions` — DeepSeek's
OpenAI-compatible surface is rooted at the bare host, with no `/v1` segment.
Override `api_url` to point at any other OpenAI-compatible endpoint; the
models listing is always read from that endpoint's `/models` sibling.

## Behavior

- **Registration:** self-declares via `router::provider::register` with
  backoff until acked, and re-declares on the `router::ready` trigger type.
  The declaration carries no models and
  `credential_env_var: DEEPSEEK_API_KEY`; the post-register refresh
  discovers the catalog, gated on a configured credential (no key → empty
  slice, so the picker never shows unusable rows).
- **Identity binding:** the router returns a `registration_token` on first
  registration; it is persisted in state (scope `provider-deepseek`,
  key `registration_token`) and presented on every later
  `register`/`resolve`/`reconcile`. If that state is lost the router rejects
  re-registration — the operator must clear the binding on the router side.
- **Credentials:** resolved per request via `router::provider::resolve`
  (config slice → `DEEPSEEK_API_KEY` env on the router → none). Both
  `api_key` and `oauth` credential shapes are sent as `Authorization:
  Bearer`; v1 performs no OAuth refresh.
- **Catalog:** `GET /models` owns the id list; `src/curated.rs` supplies what
  the listing does not carry — display names, context windows, output
  ceilings, pricing (USD per MTok) and the `reasoning_effort` vocabulary
  from api-docs.deepseek.com (snapshot 2026-09-10). The live ids are
  `deepseek-flash` (DeepSeek-V4.1-Flash) and `deepseek-v4-pro`; the retired
  `deepseek-v4-flash` names are still accepted by the API and get the Flash
  row's metadata under their own id. An id the table does not know still
  lands in the catalog on conservative defaults (64K context, 8K output, no
  pricing) rather than disappearing, so a model DeepSeek ships tomorrow is
  routable today. Prices are the **peak** rate — DeepSeek's list price;
  off-peak (every hour outside 01:00–04:00 and 06:00–10:00 UTC, Monday to
  Friday) is half of it, so reported cost is exact at peak and a 2x ceiling
  off-peak — a time-of-day multiplier is not something the catalog can
  express. DeepSeek retires V4 Pro on 2026-09-14 12:00 Beijing time: from
  then `deepseek-v4-pro` requests are answered by V4.1 Flash and billed at
  the Flash price until V4.1 Pro ships; the row stays while the id is listed.
- **Liveness:** `ping` at least every 30s of upstream silence; a failed
  channel write (caller gone / `router::abort`) drops the SSE receiver and
  aborts the in-flight HTTP request. DeepSeek holds an overloaded request
  open with `: keep-alive` SSE comments, which the decoder ignores.
  Generation must start within **120s** of the upstream request, including
  the wait for HTTP headers. Keepalives, empty deltas, and usage-only chunks
  cannot extend that deadline. On expiry the provider closes the HTTP
  request and emits one `transient` error explaining that generation did
  not start; the model selection and catalog stay unchanged. The first
  text, reasoning, or tool-call content disarms this startup deadline, so
  a long answer continues under the normal router stream/idle budgets.
  Set `PROVIDER_READ_TIMEOUT_SECS` on the **provider worker** to change both
  this startup budget and the existing HTTP read-silence timeout (for example,
  when a self-hosted `api_url` needs a longer prompt evaluation).
- **Errors:** 401/403 → `auth_expired`, **402 (out of balance) →
  `permanent`** (a billing wall retry cannot fix), 429 → `rate_limited`,
  400/422 → `permanent` unless the message says the prompt did not fit,
  `context_length_exceeded` → `context_overflow`, 500/503 and network →
  `transient`. `finish_reason: insufficient_system_resource` — the upstream
  running out of capacity mid-generation — terminates the stream as a
  `transient` error so the router retries instead of returning a silently
  truncated answer. No transport retries here: the router owns retry policy.
  Pre-generation diagnostics stay in `error_message`, with empty assistant
  content. A read failure after generation starts retains the streamed content,
  usage, and warnings so the Harness can preserve and resume the partial response.
- **Structured output:** `json_object` mode only — DeepSeek documents no
  strict json_schema mode. A `response_format` schema is dropped with a
  report-and-continue warning; every catalog record declares
  `supports_structured_output: false`. The caller must mention "json" in the
  prompt per DeepSeek's rules.
- **Reasoning:** with a requested thinking level the request carries
  `thinking: {type: enabled}` plus the top-level `reasoning_effort` param —
  the router's five levels collapse onto DeepSeek's three-wide vocabulary as
  `minimal`/`low` → `low`, `medium`/`high` → `high`, `xhigh` → `max`
  (`src/reasoning.rs`; the API would coerce a literal `xhigh` down to `high`,
  so the mapping is done here). With **no** level both params are omitted, so
  each model runs its own documented default: the V4 family reasons at `high`
  effort — an unconfigured console chat streams its chain of thought out of
  the box, with reasoning tokens billed as output — while a legacy
  non-thinking alias (`deepseek-chat`) keeps the behavior its name encodes.
  **Off** is the one knob the ladder cannot express: pass
  `provider_options: { deepseek: { thinking: "disabled" } }` on the send and
  the request carries `thinking: {type: disabled}` with no `reasoning_effort`
  (a `thinking_level` on the same send is reported as ignored). It is never
  synthesised — an off-by-default would blank the console's thinking pane —
  and any value other than `enabled`/`disabled` fails the turn instead of
  silently running at full effort. Each documented catalog row advertises the
  effort vocabulary as `reasoning_efforts` (`low`, `high`, `max`) so
  `router::models::get` shows what a level lands on.
  Reasoning models stream their chain of thought as `reasoning_content`
  deltas, which the worker surfaces as `thinking` blocks (`src/sse.rs`), and
  `completion_tokens_details.reasoning_tokens` lands on `usage.reasoning`.
  Assistant thinking is **replayed** to the API as `reasoning_content` only
  on tool-calling messages — DeepSeek 400s a tool round whose intermediate
  reasoning was dropped — and nowhere else: the API documents replayed
  reasoning as ignored outside tool rounds, where it would only re-bill the
  whole chain as input on every later turn.
- **Block ordering:** the assembled message is a *sequence* of blocks in the
  order the model produced them, not one merged block per kind. A turn that
  reasons, answers, reasons again and answers again lands as four blocks, and
  a tool call sits between the blocks it actually fell between — matching what
  the `thinking_start`/`text_start`/… frames already described on the wire, and
  the same shape `provider-anthropic` assembles. DeepSeek's own response
  carries one `reasoning_content` and one `content` per message, so in practice
  a turn is one thinking block then one text block; the ordering matters behind
  an `api_url` override onto a gateway that genuinely interleaves. On the
  turns that replay at all (tool-calling ones — see Reasoning), replay
  flattens the blocks back to the two scalar fields the request shape has
  (thinking blocks joined, text blocks joined) — the only representation
  DeepSeek accepts.
- **Text only:** DeepSeek documents no multimodal content-part array, so
  image blocks are replaced with a text marker instead of being sent as
  `image_url` parts (which would fail the whole turn); the caller gets a
  report-and-continue warning and every known catalog record declares
  `supports_vision: false`.
- **Prompt caching:** on by default for every account, with no request
  markers (context caching on disk). Hits land on shared request *prefixes*,
  so the wire layer keeps a session's transcript append-only by
  construction: every serialization rule depends only on a message's own
  content (never on what follows it), which makes turn N's request body a
  byte-stable prefix of turn N+1's; ignorable reasoning is dropped instead
  of resent (see Reasoning); and no `user_id` is sent, so all traffic shares
  one account-wide cache instead of partitioning it. The two deliberate
  exceptions mutate history for correctness and cost one cache bust each: a
  late tool result replacing its orphan placeholder, and latest-wins dedup
  of a duplicated result. DeepSeek reports the resulting prompt split
  directly, and it maps straight onto the spec's disjoint usage splits:
  `prompt_cache_hit_tokens` → `usage.cache_read` (billed at the cached-input
  rate) and `prompt_cache_miss_tokens` → `usage.input` (billed at the input
  rate). Because the router's cost fill adds the two, `input` is deliberately
  the miss slice rather than the `prompt_tokens` total — with a ~120x cache
  discount, billing the cached prefix at both rates would roughly double the
  reported cost of a long agent loop.

## Tests

```bash
cargo test                                            # unit (pure modules + TCP stubs)
III_ENGINE_BIN=$(which iii) cargo test --test integration -- --test-threads=1
```

The integration suite spawns a real engine, the real router (path dep), this
provider, and a local stub upstream serving both `/models` and
`/chat/completions` — no external API calls anywhere.

## Running

The binary takes the standard worker CLI flags: `--url` (engine WebSocket,
default `ws://127.0.0.1:49134`, falls back to the `III_URL` environment
variable), `--manifest` (print the registry manifest and exit), and
`--config` (accepted but ignored with a warning — provider config comes
from the `llm-router` configuration entry).