The route used the enforceBudget + recordSpend read-then-write pair and
passed no cap, so every call measured against the operator ceiling — a
self-hoster on their own key was rationed by the operator's bill while
paying for their own usage. It also left the TOCTOU window the rest of
the codebase closed: concurrent calls could each read a sub-cap total
and all proceed.
Move the gate after the chain resolves and reserve atomically against
resolveDailyCap(chain), matching runStatusCompletion and the document
paths. Reconcile on every exit: the provider's reported count on a
completed call, a full refund when nothing was generated.
The OpenAI client pinned `response_format: { type: "json_object" }` on every
call. The Coach tool loop's forced-final round runs with `tool_choice: "none"`
and expects plain prose, but JSON mode coerced the reply into a JSON object —
the user saw a JSON blob or an empty reply instead of the coach's answer.
Send `response_format` only when the caller opts in with `responseFormat: "json"`
and no tools are present, mirroring the Anthropic client's prefill gate. The
JSON-consuming callers that relied on the implicit mode (insight generate +
briefing retry, medication extraction, the connection-test probe) now pass the
flag explicitly; the prose Coach path and every tool round stay out of JSON
mode. Codex already omitted the flag and is unaffected.
Replace the CompletionParams { systemPrompt, userPrompt } string pair with a
structured { system, messages[] } request across all five AI clients. One
refactor unlocks three things: provider prompt-caching (Anthropic cache_control
on the stable system prefix, OpenAI/Codex automatic prefix cache, with
cachedInputTokens surfaced for observability), the tool-calling wire surface,
and clean multi-turn.
Each client maps the new shape to its own wire and degrades unsupported
features silently: the local OpenAI-compatible server never receives a tools
field (most reject unknown keys), and prompt-caching is best-effort everywhere.
CompletionResult gains toolCalls, finishReason and cachedInputTokens; the tool
types and per-provider mapping ship now, the tool-use loop does not.
Vision inputs fold into the user turn's content parts via singleUserTurn, with
each provider's pre-existing block ordering preserved. The corrective-retry
paths append to the last user message rather than a flat string. The Coach
keeps assembling one user turn so its snapshot-once and grounding behaviour is
unchanged. No wire schema visible to API consumers changed.
Behaviour is unchanged across every surface; the full suite stays green.
One-shot prettier pass so the new format:check gate starts from a clean
baseline. Formatting only — no behaviour change. Generator-owned and
vendored artifacts stay untouched via .prettierignore.
The medication NL-extraction route egresses user-typed free text (PHI) to the
operator's server-managed provider key; require an active consent receipt for
the coach surface before the chain runs, matching the other server-managed
egress sites.
Adds an optional Coach-backed overlay that extracts a structured
medication payload from a free-text description so the wizard can
pre-fill its steps. The route is rate-limited and budget-gated; the
prompt carries a citation-coverage guard so the extraction cannot
hallucinate a name + dose pair the user did not write.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Marc-André Bombeck