Three characterisation tests for grounding gaps found while verifying that
every figure the Coach states comes from the record.
The numeric grounding guard rewrites any figure the turn's Grounding Ledger
cannot account for, and it holds: removing the strip reddens the existing
cross-turn ledger test. What is not covered is its activation. On the tool
path the guard runs only when `loop.toolResults` is non-empty, and a tool that
found nothing resolves to `{ present: false, reason: "none" }` with no
`available` payload, which the loop drops. So the guard disarms itself exactly
when the record is empty: ask about a metric with no readings, and an invented
figure is persisted and streamed verbatim, with no withheld-figure marker to
tell the reader it was never checked. The no-tools snapshot fallback is
populated only in the non-tool-mode branch, so the tool path has nothing to
fall back to, and every cloud provider runs the tool path.
Separately, a rollup bucket outlives the rows it was folded from. The read-swap
falls back to live SQL only on an empty band, `ensureUserRollupsFresh` repairs
the DAY tier over the trailing 90 days only, and it keys off
`Measurement.updatedAt`, which a hard delete does not bump. The integration
test reproduces the hard `deleteMany` in the WHOOP body sync against a
MONTH-band bucket: the deleted value keeps reaching the Coach's coarse tail
while the all-time extremes beside it report only the surviving value, and the
block is stamped current because `asOf` is derived from raw reading age and
knows nothing about rollup age.
All three assert current behaviour and carry a docblock saying so. Closing
either gap will redden them, which is the point.
Navigating away mid-reply aborts the SSE fetch; the provider call tears
down, but the conversation was left with the user message dangling and
no trace of the interruption. Persist an empty assistant row tagged
providerType "cancelled" (the refusal/nudge sentinel channel) when the
request signal is aborted, refund stays intact, and the abort is
annotated as insights.coach.cancelled instead of a provider failure or
a GlitchTip defect. Cancelled markers are filtered out of the provider
transcript on later turns. Refs #781
A Coach retrieval tool that read the window and found nothing returned
`reason: "no_data"` — the same code for two situations that have nothing in
common: the record holds nothing for that metric, and the record holds a
substantial history for it, all of it older than the window that was searched.
Nothing downstream could tell them apart, so the Coach said the honest-sounding
sentence for the wrong case, and then, having been told there was nothing there,
changed the subject. Someone with a year of imported glucose readings was told
the readings did not exist, on the same account whose glucose page was showing
them.
The window cap stays. It exists because a multi-year history does not fit a
prompt budget, and widening it would only move the same failure. What was
missing is the ability to say a different true sentence.
- `tools/availability.ts` — one grouped aggregate per miss, no rows
materialised, resolving four states: `no_data` (probe confirmed empty),
`outside_window` (rows exist, older than the searched window; carries the
count, the date range in the user's own timezone, and a bounded per-series
mean/min/max), `unavailable_in_scope` (rows exist inside the window and the
domain still produced none — a module is off), and `no_data_unconfirmed` (the
probe itself failed; absence was never established, so it is not asserted).
- Every empty read in the executor routes through it. A bare `no_data` can no
longer be returned without the record having been checked.
- The DATA INVENTORY carries the same distinction, so a domain the window
cannot reach is advertised with what it holds instead of as absent — the model
no longer skips the fetch, and a vague follow-up keeps the subject.
- Both instructions that told the model to report absence and move on now
branch on the reason code, and forbid answering about one metric by reporting
another.
- The out-of-window aggregate joins the authoritative set for the prose
verifier, so the sentence the fix exists to make sayable is not elided.
- A cross-cutting narration block that throws is now counted rather than
vanishing into a `null` that looks like honest absence.
- The medication-compliance page tells a failed read apart from an account with
no medications, instead of inviting someone to re-enter what it already holds.
Refs #648
Replace the internal "[unverified]" token the grounding guard left in a Coach
reply with the editorial elision mark […], and add a quiet muted caption
under the bubble explaining that some figures could not be checked against the
user's data. The mark holds the slot grammar-safely mid-sentence, before a
surviving unit, and inside a range; the notice carries the meaning. The withheld
count rides the existing provenance envelope (unverifiedFigures) through the
stream, persistence, and reload, and the fenced document-chat path carries a
minimal envelope so its notice survives too. New i18n key across all six locales.
Also fix an adjacent reload gap: provenanceFromJson restored only a subset of the
serialised envelope, so a persisted suggestion card, confirm action card, and the
tool trace were dropped on conversation reload despite the render layer expecting
them to survive. Restore them defensively alongside the new field.
Promote the per-turn magnitude bag to a per-conversation typed Grounding
Ledger built from every source the model was actually shown: this turn's
tool payloads, prior-turn tool figures, coach-memory goals, the medication
schedule, the reference-grounding block, and guided blocks. Assistant prose
is never a registration source, so a figure that slipped a rung cannot
launder into an authoritative value on the next turn.
Complete the action ladder: a reconciled number passes, a reference or
education sentence is exempt across all six locales, a positively-typed
benign number (date, time, year, ordinal, list marker) passes while an
untyped unreconciled one stays on the strip path, and an unreconciled
health figure is soft-stripped.
Gate the dose continuation exemption on the user's actual schedule: a
"keep taking your N mg" is trusted only when N is a dose the user is on;
an off-schedule maintenance dose is caught. Extend the risk and percent
classifiers to fr/es/it/pl to match the EN/DE depth.
No schema migration: the ledger is rebuilt per turn from persisted tool
traces and re-rendered prompt blocks.
Replace the untyped magnitude bag in the Coach numeric verifier with a typed
tokenizer and a normal-form reconciler, and narrow the outbound risk and dose
banks so correct figures and correct refusals stop being blocked while every
fabrication shape still blocks.
Grounding verifier (coach-prose-grounding.ts):
- Classify each prose number before grading — ISO/written dates, clock times,
bare years, ordinals, list markers, hyphenated time spans, ranges, percents,
thousands-separated integers, unit-suffixed measurements. Benign types are
never graded or stripped; ranges decompose into per-endpoint magnitudes.
- Grade against typed authoritative entries (value + kind + aggregation).
Sign-insensitive, canonical rounding, ratio to percent in percent space, and
minutes to hours widen the match only for an aggregate, kinded entry; kind
scoping stops a weight value grounding a blood-pressure claim.
- Thousands/decimal separators follow the reply language.
- Boundary-safe strip so a flagged token never clips a larger grounded number.
Outbound screen (outbound-screen.ts):
- Risk bank requires an asserted figure or a categorical engine result; a bare
engine or horizon mention (education/refusal) passes. Spelled-out percents
count as figures. No refusal-context exemption. EN + DE.
- Dose bank keeps a maintenance-continuation restatement, voided by any change
stem. Taper suffix fixed.
Route + fenced-chat thread the reply locale into the verifier.
The previous commit spliced the DATA INVENTORY manifest into the tool-mode
authoritative payload set unconditionally. That made the payload set
non-empty on a turn where the model answered WITHOUT calling any tool
(empty tool results), which flipped the verifier from dormant to active —
grading the reply against an inventory that carries per-domain sample counts
only, never the snapshot's pre-computed averages. A figure the model
correctly narrated from the snapshot (a 30-day systolic average) was then
stripped to "[unverified]" — the exact false positive this work exists to
prevent, and a regression from main's "no tool figures this turn → skip,
prompt-level grounding is the backstop" contract.
The inventory must only WIDEN an already-active authoritative set, never
ACTIVATE it on its own. Gate activation on a real present figure block
(a pinned workout-evidence block or a present tool result) exactly as main
did; when active, the inventory counts still join the set so a plain count
restatement stays grounded.
Adds a route-level regression test (fast suite) proving the verifier stays
dormant when no tool returned figures, plus companion unit tests documenting
that a real tool section grounds the average while a counts-only inventory
does not.
Three confirmed false-positive shapes in findUnverifiedCoachNumbers/
stripUnverifiedNumbers were replacing legitimate prose with the literal
"[unverified]" placeholder even though the underlying data was grounded:
- A correctly-computed delta, midpoint average, or percent change between
two of a payload's own headline aggregate figures (e.g. "down 3.5 kg from
your 7-day average") was never itself a raw leaf in any tool result, so it
was flagged as invented. The authoritative set now also carries the
pairwise delta/average/percent-change between a SINGLE payload's own
named aggregate fields (latest, avg7, avg30, avgSys30, allTimeAvg, ...),
scoped per payload so a weight delta cannot ground itself against an
unrelated metric's figure.
- The always-sent DATA INVENTORY manifest (tool-mode base context) carries a
sample count per domain, but that count was never added to the
authoritative set the way the pinned workout-evidence block already is —
so a plain restatement of a domain's count ("you've logged 42 BP
readings") was flagged whenever the model didn't also call that domain's
retrieval tool in the same turn. The inventory entries now splice into
toolResultPayloads exactly like workoutEvidence.
- A calendar-date ordinal ("July 21st") or a numbered-list marker ("1. Cut
back on sodium") tokenised as a bare number and, when ungrounded, got
soft-stripped mid-word into visibly broken text ("July [unverified]st").
Prose is now masked for these two narrow, high-precision shapes before
numeric extraction; the original text used for stripping is untouched.
A genuinely fabricated figure (unrelated to any grounded point, inventory
count, date, or list marker) is still caught and stripped — covered by new
adversarial cases alongside each fix.
Investigated and ruled out: multi-round tool-loop result accumulation (all
present results across every round already reach the verifier); unit
conversion (the app resolves and serves one consistent unit, so there is no
independent conversion step for the model to get wrong). The Daily
Briefing's separate withhold-a-whole-section grounding gate is unrelated to
the reported symptom (a different failure mode, whole-line omission vs.
inline replacement) and was not touched.
Four suites, each mutation-checked against a reverted protection:
- route-workout-scope: the workout row is read exactly once across a ten-turn
conversation that keeps sending the id, the where clause always names
userId, another user's id resolves to nothing and the chat proceeds, and
the section sits inside the health-data fence.
- workout-evidence: the numbers-only closure. An instruction-shaped
sport_type — reachable through the restore path, which types the column as
free text — folds to "other" and never reaches the block; a free-text leaf
at any depth throws.
- page-launch-params: an unknown ?scope= is ignored silently, and ?workout=
yields to both ?c= and the fenced ?doc= path.
- composite-score-anatomy-coach: every score sheet carries an opener, a
scope narrowed to the composite's inputs, and auto-send.
resolveLaunchState's shape assertion gains the workoutId field.
Status cards and derived-score sheets read one way: they state a verdict and
end there. The metric pages already carried the hand-off; the score sheets
passed neither an opener nor a scope, so they painted no action at all.
- Score sheets (sleep / readiness / recovery / stress / strain) gain the same
auto-sent opener the metric pages use, each scoped to the inputs that drive
the composite rather than to a block it has no equivalent of.
- Workout detail's coach launch carries the workout itself. The chat request
gains an optional workoutId; on the FIRST turn the route narrows by
{ id, userId } and pins ONE bounded evidence section onto the snapshot.
Later turns reuse the pinned snapshot, so per-turn work stays constant in
conversation length.
- /coach?workout=<id> covers the full-page and maximize paths.
The workout block deliberately does not take the fenced transport the stored-
document path takes: a workout row is the user's own sensor record, and the
projection admits no free text. That is enforced, not assumed — sport_type is
a TEXT column, so it is re-asserted against the closed sport union, metadata
is excluded apart from its numeric zone durations read through a Zod slice,
and a recursive guard throws on any non-numeric leaf.
Own-history sport averages move to src/lib/workouts/sport-context.ts so the
narrative and the detail page cannot quote different figures.
Four locales received English on surfaces whose translations were
already complete, because the resolving code was a de/en binary.
- Medication reminder push and the Telegram inline-keyboard labels
admitted only de/en, so es/fr/it/pl got the English template on every
channel. Route the resolver through a shared coerceLocale().
- The Coach's trailing "reply now in X" clause said English for any
non-German locale, overriding the correct system-prompt directive.
Read the language name from a shared map instead.
- Coach refusal copy was a de/en constant pair, answering four locales
in English at the one turn the Coach is declining to help. Move the
copy into the bundles; the EN/DE constants now derive from them.
- The sleep-stage chart formatted axis ticks as en-US for four locales;
the cycle calendar and the device-score tile followed the *browser*
locale rather than the app locale.
- Three components carried a byte-identical duration formatter that
branched on German. Lift it to one key-driven helper.
Polish needs a third plural tier: 2-4 takes a "few" form a two-form
scheme cannot express, so "2 godzin temu" read as "2 of-hours ago".
Add a pluralKey() helper driven by Intl.PluralRules and a Few tier to
the relative-time and stale-hint keys. The five locales whose rules
have no "few" category resolve exactly as before by construction, and
their Few values mirror Other. Teach the reverse-coverage guard to
expand a pluralKey() base into its three tiers.
enforceBudget and recordSpend have no callers left. Both defects they
carried recurred every time a surface copied them: a TOCTOU window
between the read and the bump, and a cap parameter defaulting to the
operator ceiling so an omitted argument rationed a self-hoster paying
for their own key. Deleting them means a future surface cannot reach
for the older, simpler-looking helper and reintroduce either.
getDailyTokenSpend goes with them — it existed to feed the read half.
The ledger-poisoning clamps recordSpend carried (NaN, negative,
fractional token counts) move to reserveBudget, where the property now
lives. Refresh the doc comments that still narrated the old flow.
The Coach SNAPSHOT was spliced into the prompt bare. Its JSON leaves are not
all server-computed: lab analyte, panel and unit names are transcribed by a
model out of an uploaded PDF and stored verbatim as the biomarker name, so a
hostile document chooses those strings. The about-me block has been fenced
since v1.16.1; extend the same pattern rather than adding per-field escapes.
- Move the fence markers and the wrapping primitive into a shared data-fence
module. Content is scrubbed against every known marker, not just its own
pair, so text in one block cannot forge another block's boundary.
- Fence the SNAPSHOT with an explicit data/instruction frame.
- Fence the OCR text handed to the document extractor; the previous
'OCR TEXT:' label was a label, not a boundary.
- Sanitise analyte, panel and unit at the labs source, which feeds both the
snapshot prompt and the labs_read tool result, and the medication label in
the adherence storyline.
- Capture a ---REMEMBER--- sentinel as proposed rather than active. It was
the only model-driven write with no confirm step, and the reminder recall
block fed it back into every later turn. Proposed reminders are already
excluded from that block and already carry a confirm control.
The status-provider suite drives the real budget module against an
in-memory stand-in for the day's usage row, so it exercises the atomic
reserve/reconcile path rather than asserting a mock was called: spend
lands on the ledger, an exhausted day is refused before any egress, a
failed generation is refunded, and a chain on the user's own key is
measured against the user-plan ceiling rather than the operator's.
The snapshot suite walks a 60-turn conversation and pins the full
snapshot to exactly two turns — the first and the elision crossing —
including the case where a lost reply shifts the turn parity.
The provider chokepoint every status and reference generator funnels
through ran with no budget accounting at all. Nine generator families
egressed behind it — the specialised status cards, the generic metric
cards, biomarker cards, the batched assessment, the derived scores, the
period narrative, and the off-request Coach memory workers (rolling
summary, fact extraction, plan proposals) — and none of that spend landed
on the daily ledger or met any ceiling. On an operator-key account it was
unmetered operator spend.
Reserve and reconcile around the provider call, using the same atomic
upsert the Coach and document paths use. The ceiling follows the cost
owner: a chain whose primary is the operator's own credential draws the
operator ceiling, everything else draws the user-plan ceiling, so a
self-hoster on their own key is never rationed by the operator's bill.
The status enqueue keyed its de-dupe singleton on (user, metric, locale).
The locale on a status read is client-chosen, so each of the six
supported locales opened its own slot and one legitimate generation pass
could be multiplied into six. Key on (user, metric) and carry the locale
in the payload instead.
The Coach re-shipped the full snapshot on every turn past the history
cap, not just the turn that crosses it, so a long conversation paid the
per-turn snapshot cost the snapshot-once model exists to remove. Re-ground
on the crossing only, tested as a narrow window rather than an equality
because a turn advances the count by one or two depending on whether the
reply persisted.
Give /api/ai/test a daily ceiling and a ledger entry. Its burst bucket
bounded rate but nothing cumulative, and with an empty body it resolves
the same chain generation uses — which can be the operator's credential.
Correct the memory-refresh worker comment, which asserted a budget check
inside the chokepoint that did not exist.
The outbound safety screen ran on the Coach only. A dose-change imperative or
a fabricated clinical risk score written onto a per-metric, biomarker or
derived-score status card persisted as that day's cached assessment, while the
identical sentence in the Coach was caught and replaced -- the only transform
between provider and persistence was whitespace normalisation.
Move the pattern banks into a shared screen that takes the reader's locale.
The old signature took no locale, so the dose-refusal and risk-score contracts
had no enforcement for fr/es/it/pl even though the ground rules ship in all six
contract files. Banks now cover all six locales and each screen runs the
reader's bank plus English, since a provider often answers in English
regardless of the language directive.
Status cards withhold on a trip: serve the deterministic stub and persist no
model text. Nobody waits on a background-generated card, and persisting a
refusal would pin it to the card for the day.
The narrative read route narrowed every non-English locale to German, so a
French, Spanish, Italian or Polish account read the German narrative row and
warmed it from the German prompt body. Carry the resolved locale through
instead: the row is keyed (user, period, locale) across the full UI locale
union, and the four locales now compose the English instruction body with
their language named and the locale's own reply-language directive appended
last. The German and English prompts are unchanged apart from the version
string.
The nightly warm keeps the user's stored locale for the same reason — it has
to write the key the read route reads — and the coach and dashboard narrative
recalls read that same key. Coach memory stays de/en: it is model-facing
context, not user-facing prose, so the four locales compose and key English
rather than German.
NARRATIVE_PROMPT_VERSION 1.13.0 -> 1.14.0.
Coach conversation history was capped at the rail's first ~20 threads and
searched only that loaded page. The rail and the standalone conversations
page now walk the caller's entire history via cursor pagination
(useInfiniteQuery + a scroll sentinel), and search runs server-side against
the conversation title (GET /api/insights/chat?q=) instead of filtering an
already-truncated client array.
Also extends the "automatic screening" info line in the metric catalog to
name the labs "what changed since your last panel" card alongside the
rhythm/breathing/health-status watchdogs it already covered — it un-mounts
the same way when there's nothing to flag and was missing from the copy.
Replace the scalar document_id discriminator on a coach conversation with a
CoachConversationDocument join table and a sticky documentScoped flag, so a
conversation can be grounded on several stored documents at once while the
prompt-injection fence stays intact across the data-model change.
The fence is re-established at both enforcement points, fail-closed:
- Client: resolveCoachSendTarget routes any turn on a fenced conversation (or a
fresh chat with staged attachments) to the new /api/insights/chat/fenced
endpoint; the fenced body is strict and never carries scope, guided-question,
or prefill hints.
- Server: the tool route fetches with documentScoped:false AND 404s when the
fetched row still carries an attachment (flag/join drift), emitting a
fence_drift audit alarm. Document text now enters an LLM prompt in exactly one
code path — the fenced pipeline — with no tools and no health snapshot; a
module-graph test pins that the tool loop is unreachable from it.
The fenced turn fences each document separately with a server-built, scrubbed
header line (title and filename are untrusted text), runs a per-document egress
consent check on the single picked provider, and grounds numerically over the
live attachment union only. Attachments are capped at five documents and 192 KB
of combined text with honest truncation.
Attach and detach ride dedicated endpoints: attaching validates the document is
owned, live, indexed, and within cap, then flips the conversation fenced (the
one legal, privilege-reducing transition, audit-logged); detaching removes the
document but never clears the flag. The document-sheet chat keeps its public
wire contract and shares the same fenced pipeline over the join.
Migration 0245 adds the join table and the flag, backfills every legacy
single-document chat into a one-attachment fenced conversation, and drops the
scalar column in the same transaction — no dual-truth window.
Asking the Coach about a document opened a separate, half-finished chat panel
that never appeared in Coach history. The bespoke drawer is gone; a document
conversation now opens the real Coach view, appears in the Coach list with a
document badge and a "Chatting about: <title>" note, and is read back through
the same history rail as any other thread.
The prompt-injection fence is unchanged and stays server-side. A document turn
is still sent through the hardened fenced endpoint (no tools, no health
snapshot, grounded in the document text only); the coach tool route continues
to fetch with documentId null, so a document-scoped conversation can never be
continued through the tool loop even from a hostile client. Only the surface
moved into the Coach; the boundary did not.
The relaxed conversation-list and detail filters drop only the documentId
predicate — userId scoping is unchanged, so a document thread is still visible
only to its owner. CoachConversationDTO gains documentId + documentTitle
(server-derived, title/filename only); no schema change (documentId exists
since 0230). OpenAPI regenerated.
Code-review and design follow-ups before the v1.22 cut. No behaviour change
beyond the items below.
Coach / insights:
- Coach SSE route: a non-provider error raised inside the open producer (e.g. a
DB failure persisting the reply) now records on the wide event and forwards to
GlitchTip instead of masquerading as a provider outage; it still surfaces an
error frame since the stream is open.
- Lab ↔ outcome correlations: hedge faint-tier pairs ("a faint hint, if
anything … too small to lean on") so the narrated confidence never outruns the
effect, matching the daily engine.
- Reminder sweep: mint the plan-review reminder and clear the plan's reviewDate
in one transaction so a half-applied write can't duplicate the reminder.
- Fold the two raw provider-fallback runners onto one shared chain walker.
Design / accessibility:
- Lift sub-44px tap targets to the 44px mobile floor (suggested-action card,
Coach bubble affordances), brighten the resting timestamp clock for non-text
contrast, and route the lab chart height through the shared constant.
- Replace ad-hoc arbitrary type sizes with the scale token and static size-11
icon buttons with the named icon-lg variant.
- Settings: add the missing About card-header descriptions, normalise the
low-stock numeric widths, and relax the notification-status rows from nested
cards to a divided list.
Housekeeping:
- Drop three unused exported types (and the now-orphaned source const), re-stamp
v1.21.5 comments to v1.22, and register the live-judge eval harness as a knip
entry so the script-only module resolves.
Add three per-user AI settings, all configured in the app (no new env vars),
behind one migration (0198):
- Response timeout (seconds). A per-user upstream timeout threaded onto the
provider call. A local/self-hosted backend can take more than the legacy
60 s default to load a model on the first request, which timed the Coach
out before the first token. Defaults to a generous ~180 s when unset;
validated to 10–600 s.
- True token streaming for the local OpenAI-compatible client. The client now
POSTs with stream:true and parses the SSE delta chunks, emitting tokens as
they arrive, with a per-idle-gap timeout instead of a whole-call ceiling and
a graceful fallback to the buffered path when a server cannot stream. The
Coach route runs the provider call inside the response stream behind a
keepalive heartbeat so a reverse proxy no longer idle-drops the connection
during model load or generation. Grounding, refusal, and sentinel-stripping
still run on the complete reply before any token reaches the client.
- Dedicated document-scan (Lab-OCR) provider. Off by default — the OCR path
uses the main provider chain. When enabled, lab-report processing resolves a
separate provider/model/key, with a hint when the chosen model is not
vision-capable. The new key column is AES-256-GCM at rest and registered in
the encrypted-columns registry and the rotation script.
i18n: EN + DE translations for the new Settings → AI strings (other locales
carry interim copies to keep locale parity green).
Feed the user's own usual blood-pressure range — the median plus or minus
MAD band, computed from the readings already in the snapshot — into the Coach
context with a clause to quote those bounds verbatim, so a band is never
invented from window means. Thread the stored first name into the prompt and
surface it sparingly on roughly one turn in three, never as a rote greeting.
Add the context-layer rule that names the relevant context before a
suggestion, a single-chart-token allowance that works on every provider, and
a per-turn opener hint.
Wire the already-shipped suggested-action SSE frame to render: the streaming
hook collects it and the message thread shows the action card, mirroring the
reminder-suggestion path. Extend the eval golden set with band-verbatim,
chart-token, and no-overclaim cases, and pin a red-team case proving the
chart-token syntax cannot smuggle a dose past the outbound screen.
Capture the casual "remember this / remind me about X" utterance the Coach
used to drop. The model emits a provider-neutral `---REMEMBER---` sentinel
that the chat route strips inline on EVERY turn — so a short conversation,
where the request usually happens, no longer waits for the long-chat memory
worker. The note is the user's own explicit request, so it is stored
silently as active; a validated closed grammar (ISO date, relative +Nd/+Nw,
or a context cue) sets the trigger, and a malformed block is dropped.
Surface the stored reminders two ways. A snapshot recall sub-block injects
near, overdue, and already-surfaced reminders so the Coach references them
unprompted next turn. A daily sweep flips overdue reminders to due and
mints a one-off review reminder from a plan whose review date has passed,
finally activating the plan review-date column. In-app surfacing only here;
push delivery follows later.
Also generalise the propose-then-confirm action card: a closed
`---SUGGEST-ACTION---` allowlist (a preventive-care item or a reminder note)
rides an additive SSE frame. Nothing is created until the user confirms.
Hotfix: the Coach and daily briefing failed for ChatGPT-provider accounts —
a provider tool-call rejection surfaced as a 500. Provider failures now
degrade gracefully, the Codex tool-call wire format is corrected, and the
provider error body is captured for diagnosis. Also: the thinking bubble
shows only the animated dots.
A Codex backend 400 on the richer Coach tool-call request (and the daily
briefing) propagated out of the provider chain runner unwrapped: a 4xx other
than 401/403/429 was not classified as a hard provider failure, so it bypassed
AllProvidersFailedError and rethrew from the chat route as an HTTP 500. Plain
status-card completions, which never send a tools array, were unaffected.
Harden the failure path so a provider error can never surface as a 500:
- Treat any 4xx a provider wire returns as a hard failure so it cascades and is
wrapped in AllProvidersFailedError. InsightSchemaError (the strict-JSON 422
surface) stays the only non-hard class, so that diagnostic is preserved.
- The chat route now also classifies a tagged provider error that reaches its
catch unwrapped and emits the graceful coach.provider.* SSE frame instead of
rethrowing. Only an untagged error keeps the 500 + error-reporter path.
Make the Codex 400 diagnosable: fold the redacted upstream response body into
the thrown error message and the chain wide-event, so the field the backend
rejects is visible in logs (it was previously dropped).
Fix the Codex request shape behind the tool-call 400:
- Send the required strict flag on each Responses-API function tool.
- Replay prior assistant turns as output_text, not input_text, per the
Responses API multi-turn contract — the violation only surfaced on round two
of the tool loop, which is the path that broke.
Codex-only; the OpenAI, Anthropic, local and mock providers are untouched.
The post-hoc numeric verifier only graded the tool path, so local /
no-tools providers got no grounding net. Grade the no-tools prose
against the snapshot the model was shown — its structured sections,
which already carry the correlations-snapshot block — and flag any
cited number that traces to no snapshot leaf. Non-blocking: the count
is annotated under coach.prose.number_unverified and the unverified
token soft-stripped, never a stream-killer. Only runs when the full
figures were delivered this turn; a cheap follow-up has no fresh set
to grade and no-ops. Tool-path behaviour and tolerance unchanged.
The Coach may point at a published /learn guide, but only a slug in the
catalog. That was a prompt instruction with no enforcement — a model could
hallucinate /learn/lower-your-cortisol and ship a dead link. Add a deterministic
post-filter on the assembled reply that keeps a published slug verbatim and
strips any reference whose slug is not in the catalog, tidying the prose around
the removed link. Makes the catalog's impossible-by-construction claim an
enforced guarantee.
Each retrieval tool re-scoped the snapshot to a single source, so its cache key
never matched the inventory's full-source build — a four-tool turn paid four
extra full snapshot builds on top of the inventory's. Thread the inventory's
probe scope to every tool so a per-tool read lands the one cache entry the
inventory already primed, collapsing the turn to a single build (a window
override still gets its own correct build). Raise the loop's round cap from two
to three so a sequential cross-metric why-chain isn't starved; the per-round
token budget is unchanged and the absolute ceiling tracks one above so the
final round still forces prose.
The tool-mode path dropped the launch scope's sources: the inventory probes the
full source set, so a Coach opened from a metric page saw the whole inventory
and could roam, while only the no-tools path honoured the narrowing. Inject a
one-line FOCUS hint naming the launched domain(s) into the tool-mode user turn
so the model prioritises that metric and fetches its figures first, with every
other domain still reachable if the question leads there. Empty on a generic
open, so the prompt prefix is unchanged on that path.
The daily AI-token gate was a flat 25,000/day for every provider. That cap
exists to bound the OPERATOR's API bill, but a ChatGPT-OAuth (Codex) or BYOK
chain egresses on the USER's own plan/key and costs the operator nothing, so
gating it on the operator-cost ceiling is wrong. A single gpt-5.x reasoning
turn legitimately reports 20k-40k gross total_tokens (re-sent system prompt +
inventory + tool defs per round + hidden reasoning, summed across rounds), so
one or two turns exhausted the flat cap and locked the user out of a plan they
pay for.
Classify the chain's cost owner by its primary provider via resolveDailyCap:
an admin-openai primary (operator pays) keeps the operator-cost cap; every
user-egress primary (codex/openai/anthropic/local) gets a generous
abuse-only ceiling. Threaded into the Coach chat gate and both OCR-extract
modes.
Raise the operator-cost cap 25,000 -> 200,000 so the operator-key path also
survives a normal day of reasoning turns, keeping gross-token accounting
rather than reworking the reconcile math.
Subtract cached input tokens at reconcile: the Responses-API gross total still
includes prompt-cached input the user did not re-pay for. The codex client
already parsed cached_tokens; sum it through the tool loop and bill
total_tokens minus cached.
The Daily Briefing strips any number absent from its server-computed
figures; the Coach's tool path had no equivalent, so a transcription or
paraphrase drift (a tool returns systolic 128, the reply says ~138) could
ship. Collect the numeric leaves from every present tool result this turn,
extract the numbers the reply asserts, and soft-correct any that match no
fetched figure to a neutral placeholder — annotated, non-blocking, and a
no-op on a qualitative turn or the no-tools path where there is nothing to
grade against. The bounded retrieval loop now returns the present results'
payloads so the check has an authoritative figure set.
Thread the request abort signal through runCoachToolLoop into each
per-round completion so a mid-generation client disconnect tears down
the upstream call instead of paying the full token cost. Completes the
no-tools-path wiring to cover the tool-retrieval path as well.
Regenerate the OpenAPI spec for the trimmed OCR-capability reason enum.
The Coach SSE turn ran the provider call to completion before the stream
existed and never wired it to the request's abort signal, so a mid-generation
client disconnect still paid the full upstream token cost. Thread an optional
`signal` through CompletionParams and singleUserTurn into each client's
safeFetch (composed with the per-call timeout via AbortSignal.any), and pass
`request.signal` from the chat route's no-tools path. An early disconnect now
tears the upstream request down instead of generating into a closed
connection. Omitting the signal leaves behaviour unchanged (timeout-only).
Replace snapshot-stuffing in the Coach turn with a small base context plus
on-demand retrieval. The model receives the full system prompt, a tool-mode
grounding addendum, and a tiny DATA INVENTORY manifest listing which domains the
user has logged and which tool fetches each; the figures themselves are pulled
only when a question needs them.
Six read-only tools wrap the existing server-authoritative snapshot builder:
get_metric_series (blood pressure / weight / pulse + the additive series),
get_glucose_panel, get_sleep, get_medication_compliance, get_labs, and
get_illness_recovery. Each returns a structured { present: false } for an absent
or opted-out domain rather than throwing or returning an ambiguous empty list,
so the model can always tell missing data from real data. Module and cycle gates
are enforced at the builder, so opted-out domains are unfetchable.
A bounded loop runs the call -> tool-result -> answer cycle: at most two rounds
with a hard cap of three that forces a final answer, parallel tool execution,
and only the final round streamed. The day's budget reserves the per-call
ceiling times the round count and reconciles the summed tokens; the atomic
reserve and reconcile primitives are unchanged. All inbound gates (rate limit,
refusal, replay scan, consent, budget) stay before the loop.
The legacy snapshot path stays alive as the no-tools fallback: tool mode is on
only when every provider in the chain supports tool-calling, so a chain that
includes a local model falls back to the prior behaviour verbatim. The tool
trace is persisted onto the assistant message provenance.
Replace the CompletionParams { systemPrompt, userPrompt } string pair with a
structured { system, messages[] } request across all five AI clients. One
refactor unlocks three things: provider prompt-caching (Anthropic cache_control
on the stable system prefix, OpenAI/Codex automatic prefix cache, with
cachedInputTokens surfaced for observability), the tool-calling wire surface,
and clean multi-turn.
Each client maps the new shape to its own wire and degrades unsupported
features silently: the local OpenAI-compatible server never receives a tools
field (most reject unknown keys), and prompt-caching is best-effort everywhere.
CompletionResult gains toolCalls, finishReason and cachedInputTokens; the tool
types and per-provider mapping ship now, the tool-use loop does not.
Vision inputs fold into the user turn's content parts via singleUserTurn, with
each provider's pre-existing block ordering preserved. The corrective-retry
paths append to the last user message rather than a flat string. The Coach
keeps assembling one user turn so its snapshot-once and grounding behaviour is
unchanged. No wire schema visible to API consumers changed.
Behaviour is unchanged across every surface; the full suite stays green.
The full SNAPSHOT block (every cluster plus labs, illness, cycle and the
reference-grounding lines) was interpolated into the prompt on every turn,
so a one-word follow-up re-paid the entire grounding prefix. Ship it only on
the first turn, or once the history window starts eliding turns; on a cheap
follow-up send a short pointer back to the snapshot the model already received
earlier in the same conversation. The prior turns ride the transcript, so
grounding and cross-metric correlation are preserved while a follow-up turn
sheds about 6,000 tokens. Record snapshotSent and promptChars on the reply
annotation so the cut is observable.
Apply the outbound safety screen to the assembled Coach reply before
persistence and streaming, with a per-turn audit row on a block.
createSseStream gains a cancel handler + a shared AbortSignal: a client
disconnect now flips the signal so the token-emit loop stops enqueuing
into a closed connection and any provider fetch wired to the signal is
torn down. The chat emit loop honours it between frames.
Carry an additive usage envelope on the SSE done frame
({ totalTokens, model }) and persist tokensUsed + model onto
coach_messages (migration 0182, nullable, no backfill) so the
per-message token footer survives a conversation reload. Add an
optional reasoning frame the client can render inside the thinking
disclosure when a reasoning-capable provider emits a summary; providers
without reasoning emit none. Extend CoachMessageDTO and regenerate the
OpenAPI contract. Server-authoritative: clients display the count, never
recompute it.
One-shot prettier pass so the new format:check gate starts from a clean
baseline. Formatting only — no behaviour change. Generator-owned and
vendored artifacts stay untouched via .prettierignore.
The daily token cap was read before the provider call and recorded only
after a successful reply, so concurrent requests could all read
`spent < cap` and all hit the provider, and an empty/sentinel reply
recorded zero spend despite burning upstream tokens.
Reserve the budget atomically before the call: a single upsert-increment
returns the new total, so concurrent requests serialise on the row and
cannot both pass the cap; over-cap refunds the reservation and surfaces
`coach.budget.exceeded` as an SSE error frame. Reconcile the reservation
to the actual token count immediately after the provider returns —
before any empty/sentinel short-circuit — so burned tokens are always
recorded. An integration test proves two concurrent requests cannot both
exceed the cap and that an empty reply still records its cost.
Replace the scattered `?? 0.3` / `?? 1000` temperature and maxTokens
magic numbers across the status provider, Coach route, comprehensive
generator and period-narrative generator with a single typed
`ai-budgets.ts` map. Each entry documents the output contract that
fixes its token ceiling.
Right-size the per-metric status output from 1000 to 250 tokens: the
card renders one `{ "summary": "..." }` of 2-4 sentences (30-60 words),
so 1000 was an order of magnitude over the real ceiling.
Surface the population reference bands behind the metrics a Coach turn
carries as a compact, citation-aware grounding block appended after the
SNAPSHOT. For each present metric the block states the headline band, its
source (blood pressure on the ESH 2023 line, US/ESC framings as context
only), and where the user's value sits via the shared four-state
placement contract.
Framed strictly as general guidance: the block opens and closes with a
not-a-diagnosis caveat, never overrides the existing refusal/dose/drug-
level safety discipline, and carries no commercial brand name. Glucose
respects the diabetes opt-in — the tighter ADA management goal only when
the user has declared it, otherwise the general non-diabetic band.
The block is deterministic and inspectable: built purely from the
reference-range backbone over the same scalars the snapshot already
surfaces, so it can never cite a value or band the snapshot does not
carry.
Marc-André Bombeck