Replace the Coach snapshot-stuffing with a small base context plus a data
inventory and six on-demand retrieval tools (metric series, glucose,
sleep, medication compliance, labs, illness recovery) the model calls as
needed, then answers. Cuts first-turn input ~35-65% while preserving
grounding and every gate; tools return a structured present:false rather
than throwing, so the model cites only what it fetched. Falls back to the
snapshot path when a provider lacks tool support.
Replace snapshot-stuffing in the Coach turn with a small base context plus
on-demand retrieval. The model receives the full system prompt, a tool-mode
grounding addendum, and a tiny DATA INVENTORY manifest listing which domains the
user has logged and which tool fetches each; the figures themselves are pulled
only when a question needs them.
Six read-only tools wrap the existing server-authoritative snapshot builder:
get_metric_series (blood pressure / weight / pulse + the additive series),
get_glucose_panel, get_sleep, get_medication_compliance, get_labs, and
get_illness_recovery. Each returns a structured { present: false } for an absent
or opted-out domain rather than throwing or returning an ambiguous empty list,
so the model can always tell missing data from real data. Module and cycle gates
are enforced at the builder, so opted-out domains are unfetchable.
A bounded loop runs the call -> tool-result -> answer cycle: at most two rounds
with a hard cap of three that forces a final answer, parallel tool execution,
and only the final round streamed. The day's budget reserves the per-call
ceiling times the round count and reconciles the summed tokens; the atomic
reserve and reconcile primitives are unchanged. All inbound gates (rate limit,
refusal, replay scan, consent, budget) stay before the loop.
The legacy snapshot path stays alive as the no-tools fallback: tool mode is on
only when every provider in the chain supports tool-calling, so a chain that
includes a local model falls back to the prior behaviour verbatim. The tool
trace is persisted onto the assistant message provenance.
The Coach snapshot builder returns the structured, post-degrade record it
serialises to JSON alongside the JSON string, so a single domain block can be
read out without re-parsing. Add an optional tool-call trace to the provenance
envelope (which retrieval tools ran + whether each found data) and mirror it in
the OpenAPI provenance schema.
Groundwork for on-demand retrieval: the retrieval tools slice one section out
of this record, reusing the builder's gates and I/O verbatim.
Retarget the Fitbit integration from the Google Health API (which needs
annual CASA review — a wall for self-hosters) to the classic Fitbit Web
API with PKCE and rotating refresh tokens, so a self-hoster can connect
with an instant developer app (migration 0191 adds the PKCE verifier).
Transport-only fork; the ingest, dedup, source-priority, backfill and
poll paths are reused. Intraday HR and skin temperature are not mapped
(no honest slot); the integration stays experimental.
Store additive per-bucket regression accumulators (sum_x, sum_xy, sum_xx,
sum_yy) so windowed slope, r2 and standard deviation compose from the
rollup tier instead of a live raw-table scan on every warm request
(migration 0190, backfilled by the boot queue). Median and the exact
anomaly count stay on a minimal live query. Parity with the live REGR_*
path within float tolerance, well below the read rounding.
Add the optional FITBIT_REDIRECT_URI to .env.production.example (BYO-key shape,
derives from NEXT_PUBLIC_APP_URL) so check-env documents it. Rename the
settings-card copy from "Google Health" to "Fitbit" across all six locales now
that the transport is the classic Fitbit Web API.
Connect mints the PKCE pair, stashes the verifier on the OAuth-state row, and
sends the S256 challenge to the authorize endpoint. Callback reads the verifier
back and presents it on the token exchange, refusing cleanly when it is absent.
The connection-test probe hits the classic /1/user/-/profile.json endpoint.
The shipped Fitbit integration spoke the Google Health API, whose Restricted
scopes need brand verification plus an annual CASA assessment before leaving
testing mode — a wall most self-hosters cannot clear. Retarget the transport to
the classic api.fitbit.com Web API: a self-hoster registers an app at
dev.fitbit.com in minutes, no CASA.
Forks only the transport; the upsert/dedup/rollup tail, source-priority,
backfill, poll worker, and status ledger are reused unchanged.
- OAuth: www.fitbit.com/oauth2/authorize + api.fitbit.com/oauth2/token, PKCE
(S256), classic self-serve scopes.
- Refresh tokens rotate (one-time use): persist the new token every refresh,
with a defensive keep-existing guard on a malformed reply.
- Per-endpoint date-range fetchers chunked to the API range caps; the backfill
horizon and poll concurrency are tuned for the 150 req/h-per-user budget.
- Verified classic-shape extractors for weight, body fat, SpO2, HRV (dailyRmssd),
resting HR, respiratory rate, steps, distance, active calories, floors, VO2 max
(range midpoint), per-segment sleep stages, and workouts.
- Skips intraday HR (needs a per-app grant; daily resting HR covers it) and skin
temperature (a baseline delta with no honest absolute slot).
The classic Fitbit Web API authorises with PKCE (Authorization Code + S256),
so the connect route mints a code_verifier and the callback presents it on the
token exchange. Carry it on the short-lived FitbitOAuthState row.
Additive + nullable (migration 0191): a legacy in-flight row carries no
verifier, so the change is reversible and rerun-safe with no data loss.
Two suites pin that the windowed slope / r2 / population-sd composed from
the rollup accumulators equals the live Postgres REGR_SLOPE / REGR_R2 /
STDDEV_POP over the same raw rows.
compose-regression.test.ts (container-free) builds per-DAY-bucket
accumulators from a raw fixture exactly as the populator's SUM terms do,
then asserts composeRegression over those buckets equals a direct OLS over
the flat raw rows - the same closed form Postgres folds - on an uneven
multi-bucket fixture, plus the coverage contract (NULL accumulator,
< 2 readings, and zero-x-variance all return a full miss so the reader
falls back to live SQL).
rollup-regression-parity.integration.test.ts (real Postgres testcontainer)
seeds raw measurements across many uneven DAY buckets, folds the rollups,
and compares the accumulator compose to live REGR_*/STDDEV_POP to <= 1e-9:
a single-source WEIGHT series, a dual-source RHR series proving the
source-collapse runs BEFORE accumulator summation (composed == live WHOOP
-only, != the cross-source blend), and a DST-boundary series confirming
the UTC epoch-day x-axis parity-matches across the transition.
The residual is pure floating-point reassociation (per-bucket-then-window
vs flat summation), orders of magnitude below the read tier's rounding.
Move the AI client layer from a single system/user string pair to a
structured system + messages[] shape, with optional tools/toolChoice and
a result carrying toolCalls/finishReason/cachedInputTokens. Per-provider
prompt-caching is wired (Anthropic cache_control, OpenAI/local auto
prefix); tool plumbing is in place for the Coach to use. Behaviour-
preserving: no streaming change, snapshot-once retained.
Replace the CompletionParams { systemPrompt, userPrompt } string pair with a
structured { system, messages[] } request across all five AI clients. One
refactor unlocks three things: provider prompt-caching (Anthropic cache_control
on the stable system prefix, OpenAI/Codex automatic prefix cache, with
cachedInputTokens surfaced for observability), the tool-calling wire surface,
and clean multi-turn.
Each client maps the new shape to its own wire and degrades unsupported
features silently: the local OpenAI-compatible server never receives a tools
field (most reject unknown keys), and prompt-caching is best-effort everywhere.
CompletionResult gains toolCalls, finishReason and cachedInputTokens; the tool
types and per-provider mapping ship now, the tool-use loop does not.
Vision inputs fold into the user turn's content parts via singleUserTurn, with
each provider's pre-existing block ordering preserved. The corrective-retry
paths append to the last user message rather than a flat string. The Coach
keeps assembling one user turn so its snapshot-once and grounding behaviour is
unchanged. No wire schema visible to API consumers changed.
Behaviour is unchanged across every surface; the full suite stays green.
Migration 0190 left existing DAY rollup rows with NULL regression
accumulators until a re-fold refills them. Add a second discovery arm to
enqueueBootTimeRollupBackfill, UNION-ed with the per-day missing-coverage
arm: a user whose DAY buckets carry sum_xy IS NULL inside the fold window
needs a one-time re-fold so the windowed slope/r2/sd readers compose from
the accumulators instead of falling back to the live REGR_* scan.
New writes self-populate the accumulators (they ride the same aggregate
as mean/sum), so the arm drains to empty once the boot re-fold catches up
- the same convergence the v1.4.39 sum_value rollout reached through this
queue. Bounded to ROLLUP_FOLD_WINDOW_MS so days outside the window never
re-flag a user on every boot.
The summaries slice (computeFromRollups) and the comprehensive aggregator
(buildFromRollups) each ran a narrow live $queryRaw over the raw
measurements partition on EVERY warm-rollup request to fold slope7/30/90
and r2_7/30/90 via REGR_SLOPE / REGR_R2 - the un-eliminated live scan on
the happy path.
Drop the six REGR_* columns from both narrow queries and compose the
windowed slope / r2 (and the population sd via composeRegression) from
the per-bucket accumulators stored in measurement_rollups. The 90-day
DAY buckets are loaded once (already loaded in comprehensive; one bounded
findMany added in the slim slice), source-collapsed to the canonical
source, then summed over the day-aligned 7/30/90-day sub-windows. A
window whose buckets lack accumulators (pre-backfill) composes to null,
the same shape the empty REGR_* FILTER returned, so a transitional read
never returns a wrong partial regression.
median (PERCENTILE_CONT) and the exact |z|>2 anomaly count stay on the
minimal live query - both are non-composable across buckets. The cold
coverage-miss path keeps the live REGR_* regression. Annotate
slope_source:"rollup" + median_source/anomaly_source:"live" so a
regression to the heavy path stays observable (no-silent-cap rule).
Make the physiological return relapse-aware (the start of the final
sustained in-band run, not the first), so a vital that relapsed and
recovered again reports its true recovery. Add an optional sleep-context
sub-line (baseline vs episode asleep-minutes), withheld on thin data and
never feeding the gap median.
Adds a neutral sleep observation alongside the per-episode recovery gap: the
user's usual asleep minutes over their well baseline vs the mean over the
episode window, narrated as 'you slept more/less than usual' when the two
diverge. It is an observation only — never a recovery claim, never a return
track, and never part of the headline recovery-gap number.
Per-night asleep totals come from the canonical night reconstruction over the
baseline and episode windows, so a multi-source night (two writers for one
sleep) is counted once, matching every other sleep surface. The observation
withholds entirely on thin data — too few baseline or episode nights, or a
delta below a 30-minute floor — so day-to-day jitter is never narrated as a
finding.
Additive optional DTO field; no schema, no migration. Two locale keys across
the six bundles. Regenerate the OpenAPI contract from the Zod source.
Windowed slope, r2 and population sd still ran a live REGR_*/STDDEV_POP
query over the raw measurements partition on every warm-rollup request,
because the per-bucket mean cannot reconstruct an OLS regression over the
raw points in a window. The per-bucket sd/slope/r2 columns are a
within-bucket fold and do not compose linearly across DAY buckets.
Add the four regression-accumulator sums Postgres folds for REGR_* /
STDDEV_POP - sum_x, sum_xy, sum_xx, sum_yy - per (user, type,
granularity, bucketStart, source) over the same canonical-source rows
the live query reads, with the x-axis in epoch days. With n = count and
sum_y = mean*count already stored, all six terms are additive across
buckets, so a 7/30/90-day slope/r2/sd composes from the summed
accumulators with a bit-identical result to the live closed form.
Migration 0190 adds the four nullable columns; nullable keeps it
additive so existing rows read NULL until the boot re-fold refills them
(mirrors the v1.4.39 sum_value rollout). The populator emits the four
SUM terms inside the existing aggregate (no extra round-trip) and
persistRollupRows writes them. composeRegression / composeWindowedRegression
evaluate the closed form from already-collapsed buckets and return a full
miss when any in-window bucket lacks accumulators, so the reader falls
back to live SQL rather than composing a partial window.
Default the built, data-gated sleep, steps and glucose dashboard tiles to
visible (each self-hides without data) and surface the Vorsorge chart row,
so shipped capabilities are discoverable on the home dashboard. Seed the
demo with an explicit showcase layout.
The demo account seeds rich sleep, steps, glucose, VO2max and workout
data but inserted no dashboard layout, so the home page rendered the
new-account default and under-sold the product on its primary surface.
Persist an explicit dashboard layout derived from the default with the
seeded-rich strip tiles forced on and the daily-verdict hero enabled, so
a fresh seed renders a full, lived-in dashboard for screenshots.
The sleep, steps and glucose strip tiles shipped default-hidden even
though their modules are on by default and rich data flows in from the
sync integrations, so accounts had to hunt through Settings to see them.
Flip the three strip tiles to tileVisible:true and the vorsorge chart-row
card to visible:true, matching the VO2max precedent: each tile self-gates
on having data, so empty accounts keep a clean dashboard. Chart rows for
the three metrics stay off — those charts live on the /insights sub-pages.
totalBodyWater, boneMass and oxygenSaturation stay hidden: they carry as
catalogue ids for the iOS layout round-trip only and have no web render
path, so a flip would surface nothing. Note that at the splice point.
Update the silhouette skeleton and layout default tests for the new
counts.
A vital or symptom curve that settles, re-deviates mid-episode, and settles
again now reports the FINAL sustained in-band run as the recovery return rather
than the first. The earlier first-return rule locked onto an interim settle and
under-reported the gap whenever a relapse followed. The change is monotone — a
last return is always at or after the first — so no existing recovery number
shifts; it only extends the gap for episodes that genuinely relapsed.
The anchor-after-first-deviation guard is unchanged: an early in-band run-up is
still never read as a return. The symptom-burden track honours relapse
identically over logged days so the headline median never mixes first- and
last-return semantics.
Adds relapse, multiple-relapse, too-short-final-run, ends-out-of-band, and
symptom-relapse engine fixtures plus a cross-episode integration fixture.
Add per-bucket HR min/max (migration 0188) threaded through batch ingest
and the series reader; make telegram_chat_id unique (0189) with a
defensive pre-clean; make migration 0187 rerun-safe. Capture a numeric
measurement from a Telegram reminder reply (source=TELEGRAM, strict chat
binding, self-cleaning); blood pressure stays satisfy-only pending a
guided two-number prompt.
Fold the illness journal's per-day functional-impact curve (falling back
to linked symptom severity) into the recovery-gap as a return track
against the known-constant healthy baseline. It is adverse by
construction, feeds the qualifying-days floor and the gap median, and
withholds on sparse logging. The card names the driver when symptoms
lead ('…before your logged symptoms eased').
Two related changes to the Telegram bot's data path.
One chat ⇄ one account. `User.telegramChatId` is now UNIQUE (migration
0189, defensive pre-clean of any pre-existing duplicate binding). The
inbound webhook resolves the user from the chat id, so a chat shared
across two accounts would route a reply or button tap ambiguously. The
index is over a nullable column, so an unlinked account (null chat id)
never collides — only two accounts binding the same non-null chat id are
rejected.
Numeric measurement capture. A numeric reply to a measurement reminder is
now captured as a Measurement (`source = TELEGRAM`, added to the enum in
0189). On "Erledigt" for a reminder that names a single-value metric, the
bot offers an optional value prompt via force_reply; a numeric reply is
written through the shared `logTelegramMeasurement` helper — same range
guard, canonical unit, cache + rollup refresh as the manual route. The
reply binds to the reminder and user through a TelegramPromptContext keyed
on the prompt message id (the same strict chat/message binding the
mood-note path uses); the userId is never read from the payload. Comma
decimals are tolerated; non-numeric and out-of-range replies get a calm
hint and write nothing; the reply self-cleans on the ~30-min sweep. Blood
pressure (two values) is out of scope for the single-numeric path and
keeps its satisfy-only flow.
Guard the CREATE TABLE + both CREATE INDEX statements with IF NOT EXISTS so
a re-run on a database that already has the table is a no-op, matching the
guard convention every other recent migration ships with. Semantics are
unchanged — same table, same columns, same indexes.
When the logged symptom curve drives the recovery gap, the card names it
('you felt better N days before your logged symptoms eased') instead of
the vital-driven phrasing. Mirror the new gapDriverType and the return
track's adverse flag into the client DTO, and add the driver-aware copy
across all six locales.
The v1.19.0 hourly heart-rate bucket
(`stats:HKQuantityTypeIdentifierHeartRate:<UTC-hour>`) shipped avg-only:
one PULSE row carrying the hour's mean bpm. Add the hour's min/max so a
client can render the intra-hour range without re-uploading raw samples.
- Two nullable columns on `measurements` (`value_min` / `value_max`),
migration 0188. Set only on a well-formed hourly HR bucket row; null for
per-sample readings, per-day cumulative totals, and manual entries.
- Batch ingest threads the spread through the same unit conversion as the
average and persists / overwrites it alongside `value` on the bucket
row, pinning it null everywhere else.
- The series reader returns the spread for `kind=pulse` (null on a
per-sample PULSE row); the OpenAPI Zod contract documents both fields on
the batch entry and the series point. The avg path is unchanged.
The recovery-gap engine reasoned only over passive vitals and ignored
the journal's own per-day symptom series. Add the functional-impact
(symptom-burden) return track: read functionalImpact per episode day
(falling back to the strongest linked symptom severity) and treat it as
one more return against the known-constant baseline (healthy = 0).
The symptom track needs none of the MAD-band / contamination machinery
the passive vitals carry — the band is the constant 0 — so it is more
reliable to fold in than any passive vital. It is adverse by
construction, so it contributes to adverseCoverageDays and the headline
gap median, giving the qualifying-days floor a genuinely
illness-relevant contributor where a weight-only episode had none.
Stability is logged-days-only: a sparse journal that logs impact-0 once
and stops withholds the return rather than fabricating an all-clear from
the absence of further logs. gapDriverType names the dominant return
track, with the symptom curve winning ties as the most illness-specific
signal.
A multi-year All range was silently truncated to the most recent year by
the DAY-only 365-row cap. Route windows wider than the cap through a new
tiered rollup reader (WEEK/MONTH by span, finer on coverage miss,
uncapped by tier choice); narrow windows stay byte-identical. No silent
truncation — a coverage miss logs and falls through to the daily path.
The chart still requests aggregate=daily for windows over 7 days, but
the server now steps the rollup tier up to WEEK / MONTH for ranges past
the DAY cap and returns whole-history coverage. Update the stale
comments that claimed the "All" range was still capped at ~365 daily
buckets; the client fold + bucketTimeSeries downsample the coarse
buckets to the visible range as before.
A multi-year "All" range read its series through the DAY-only daily
reader, which caps the result at BUCKET_CAP.daily (365) buckets. The
older history was silently dropped — the range collapsed to roughly the
most recent year instead of the whole span.
Add readTieredRollupSeries, which reads the bucket tier matching the
charts own display downsampler (WEEK for 1-2 years, MONTH beyond) and
returns whole-history coverage at that tier. readDailySeries now routes
any window wider than the DAY cap through it, falling finer on a coverage
miss and logging the miss rather than truncating silently. Short and
normal ranges never enter the branch, so the common path is unchanged.
Surface the captured Oura resilience band (limited…exceptional) as a calm
tile on the recovery surface, with a quiet above/in-line/below-average cue
and a learning state while the series is sparse. Server-authoritative read
of the existing summaries slice; neutral card chrome.
Correct the sleep-rhythm peer-card comment to state plainly that each
card reads its own window (the captions disclose the per-card night
count); no computation changed. Remove the dead streaming.startedAt and
streaming.reasoning state left after the thinking-disclosure removal.
v1.19.0 ingests Oura's daily resilience level end-to-end (RESILIENCE,
ordinal-encoded limited=1 … exceptional=5) but no surface rendered it.
Add a calm resilience tile to the recovery page: it names the latest
band, gives a one-line description, and a quiet "vs the recent average"
cue once a few days exist, with a learning note while the series is
still sparse. Resilience is a categorical band, not a continuous score,
so it reads as a dedicated readout rather than a misleading line chart.
Server-authoritative — the tile reads the stored summaries slice (latest
+ mean, both computed server-side) and never re-derives the value. Calm
posture: one neutral card, no green-when-good / red-when-low tint. The
band label and trend cue carry the read in muted text only.
Self-gating: renders nothing without a reading, and counts toward the
recovery-page data gate so an Oura-only account no longer sees the empty
note. Copy lands in all six locales.
The thinking disclosure was removed, so the coach streaming bubble no
longer reads startedAt or reasoning. Both fields stayed populated by the
send hook but nothing consumed them. Remove them from
CoachStreamingMessage and every writer, accept the additive reasoning
SSE frame as an explicit no-op so the wire type stays recognised, and
update the thread-test fixture. Streaming is otherwise unchanged.
The comment claimed the average-per-night, sleep-debt, and chronotype
cards share one window and cannot disagree. They each read their own
sub-window — the average over the full scorable span, sleep-debt over
its trailing 14 nights, chronotype over CHRONOTYPE_WINDOW_NIGHTS — so
the figures are each internally honest rather than identical, and every
card's caption discloses its own nightsCounted. State that plainly; no
computation changes.
The drawer's Conversations button now hands off to
/coach?view=conversations so the full page opens the conversation list on
arrival; widen the navigation assertion to accept the query string.
Add averagePerNight to the SleepRhythmResource OpenAPI schema so the new
average-per-night card's field is part of the iOS-consumed contract (the
endpoint already returned it; the Zod source had not been updated, so the
in-sync check could not catch the omission).
Tag each vital return with whether it deviated in the illness-adverse
direction and tally only adverse returns when naming the recovery driver,
so a neutral move (e.g. weight drift) can no longer be surfaced as the
vital someone recovered ahead of.
Tighten the cross-episode typical-gap floor to count only active days
carrying an adverse-direction reading (a neutral weight-only episode can
no longer clear it), via a single shared adverse-direction predicate.
Surface the most-frequent returning vital so the card names the driver
('…before your resting heart rate settles') instead of a bare line.
The cross-episode summary already computes per-vital physiological returns
but the card surfaced only one bare line. Thread the dominant returning
vital through the aggregate and the DTO so the copy reads 'you tend to feel
better before your resting heart rate settles' instead of a context-free
gap figure.
The aggregate tallies which vital produced a real return across the
episodes that qualified for the typical-gap median and reports the most
frequent one (deterministic alphabetical tiebreak); the driver is named
only when a gap actually clears the magnitude gate. Zero new compute — the
returns were already in hand. Adds the localized copy in all six locales.
The cross-episode typical-gap median counts an episode toward its
qualifying-days floor via the per-episode coverage `historyDays`, which
counts a day if ANY banded vital has a reading. A WEIGHT-only episode (no
illness-adverse direction) could therefore clear the floor and feed the
median despite carrying no illness-relevant signal, weakening the honest-
sample claim.
Compute a distinct-adverse-day count in the engine — days with at least
one notable reading in the metric's adverse direction — and feed that as
the qualifying-days floor instead. Reuses the engine's single adverse-
direction predicate (now exported as `isAdverseDeviation`); a neutral-
direction vital never counts a day. The insufficient arm contributes 0,
which is correct since its gap is null and never reaches the median.
S1: add an average-per-night sleep card as a third peer beside debt and
chronotype, computed by a shared helper over the same scorable-night set
so the three cannot disagree; grid widens to three columns. S2: restore
full saturation on the two missed mood surfaces (calendar heatmap and the
time-of-day chart), keeping the quiet tint only for no-data cells.
The v1.19.0 saturation fix missed two surfaces: the mood-by-time-of-day
bars and the mood calendar still rendered populated values at 0.55
opacity, reading as a dimmed glitch. Drop the dimming on populated bars
and cells (and the legend swatches); keep the quiet tint only on the
no-data empty state.
Marc-André Bombeck