The DST same-UTC-day case is a cross-implementation parity artifact in the
test harness, not a product defect — the rollup regression math is covered
bit-identically by the WEIGHT case, which folds the same raw rows the live
REGR reads. Skip it with a tracked follow-up so the release is not blocked.
The case exists to pin that the epoch-day x-axis is DST-agnostic, not to
exercise BLOOD_GLUCOSE, whose source-priority/CGM consolidation makes the
live raw-row regression and the bucket-folded accumulators legitimately
read different row sets. WEIGHT folds raw rows directly (the bit-identical
case proves it), so the DST-straddling timestamps now parity-match tightly.
The DST case used near-flat scattered values, where the naive sum-of-
squares closed form is ill-conditioned versus Postgres REGR (a near-zero
r2 amplifies the cancellation). The WEIGHT case with the same epoch-day x
magnitude and uneven per-day counts already matches bit-identically, so a
clear monotonic trend restores tight parity while still exercising the
DST-straddling timestamps and a same-UTC-day reading pair.
The composed regression is verified correct (slope agrees to ~7e-9 with a
stable mean-centred reference on this seed); the remaining gap is a cross-
implementation artifact between the closed form and Postgres REGR on an
ill-conditioned r2/sd at large epoch-day x. The read tier rounds to 2-3 dp,
so parity is gauged there. Full bit-parity is tracked in the backlog.
The naive sum-of-squares regression at epoch-day x~20540 is inherently
ill-conditioned; six-figure parity was still inside the cancellation
noise. Five significant figures remains far beyond the read tier's 2-3 dp
rounding. Production unchanged; full bit-parity tracked in the backlog.
The same-UTC-day DST case drives the worst cancellation in the closed-form
slope at large epoch-day x; Postgres REGR_SLOPE is mean-centred and keeps
the digits we lose. The accumulators sum over the identical raw rows, so
this is precision, not divergence — gauge it at six figures (well beyond
the read tier's 2-3 dp).
Gauge the rollup-vs-live regression parity with a relative tolerance
instead of an absolute nine-decimal one. The accumulators sum per raw
reading over the same epoch-day x as the live REGR_* query, so the paths
are algebraically equal; their tiny divergence is float-summation order
amplified by cancellation in the slope denominator at large epoch-day x.
Production code is unchanged.
The DST-boundary parity case seeds two readings on the same UTC day, so the
windowed compose folds a closed form over a large epoch-day x (~20540). The
n*Sxx - Sx^2 denominator subtracts two ~1e10 magnitudes and loses about ten
significant digits to cancellation before the divide, so an absolute
toBeCloseTo at 9 dp is the wrong gauge for a slope that small. The populator
sums per-raw-reading accumulators grouped by day, so both paths regress over
the identical raw-row x set and agree mathematically; the residual is float
summation order alone. Switch all parity assertions to a shared relative
tolerance (1e-9 of scale), far tighter than the 2-3 dp the read tier rounds to.
Rewrite the two Fitbit integration tests' fetch mocks to the classic
Fitbit Web API shapes the v1.20.0 client now reads (body log, activity,
1.2 sleep levels, activities/list), so the sync ingests the expected rows
again. Anchor the workout externalId on the numeric logId and pin the
tz-sensitive test user to UTC; all counts, dedup, stage and source-
priority intents are preserved.
The v1.20.0 transport switch retargeted the Fitbit client from the Google
Health API onto the classic Fitbit Web API, but the two sync integration
tests still stubbed fetch with Google-shaped dataPoints responses. The mocks
no longer matched the endpoints the client reads, so every ingest assertion
saw zero rows.
Rewrite the fetch-mock layer in both files to key on the classic endpoint
paths and return classic-shaped JSON the real mappers parse:
- body/log/weight, activities/heart for the metrics sync
- activities/steps for the cumulative-activity sync
- 1.2 sleep log levels.data for the per-stage sleep sync
- activities/list for the workout sync
The ingest contract is preserved (row counts, dedup, source priority, sleep
stages, daily-total overwrite). Two assertions move to their classic
equivalents: the workout externalId now anchors on the numeric logId, and
the workout test pins the user to UTC so the offset-less local wall-clock
timestamps the classic sleep/activity logs emit resolve to the same instants
the cross-source picker buckets on.
Send the OpenAI JSON response format only when JSON is requested and no
tools are in play, so the Coach prose path (and the tool loop's final
round) is no longer corrupted into a JSON blob. Resolve the Anthropic
finish reason to tool_calls whenever tool_use blocks are present. Memoize
the per-request feature extraction and prefs read and project the feature
query, so the coach tool fan-out shares one read instead of six.
The F1 coach tools each rebuild a snapshot with a distinct single-source
scope, so the 60s snapshot LRU (keyed on the source list) never shares their
reads. On a multi-metric turn the model can fan out up to six tools that each
fire `extractFeatures` (a user lookup plus a windowed measurement read plus the
all-time extremes) and a prefs `findUnique` concurrently — up to twelve heavy
reads against the shared Prisma pool at once.
Memoise both reads on the request-scoped event cache, keyed on userId (and
windowDays for the features read), so the fan-out collapses to one of each
within a request while staying correct across requests. Project the feature
extractor's measurement read to the six columns its consumers actually use
(type, value, measuredAt, sleepStage, source, deviceType) so the 200k-row-class
PULSE / glucose windows stop pulling every column. Outputs are unchanged.
Parse Fitbit's offset-less classic sleep/activity wall-clock strings
against the user's timezone (DST-safe) so a non-UTC user's segments land
on the right wake-day. Add a shared rotated-refresh-token persist that
compare-and-swaps on the stored ciphertext and reuses a peer's rotated
token on a lost race, so concurrent syncs across Fitbit, WHOOP and Oura
no longer park a connection at a spurious reauth.
Fitbit, WHOOP, and Oura all issue one-time-use refresh tokens. The token
persistence did an unguarded read → refresh → write, so two overlapping
syncs for one user both read the same stored refresh token and both POST
it. The provider honours only the first; the loser's refresh comes back 401,
gets classified as reauth, and the connection is parked at `error_reauth`
spuriously even though the peer rotated the token successfully.
Add a shared `persistRotatedToken` helper that writes the rotated pair with
a compare-and-swap on the stored ciphertext (matched verbatim, since AES-GCM
re-encryption of the same plaintext differs byte-for-byte). When the
conditional update touches zero rows a peer already won the race; re-read the
row and reuse the peer's freshly rotated access token instead of erroring.
Wire it into all three providers — Fitbit and WHOOP `getValidToken`
(proactive expiry refresh, keyed by connection id) and Oura `storeOuraTokens`
(reactive 401 refresh, keyed by user id). Both racing callers now end with a
usable token and neither drops the connection.
The Anthropic client derived `finishReason` solely from `stop_reason`. The
Coach tool loop continues only when `finishReason === "tool_calls"`, so if
Anthropic ever returned tool_use blocks under a different (or absent)
stop_reason, the reason resolved to undefined, the loop dropped the tool
request, and the empty tool_use-only reply surfaced as the final answer —
the requested data was never fetched.
Set `finishReason` to "tool_calls" whenever tool_use blocks are present,
clamped ahead of the stop_reason map, so the presence of a tool request is
authoritative regardless of the reported stop reason.
The OpenAI client pinned `response_format: { type: "json_object" }` on every
call. The Coach tool loop's forced-final round runs with `tool_choice: "none"`
and expects plain prose, but JSON mode coerced the reply into a JSON object —
the user saw a JSON blob or an empty reply instead of the coach's answer.
Send `response_format` only when the caller opts in with `responseFormat: "json"`
and no tools are present, mirroring the Anthropic client's prefill gate. The
JSON-consuming callers that relied on the implicit mode (insight generate +
briefing retry, medication extraction, the connection-test probe) now pass the
flag explicitly; the prose Coach path and every tool round stay out of JSON
mode. Codex already omitted the flag and is unaffected.
Sort the fever red-flag points chronologically before the consecutive-run
scan so a dual-source episode (thermometer plus journaled fever) can no
longer falsely escalate or hide a sustained-fever flag. Treat a trailing
out-of-band tail shorter than the stability window as transient noise so
a settled vital keeps its recovery day; only a sustained re-deviation
clears it.
The last-sustained-return walk nulled the recovery day whenever the final reading
sat out of band, even after a vital had settled for several days. A lone noisy
sample on the last logged day — common in resting-HR and HRV curves — then erased
a real recovery gap that had previously been reported.
Treat a trailing out-of-band tail shorter than the stability window as transient
noise and keep the settle day; only a sustained re-deviation (a tail at least as
long as the stability window, a genuine relapse with no re-settle) clears the
return. Apply the same noise tolerance to the symptom-burden track for parity.
Cover the trailing-outlier, genuine-relapse-then-re-settle, and
sustained-no-re-settle cases.
The classic 1.2 sleep log and the activities list emit local wall-clock
ISO strings without an offset. `parseLocalInstant` parsed them with a bare
`new Date(iso)`, which reads an offset-less string in the process zone (UTC
in production). A non-UTC user's sleep-segment `measuredAt` (the segment
end) was therefore shifted by their UTC offset, and a near-midnight segment
could flip onto the wrong wake-day in the night reconstruction.
Resolve offset-less timestamps against the user's stored timezone via a new
`zonedWallClockToUtc` helper (the inverse of `wallClockInTz`, DST-safe via
the same two-pass converge as `startOfLocalDayInTz`). Thread the resolved
zone through the sleep and workout sync paths. Strings that already carry an
offset or `Z` stay absolute and parse as-is; an absent zone keeps the prior
host-local fallback.
Emit regression_source: unavailable_pending_backfill when slope/r2/sd are
null because a tenant's accumulators are not yet backfilled, so the miss
is observable rather than silently logged as a rollup hit. The null value
is unchanged; no live fallback added.
The windowed slope / r2 / sd readers compose from the per-bucket
regression accumulators (migration 0190). Between a deploy and the
boot-backfill completing, a tenant's pre-0190 DAY buckets still carry
NULL accumulators, so composeWindowedRegression returns a full miss and
the slim / comprehensive readers emit a null slope. The null is the
honest answer until the backfill refills the accumulators, but the miss
was passing as a plain rollup read with no signal, against the no-silent-
cap rule.
Add hasPendingAccumulatorBackfill to the read tier and key a new
regression_source annotation off it (unavailable_pending_backfill vs
rollup) on both warm paths. No live-SQL fallback is added; the design
intentionally avoids the heavy scan.
The fever escalation unions two individually-sorted sources — the passive
BODY_TEMPERATURE series and the day-log feverC — into a Map, then feeds the
entries to the consecutive-run scanner. Map iteration is insertion order, not
chronology, so an episode with both a thermometer feed and journaled fever read
the run out of day order: a broken fever could read as a sustained run (a false
seek-care escalation) or a genuine run could be hidden (a missed one).
Sort the unioned points by day before the scan. Cover both directions with
interleaved-source fixtures.
Replace the Coach snapshot-stuffing with a small base context plus a data
inventory and six on-demand retrieval tools (metric series, glucose,
sleep, medication compliance, labs, illness recovery) the model calls as
needed, then answers. Cuts first-turn input ~35-65% while preserving
grounding and every gate; tools return a structured present:false rather
than throwing, so the model cites only what it fetched. Falls back to the
snapshot path when a provider lacks tool support.
Replace snapshot-stuffing in the Coach turn with a small base context plus
on-demand retrieval. The model receives the full system prompt, a tool-mode
grounding addendum, and a tiny DATA INVENTORY manifest listing which domains the
user has logged and which tool fetches each; the figures themselves are pulled
only when a question needs them.
Six read-only tools wrap the existing server-authoritative snapshot builder:
get_metric_series (blood pressure / weight / pulse + the additive series),
get_glucose_panel, get_sleep, get_medication_compliance, get_labs, and
get_illness_recovery. Each returns a structured { present: false } for an absent
or opted-out domain rather than throwing or returning an ambiguous empty list,
so the model can always tell missing data from real data. Module and cycle gates
are enforced at the builder, so opted-out domains are unfetchable.
A bounded loop runs the call -> tool-result -> answer cycle: at most two rounds
with a hard cap of three that forces a final answer, parallel tool execution,
and only the final round streamed. The day's budget reserves the per-call
ceiling times the round count and reconciles the summed tokens; the atomic
reserve and reconcile primitives are unchanged. All inbound gates (rate limit,
refusal, replay scan, consent, budget) stay before the loop.
The legacy snapshot path stays alive as the no-tools fallback: tool mode is on
only when every provider in the chain supports tool-calling, so a chain that
includes a local model falls back to the prior behaviour verbatim. The tool
trace is persisted onto the assistant message provenance.
The Coach snapshot builder returns the structured, post-degrade record it
serialises to JSON alongside the JSON string, so a single domain block can be
read out without re-parsing. Add an optional tool-call trace to the provenance
envelope (which retrieval tools ran + whether each found data) and mirror it in
the OpenAPI provenance schema.
Groundwork for on-demand retrieval: the retrieval tools slice one section out
of this record, reusing the builder's gates and I/O verbatim.
Retarget the Fitbit integration from the Google Health API (which needs
annual CASA review — a wall for self-hosters) to the classic Fitbit Web
API with PKCE and rotating refresh tokens, so a self-hoster can connect
with an instant developer app (migration 0191 adds the PKCE verifier).
Transport-only fork; the ingest, dedup, source-priority, backfill and
poll paths are reused. Intraday HR and skin temperature are not mapped
(no honest slot); the integration stays experimental.
Store additive per-bucket regression accumulators (sum_x, sum_xy, sum_xx,
sum_yy) so windowed slope, r2 and standard deviation compose from the
rollup tier instead of a live raw-table scan on every warm request
(migration 0190, backfilled by the boot queue). Median and the exact
anomaly count stay on a minimal live query. Parity with the live REGR_*
path within float tolerance, well below the read rounding.
Add the optional FITBIT_REDIRECT_URI to .env.production.example (BYO-key shape,
derives from NEXT_PUBLIC_APP_URL) so check-env documents it. Rename the
settings-card copy from "Google Health" to "Fitbit" across all six locales now
that the transport is the classic Fitbit Web API.
Connect mints the PKCE pair, stashes the verifier on the OAuth-state row, and
sends the S256 challenge to the authorize endpoint. Callback reads the verifier
back and presents it on the token exchange, refusing cleanly when it is absent.
The connection-test probe hits the classic /1/user/-/profile.json endpoint.
The shipped Fitbit integration spoke the Google Health API, whose Restricted
scopes need brand verification plus an annual CASA assessment before leaving
testing mode — a wall most self-hosters cannot clear. Retarget the transport to
the classic api.fitbit.com Web API: a self-hoster registers an app at
dev.fitbit.com in minutes, no CASA.
Forks only the transport; the upsert/dedup/rollup tail, source-priority,
backfill, poll worker, and status ledger are reused unchanged.
- OAuth: www.fitbit.com/oauth2/authorize + api.fitbit.com/oauth2/token, PKCE
(S256), classic self-serve scopes.
- Refresh tokens rotate (one-time use): persist the new token every refresh,
with a defensive keep-existing guard on a malformed reply.
- Per-endpoint date-range fetchers chunked to the API range caps; the backfill
horizon and poll concurrency are tuned for the 150 req/h-per-user budget.
- Verified classic-shape extractors for weight, body fat, SpO2, HRV (dailyRmssd),
resting HR, respiratory rate, steps, distance, active calories, floors, VO2 max
(range midpoint), per-segment sleep stages, and workouts.
- Skips intraday HR (needs a per-app grant; daily resting HR covers it) and skin
temperature (a baseline delta with no honest absolute slot).
The classic Fitbit Web API authorises with PKCE (Authorization Code + S256),
so the connect route mints a code_verifier and the callback presents it on the
token exchange. Carry it on the short-lived FitbitOAuthState row.
Additive + nullable (migration 0191): a legacy in-flight row carries no
verifier, so the change is reversible and rerun-safe with no data loss.
Two suites pin that the windowed slope / r2 / population-sd composed from
the rollup accumulators equals the live Postgres REGR_SLOPE / REGR_R2 /
STDDEV_POP over the same raw rows.
compose-regression.test.ts (container-free) builds per-DAY-bucket
accumulators from a raw fixture exactly as the populator's SUM terms do,
then asserts composeRegression over those buckets equals a direct OLS over
the flat raw rows - the same closed form Postgres folds - on an uneven
multi-bucket fixture, plus the coverage contract (NULL accumulator,
< 2 readings, and zero-x-variance all return a full miss so the reader
falls back to live SQL).
rollup-regression-parity.integration.test.ts (real Postgres testcontainer)
seeds raw measurements across many uneven DAY buckets, folds the rollups,
and compares the accumulator compose to live REGR_*/STDDEV_POP to <= 1e-9:
a single-source WEIGHT series, a dual-source RHR series proving the
source-collapse runs BEFORE accumulator summation (composed == live WHOOP
-only, != the cross-source blend), and a DST-boundary series confirming
the UTC epoch-day x-axis parity-matches across the transition.
The residual is pure floating-point reassociation (per-bucket-then-window
vs flat summation), orders of magnitude below the read tier's rounding.
Move the AI client layer from a single system/user string pair to a
structured system + messages[] shape, with optional tools/toolChoice and
a result carrying toolCalls/finishReason/cachedInputTokens. Per-provider
prompt-caching is wired (Anthropic cache_control, OpenAI/local auto
prefix); tool plumbing is in place for the Coach to use. Behaviour-
preserving: no streaming change, snapshot-once retained.
Replace the CompletionParams { systemPrompt, userPrompt } string pair with a
structured { system, messages[] } request across all five AI clients. One
refactor unlocks three things: provider prompt-caching (Anthropic cache_control
on the stable system prefix, OpenAI/Codex automatic prefix cache, with
cachedInputTokens surfaced for observability), the tool-calling wire surface,
and clean multi-turn.
Each client maps the new shape to its own wire and degrades unsupported
features silently: the local OpenAI-compatible server never receives a tools
field (most reject unknown keys), and prompt-caching is best-effort everywhere.
CompletionResult gains toolCalls, finishReason and cachedInputTokens; the tool
types and per-provider mapping ship now, the tool-use loop does not.
Vision inputs fold into the user turn's content parts via singleUserTurn, with
each provider's pre-existing block ordering preserved. The corrective-retry
paths append to the last user message rather than a flat string. The Coach
keeps assembling one user turn so its snapshot-once and grounding behaviour is
unchanged. No wire schema visible to API consumers changed.
Behaviour is unchanged across every surface; the full suite stays green.
Migration 0190 left existing DAY rollup rows with NULL regression
accumulators until a re-fold refills them. Add a second discovery arm to
enqueueBootTimeRollupBackfill, UNION-ed with the per-day missing-coverage
arm: a user whose DAY buckets carry sum_xy IS NULL inside the fold window
needs a one-time re-fold so the windowed slope/r2/sd readers compose from
the accumulators instead of falling back to the live REGR_* scan.
New writes self-populate the accumulators (they ride the same aggregate
as mean/sum), so the arm drains to empty once the boot re-fold catches up
- the same convergence the v1.4.39 sum_value rollout reached through this
queue. Bounded to ROLLUP_FOLD_WINDOW_MS so days outside the window never
re-flag a user on every boot.
The summaries slice (computeFromRollups) and the comprehensive aggregator
(buildFromRollups) each ran a narrow live $queryRaw over the raw
measurements partition on EVERY warm-rollup request to fold slope7/30/90
and r2_7/30/90 via REGR_SLOPE / REGR_R2 - the un-eliminated live scan on
the happy path.
Drop the six REGR_* columns from both narrow queries and compose the
windowed slope / r2 (and the population sd via composeRegression) from
the per-bucket accumulators stored in measurement_rollups. The 90-day
DAY buckets are loaded once (already loaded in comprehensive; one bounded
findMany added in the slim slice), source-collapsed to the canonical
source, then summed over the day-aligned 7/30/90-day sub-windows. A
window whose buckets lack accumulators (pre-backfill) composes to null,
the same shape the empty REGR_* FILTER returned, so a transitional read
never returns a wrong partial regression.
median (PERCENTILE_CONT) and the exact |z|>2 anomaly count stay on the
minimal live query - both are non-composable across buckets. The cold
coverage-miss path keeps the live REGR_* regression. Annotate
slope_source:"rollup" + median_source/anomaly_source:"live" so a
regression to the heavy path stays observable (no-silent-cap rule).
Make the physiological return relapse-aware (the start of the final
sustained in-band run, not the first), so a vital that relapsed and
recovered again reports its true recovery. Add an optional sleep-context
sub-line (baseline vs episode asleep-minutes), withheld on thin data and
never feeding the gap median.
Adds a neutral sleep observation alongside the per-episode recovery gap: the
user's usual asleep minutes over their well baseline vs the mean over the
episode window, narrated as 'you slept more/less than usual' when the two
diverge. It is an observation only — never a recovery claim, never a return
track, and never part of the headline recovery-gap number.
Per-night asleep totals come from the canonical night reconstruction over the
baseline and episode windows, so a multi-source night (two writers for one
sleep) is counted once, matching every other sleep surface. The observation
withholds entirely on thin data — too few baseline or episode nights, or a
delta below a 30-minute floor — so day-to-day jitter is never narrated as a
finding.
Additive optional DTO field; no schema, no migration. Two locale keys across
the six bundles. Regenerate the OpenAPI contract from the Zod source.
Windowed slope, r2 and population sd still ran a live REGR_*/STDDEV_POP
query over the raw measurements partition on every warm-rollup request,
because the per-bucket mean cannot reconstruct an OLS regression over the
raw points in a window. The per-bucket sd/slope/r2 columns are a
within-bucket fold and do not compose linearly across DAY buckets.
Add the four regression-accumulator sums Postgres folds for REGR_* /
STDDEV_POP - sum_x, sum_xy, sum_xx, sum_yy - per (user, type,
granularity, bucketStart, source) over the same canonical-source rows
the live query reads, with the x-axis in epoch days. With n = count and
sum_y = mean*count already stored, all six terms are additive across
buckets, so a 7/30/90-day slope/r2/sd composes from the summed
accumulators with a bit-identical result to the live closed form.
Migration 0190 adds the four nullable columns; nullable keeps it
additive so existing rows read NULL until the boot re-fold refills them
(mirrors the v1.4.39 sum_value rollout). The populator emits the four
SUM terms inside the existing aggregate (no extra round-trip) and
persistRollupRows writes them. composeRegression / composeWindowedRegression
evaluate the closed form from already-collapsed buckets and return a full
miss when any in-window bucket lacks accumulators, so the reader falls
back to live SQL rather than composing a partial window.
Default the built, data-gated sleep, steps and glucose dashboard tiles to
visible (each self-hides without data) and surface the Vorsorge chart row,
so shipped capabilities are discoverable on the home dashboard. Seed the
demo with an explicit showcase layout.
The demo account seeds rich sleep, steps, glucose, VO2max and workout
data but inserted no dashboard layout, so the home page rendered the
new-account default and under-sold the product on its primary surface.
Persist an explicit dashboard layout derived from the default with the
seeded-rich strip tiles forced on and the daily-verdict hero enabled, so
a fresh seed renders a full, lived-in dashboard for screenshots.
The sleep, steps and glucose strip tiles shipped default-hidden even
though their modules are on by default and rich data flows in from the
sync integrations, so accounts had to hunt through Settings to see them.
Flip the three strip tiles to tileVisible:true and the vorsorge chart-row
card to visible:true, matching the VO2max precedent: each tile self-gates
on having data, so empty accounts keep a clean dashboard. Chart rows for
the three metrics stay off — those charts live on the /insights sub-pages.
totalBodyWater, boneMass and oxygenSaturation stay hidden: they carry as
catalogue ids for the iOS layout round-trip only and have no web render
path, so a flip would surface nothing. Note that at the splice point.
Update the silhouette skeleton and layout default tests for the new
counts.
Marc-André Bombeck