The snapshot's second phase (blood pressure in target, the per-context
glucose tiles, the Health Score) waits until weight, both blood-pressure
halves and steps have DAY buckets. The fold writes buckets only inside
the five-year window, so a type whose every reading is older than that
never gets one and the phase stayed null for good. Since the Apple Health
import now folds only inside the window too, an account that imported
years-old blood pressure or steps and nothing since lost those tiles and
its score with no explanation.
An uncovered gate type now holds the phase back only when it has a
reading inside the window, which is the case the backfill will still
reach. The extra query runs on the miss path alone and asks one indexed
EXISTS per uncovered type.
The runtime accepts "+05:30" as a zone id, so isValidTimezone let it through,
but Postgres reads a bare offset in AT TIME ZONE with the POSIX sign (east
of UTC is negative), and the day-grain SQL aggregates would cut every day on
the wrong side of UTC.
isValidTimezone now accepts named zones only: whatever the runtime resolves
to a bare offset is refused, while named zones, their aliases, UTC and the
Etc/GMT family stay accepted. Every write that already validated through it
refuses offsets now, and the screener and mood linked-context request schemas
validate their zone too. A value stored before this change can still sit in
the database, so the SQL call sites that read the stored zone directly (the
dashboard summary, achievements, the doctor report, the tiered AI context)
fall back rather than splice it, as the resolver and the day aggregates
already did.
A single account syncing heart rate from a watch took the worker down
within half an hour. The status cards' graded series read the last 91
days of raw rows, bounded in time but not in rows, so a stream sampled
every minute or faster materialised six or seven figures of objects on
every refresh and the read ran into the statement timeout.
Add readDayAggregates, which folds one type into per-day sum, count, min
and max in Postgres (optionally split into age segments and filtered to
the metric's plausibility domain), and move the readers that folded raw
rows into days in memory onto it: the graded series (recent, weekly and
the cold-tier fallback, for every type), the vitals day-mean fallback
behind the personal band, the tiered context's recent days and the plan
review. Recent, weekly and monthly buckets are identical to the raw
fold; the cold-tier yearly slope now runs over day means.
On 91 days of per-minute pulse the graded series now allocates 1.6 MB
where it allocated 62 MB, while the raw read alone holds 22 MB.
A structural guard freezes the remaining untaken measurement reads in
the worker and insight directories: each must name a sparse literal
type or carry a listed reason.
Refs #1023
The three score engines' unit tests hand `persist*Score` a fake Prisma, but
`upsertScoreRow` routes the written row through `afterMeasurementMutation`,
which reads the `@/lib/db` singleton instead of the injected client. Both legs
of that tail therefore opened a socket to a Postgres the unit job does not run:
on CI the attempt failed with ECONNREFUSED a few hundred milliseconds later and
logged on the way out, after the test that started it had ended. A console write
that lands while the worker is closing its RPC channel fails the whole run with
`EnvironmentTeardownError: Closing rpc while "onUserConsoleLog" was pending` —
four of nine unit-suite failures in thirty days, every one of them with all
tests green, blamed on whichever score file the worker happened to be tearing
down. Run 31961548185 carries the full stack: recomputeBucketsForMeasurement <-
afterMeasurementMutation <- upsertScoreRow <- persistStressScore <-
stress-score.test.ts:105.
It stayed invisible on a maintainer's machine because a local Postgres answers
on 5432: the same call succeeded quietly, and the suite was writing rollup rows
for its fixture users into a real database.
The four files that drive `upsertScoreRow` stub the tail. That removes the
doomed I/O rather than the log line, and the call they were making by accident
is now asserted in score-row.test.ts instead: the v1.37.19 wiring is pinned by
an expectation rather than by a stack trace. Verified by mutation — dropping the
`afterMeasurementMutation` call from score-row.ts fails the new assertion.
The settle guard's sweep is widened while here. Its eight-line handler window
and five-line registration preamble were both measured against the sites that
existed when it was written and both cut through real code: the console write in
mood-rollups.ts sits nine lines into its handler, so the one genuinely
unregistered detached promise in the tree was never seen, and the registration in
fire-and-forget.ts sits below its handler, so a correct site read as a violation.
Registration is now looked for wherever it wraps the statement, by paren depth
rather than by line count, and mood-rollups.ts registers its WEEK/MONTH/YEAR
enqueue. Verified by mutation: unwrapping that call fails the guard.
Gate: the four files, twenty consecutive runs, 20/20 green (53 tests each).
Test time across the three engine files falls from 708 ms to 20 ms — the
round-trips that are no longer made.
The unit suite reported all 1937 files green and then failed the job with
`EnvironmentTeardownError: Closing rpc while "onUserConsoleLog" was
pending`, naming a different scoring test each time.
Vitest replaces globalThis.console with a sink that forwards every write
to the main thread as an onUserConsoleLog RPC call. When a worker
finishes a test file it awaits the calls in flight and then rejects
whatever was started after that point, so a console write from work that
outlived its test lands in the gap and fails the run. The blame goes to
the file being torn down rather than to the test that started the work,
which is why the named file never logged anything itself.
Three paths detach a promise and log on the handle: fireAndForget, the
stale-while-revalidate cache refresh, and the offline query persister.
They now register the handled handle in a set that both suite setups
drain in afterEach, so no test leaves work in flight. The registry is
inert outside test: nothing there awaits it, and holding a reference to
an unsettled promise would retain it for the life of a worker. No
production behaviour changes, so there is no changelog entry.
Evidence. A focused harness that starts failing background tasks and
returns hit the teardown error 10 runs out of 10 before the change and
0 out of 10 after. Instrumenting the real suite showed two detached
console writes per run escaping their test, both from the rollup tail;
after the change, none, and thirteen writes per run now complete inside
the test that started them.
The guard asserts the behaviour, not a count: a registered task must
keep settleBackgroundTasks from resolving ahead of it, and the set must
drain back to empty. It goes red on unregistered work, on a disabled
registry, on an unwrapped call site, on a sweep that matches nothing,
and on a setup that drops the hook.
The tier for a chart series keyed off the requested window width. The
All range always requests ~3650 days, so every account got MONTH
buckets regardless of how much history exists — a record spanning four
months came back as four monthly means, and the chart captioned them
from the real span as a weekly average. The reader now re-reads at the
tier the returned rows' span calls for; a coverage miss on the finer
tier keeps the coarser rows.
The ladder comment claimed to mirror the client's pickBucket
thresholds and misquoted them. The two ladders are deliberately not
identical; a test now pins the direction that keeps the caption honest
(server never coarser than the client captions) for every span up to
ten years.
The knip blind spot (ignoreExportsUsedInFile) left eight symbols carrying
an export nothing outside their file consumes — the modifier goes, the
symbols stay. The score series module goes whole: its only importer was
its own guard test, whose composite- and config-guarding suites survive
in place.
DocumentContentIndex.tokenizerVersion gains the consumer that was the
point of storing it: the content-index job now re-tokenises rows whose
version trails CONTENT_TOKENIZER_VERSION, from the decrypted index text
along the same path the rotation script uses, keyset-paginated and
fault-isolated per row — a future tokenizer bump migrates the index
instead of silently splitting it. ImportJob.uploadSha256 turned out to
already have its dedupe consumer (both kick-off routes short-circuit on
same-bytes non-failed jobs); its comment now names that instead of
pretending to be write-only. The remaining write-only columns carry
their honest annotations in the shape the audit sweep parses.
- The tiered AI context's band reader now falls back to a live grouped
aggregate when a rollup band is empty (the module doc promised
replace-with-fallback from day one; only the replace half existed), so
a coverage-miss or never-folded YEAR partition no longer hands the
model empty history. Far-from-UTC users get the day-grain band keyed
on their own calendar (the same ±3h guard the correlation readers
carry).
- The slim analytics slice splices measurements older than the 5-year
fold window into its all-time count/min/max/mean: the rollup tier
only covers the window, so "all time" silently truncated for long
histories while the live fallback told the truth. A type whose only
rows are pre-fold no longer reads as never logged.
A soft-deleted measurement never reached the rollup tier: the WHOOP
delete webhook tombstoned rows without recomputing the dirtied buckets,
and the freshness watermark probe filtered tombstones out, so the
deletion's updatedAt bump was invisible and the ghost aggregate read as
fresh forever.
Extract the post-write tail every measurement surface repeats into
afterMeasurementMutation (rollup recompute + status-insight
invalidation, both best-effort), call it from the WHOOP delete webhook
with the (type, measuredAt) pre-image collected before the updateMany,
and drop the deletedAt filter from the watermark probe so a tombstone
counts as staleness.
The persistent mood-rollup writer bucketed on the UTC truncation of
`mood_logged_at` and ignored the canonical per-row `MoodEntry.date`
label. Every coverage-miss fallback keys on `entry.date`, so the rollup
tier disagreed with the fallback and with the day the user logged the
mood: a late-evening or after-midnight local mood landed on the wrong
calendar day, and the label shifted the moment the boot backfill flipped
a user from the live fallback onto the rollup tier.
Key every bucket off the stored `date` label instead — DAY, WEEK, MONTH
and YEAR — with `bucket_start` set to that label's UTC-midnight instant.
The rollup is now byte-identical to the live `entry.date` fallback and
timezone-independent by construction: DST correctness stays where it
already lives, in `moodDateKey` at write time. No consumer changes, no
schema change; every rollup reader already formats `bucket_start` as a
UTC day label.
Migration 0269 empties `mood_entry_rollups` so the boot-time
full-backfill worker re-mints every user's buckets under the corrected
keying; the worker now deletes-then-refolds so it is self-cleaning
against any row left by an older instance. While the table is empty,
reads fall back to the live `date`-keyed walk, which was already
correct, so there is no window where a wrong day is shown.
persistRollupRows scoped its delete to the [min, max] bucket range of the rows
the aggregate RETURNED. That made the repair self-limiting in exactly the case
that needs it: a recompute could only overwrite a bucket the live aggregate
still produces, so a bucket whose source rows had all gone was never in the
delete range and kept its stale aggregate.
Three shapes of the same hole. An empty aggregate skipped the write entirely
(the caller short-circuited on rows.length === 0). A returned range narrower
than the requested window left the edge buckets untouched. A type that lost
every reading in the window dropped out of the returned rows, so the per-type
delete filter could never reach it.
The DAY write hook masked it — it deletes its own bucket by hand when the day
empties — but nothing covered the WEEK / MONTH / YEAR worker, which recomputes
one bucket span at a time: deleting the last reading of a type in a week
dropped the DAY bucket and left the coarser ones stale forever. The coverage
probe still reported hasBuckets = true, so the read-swap served them.
A recompute now prunes the window it was asked for and writes its rows in the
same transaction. The single-bucket write hook keeps its own scope, which is
correct there because it already deletes its bucket when the aggregate empties.
Still one tier: the rollup replaces the legacy read and falls back to live SQL
only on a coverage miss.
Each case fails on the pre-fix code: the rollup tests on an unaligned
recompute window, the batch tests on the older stats: snapshot winning,
and the resting test on a series that stops at the fold horizon.
recomputeUserRollups filtered measurements by a caller-supplied from/to
but grouped them by date_trunc, and persistRollupRows then deletes every
rollup row in the bucket range it is about to write. A from that landed
mid-bucket therefore replaced a complete bucket with an aggregate
computed only from the readings after that instant.
Every caller passes a moving instant — now minus ninety days on the
read-path refresh, an arbitrary instant on the coverage fallback, now
minus five years on the boot backfill and the restore path — so a
different oldest bucket was corrupted on each invocation and nothing
repaired it. Snap both edges out to whole buckets via bucketSpan, whose
own contract note already required exactly this of every recompute.
Two layers. The integration suite runs a real pg-boss against a real
Postgres and proves the semantics the policies rest on: that a bare
singletonKey under the default policy de-duplicates nothing, that
exclusive suppresses a duplicate while the first job is queued AND while
it is active, that short suppresses only while queued and admits a send
once the job is active, and that a distinct key always enqueues.
It also pins the two facts that make the fix reach a running instance:
createQueue does not change the policy of a queue that already exists,
and updateQueue refuses a policy change, so the reconcile is required.
The unit guard couples each enqueue site to its registrar entry. The key
and the policy live in different files with nothing in the type system
holding them together, so deleting an entry would leave the call site
looking correct while it silently stopped working.
Refine the go-forward aggregated HR wire contract (iOS #34) from the never-
adopted hourly bucket to a 10-minute UTC bucket, so one contract serves both
the measurements-list de-explosion and the intraday-pulse 10-min curve.
- Batch route accepts stats:HKQuantityTypeIdentifierHeartRate:<10-min ISO-Z>
with avg + min/max, overwrite semantics, malformed-suffix rejection; the
validator is identifier-generic (allowlist) so respiratoryRate/SpO2 can join
later without a grammar change.
- The intraday reader classifies a day's PULSE rows three ways by externalId
shape (raw / uploaded 10-min bucket / server-folded hourly), builds the
10-min curve directly from uploaded buckets with the min/max envelope and
tension detection enabled, and overlays buckets over raw on the one
cutover-straddling day.
- DAY rollup min/max read COALESCE(value_min, value) so a bucket-native day's
band reflects the true intra-bucket extremes.
No schema change (value_min/value_max exist since v1.19.2). Reconciliation is
go-forward cutover; the nightly retention fold handles the raw backlog.
Rollup writes were delete-then-createMany inside a transaction; under READ
COMMITTED a concurrent recompute racing the import's full fold on the same
(user, type, granularity, bucket, source) partition tripped the composite
PK — and the P2002 swallow dropped the loser's freshly computed aggregate,
leaving a stale bucket. persistRollupRows now writes chunked
INSERT … ON CONFLICT DO UPDATE with deterministic (type, source, bucket)
lock ordering inside one transaction: a race can neither fail the import
nor lose an aggregate, and previously-staled buckets self-heal on their
next recompute. The vanished-source delete and in-transaction guarantees
are preserved; the now-unreachable P2002 swallow is removed.
The masked ENOENT is gone with it: long imports outlived the queue's
15-minute expiry, pg-boss redelivered the still-running job, and the
redelivery re-opened the consumed staging file — overwriting the real
terminal state with a raw ENOENT. Import sends now carry retryLimit 0 +
6h expiry (a retry after the staging file is consumed can only mask),
deliveries for terminal ImportJob rows are no-ops, a genuinely missing
staging file reports an honest re-upload message, and the extracted XML
is cleaned up on failure too.
Long-term retention for the dense intra-day tier (PULSE, HRV, SpO2) folds
to per-local-hour means instead of a single daily mean: the last 14 days
keep every raw sample, older days keep up to 24 points per metric — the
day's shape survives permanently at ~9k rows/metric/year. Hourly rows key
as stats:<HK>:<YYYY-MM-DD>T<HH> anchored at local HH:30, adopt-in-place
against both unique indexes with the P2002 retry outside the transaction.
The pre-fold DAY-rollup recompute extends to HRV (mean fidelity, not only
min/max) and the post-fold recompute is gone for the whole tier. Derived
resting rows for proxy users still mint from the day's raw samples.
A one-shot rebuild reconstructs the hourly shape for every already-folded
day from the soft-deleted raw samples and retires the daily row in the
same transaction — at no instant are both grains live, so no
AVG-over-live-rows reader can double-count. Boot discovery pairs live
daily-grain rows with surviving tombstones (parameter-bound SQL, pinned
against real Postgres) and converges to zero once converted; days whose
tombstones were pruned keep their daily mean. PULSE/SpO2 DAY buckets keep
the true pre-fold stats; HRV DAY buckets recompute from the hourly rows
(strictly better than the collapsed fold-time bucket); WEEK/MONTH/YEAR
respans enqueue whole-bucket recomputes.
The week/month/year rollup read anchored on now and always returned the
trailing window, so a historic from/to range (API and iOS callers) came
back with buckets outside the requested span. Thread the resolved from/to
bounds through so the read is bounded on both ends, matching the day-tier
and live-SQL fallback contract; tier selection is unchanged.
Cap and page the unbounded reads that let a multi-year tenant pull its
whole measurement history into memory on hot paths.
- extractFeatures: read newest-first with a 6000-row cap (mirrors the
Coach snapshot) and reverse to ascending, so a 365-400 day window no
longer materialises hundreds of thousands of PULSE / glucose rows per
Coach turn and nightly briefing. Full-history extremes still come from
readAllTimeExtremes, so the cap never relabels an extreme.
- Add a (userId, measuredAt) index on measurements (migration 0210) for
the type-less range scans that extractFeatures and the consolidation
passes drive; the existing (userId, type, measuredAt) cannot serve them
as a prefix.
- runConsolidation: keyset-paginate the per-(user, type) source scan on
(measuredAt, id) instead of one unbounded findMany, removing the giant
single result-set allocation that drove the boot storm. Output is
identical; covered by a paging unit test.
- Stagger the boot-time backfill enqueues with increasing startAfter
offsets so the heavy full-history loads no longer all drain in parallel
onto one tenant at startup. Non-boot callers keep their immediate send.
- Backups: replace load-all-then-stringify with paged narrow-select reads
for the weekly and off-host writers, bounding transient heap. Off-host
still includes soft-deleted rows. Covered by a serialize/restore
round-trip test.
- Drop the per-request auth.bearer.success audit insert; the wide event
already records auth_method and user_id, so the polling client no longer
drives a write + pool checkout on every request. Failure-path audit
stays.
- Coach conversation detail: cap the message include to the newest 200 so
a long thread no longer decrypts every message on each open; response
shape unchanged.
- Admin audit-log: floor the scan at the retention horizon so the listing
and its count stop sweeping open-ended over the substring filters.
The per-bucket OLS accumulators stored x on the raw epoch-day axis (x ~ 20540),
so sum_xx accumulated past ~1e10 and the square shed ~10 of a double's
significant digits before the value was ever stored. The sub-day x detail was
lost at accumulation time, which no read-side identity can recover; an
ill-conditioned window's composed slope/r2 then drifted from the live REGR_*
probe past the parity gauge.
Rebase x to a fixed origin (epoch-days of 2020-01-01) at write time so the
squared terms stay O(1e7) and exact. Slope, r2, and population sd are invariant
under the affine x-shift, so the live raw-epoch probe still parity-matches the
cross-bucket compose; composeRegression needs no formula change. The per-bucket
slope/r2 columns stay on the raw axis (shift-invariant).
Migration 0193 recomputes sum_x/sum_xy/sum_xx from measurements on the rebased
basis, one set-based update per granularity. Recompute-from-source is idempotent
and rerun-safe; it touches only rows that already carry the accumulators and
leaves pre-migration NULLs for the boot re-fold.
Un-quarantine the DST-boundary parity test: it now matches live REGR_* within
the existing 1e-9 relative bound, with the WEIGHT case still bit-identical.
Reformulate composeRegression from the determinant form (n*Sxx - Sx^2)
to the mean-centered corrected-sum identities (Sxx = Sxx - Sx^2/n, ...).
Algebraically identical to the prior form and to Postgres REGR_* /
STDDEV_POP, but it subtracts the x mean at matched scale and avoids the
catastrophic cancellation the determinant form suffers on the ~1e10
epoch-day x-axis. Also read the exact stored sum_value accumulator for
the Sy term instead of reconstructing it as mean*count, dropping the
AVG-to-multiply float residual; fall back to mean*count when the column
is null.
Composition-only: no migration, no schema change. The WEIGHT live-parity
case stays bit-identical at the 1e-9 relative bound. The near-flat DST
parity seed improves by ~30x but still exceeds 1e-9, so it remains
quarantined with an updated note pointing at a window-local x-rescale as
the real fix.
The windowed slope / r2 / sd readers compose from the per-bucket
regression accumulators (migration 0190). Between a deploy and the
boot-backfill completing, a tenant's pre-0190 DAY buckets still carry
NULL accumulators, so composeWindowedRegression returns a full miss and
the slim / comprehensive readers emit a null slope. The null is the
honest answer until the backfill refills the accumulators, but the miss
was passing as a plain rollup read with no signal, against the no-silent-
cap rule.
Add hasPendingAccumulatorBackfill to the read tier and key a new
regression_source annotation off it (unavailable_pending_backfill vs
rollup) on both warm paths. No live-SQL fallback is added; the design
intentionally avoids the heavy scan.
Two suites pin that the windowed slope / r2 / population-sd composed from
the rollup accumulators equals the live Postgres REGR_SLOPE / REGR_R2 /
STDDEV_POP over the same raw rows.
compose-regression.test.ts (container-free) builds per-DAY-bucket
accumulators from a raw fixture exactly as the populator's SUM terms do,
then asserts composeRegression over those buckets equals a direct OLS over
the flat raw rows - the same closed form Postgres folds - on an uneven
multi-bucket fixture, plus the coverage contract (NULL accumulator,
< 2 readings, and zero-x-variance all return a full miss so the reader
falls back to live SQL).
rollup-regression-parity.integration.test.ts (real Postgres testcontainer)
seeds raw measurements across many uneven DAY buckets, folds the rollups,
and compares the accumulator compose to live REGR_*/STDDEV_POP to <= 1e-9:
a single-source WEIGHT series, a dual-source RHR series proving the
source-collapse runs BEFORE accumulator summation (composed == live WHOOP
-only, != the cross-source blend), and a DST-boundary series confirming
the UTC epoch-day x-axis parity-matches across the transition.
The residual is pure floating-point reassociation (per-bucket-then-window
vs flat summation), orders of magnitude below the read tier's rounding.
Migration 0190 left existing DAY rollup rows with NULL regression
accumulators until a re-fold refills them. Add a second discovery arm to
enqueueBootTimeRollupBackfill, UNION-ed with the per-day missing-coverage
arm: a user whose DAY buckets carry sum_xy IS NULL inside the fold window
needs a one-time re-fold so the windowed slope/r2/sd readers compose from
the accumulators instead of falling back to the live REGR_* scan.
New writes self-populate the accumulators (they ride the same aggregate
as mean/sum), so the arm drains to empty once the boot re-fold catches up
- the same convergence the v1.4.39 sum_value rollout reached through this
queue. Bounded to ROLLUP_FOLD_WINDOW_MS so days outside the window never
re-flag a user on every boot.
Windowed slope, r2 and population sd still ran a live REGR_*/STDDEV_POP
query over the raw measurements partition on every warm-rollup request,
because the per-bucket mean cannot reconstruct an OLS regression over the
raw points in a window. The per-bucket sd/slope/r2 columns are a
within-bucket fold and do not compose linearly across DAY buckets.
Add the four regression-accumulator sums Postgres folds for REGR_* /
STDDEV_POP - sum_x, sum_xy, sum_xx, sum_yy - per (user, type,
granularity, bucketStart, source) over the same canonical-source rows
the live query reads, with the x-axis in epoch days. With n = count and
sum_y = mean*count already stored, all six terms are additive across
buckets, so a 7/30/90-day slope/r2/sd composes from the summed
accumulators with a bit-identical result to the live closed form.
Migration 0190 adds the four nullable columns; nullable keeps it
additive so existing rows read NULL until the boot re-fold refills them
(mirrors the v1.4.39 sum_value rollout). The populator emits the four
SUM terms inside the existing aggregate (no extra round-trip) and
persistRollupRows writes them. composeRegression / composeWindowedRegression
evaluate the closed form from already-collapsed buckets and return a full
miss when any in-window bucket lacks accumulators, so the reader falls
back to live SQL rather than composing a partial window.
A multi-year "All" range read its series through the DAY-only daily
reader, which caps the result at BUCKET_CAP.daily (365) buckets. The
older history was silently dropped — the range collapsed to roughly the
most recent year instead of the whole span.
Add readTieredRollupSeries, which reads the bucket tier matching the
charts own display downsampler (WEEK for 1-2 years, MONTH beyond) and
returns whole-history coverage at that tier. readDailySeries now routes
any window wider than the DAY cap through it, falling finer on a coverage
miss and logging the miss rather than truncating silently. Short and
normal ranges never enter the branch, so the common path is unchanged.
The 0–14d raw band folded same-day readings on a UTC calendar key, so a
late-evening reading slipped onto the wrong day for non-UTC users and
across DST transitions. Key it on userDayKey(tz) instead and make the
timezone a required option so a caller cannot silently fall back to UTC.
The coarser DAY/WEEK/MONTH/YEAR bands stay UTC-anchored on purpose: the
rollup writer and the live aggregator both bucket on the UTC calendar
day and are parity-locked for the read-swap fallback. Document the split
and thread the resolved user timezone from the coach snapshot caller.
One-shot prettier pass so the new format:check gate starts from a clean
baseline. Formatting only — no behaviour change. Generator-owned and
vendored artifacts stay untouched via .prettierignore.
Add a coarseOnly option to buildTieredSeries that skips the raw 0–14d
read and the DAY (14–30d) / WEEK (30–90d) band reads, returning only the
MONTH / YEAR bands plus the anomaly envelope. The Coach snapshot's
coarse timeline tail consumes only month/year + anomalies, so the
near-term bands were read and discarded every turn; it now passes
coarseOnly. The briefing path is unchanged. Look the learn-guide prompt
example up by slug rather than by array index so reordering the catalog
cannot silently change the example link.
Wire the Coach snapshot's core-metric history onto the shared tiered
series so blood pressure, weight, and pulse carry a coarse tail the
in-window weekly fold could not produce — 90d-1y monthly and >1y yearly
buckets plus the preserved peak/trough envelope, so a spike from months
ago survives a coarse bucket. The tiered read skips the raw 0-14d band
the snapshot already holds, so no duplicate round-trip; the coarse tail
is bounded per band and sheds first under the prompt-budget degrader, so
token size holds.
Surface the active illness episodes and Rest Mode state to the Coach
prompt: the snapshot already carries the illness block, and the system
prompt now instructs the Coach to read an off-band vital as plausibly
illness-explained and avoid pushing a heavier cadence while the user is
in Rest Mode.
Add a static catalog of the public /learn guides and inject a compact
list into the system prompt. The Coach may reference at most one guide
by topic and URL when genuinely helpful, and only a slug the catalog
lists — a fabricated /learn URL is impossible by construction.
Introduce buildTieredSeries(): a five-band progressive data-context helper —
0-14d raw daily, 14-30d DAY, 30-90d WEEK, 90d-1y MONTH, >1y YEAR — plus a
per-metric min/max-envelope anomaly list so a peak from months ago survives a
coarse bucket. Reuses probeRollupCoverage / ensureUserRollupsFresh with the
existing replace-fall-back-on-miss contract; readers never throw, so a
coverage gap simply yields an empty band. Caps each metric near 600 tokens to
stay within the existing context budget. Shared across the Coach snapshot and
the daily briefing.
The snapshot's thick phase gated on every type the account had ever
logged having rollup coverage — one fresh reading from a new
integration nulled the health score and the blood-pressure extras
while the insights page showed a score for the same account. The gate
now checks only the three types the phase reads, per the established
per-type pattern. Underneath sat the destabiliser: the large rollup
fold deleted the window's buckets and re-inserted without a
transaction or duplicate tolerance, so a racing sync write aborted the
insert and left the delete committed — whole types lost their buckets
until the next clean fold, and accounts with data older than the
five-year fold window re-ran the full backfill on every boot because
discovery scanned unbounded history. Delete and insert now commit
atomically with duplicate tolerance, discovery shares the fold's
bound, the provisional ring says the score is being computed when the
inputs exist and reserves the no-data wording for accounts that have
none, and the greeting leads the band typographically.
A typed apiFetch wrapper owns the envelope unwrap, the ok-check and the
error read for every client call — two hundred call sites across the
app migrate onto it and a lint rule keeps raw api fetches out for good.
The legacy timezone helpers fold into the tz module and test fixtures
drop personal names.
A take whose slot the legacy resolver rejected fell back to a standalone
insert that kept the client's scheduledFor, parking a second live row on
an anchor the reminder projector had already minted — the partial unique
index permits that because it spans source. Both intake write routes now
probe the slot source-agnostically and converge onto the existing row
before inserting, and a standalone insert anchors on the take time per
the ad-hoc contract. The compliance rollup counts distinct slots instead
of rows, which also corrects historical duplicates on the next recompute,
and the slot dedup pass runs nightly instead of only at worker boot.
The medication compliance percentage and the history view drew their
per-dose attribution from four different matchers — the ±12h pairDoses
proximity used by calculateCompliance, the deriveDoseStatus bands (open
dose only), the 3h+120min classifyIntakeTiming heatmap heuristic, and a
raw takenAt-IS-NOT-NULL per-day rollup. They could and did disagree: a
dose could read "taken late" in the rate while the history called it an
off-schedule ad-hoc take.
Route the rate through the shared cadence-aware band minter +
reconstructDoseHistory so the percentage is a tally over the SAME ledger
the history view renders. The same intake can no longer be taken in the %
and ad-hoc in the ledger.
- calculateCompliance: when a medication context is supplied, the
count fields (taken / skipped / missed / rate) come from
tallyComplianceFromLedger. Numerator = taken_on_time + taken_late (a
late dose still counts as taken — the taking-vs-timing split is kept
available); denominator = taken + missed; skipped, ad-hoc, upcoming and
whole PRN groups are excluded; rate capped at 100. The streak still
walks the cadence timeline. Context-less pure-math callers keep the
legacy timeline tally byte-stable.
- complianceChips + missedDoses: adherence rate + missed count come from
the same ledger when an engine context is threaded.
- /api/medications/[id]/compliance heatmap: the per-day timing split and
missed marks are bucketed from the one ledger instead of the per-day
classifyIntakeTiming + expectedSlotCountForDay walk, so the cells agree
with the headline rate.
- medication_compliance_rollups stays a raw cross-medication daily
coverage count, documented as distinct from the band-attributed % (the
band minter is per-cadence and cannot run inside a single SQL tally).
Downstream readers (health score, insight compliance status) keep calling
calculateCompliance; their numbers shift to the honest band-attributed
values. Some currently-counted off-time takes (e.g. an 11:29 take of a
07:00 dose) now read ad-hoc with the slot missed rather than taken-late —
the configurable per-dose window is the lever to widen the on-time band.
Tests: the irregular twice-daily, single-time-late heatmap, and legacy
parity fixtures are updated with rationale for the intended number
changes; new tally tests pin the taking/timing split, the skip / ad-hoc /
upcoming / PRN exclusions, auto-missed=missed, the 100% cap, rolling
retrospective families, and the cross-consistency invariant.
The Prisma unique-constraint (P2002) narrowing predicate had been
hand-copied into five call sites. Fold them into a single isP2002 in
src/lib/prisma-errors.ts and import it everywhere (the two step-drain
sites keep their local isUniqueConstraintViolation name via an import
alias). Behaviour-identical; one place to chase a future P2002-semantics
change instead of five.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The synchronous DAY rollup hook carries no pg-boss singleton, so two
measurement writes landing on the same (user, type, day) can interleave
their delete-then-insert and the losing createMany trips the per-source
unique index. The peer rewrote the identical source-collapsed partition
— the aggregate is a pure function of the same underlying rows — so the
rollup is already correct. Catch the P2002 in the single-chunk
transactional path, annotate it, and return rather than failing the
recompute. Any other Prisma error still bubbles.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Drop the unused `now` field from the coach `ExtractOpts` type; fact
extraction never read it (conversation-summary keeps its own `now`).
- Correct the no-ladder tiebreak docstring in measurement-read: the
collapse picks the alphabetically smallest source name, matching the
live-SQL `ORDER BY … source` tiebreak, not max count.
- Replace the hand-inlined canonical-source subquery in the summaries
rollup-path narrows query with `canonicalMeasurementsFrom(rank, "90 days")`
for live/rollup parity with its siblings.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- conversation-summary: fault-isolate the per-message decrypt in the
fold loop so one undecryptable historic turn (e.g. mid key-rotation)
no longer permanently stalls a conversation's summary refresh.
- source collapse: unify the no-ladder tiebreak on alphabetically
smallest source across the JS reader and the all-time rollup SQL, so a
ranked vital whose day carries only off-ladder sources resolves to the
same canonical source on the warm and the live-SQL read paths.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Complete the source-aware rollup read path: the slim summaries slice
(rollup + live), the comprehensive aggregator (both paths, including the
dailyByType correlation feed), and the /api/measurements route (rollup
read + the live date_trunc fallback) now resolve overlapping sources to
the source-priority ladder's canonical reading per (type, day) before
aggregating. A dual-source vital (e.g. WHOOP + Apple Watch resting heart
rate) surfaces one canonical series instead of a blend or a doubled
count; cumulative metrics keep summing per canonical source.
The collapse runs in SQL where the aggregate must stay a single
round-trip (buildSourceRankCase emits a per-user ladder-rank CASE;
canonicalMeasurementsFrom restricts a live read to canonical-source rows)
and in application code (collapseRollupRowsBySource) for the row readers,
both driven by the same resolved ladders so live and rollup agree.
Also widens the enum-literal SQL whitelist to accept digits so VO2_MAX
no longer throws on the typed write-hook recompute or the rank CASE.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
readRollupBuckets and the WEEK/MONTH/YEAR readers now select the source
column and run collapseRollupRowsBySource before composing, so every
rollup consumer (dashboard sparkline, fast-paths, derived metrics, range
deltas) sees one canonical row per bucket. loadUserSourcePriority loads
the ladder blob once; readBestGranularityRollups threads it through each
granularity probe so the per-floor walk never re-queries the user.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
collapseRollupRowsBySource resolves the per-source rollup rows minted by
the source-aware writer down to one row per bucket, using the user's
source-priority ladder (the same ordering the raw-row picker uses). A
metricKeyForType superset maps every overlapping non-cumulative vital
(RHR, HRV-SDNN, SpO2, body/skin temperature, respiratory rate, weight,
body fat, blood pressure, pulse, VO2max, recovery) plus the cumulative
keys onto a SourcePriorityMetricKey. Unlisted sources fall back to the
highest-count row so a bucket never doubles or goes dark.
Wired into the rollup readers in the following commits.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rollup rows were grouped by (type, day) only, so two sources reporting
the same standard vital for one day (e.g. WHOOP + Apple Watch resting
heart rate) were AVG-blended into a single row instead of resolving to
the source-priority ladder's canonical reading.
Re-grain the rollup table to (type, day, source): the writer now groups
by source and delete-then-inserts each affected partition (so a source
that stops reporting a bucket cannot strand a stale row), and the read
path will collapse overlapping sources to the canonical reading in a
follow-up commit. Cumulative metrics (steps, energy) keep summing per
source. Migration 0115 purges the legacy source-blind aggregates; the
boot backfill + read-time self-heal re-mint per-source rows.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Six touch-disjoint waves landed on develop before this release:
- W1-KNIP — exports/types tiers flipped to enforcing-mode on main;
35 unused exports + 52 unused types triaged to 0/0; shadcn surface
+ zod-infer contract types scoped via knip.json ignore-block.
- W2-ZOD-MULTI-ISSUE — `returnAllZodIssues` shared helper +
`/api/dashboard/widgets` 422 now returns every Zod issue + worker
audit-ledger breadcrumb. 41 sibling routes documented for v1.4.43.
- W3-QUERYKEY-LONGTAIL — 40 settings/medications/admin/hooks files
migrated to the queryKeys factory; 9 new factory entries; ESLint
rule + test-guard scope extended in lockstep.
- W4-ARCH-HYGIENE — BERLIN_DAY_FORMATTER consolidated across nine
sites (seven status helpers + bucket-series + bp-in-target);
Suspense double-comment consolidated; doctor-report sanitiser regex
rewritten to escape-form so diffs are reviewable; pr-detection-
worker soft-delete filter; offhost-backup DR-intent comment.
- W5-IOS-WORKOUTS — `dedupeWorkoutBatch` write-time cross-source
dedup helper for the v1.5 iOS HealthKit observer ingest; dashboard
tile-strip placeholder parity with the live TrendCard chrome.
- W6-WITHINGS-OFF + ENV-CHECK — Withings off-response classifier
(transient/reauth/persistent) + WithingsApiError typed throw + the
`pnpm check-env` CLI. APNs `allOrNone: true` closes the AP-2
silent-disable gap the wave was conceived to close.
No migration, no env-var change, no API contract break for iOS v0.5.4.
The new `/api/dashboard/widgets` 422 envelope is additively extended.
pnpm test --run green at 4815 passing / 1 skipped.
pnpm typecheck, pnpm lint, pnpm knip (no --include flag) green.
Eight touch-disjoint waves landed on develop before this release:
- W-INSIGHTS-HOT + W-RECONCILE-INSIGHTS — extract the timeout-stub
persist helper and wire all seven *-status routes through it, closing
the recurring ~14 s warm response the iOS client sees on dashboard
open once the AI provider has stalled once that day.
- W-DELETED-2 — close the remaining three soft-delete reader-tier gaps
(export bundle, gamification achievements, doctor-report aggregator)
with a five-route integration-test pin.
- W-ORG — types consolidation into src/types/analytics.ts and
src/types/backups.ts; prompt directory unification.
- W-FRONTEND-FACTORY — queryKey factory expansion + ESLint rule lift.
- W-IOS-COORD — POST /api/auth/check-user four-branch onboarding
discovery, identifier matched verbatim, per-IP rate-limited.
- W-SIMPLIFIER — today-intake projection helper extract; unused tx?
params dropped; 13 knip-flagged dead exports trimmed.
- W-PERF-OPS — v1.4.39 W-SUM legacy-NULL UNION discovery arm retired.
- W-PROCESS-DOCS — pg.Pool sizing operator doc; ESLint rule
formalisation; knip CI gate staged for v1.4.42 flip.
No migration, no env-var change, no API contract break for iOS v0.5.4.
The new check-user endpoint is an additive surface. AP-2 .p8 key is
installed; SB-5 time-sensitive APNs is live as of this deploy.
pnpm test --run green at 4732 passing / 1 skipped.
pnpm typecheck, pnpm lint, pnpm knip green.
Eleven touch-disjoint waves landed on develop plus a six-axis QA
reconcile round before tagging. See CHANGELOG.md for the user-facing
prose; this commit message documents the wave + report inventory for
release-management traceability.
Waves shipped (each with a phase report under .planning/):
- phase-W-POOL-v1440-report.md — p-limit(4) wrapper around the
analytics 15-way fan-out + pg.Pool max raised 10 -> 20. Closes the
v1.4.39 empirical-trace finding #1 (Prisma pool starvation).
- phase-W-DELETED-v1440-report.md — 11 reader-tier helpers + 3
integration contracts close audit Critical Finding #3 (soft-delete
invisibility) across dashboard, analytics, summaries, rollup tiers,
and the Coach snapshot path.
- phase-W-INSIGHTS-v1440-report.md — mood-rollup swap on the three
/api/insights/{features,targets,cards} routes plus the
compliance-rollup hook gap closure for bulk-projection mints; the
post-QA reconcile patch closes the six remaining measurement.findMany
sites in the cluster that still leaked tombstoned rows.
- phase-W-INFRA-v1440-report.md — rollup helper umbrella consolidation
under src/lib/rollups/* with zero orphan re-exports; lint, typecheck,
knip, and 93/93 rollup tests green post-restructure.
- phase-W-RSC-v1440-report.md — per-tile Suspense boundaries plus the
queryKey factory-bypass walker test (extended to comparison/ in the
reconcile pass).
- phase-W-PRIVACY-v1440-report.md — bilingual /privacy page with 17
paired-section tests pinning the 9 SB-3 disclosure requirements.
- phase-W-AASA-v1440-report.md — /.well-known/apple-app-site-association
endpoint pinned to S8WDX4W5KX.dev.healthlog.app with app-ID parity
assertions guarding against split rotation.
- phase-W-APNS-NOTIFY-v1440-report.md — MEDICATION_REMINDER
time-sensitive APNs payload (priority 10 + interruption-level)
scoped via a parameterised test that pins all 6 other event-types
do not bypass Focus. AP-2 .p8 install gates real delivery.
- phase-W-CONSENT-v1440-report.md — AI consent receipts CRUD with
append-only invariant pinned in test; revoke-then-regrant leaves
both rows; idempotent DELETE returns 200 + receipt: null.
- phase-W-WMY-WIRE-v1440-report.md — three previously-dead WMY readers
(week/month/year) wired through summaries-slice and
health-score-fast-path with a linear-composability parity test
pinning the cross-consumer routing contract.
- phase-W-GHOSTS-v1440-report.md — ~1 177 lines of dead code removed
across 9 atomic commits; startOfUtcDay + wallClockInTz consolidation
eliminates a sibling-drift risk; knip whitelist trimmed to three
files pending follow-up.
QA reconcile patches applied per the v1.4.40 QA round (six reports
under .planning/round-v1440-QA-*-findings.md):
- senior C1 — dashboard-suspense-boundaries regex aligned with the
post-W-INFRA Thread 2 userTimezone-local lift.
- senior H1 — soft-delete filter added to the six remaining
insights measurement.findMany sites.
- specialist H1 — Promise.allSettled around bulk-projection
compliance recompute on both /api/medications/intake and
/api/dashboard/summary.
- ux H1 — compare-toggle migrated to queryKeys.dashboardWidgets
factory; walker test extended to src/components/comparison/.
- security M1 — consent artefact 64 KB cap enforced via
Buffer.byteLength (UTF-8 bytes, not UTF-16 code units).
Quality gates green at tag: pnpm typecheck, pnpm lint, pnpm knip
--include files,dependencies,binaries,unlisted, pnpm test --run
(4726 passing / 1 skipped). Tests 4525 -> 4726 unit + 1 skipped over
the v1.4.39 -> v1.4.40 span.
User-facing prose: see CHANGELOG.md.
