The persistent mood-rollup writer bucketed on the UTC truncation of
`mood_logged_at` and ignored the canonical per-row `MoodEntry.date`
label. Every coverage-miss fallback keys on `entry.date`, so the rollup
tier disagreed with the fallback and with the day the user logged the
mood: a late-evening or after-midnight local mood landed on the wrong
calendar day, and the label shifted the moment the boot backfill flipped
a user from the live fallback onto the rollup tier.
Key every bucket off the stored `date` label instead — DAY, WEEK, MONTH
and YEAR — with `bucket_start` set to that label's UTC-midnight instant.
The rollup is now byte-identical to the live `entry.date` fallback and
timezone-independent by construction: DST correctness stays where it
already lives, in `moodDateKey` at write time. No consumer changes, no
schema change; every rollup reader already formats `bucket_start` as a
UTC day label.
Migration 0269 empties `mood_entry_rollups` so the boot-time
full-backfill worker re-mints every user's buckets under the corrected
keying; the worker now deletes-then-refolds so it is self-cleaning
against any row left by an older instance. While the table is empty,
reads fall back to the live `date`-keyed walk, which was already
correct, so there is no window where a wrong day is shown.
persistRollupRows scoped its delete to the [min, max] bucket range of the rows
the aggregate RETURNED. That made the repair self-limiting in exactly the case
that needs it: a recompute could only overwrite a bucket the live aggregate
still produces, so a bucket whose source rows had all gone was never in the
delete range and kept its stale aggregate.
Three shapes of the same hole. An empty aggregate skipped the write entirely
(the caller short-circuited on rows.length === 0). A returned range narrower
than the requested window left the edge buckets untouched. A type that lost
every reading in the window dropped out of the returned rows, so the per-type
delete filter could never reach it.
The DAY write hook masked it — it deletes its own bucket by hand when the day
empties — but nothing covered the WEEK / MONTH / YEAR worker, which recomputes
one bucket span at a time: deleting the last reading of a type in a week
dropped the DAY bucket and left the coarser ones stale forever. The coverage
probe still reported hasBuckets = true, so the read-swap served them.
A recompute now prunes the window it was asked for and writes its rows in the
same transaction. The single-bucket write hook keeps its own scope, which is
correct there because it already deletes its bucket when the aggregate empties.
Still one tier: the rollup replaces the legacy read and falls back to live SQL
only on a coverage miss.
Each case fails on the pre-fix code: the rollup tests on an unaligned
recompute window, the batch tests on the older stats: snapshot winning,
and the resting test on a series that stops at the fold horizon.
Rollup writes were delete-then-createMany inside a transaction; under READ
COMMITTED a concurrent recompute racing the import's full fold on the same
(user, type, granularity, bucket, source) partition tripped the composite
PK — and the P2002 swallow dropped the loser's freshly computed aggregate,
leaving a stale bucket. persistRollupRows now writes chunked
INSERT … ON CONFLICT DO UPDATE with deterministic (type, source, bucket)
lock ordering inside one transaction: a race can neither fail the import
nor lose an aggregate, and previously-staled buckets self-heal on their
next recompute. The vanished-source delete and in-transaction guarantees
are preserved; the now-unreachable P2002 swallow is removed.
The masked ENOENT is gone with it: long imports outlived the queue's
15-minute expiry, pg-boss redelivered the still-running job, and the
redelivery re-opened the consumed staging file — overwriting the real
terminal state with a raw ENOENT. Import sends now carry retryLimit 0 +
6h expiry (a retry after the staging file is consumed can only mask),
deliveries for terminal ImportJob rows are no-ops, a genuinely missing
staging file reports an honest re-upload message, and the extracted XML
is cleaned up on failure too.
The week/month/year rollup read anchored on now and always returned the
trailing window, so a historic from/to range (API and iOS callers) came
back with buckets outside the requested span. Thread the resolved from/to
bounds through so the read is bounded on both ends, matching the day-tier
and live-SQL fallback contract; tier selection is unchanged.
The windowed slope / r2 / sd readers compose from the per-bucket
regression accumulators (migration 0190). Between a deploy and the
boot-backfill completing, a tenant's pre-0190 DAY buckets still carry
NULL accumulators, so composeWindowedRegression returns a full miss and
the slim / comprehensive readers emit a null slope. The null is the
honest answer until the backfill refills the accumulators, but the miss
was passing as a plain rollup read with no signal, against the no-silent-
cap rule.
Add hasPendingAccumulatorBackfill to the read tier and key a new
regression_source annotation off it (unavailable_pending_backfill vs
rollup) on both warm paths. No live-SQL fallback is added; the design
intentionally avoids the heavy scan.
Two suites pin that the windowed slope / r2 / population-sd composed from
the rollup accumulators equals the live Postgres REGR_SLOPE / REGR_R2 /
STDDEV_POP over the same raw rows.
compose-regression.test.ts (container-free) builds per-DAY-bucket
accumulators from a raw fixture exactly as the populator's SUM terms do,
then asserts composeRegression over those buckets equals a direct OLS over
the flat raw rows - the same closed form Postgres folds - on an uneven
multi-bucket fixture, plus the coverage contract (NULL accumulator,
< 2 readings, and zero-x-variance all return a full miss so the reader
falls back to live SQL).
rollup-regression-parity.integration.test.ts (real Postgres testcontainer)
seeds raw measurements across many uneven DAY buckets, folds the rollups,
and compares the accumulator compose to live REGR_*/STDDEV_POP to <= 1e-9:
a single-source WEIGHT series, a dual-source RHR series proving the
source-collapse runs BEFORE accumulator summation (composed == live WHOOP
-only, != the cross-source blend), and a DST-boundary series confirming
the UTC epoch-day x-axis parity-matches across the transition.
The residual is pure floating-point reassociation (per-bucket-then-window
vs flat summation), orders of magnitude below the read tier's rounding.
A multi-year "All" range read its series through the DAY-only daily
reader, which caps the result at BUCKET_CAP.daily (365) buckets. The
older history was silently dropped — the range collapsed to roughly the
most recent year instead of the whole span.
Add readTieredRollupSeries, which reads the bucket tier matching the
charts own display downsampler (WEEK for 1-2 years, MONTH beyond) and
returns whole-history coverage at that tier. readDailySeries now routes
any window wider than the DAY cap through it, falling finer on a coverage
miss and logging the miss rather than truncating silently. Short and
normal ranges never enter the branch, so the common path is unchanged.
The 0–14d raw band folded same-day readings on a UTC calendar key, so a
late-evening reading slipped onto the wrong day for non-UTC users and
across DST transitions. Key it on userDayKey(tz) instead and make the
timezone a required option so a caller cannot silently fall back to UTC.
The coarser DAY/WEEK/MONTH/YEAR bands stay UTC-anchored on purpose: the
rollup writer and the live aggregator both bucket on the UTC calendar
day and are parity-locked for the read-swap fallback. Document the split
and thread the resolved user timezone from the coach snapshot caller.
One-shot prettier pass so the new format:check gate starts from a clean
baseline. Formatting only — no behaviour change. Generator-owned and
vendored artifacts stay untouched via .prettierignore.
Introduce buildTieredSeries(): a five-band progressive data-context helper —
0-14d raw daily, 14-30d DAY, 30-90d WEEK, 90d-1y MONTH, >1y YEAR — plus a
per-metric min/max-envelope anomaly list so a peak from months ago survives a
coarse bucket. Reuses probeRollupCoverage / ensureUserRollupsFresh with the
existing replace-fall-back-on-miss contract; readers never throw, so a
coverage gap simply yields an empty band. Caps each metric near 600 tokens to
stay within the existing context budget. Shared across the Coach snapshot and
the daily briefing.
The snapshot's thick phase gated on every type the account had ever
logged having rollup coverage — one fresh reading from a new
integration nulled the health score and the blood-pressure extras
while the insights page showed a score for the same account. The gate
now checks only the three types the phase reads, per the established
per-type pattern. Underneath sat the destabiliser: the large rollup
fold deleted the window's buckets and re-inserted without a
transaction or duplicate tolerance, so a racing sync write aborted the
insert and left the delete committed — whole types lost their buckets
until the next clean fold, and accounts with data older than the
five-year fold window re-ran the full backfill on every boot because
discovery scanned unbounded history. Delete and insert now commit
atomically with duplicate tolerance, discovery shares the fold's
bound, the provisional ring says the score is being computed when the
inputs exist and reserves the no-data wording for accounts that have
none, and the greeting leads the band typographically.
A take whose slot the legacy resolver rejected fell back to a standalone
insert that kept the client's scheduledFor, parking a second live row on
an anchor the reminder projector had already minted — the partial unique
index permits that because it spans source. Both intake write routes now
probe the slot source-agnostically and converge onto the existing row
before inserting, and a standalone insert anchors on the take time per
the ad-hoc contract. The compliance rollup counts distinct slots instead
of rows, which also corrects historical duplicates on the next recompute,
and the slot dedup pass runs nightly instead of only at worker boot.
The synchronous DAY rollup hook carries no pg-boss singleton, so two
measurement writes landing on the same (user, type, day) can interleave
their delete-then-insert and the losing createMany trips the per-source
unique index. The peer rewrote the identical source-collapsed partition
— the aggregate is a pure function of the same underlying rows — so the
rollup is already correct. Catch the P2002 in the single-chunk
transactional path, annotate it, and return rather than failing the
recompute. Any other Prisma error still bubbles.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- conversation-summary: fault-isolate the per-message decrypt in the
fold loop so one undecryptable historic turn (e.g. mid key-rotation)
no longer permanently stalls a conversation's summary refresh.
- source collapse: unify the no-ladder tiebreak on alphabetically
smallest source across the JS reader and the all-time rollup SQL, so a
ranked vital whose day carries only off-ladder sources resolves to the
same canonical source on the warm and the live-SQL read paths.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
readRollupBuckets and the WEEK/MONTH/YEAR readers now select the source
column and run collapseRollupRowsBySource before composing, so every
rollup consumer (dashboard sparkline, fast-paths, derived metrics, range
deltas) sees one canonical row per bucket. loadUserSourcePriority loads
the ladder blob once; readBestGranularityRollups threads it through each
granularity probe so the per-floor walk never re-queries the user.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
collapseRollupRowsBySource resolves the per-source rollup rows minted by
the source-aware writer down to one row per bucket, using the user's
source-priority ladder (the same ordering the raw-row picker uses). A
metricKeyForType superset maps every overlapping non-cumulative vital
(RHR, HRV-SDNN, SpO2, body/skin temperature, respiratory rate, weight,
body fat, blood pressure, pulse, VO2max, recovery) plus the cumulative
keys onto a SourcePriorityMetricKey. Unlisted sources fall back to the
highest-count row so a bucket never doubles or goes dark.
Wired into the rollup readers in the following commits.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rollup rows were grouped by (type, day) only, so two sources reporting
the same standard vital for one day (e.g. WHOOP + Apple Watch resting
heart rate) were AVG-blended into a single row instead of resolving to
the source-priority ladder's canonical reading.
Re-grain the rollup table to (type, day, source): the writer now groups
by source and delete-then-inserts each affected partition (so a source
that stops reporting a bucket cannot strand a stale row), and the read
path will collapse overlapping sources to the canonical reading in a
follow-up commit. Cumulative metrics (steps, energy) keep summing per
source. Migration 0115 purges the legacy source-blind aggregates; the
boot backfill + read-time self-heal re-mint per-source rows.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Eight touch-disjoint waves landed on develop before this release:
- W-INSIGHTS-HOT + W-RECONCILE-INSIGHTS — extract the timeout-stub
persist helper and wire all seven *-status routes through it, closing
the recurring ~14 s warm response the iOS client sees on dashboard
open once the AI provider has stalled once that day.
- W-DELETED-2 — close the remaining three soft-delete reader-tier gaps
(export bundle, gamification achievements, doctor-report aggregator)
with a five-route integration-test pin.
- W-ORG — types consolidation into src/types/analytics.ts and
src/types/backups.ts; prompt directory unification.
- W-FRONTEND-FACTORY — queryKey factory expansion + ESLint rule lift.
- W-IOS-COORD — POST /api/auth/check-user four-branch onboarding
discovery, identifier matched verbatim, per-IP rate-limited.
- W-SIMPLIFIER — today-intake projection helper extract; unused tx?
params dropped; 13 knip-flagged dead exports trimmed.
- W-PERF-OPS — v1.4.39 W-SUM legacy-NULL UNION discovery arm retired.
- W-PROCESS-DOCS — pg.Pool sizing operator doc; ESLint rule
formalisation; knip CI gate staged for v1.4.42 flip.
No migration, no env-var change, no API contract break for iOS v0.5.4.
The new check-user endpoint is an additive surface. AP-2 .p8 key is
installed; SB-5 time-sensitive APNs is live as of this deploy.
pnpm test --run green at 4732 passing / 1 skipped.
pnpm typecheck, pnpm lint, pnpm knip green.
Eleven touch-disjoint waves landed on develop plus a six-axis QA
reconcile round before tagging. See CHANGELOG.md for the user-facing
prose; this commit message documents the wave + report inventory for
release-management traceability.
Waves shipped (each with a phase report under .planning/):
- phase-W-POOL-v1440-report.md — p-limit(4) wrapper around the
analytics 15-way fan-out + pg.Pool max raised 10 -> 20. Closes the
v1.4.39 empirical-trace finding #1 (Prisma pool starvation).
- phase-W-DELETED-v1440-report.md — 11 reader-tier helpers + 3
integration contracts close audit Critical Finding #3 (soft-delete
invisibility) across dashboard, analytics, summaries, rollup tiers,
and the Coach snapshot path.
- phase-W-INSIGHTS-v1440-report.md — mood-rollup swap on the three
/api/insights/{features,targets,cards} routes plus the
compliance-rollup hook gap closure for bulk-projection mints; the
post-QA reconcile patch closes the six remaining measurement.findMany
sites in the cluster that still leaked tombstoned rows.
- phase-W-INFRA-v1440-report.md — rollup helper umbrella consolidation
under src/lib/rollups/* with zero orphan re-exports; lint, typecheck,
knip, and 93/93 rollup tests green post-restructure.
- phase-W-RSC-v1440-report.md — per-tile Suspense boundaries plus the
queryKey factory-bypass walker test (extended to comparison/ in the
reconcile pass).
- phase-W-PRIVACY-v1440-report.md — bilingual /privacy page with 17
paired-section tests pinning the 9 SB-3 disclosure requirements.
- phase-W-AASA-v1440-report.md — /.well-known/apple-app-site-association
endpoint pinned to S8WDX4W5KX.dev.healthlog.app with app-ID parity
assertions guarding against split rotation.
- phase-W-APNS-NOTIFY-v1440-report.md — MEDICATION_REMINDER
time-sensitive APNs payload (priority 10 + interruption-level)
scoped via a parameterised test that pins all 6 other event-types
do not bypass Focus. AP-2 .p8 install gates real delivery.
- phase-W-CONSENT-v1440-report.md — AI consent receipts CRUD with
append-only invariant pinned in test; revoke-then-regrant leaves
both rows; idempotent DELETE returns 200 + receipt: null.
- phase-W-WMY-WIRE-v1440-report.md — three previously-dead WMY readers
(week/month/year) wired through summaries-slice and
health-score-fast-path with a linear-composability parity test
pinning the cross-consumer routing contract.
- phase-W-GHOSTS-v1440-report.md — ~1 177 lines of dead code removed
across 9 atomic commits; startOfUtcDay + wallClockInTz consolidation
eliminates a sibling-drift risk; knip whitelist trimmed to three
files pending follow-up.
QA reconcile patches applied per the v1.4.40 QA round (six reports
under .planning/round-v1440-QA-*-findings.md):
- senior C1 — dashboard-suspense-boundaries regex aligned with the
post-W-INFRA Thread 2 userTimezone-local lift.
- senior H1 — soft-delete filter added to the six remaining
insights measurement.findMany sites.
- specialist H1 — Promise.allSettled around bulk-projection
compliance recompute on both /api/medications/intake and
/api/dashboard/summary.
- ux H1 — compare-toggle migrated to queryKeys.dashboardWidgets
factory; walker test extended to src/components/comparison/.
- security M1 — consent artefact 64 KB cap enforced via
Buffer.byteLength (UTF-8 bytes, not UTF-16 code units).
Quality gates green at tag: pnpm typecheck, pnpm lint, pnpm knip
--include files,dependencies,binaries,unlisted, pnpm test --run
(4726 passing / 1 skipped). Tests 4525 -> 4726 unit + 1 skipped over
the v1.4.39 -> v1.4.40 span.
User-facing prose: see CHANGELOG.md.
