Every ingest path in the product can now emit a typed, salience-classified
event into pg-boss, consumed by one worker that claims a durable per-day
reaction marker and fans out to the reaction surfaces.
The recency classifier runs at the seam, before anything is enqueued, so a
mass backfill — a multi-year Apple Health export, a provider re-sync, a
30-day catch-up — emits nothing at all. Non-salient writes are annotated and
dropped.
The worker makes no provider call by construction: it imports no provider
module, writes only markers, and enqueues nothing that is not budget-gated
downstream. Business refusals return a status rather than throwing, so a
refusal is never retried against a ceiling that will not move until the local
day rolls over.
Seams: sleep (one hook, inherited by all seven transports), workouts (the
batch route, the four provider syncs, the export importer), weight and blood
pressure (both measurement routes), labs (manual, OCR commit, document fact
commit).
The provider workout syncs could not previously tell a created row from an
updated one, so a re-sync of a recent session would have re-fired on every
poll; they now recover the distinction from the row the upsert already
returns.
Reaction lines are AES-256-GCM at rest, following the InsightNarrative
precedent, and the column is registered for key rotation. Markers are pruned
after 14 days by a daily maintenance job.
resolveRestingPulseSeries returned the resting branch as soon as a
single RESTING_HEART_RATE row existed and never consulted the PULSE
proxy again. Since the retention fold began minting derived resting
rows for the days it folds, a proxy account carries derived rows for
old folded days and nothing but raw PULSE for the recent ones — so the
visible series stopped dead at the fold horizon.
Merge per day instead: a day with a resting row uses it, a day without
falls back to the proxy. which stays a three-value labelling signal and
reports proxy as soon as one point is estimated, so a partly-estimated
series is never labelled resting heart rate.
Widen the intraday baseline read to pull PULSE history when the newest
resting row is stale, not only when the rows are few — the count gate
alone pinned the baseline to the fold horizon.
The dense-intraday fold tombstones the raw heart-rate readings a proxy
account's resting figure is derived from, and mints one derived
RESTING_HEART_RATE row per folded day to carry the figure forward. The
mint is skipped for accounts that already hold native resting rows,
because the read resolver ignores the proxy for them.
The probe behind that skip matched every source, including the COMPUTED
rows the mint itself writes. From the second run onward it therefore
reported native resting data for every proxy account, skipped the mint,
and let the fold delete that day's raw readings anyway - losing the
resting figure permanently, once per run.
Scope the probe to non-computed sources. Adds a regression test that
reproduces the second-run state and fails on the previous code.
chore(release): v1.30.16
Kill the above-the-fold first-load flash on `/`: the hero digest and the
batched chart series still fired client-side after JS download + hydrate,
so the hero and charts painted empty and filled in seconds later (LCP
~5.5 s mobile-throttled while the snapshot already dehydrated).
Extend the existing `page.tsx` snapshot prefetch to also seed the daily
digest and the chart series batch into the same dehydrated TanStack cache,
under keys byte-identical to the client's:
- Extract the series-batch route core into `readSeriesBatch(...)` so the
RSC and the route share one code path (no HTTP-to-self); the route thins
to auth + Zod + the shared reader, behaviour unchanged.
- New client-safe pure module `batch-chart-types.ts`: `deriveBatchChartTypes`
(the single implementation of the visible-with-data type-set derivation,
order + resting-HR-vs-pulse pick preserved) and `computeBatchWindow`
(profile-tz end-of-day minus the coverage window). Server and client run
the same function over the same snapshot payload, and the window is
computed once server-side and threaded to the client as a prop, so the
`chartSeriesBatch` key matches on both sides by construction.
- Render the hero data-first so the SSR HTML carries the populated hero
(the LCP element) — hydration-safe because the digest is dehydrated and
never restored from the IndexedDB persist allowlist; bound that invariant
with comments on both the hero gate and the allowlist.
- Warm the shared chart-runtime chunk during hydration.
Both prefetches honour the `DASHBOARD_SSR_PREFETCH=false` escape and fail
soft to the client-fetch path on any error; a ~400 ms soft budget bounds
the cold-day digest cost. No contract change, no visual change.
Two data-integrity gaps found in review of the v1.30.7 contract:
- The bucket externalId guard accepted a non-canonical timestamp that V8
silently rolls over (T24:00, 04-31, non-leap 02-29). Such a row would key
under a literal string the canonical client never emits, so the two would
never collapse on the overwrite key, and it would plot on the wrong local
day. Require canonical round-trip (toISOString() === suffix) in both the
guard and the bucket-start parser.
- The per-bucket min/max bypassed the plausibility-range guard that value
passes; now that they drive the DAY min/max band and the intraday envelope,
an out-of-range or mis-ordered spread poisoned the aggregate. Persist the
spread only when both extremes are in range and ordered around value;
otherwise drop to null and keep the average.
Refine the go-forward aggregated HR wire contract (iOS #34) from the never-
adopted hourly bucket to a 10-minute UTC bucket, so one contract serves both
the measurements-list de-explosion and the intraday-pulse 10-min curve.
- Batch route accepts stats:HKQuantityTypeIdentifierHeartRate:<10-min ISO-Z>
with avg + min/max, overwrite semantics, malformed-suffix rejection; the
validator is identifier-generic (allowlist) so respiratoryRate/SpO2 can join
later without a grammar change.
- The intraday reader classifies a day's PULSE rows three ways by externalId
shape (raw / uploaded 10-min bucket / server-folded hourly), builds the
10-min curve directly from uploaded buckets with the min/max envelope and
tension detection enabled, and overlays buckets over raw on the one
cutover-straddling day.
- DAY rollup min/max read COALESCE(value_min, value) so a bucket-native day's
band reflects the true intra-bucket extremes.
No schema change (value_min/value_max exist since v1.19.2). Reconciliation is
go-forward cutover; the nightly retention fold handles the raw backlog.
The add form always defaulted to BLOOD_PRESSURE regardless of what a
user actually logs day to day, forcing a re-pick from a 15-entry
select on every entry unless arriving via a ?add= deep link. Seed the
default from the last manually-saved type (localStorage, client-only)
when no deep link overrides it, and record the type on every
successful save.
Adds prev/next-day paging to the intraday pulse "shape of the day" chart
(TileHeader right slot, next disabled at today) so a user can step back
through prior days on the existing date-scoped route.
Raises DENSE_INTRADAY_RETENTION_DAYS from 14 to 90 — a self-hoster's own
storage is the trade-off for three months of full 10-minute-resolution
history instead of two weeks. Days outside the window keep folding to
hourly-mean stats: rows as before; loadIntradayPulse now falls back to
that hourly tier when a day's raw samples are gone, so the navigator
shows a coarser shape instead of an empty chart. The tension-window
read is suppressed on hourly days — it needs per-sample resolution to
stay honest. A new resolution field on the DTO ("tenMin" | "hourly")
drives a small note in the chart.
The Apple Health ZIP import upserted each spot reading on the
(userId, type, source, externalId) index only. When two readings shared the
natural key (userId, type, measuredAt, source, sleepStage) with different
externalIds — a re-import of overlapping data, or two same-instant Apple
samples from different source apps — the externalId upsert missed and the
create collided on the second unique index (migration 0055), raising P2002
and aborting the entire import (issue #486, after the v1.28.36 dedup fix let
the upload run).
flushSpotBatch now wraps the per-row upsert: on P2002 it probes the occupying
row by the natural key (tombstone-inclusive) and adopts it in place — re-keys
to the incoming externalId, takes the new value/unit, resurrects if
soft-deleted — mirroring the v1.28.39 provider natural-key rescues (closest to
withings/sync-sleep). If no twin resolves it, the one reading is skipped and
the run continues; the catch sits in the per-row loop outside any transaction,
so a collision can never abort the batch or the import. Adopted rows count as
updated, skipped ones surface honestly. Workout + stats: paths verified unable
to hit this (no second unique index / deterministic externalId). Repro +
real-Postgres integration tests added.
Four verified HIGH data-integrity fixes, each in-place per-provider (no
shared helper), from the campaign's data-integrity audit + adversarial
verification:
F1 — drain-per-sample-cumulative buildScanWhere was the only one of four
sibling drains missing deletedAt: null; the nightly cumulative fold
re-summed soft-deleted Apple samples and hard-deleted the legacy-steps
audit tombstones. Added the filter (mirrors consolidate-daily-mean,
dense-intraday-retention, consolidate-legacy-steps).
F2 — the additive late-fold merge had no single-flight guard, so two
overlapping runs (deploy overlap, or the admin endpoint racing the cron)
produced a permanent 2x daily total. Added a pg_advisory_xact_lock per
(user, externalId) plus an in-transaction re-read of the live
contributors so the serialised-second fold contributes zero (a bare lock
alone would still re-add its stale scanned sum). Mirrors the documents
quota advisory-lock pattern.
F3 — Withings sleep externalId keyed on segment START while measuredAt +
the natural key use END; a re-scored segment missed the externalId probe,
the sweep tombstoned the old row, and the re-keyed insert collided on the
natural key and wedged. Added the natural-key rescue (re-key in place),
mirroring the same-file measure path.
F4 — Withings measure sync stamped lastSyncedAt + recordSyncSuccess
unconditionally after a warn-only per-row loop; a transient write failure
stranded the reading behind the 10-min overlap forever. Now holds the
watermark and records failure on a hard row error (mirrors
google-health/fitbit), while a benign P2002 still advances.
Regression test per finding (reverting the three source files fails
exactly four tests). Integration suite green against real Postgres.
Supplement-style intake written to Apple Health by nutrition apps —
vitamins, minerals, water, caffeine — can now sync as daily totals into a
dedicated NutrientIntakeDay store (migration 0241): composite (user, day,
nutrient) identity, re-post replaces, no soft delete, no rollup coupling.
A code-side catalog carries the 26 nutrient codes with canonical units,
plausibility caps, and EFSA dietary reference values (direction-aware;
chromium cites NIH ODS where EFSA declined a DRV).
Deliberately not a nutrition feature: no diary, no calories, no dashboard
surface. POST /api/nutrients/batch is gated behind the new opt-in
nutrients module (default off — with the module off the server refuses
ingest entirely), rate-limited, idempotency-wrapped, with per-entry
unit-mismatch and out-of-range skips. GET /api/nutrients serves a
catalog-ordered summary; the only visible surface is a read-only card
under Settings → Sources. Coach/correlation access is a later slice; the
iOS reading side is coordinated in the companion-app tracker.
Long-term retention for the dense intra-day tier (PULSE, HRV, SpO2) folds
to per-local-hour means instead of a single daily mean: the last 14 days
keep every raw sample, older days keep up to 24 points per metric — the
day's shape survives permanently at ~9k rows/metric/year. Hourly rows key
as stats:<HK>:<YYYY-MM-DD>T<HH> anchored at local HH:30, adopt-in-place
against both unique indexes with the P2002 retry outside the transaction.
The pre-fold DAY-rollup recompute extends to HRV (mean fidelity, not only
min/max) and the post-fold recompute is gone for the whole tier. Derived
resting rows for proxy users still mint from the day's raw samples.
A one-shot rebuild reconstructs the hourly shape for every already-folded
day from the soft-deleted raw samples and retires the daily row in the
same transaction — at no instant are both grains live, so no
AVG-over-live-rows reader can double-count. Boot discovery pairs live
daily-grain rows with surviving tombstones (parameter-bound SQL, pinned
against real Postgres) and converges to zero once converted; days whose
tombstones were pruned keep their daily mean. PULSE/SpO2 DAY buckets keep
the true pre-fold stats; HRV DAY buckets recompute from the hourly rows
(strictly better than the collapsed fold-time bucket); WEEK/MONTH/YEAR
respans enqueue whole-bucket recomputes.
The daily-series reader only fell back to live SQL on a coverage MISS
(zero rollup rows); a transient rollup-table error (statement_timeout,
deadlock, connection reset) was unwrapped and 500'd the tile on the
single-type route even though live SQL could serve it — and diverged from
the batched series route, which already caught. Wrap the primary daily
read and the long-range tiered read: treat a throw as a coverage miss and
fall through to the live date_trunc aggregate.
Wire the two new screening totals through the read surfaces on the
PHQ-9 / GAD-7 precedent: signal-registry entries (higher-better
direction, off the Coach snapshot and MCP by construction, no FHIR
coding attached — neither total has a verified LOINC, and the export
skips uncoded signals rather than shipping an invented code), the
scores category cluster, list labels/icons/colours, the generic
insight-chart tokens, a null PR direction (screeners are never
gamified), and the module-gated mental-health section of the doctor
report. The vitals-table exclusion rationale is documented alongside
the existing screener entries.
Two new self-report instruments join the mental-wellbeing engine on the
existing PHQ-9 / GAD-7 infrastructure:
- WHO-5 Well-Being Index: 5 items on the 6-point 5..0 scale over the last
two weeks; the reported total is raw-sum x 4 (0-100) per the official
WHO scoring (WHO/UCN/MSD/MHE/2024.1, CC BY-NC-SA 3.0 IGO). A total of
50 or below follows the WHO comment (below 50 = indication for further
assessment).
- Sleep Condition Indicator: 8 DSM-5-aligned items, 0-4 each with
item-specific anchors, total 0-32, higher = better sleep; a total of
16 or below maps to the probable-insomnia range of the source
publication (Espie et al., BMJ Open 2014;4:e004183, CC BY-NC,
(c) Sleepio Limited).
The instrument registry generalises rather than forking: per-item answer
ceilings, option order/label schemes, recall-stem captions, a reported-
score multiplier, score direction (higher-is-better inverts the follow-up
threshold), per-instrument attribution lines, and the validated-item
locale set are now instrument properties. Scoring stays server-
authoritative; the create route validates the per-instrument answer
ceiling and projects the total onto a COMPUTED WHO5_SCORE / SCI_SCORE
measurement row (never client-attributable), which also auto-satisfies a
planned screening reminder. Migration 0226 adds the enum values
(additive, idempotent). OpenAPI regenerated.
Neither instrument carries a self-harm item; the PHQ-9 item-9 crisis
routing is unchanged.
The week/month/year rollup read anchored on now and always returned the
trailing window, so a historic from/to range (API and iOS callers) came
back with buckets outside the requested span. Thread the resolved from/to
bounds through so the read is bounded on both ends, matching the day-tier
and live-SQL fallback contract; tier selection is unchanged.
HealthKit emits HKQuantityTypeIdentifierWaistCircumference as a length
quantity in metres, but the mapping table had no entry for it, so a batch
upload of that identifier was silently dropped (mapAppleHealthEntry returned
null). Add the bidirectional mapping with metre-to-cm scaling so the round
trip lands on the canonical WAIST_CIRCUMFERENCE measurement (cm), matching
getUnitForType and the WHO display bands. Extend the mapping test.
Add the opt-in PHQ-9 / GAD-7 questionnaire surface at /insights/mental-wellbeing:
instrument picker, 0–3 item scale, server-scored result with the validated
severity band, the verbatim Pfizer/PHQ attribution, the screening-not-diagnosis
disclaimer, and a calm crisis-resource card on a positive item-9. The surface
deliberately does NOT mount the shared Coach-launch card — mental-health item
content stays out of the AI.
- i18n: full mentalHealth block plus the six new measurement-type labels and the
longevity-marker catalogue names/descriptions across all six locales
(de/en/es/fr/it/pl). Instrument item wording uses the validated German text and
the canonical English elsewhere (re-wording invalidates the score).
- OpenAPI: regenerate the contract for /api/mental-health/assessments (response
excludes raw item answers by construction).
- Exclude the new manual / instrument types from the HealthKit-counterpart guard.
Add the clinical-signals wave's new measurement types and wire them through
the signal registry so FHIR, correlation eligibility, ingestion, and the
measurement-list surfaces derive from one definition each:
- MeasurementType: PHQ9_SCORE, GAD7_SCORE, GRIP_STRENGTH, PAIN_NRS,
WAIST_CIRCUMFERENCE, WAIST_TO_HEIGHT (+ AssessmentInstrument enum and the
MentalHealthAssessment model that backs the screener scores).
- Registry: the six measurement/score signals plus the longevity lab panel as
kind:"biomarker" entries; the FHIR golden fixture is regenerated in lockstep.
Grip-strength and waist-to-height carry a null LOINC (NEEDS-VERIFY) and emit a
local text concept until a stable code is confirmed.
- FHIR: add the survey observation category for the screener / pain signals.
- Labs: extend the biomarker catalogue with ApoB, Lp(a), fasting insulin, and
the omega-3 index; add LOINC codes for those plus GGT and hs-CRP; add a
computed HOMA-IR helper.
- Exhaustive maps updated to keep the enum coverage, category, list-meta, and
personal-record drift guards green.
The two cumulative drains soft-deleted per-sample rows and wrote a daily
total but never recomputed the affected DAY rollup bucket. The rollup tier
reads only `deleted_at IS NULL`, so the stored `sumValue` kept the
pre-drain double-counted total indefinitely — dashboard and every rolled-up
week/month/year step or energy figure stayed wrong and never self-healed,
because the backfill only re-flags days with no DAY row and the
daily-series read-swap only re-folds on a day-count change.
Call `recomputeBucketsForMeasurement` after each per-day write, mirroring
the daily-mean drain that already does this. Added per-drain (not hoisted)
so the mean drain, which recomputes inside its own writer, is not
double-recomputed.
Date-only daily scalars from Oura and Polar were stored at UTC midnight
(`${day}T00:00:00.000Z`). On read these rows re-bucket through
`userDayKey(measuredAt, userTz)`, so for any west-of-UTC user the day
shifted back one: a June 1 SpO2 or readiness row filed under May 31 and
fell out of alignment with the HR/sleep rows that carry true instants.
Extend the existing `canonicalDailyTimestamp` helper to accept an optional
timezone and anchor a date-only day at local noon (noon UTC when no zone
is supplied) — a full twelve hours inside the calendar day, so it
round-trips through `userDayKey` to the same day for every zone within
plus/minus twelve hours. Route every Oura day-keyed mapper (via the shared
`dayAnchor` fallback) and all five Polar daily mappers through it; real
wake/measurement instants stay untouched. Fitbit already anchored at noon
and is now consistent; Whoop and the Withings cursor anchors were verified
not to be measurement anchors.
Cap and page the unbounded reads that let a multi-year tenant pull its
whole measurement history into memory on hot paths.
- extractFeatures: read newest-first with a 6000-row cap (mirrors the
Coach snapshot) and reverse to ascending, so a 365-400 day window no
longer materialises hundreds of thousands of PULSE / glucose rows per
Coach turn and nightly briefing. Full-history extremes still come from
readAllTimeExtremes, so the cap never relabels an extreme.
- Add a (userId, measuredAt) index on measurements (migration 0210) for
the type-less range scans that extractFeatures and the consolidation
passes drive; the existing (userId, type, measuredAt) cannot serve them
as a prefix.
- runConsolidation: keyset-paginate the per-(user, type) source scan on
(measuredAt, id) instead of one unbounded findMany, removing the giant
single result-set allocation that drove the boot storm. Output is
identical; covered by a paging unit test.
- Stagger the boot-time backfill enqueues with increasing startAfter
offsets so the heavy full-history loads no longer all drain in parallel
onto one tenant at startup. Non-boot callers keep their immediate send.
- Backups: replace load-all-then-stringify with paged narrow-select reads
for the weekly and off-host writers, bounding transient heap. Off-host
still includes soft-deleted rows. Covered by a serialize/restore
round-trip test.
- Drop the per-request auth.bearer.success audit insert; the wide event
already records auth_method and user_id, so the polling client no longer
drives a write + pool checkout on every request. Failure-path audit
stays.
- Coach conversation detail: cap the message include to the newest 200 so
a long thread no longer decrypts every message on each open; response
shape unchanged.
- Admin audit-log: floor the scan at the retention horizon so the listing
and its count stop sweeping open-ended over the substring filters.
Add per-bucket HR min/max (migration 0188) threaded through batch ingest
and the series reader; make telegram_chat_id unique (0189) with a
defensive pre-clean; make migration 0187 rerun-safe. Capture a numeric
measurement from a Telegram reminder reply (source=TELEGRAM, strict chat
binding, self-cleaning); blood pressure stays satisfy-only pending a
guided two-number prompt.
Two related changes to the Telegram bot's data path.
One chat ⇄ one account. `User.telegramChatId` is now UNIQUE (migration
0189, defensive pre-clean of any pre-existing duplicate binding). The
inbound webhook resolves the user from the chat id, so a chat shared
across two accounts would route a reply or button tap ambiguously. The
index is over a nullable column, so an unlinked account (null chat id)
never collides — only two accounts binding the same non-null chat id are
rejected.
Numeric measurement capture. A numeric reply to a measurement reminder is
now captured as a Measurement (`source = TELEGRAM`, added to the enum in
0189). On "Erledigt" for a reminder that names a single-value metric, the
bot offers an optional value prompt via force_reply; a numeric reply is
written through the shared `logTelegramMeasurement` helper — same range
guard, canonical unit, cache + rollup refresh as the manual route. The
reply binds to the reminder and user through a TelegramPromptContext keyed
on the prompt message id (the same strict chat/message binding the
mood-note path uses); the userId is never read from the payload. Comma
decimals are tolerated; non-numeric and out-of-range replies get a calm
hint and write nothing; the reply self-cleans on the ~30-min sweep. Blood
pressure (two values) is out of scope for the single-numeric path and
keeps its satisfy-only flow.
The v1.19.0 hourly heart-rate bucket
(`stats:HKQuantityTypeIdentifierHeartRate:<UTC-hour>`) shipped avg-only:
one PULSE row carrying the hour's mean bpm. Add the hour's min/max so a
client can render the intra-hour range without re-uploading raw samples.
- Two nullable columns on `measurements` (`value_min` / `value_max`),
migration 0188. Set only on a well-formed hourly HR bucket row; null for
per-sample readings, per-day cumulative totals, and manual entries.
- Batch ingest threads the spread through the same unit conversion as the
average and persists / overwrites it alongside `value` on the bucket
row, pinning it null everywhere else.
- The series reader returns the spread for `kind=pulse` (null on a
per-sample PULSE row); the OpenAPI Zod contract documents both fields on
the batch entry and the series point. The avg path is unchanged.
A multi-year "All" range read its series through the DAY-only daily
reader, which caps the result at BUCKET_CAP.daily (365) buckets. The
older history was silently dropped — the range collapsed to roughly the
most recent year instead of the whole span.
Add readTieredRollupSeries, which reads the bucket tier matching the
charts own display downsampler (WEEK for 1-2 years, MONTH beyond) and
returns whole-history coverage at that tier. readDailySeries now routes
any window wider than the DAY cap through it, falling finer on a coverage
miss and logging the miss rather than truncating silently. Short and
normal ranges never enter the branch, so the common path is unchanged.
Oura's daily_resilience collection reports a categorical level — limited,
adequate, solid, strong, exceptional. Encode it ordinally (limited=1 …
exceptional=5) into the numeric measurement value so it fits the existing
model with no new categorical column: add the RESILIENCE measurement type
(migration 0186), fetch and map daily_resilience in the Oura sync, and skip
an unknown or missing level rather than coercing it to zero.
RESILIENCE_LEVELS in the Oura client is the single source of truth for the
encoding. The day-keyed externalId keeps a re-sync idempotent so a re-scored
day overwrites in place. Extend the measurement-type label, unit, range,
category, chart-token, and PR-direction maps; add the metric label across all
six locales.
Apple Watch Blood-Oxygen samples OXYGEN_SATURATION through the day and
overnight. The readings map to latest aggregation and sit in neither the
cumulative SUM drain nor the daily-mean drain, so every sample piled up raw
forever. Fold out-of-window per-sample SpO2 rows to one daily-mean stats:
row, soft-delete the raw rows, and preserve the daily min/max on the rollup
DAY bucket via a pre-fold recompute — the lowest sample is the overnight
desaturation nadir, the clinically meaningful figure the mean would hide.
Generalise the PULSE min/max preservation to a min/max-preserving set so
the pre-fold recompute and the skipped post-fold recompute cover both PULSE
and SpO2; resting-HR derivation stays PULSE-only. RESPIRATORY_RATE is left
to the existing daily-mean drain it already belongs to.
Define the go-forward aggregated heart-rate upload shape so clients can
stop posting one Measurement per raw HealthKit sample. A heart-rate row
keyed `stats:HKQuantityTypeIdentifierHeartRate:<bucket-start-hour>`
carries the hour's average bpm as a single PULSE row and rides the
existing `stats:` overwrite path: a re-post within the hour overwrites
the value instead of duplicating, identical to the per-day cumulative
rows. The hour boundary is UTC so the key is timezone-stable.
The batch route rejects a row that targets the heart-rate bucket prefix
with a malformed hour suffix as a per-entry skip
(`malformed_hr_bucket_id`) rather than persisting an un-overwriteable
row; well-formed buckets, per-sample uuid ingest, and the per-day
cumulative overwrite path are all unchanged.
Average-only for now — the measurement row carries a single value
column, so hourly min/max would need a schema change and is left as a
follow-up. Documents all three externalId shapes plus the `updated`
status in the OpenAPI contract.
The dense intra-day retention drain folds out-of-window per-sample PULSE rows
to one daily-mean row to bound the high-frequency ingest. The mean-only fold
erased the two clinically-meaningful daily figures derived from the raw
stream: the resting estimate (20th-percentile of each day's PULSE, the
fallback for users without a native resting series) and the daily min/max held
in the persistent rollup DAY bucket.
Preserve both in the PULSE facet of the fold:
- Recompute the PULSE DAY rollup from the raw rows before the fold, while they
are still live, so the bucket keeps the true intraday min/max/mean; skip the
post-fold recompute that would collapse them.
- For users with no native resting series, mint one derived RESTING_HEART_RATE
row per folded day from the day's 20th percentile, sourced COMPUTED so it
never collides with measured resting rows and reads back as the clean
resting series. Users with a native series are skipped to avoid
double-counting; the probe is memoised per user.
Add a per-day failure boundary so one poisoned day can no longer abort the
whole walk. Surface the derived-resting count through the handler return,
the wide-event annotation, and the worker log.
Capture Withings sleep vitals (HR/respiratory/SpO2/sleep-HRV/score) via the sleep
summary; collapse overlapping BP sources on the cold path for web/iOS parity; add
a STRESS_SCORE source-priority ladder; add a soft-delete-aware measurements index
and cap the all-time cold scan.
STRESS_SCORE has no source-priority ladder, so metricKeyForType returns null
and the SQL source collapse falls to an alphabetical rank-90 tiebreak. It is
safe today because the only producer is the COMPUTED HRV-derived proxy, but
the instant a device-native producer (Oura daily_stress) is enabled both rows
survive and blend nondeterministically.
Add a stress metric key with an OURA-above-COMPUTED default ladder, wire it
through metricKeyForType and RANKED_TYPES, and surface it in the Sources
settings (all six locales). A guard test asserts every RANKED_TYPES member
resolves to a non-null metric key so a future ranked type can't ship unladdered.
Regenerate the OpenAPI for the additive source-priority key.
The cumulative pass lacked the per-day error boundary and adopt-in-place
canonical-noon resolution its sibling drains carry, so the first day whose
local-noon mint collided with the (measured_at, source, sleep_stage) unique
index threw and aborted the entire global walk — leaving years of raw
step/distance rows uncollapsed.
Resolve the canonical daily row by externalId then by the canonical-noon
slot and adopt it in place, retry once on a concurrent-writer P2002, and
route per-day failures through onBucketError so one bad day is stepped over
while every other day still collapses.
One-shot prettier pass so the new format:check gate starts from a clean
baseline. Formatting only — no behaviour change. Generator-owned and
vendored artifacts stay untouched via .prettierignore.
Compliance streaks, the measurement-consolidation day key, and the
correlations weekday helper each carried their own Intl formatter
(en-CA / sv-SE / short-weekday). Delegate them to userDayKey and
wallClockInTz so every day-bucket key stays byte-for-byte aligned and the
locale decision lives in one place.
Add GET /api/measurements/series-batch returning every requested
MeasurementType's daily series in one response, so the dashboard chart
row stops issuing one self-fetch per chart. Extract the rollup-daily read
(per-source collapse, coverage probe + inline fold, live-SQL fallback)
into a shared readDailySeries reader so the single-type route and the
batch route read through the same path with no SQL duplication; the
single route now delegates to it. Cap the batch's per-type reads with
p-limit to protect the shared pool. Sleep stays on its per-night /series
adapter and is not batched.
HealthChart gains an optional preloadedSeries prop: when the dashboard
hands it a covered slice for the daily window it folds those rows instead
of self-fetching, keyed so a range-tab change re-fetches cleanly. Non-
dashboard mounts are unchanged (empty discriminator keeps the cache tuple
byte-identical). Register the endpoint in the OpenAPI table and regen the
spec; add unit + parity coverage for the batch route.
Two additive measurement_type values for the Oura coverage completion:
- SLEEP_SCORE — the headline 0-100 sleep-quality score (unit score),
kept apart from the WHOOP sleep sub-scores so two vendors' headline
numbers never share a bucket.
- BODY_TEMPERATURE_DEVIATION — a signed nightly body-temperature offset
from the personal baseline (celsius). It is a deviation, not an
absolute reading, so it cannot share the body / skin / wrist
temperature buckets.
Migration 0165 extends the enum (additive, no backfill). Register both in
the validation enum, units, plausibility ranges, category overlay, list-UI
label/icon/colour maps, chart-token allowlist, and the PR-direction switch
(both resolve to no record direction). Both are descriptive signals, not
clinical vitals, so they stay out of the doctor-report PDF. Localise the
two list labels across all six locales and regenerate the OpenAPI contract.
Polar Nightly Recharge and Training Load Pro carry two native signals with
no existing home: the HRV-based autonomic-nervous-system charge (distinct
from the 1-6 recovery band that maps to RECOVERY_SCORE) and the device-native
cardio-load strain figure (distinct from WHOOP DAY_STRAIN and the COMPUTED
STRAIN_SCORE). Add both to the canonical enum (migration 0166) and wire them
through the validation enum, canonical units, plausibility ranges, category
cluster, chart-token allowlist, PR-direction resolver, and the measurement-
list label/icon/colour maps, with labels across all six locales. Both are
device-derived composites, so they read no PR direction and stay out of the
clinical vitals PDF, matching the WHOOP-native score classes.
Regenerate the OpenAPI contract for the two new enum members.
Ladder reconciliation ranks every enum source, so the unranked
fall-through needs out-of-enum fixtures now; a stored single-source
ladder resolving one canonical row is pinned as the new contract. The
body-sync test sheds the import its retired weight path used.
Server and first client render now agree everywhere query state used to
leak into the chrome, checkboxes no longer nest inside buttons, delete
confirmations admit the undo they offer, raw step counts drop their
decimals, the truncated unit label wraps instead, streaks know their
singular, the medication header speaks the user's date format, a failed
coach turn keeps the user's message and says what went wrong, settings
status notes follow a language switch, the locale only flips once its
bundle actually loaded, and the last raw api fetches in hooks move onto
the shared client.
A typed apiFetch wrapper owns the envelope unwrap, the ok-check and the
error read for every client call — two hundred call sites across the
app migrate onto it and a lint rule keeps raw api fetches out for good.
The legacy timezone helpers fold into the tz module and test fixtures
drop personal names.
The nightly mean-consolidation upsert targets the externalId arbiter, but
the measurement table carries a second full unique on (user, type,
measured_at, source, sleep_stage) that also spans tombstones. Any row
parked exactly on the canonical local-noon instant made the insert throw
P2002, and because the global pass had no per-bucket fence the first
poisoned day aborted consolidation for every remaining user and type.
writeDay now probes the canonical slot inside the transaction and
resolves the collision deterministically: rows the pass owns (tombstones,
per-sample rows about to fold) shift to the next free second, foreign
live rows keep their instant and the mean row yields instead. The runner
gains an optional per-bucket error fence so one bad day is logged,
counted as daysFailed, and skipped.
- When the sensitive-category encryption preference is on (the default),
encrypt the intent-revealing day-log fields (sexual activity, protection,
pregnancy/progesterone tests, contraception) into an encrypted envelope and
leave the queryable columns null, decrypting on read. The toggle now does
what it says rather than storing those fields in plaintext.
- Make the day-log write a partial merge so a re-log only overwrites the
fields it carries, never nulling untouched ones (no more data loss when the
same day is logged again).
- Make the upsert collision-safe: a manual entry and an imported entry on the
same date adopt one canonical row instead of throwing, and the single POST
returns a clean conflict rather than a 500; one bad date no longer aborts a
whole import.
- Compare the decrypted note, not the ciphertext, so an unchanged re-log is a
true no-op and stops needless re-sync.
- Hoist cycle attribution out of the per-entry loop; map bulk failures to
stable reason codes; audit the import goal change; mask the event name in
discreet reminder pushes; add a one-tap hard-delete purge for all cycle data.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Promote the reproductive HealthKit category types out of the
Apple-Health-import privacy-hold list and route them into cycle day-logs
instead of measurements, and surface cycle data on the three export
surfaces.
Import promotion:
- A new HealthKit -> cycle mapping module is the single source of truth for
how MenstrualFlow, IntermenstrualBleeding, CervicalMucusQuality,
OvulationTestResult, SexualActivity, PregnancyTestResult,
ProgesteroneTestResult, Contraceptive and the symptom category types fold
into CycleDayLog fields / seeded symptom keys. The XML importer
accumulates same-day reproductive samples and writes one canonical
day-log per day with source APPLE_HEALTH and a synthetic per-day
externalId for idempotent re-import. The fold is gated on cycle tracking
being enabled, so a non-cycle export never provisions cycle rows. A
contraceptive method nudges the profile goal toward avoiding pregnancy
only when the goal is still on its default. Basal body temperature stays
a measurement and mirrors onto the day-log. Pregnancy/Lactation status
and the pregnancy-bleeding types stay deferred; the cycle awareness types
remain server-derived.
Export:
- FHIR emits LMP (LOINC 8665-2), average cycle length (64700-8), period
length (64698-4) and the current phase as opt-in reproductive-health
observations grouped into their own document section.
- The doctor PDF gains an opt-in cycle section (recent cycles, average
length and variability, current phase), privacy-default off like mood.
- The full-backup export, the weekly backup worker and the off-host DR
snapshot include the cycle tables, and the admin restore re-creates them;
the day-log note round-trips as ciphertext and is never decrypted onto a
report surface.
Mapping table documented in docs/api/cycle-healthkit-mapping.md. Adds unit
+ integration coverage for the mapping, the import routing, the FHIR
observations and the backup round-trip. OpenAPI regenerated. Six-locale
strings added.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add three MeasurementType values for WHOOP fields a coverage audit found
uncaptured: AVERAGE_HEART_RATE and MAX_HEART_RATE (the cycle score's
whole-day aggregates, kept distinct from spot PULSE / RESTING_HEART_RATE /
WALKING_HEART_RATE_AVERAGE) and SLEEP_DISTURBANCE_COUNT (the per-night
disturbance tally). Units: bpm / bpm / count.
Wiring mirrors the existing per-type surfaces, traced from VO2_MAX and the
v1.11.0 WHOOP-native score classes:
- canonical enum + unit + plausibility range in
src/lib/validations/measurement.ts (every ingest / analytics / rollup
path derives from measurementTypeEnum.options, so the mean-based rollup
tier picks the new types up automatically).
- Prisma enum + migration 0125 (ALTER TYPE ADD VALUE IF NOT EXISTS,
non-transactional, idempotent — same pattern as 0111).
- category overlay (cardiovascular for the HR pair, sleep for the count).
- list-view label key / Lucide icon / chart-colour.
- i18n label key in all six locales.
- chart-token allowlist + orphan-enum stripper.
- PersonalRecord direction (HR pair null — no clean goal axis, like
WALKING_HEART_RATE_AVERAGE; disturbance count MIN, like
BREATHING_DISTURBANCES).
- enum-coverage drift guard (count 60 -> 63, doctor-PDF exclusions for the
device-derived signals) + apple-health HK-counterpart exclusion (these
are WHOOP-only; HealthKit ships no first-class daily-aggregate HR
quantity nor a disturbance tally).
The types render on the measurement list + Insights chart surfaces; like the
v1.11.0 WHOOP-native scores they get no dedicated Insights sub-page (the
sub-page map stays curated) and no LOINC / FHIR mapping (not clinical-grade).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Marc-André Bombeck