Wire the two new screening totals through the read surfaces on the
PHQ-9 / GAD-7 precedent: signal-registry entries (higher-better
direction, off the Coach snapshot and MCP by construction, no FHIR
coding attached — neither total has a verified LOINC, and the export
skips uncoded signals rather than shipping an invented code), the
scores category cluster, list labels/icons/colours, the generic
insight-chart tokens, a null PR direction (screeners are never
gamified), and the module-gated mental-health section of the doctor
report. The vitals-table exclusion rationale is documented alongside
the existing screener entries.
Two new self-report instruments join the mental-wellbeing engine on the
existing PHQ-9 / GAD-7 infrastructure:
- WHO-5 Well-Being Index: 5 items on the 6-point 5..0 scale over the last
two weeks; the reported total is raw-sum x 4 (0-100) per the official
WHO scoring (WHO/UCN/MSD/MHE/2024.1, CC BY-NC-SA 3.0 IGO). A total of
50 or below follows the WHO comment (below 50 = indication for further
assessment).
- Sleep Condition Indicator: 8 DSM-5-aligned items, 0-4 each with
item-specific anchors, total 0-32, higher = better sleep; a total of
16 or below maps to the probable-insomnia range of the source
publication (Espie et al., BMJ Open 2014;4:e004183, CC BY-NC,
(c) Sleepio Limited).
The instrument registry generalises rather than forking: per-item answer
ceilings, option order/label schemes, recall-stem captions, a reported-
score multiplier, score direction (higher-is-better inverts the follow-up
threshold), per-instrument attribution lines, and the validated-item
locale set are now instrument properties. Scoring stays server-
authoritative; the create route validates the per-instrument answer
ceiling and projects the total onto a COMPUTED WHO5_SCORE / SCI_SCORE
measurement row (never client-attributable), which also auto-satisfies a
planned screening reminder. Migration 0226 adds the enum values
(additive, idempotent). OpenAPI regenerated.
Neither instrument carries a self-harm item; the PHQ-9 item-9 crisis
routing is unchanged.
The week/month/year rollup read anchored on now and always returned the
trailing window, so a historic from/to range (API and iOS callers) came
back with buckets outside the requested span. Thread the resolved from/to
bounds through so the read is bounded on both ends, matching the day-tier
and live-SQL fallback contract; tier selection is unchanged.
HealthKit emits HKQuantityTypeIdentifierWaistCircumference as a length
quantity in metres, but the mapping table had no entry for it, so a batch
upload of that identifier was silently dropped (mapAppleHealthEntry returned
null). Add the bidirectional mapping with metre-to-cm scaling so the round
trip lands on the canonical WAIST_CIRCUMFERENCE measurement (cm), matching
getUnitForType and the WHO display bands. Extend the mapping test.
Add the opt-in PHQ-9 / GAD-7 questionnaire surface at /insights/mental-wellbeing:
instrument picker, 0–3 item scale, server-scored result with the validated
severity band, the verbatim Pfizer/PHQ attribution, the screening-not-diagnosis
disclaimer, and a calm crisis-resource card on a positive item-9. The surface
deliberately does NOT mount the shared Coach-launch card — mental-health item
content stays out of the AI.
- i18n: full mentalHealth block plus the six new measurement-type labels and the
longevity-marker catalogue names/descriptions across all six locales
(de/en/es/fr/it/pl). Instrument item wording uses the validated German text and
the canonical English elsewhere (re-wording invalidates the score).
- OpenAPI: regenerate the contract for /api/mental-health/assessments (response
excludes raw item answers by construction).
- Exclude the new manual / instrument types from the HealthKit-counterpart guard.
Add the clinical-signals wave's new measurement types and wire them through
the signal registry so FHIR, correlation eligibility, ingestion, and the
measurement-list surfaces derive from one definition each:
- MeasurementType: PHQ9_SCORE, GAD7_SCORE, GRIP_STRENGTH, PAIN_NRS,
WAIST_CIRCUMFERENCE, WAIST_TO_HEIGHT (+ AssessmentInstrument enum and the
MentalHealthAssessment model that backs the screener scores).
- Registry: the six measurement/score signals plus the longevity lab panel as
kind:"biomarker" entries; the FHIR golden fixture is regenerated in lockstep.
Grip-strength and waist-to-height carry a null LOINC (NEEDS-VERIFY) and emit a
local text concept until a stable code is confirmed.
- FHIR: add the survey observation category for the screener / pain signals.
- Labs: extend the biomarker catalogue with ApoB, Lp(a), fasting insulin, and
the omega-3 index; add LOINC codes for those plus GGT and hs-CRP; add a
computed HOMA-IR helper.
- Exhaustive maps updated to keep the enum coverage, category, list-meta, and
personal-record drift guards green.
The two cumulative drains soft-deleted per-sample rows and wrote a daily
total but never recomputed the affected DAY rollup bucket. The rollup tier
reads only `deleted_at IS NULL`, so the stored `sumValue` kept the
pre-drain double-counted total indefinitely — dashboard and every rolled-up
week/month/year step or energy figure stayed wrong and never self-healed,
because the backfill only re-flags days with no DAY row and the
daily-series read-swap only re-folds on a day-count change.
Call `recomputeBucketsForMeasurement` after each per-day write, mirroring
the daily-mean drain that already does this. Added per-drain (not hoisted)
so the mean drain, which recomputes inside its own writer, is not
double-recomputed.
Date-only daily scalars from Oura and Polar were stored at UTC midnight
(`${day}T00:00:00.000Z`). On read these rows re-bucket through
`userDayKey(measuredAt, userTz)`, so for any west-of-UTC user the day
shifted back one: a June 1 SpO2 or readiness row filed under May 31 and
fell out of alignment with the HR/sleep rows that carry true instants.
Extend the existing `canonicalDailyTimestamp` helper to accept an optional
timezone and anchor a date-only day at local noon (noon UTC when no zone
is supplied) — a full twelve hours inside the calendar day, so it
round-trips through `userDayKey` to the same day for every zone within
plus/minus twelve hours. Route every Oura day-keyed mapper (via the shared
`dayAnchor` fallback) and all five Polar daily mappers through it; real
wake/measurement instants stay untouched. Fitbit already anchored at noon
and is now consistent; Whoop and the Withings cursor anchors were verified
not to be measurement anchors.
Cap and page the unbounded reads that let a multi-year tenant pull its
whole measurement history into memory on hot paths.
- extractFeatures: read newest-first with a 6000-row cap (mirrors the
Coach snapshot) and reverse to ascending, so a 365-400 day window no
longer materialises hundreds of thousands of PULSE / glucose rows per
Coach turn and nightly briefing. Full-history extremes still come from
readAllTimeExtremes, so the cap never relabels an extreme.
- Add a (userId, measuredAt) index on measurements (migration 0210) for
the type-less range scans that extractFeatures and the consolidation
passes drive; the existing (userId, type, measuredAt) cannot serve them
as a prefix.
- runConsolidation: keyset-paginate the per-(user, type) source scan on
(measuredAt, id) instead of one unbounded findMany, removing the giant
single result-set allocation that drove the boot storm. Output is
identical; covered by a paging unit test.
- Stagger the boot-time backfill enqueues with increasing startAfter
offsets so the heavy full-history loads no longer all drain in parallel
onto one tenant at startup. Non-boot callers keep their immediate send.
- Backups: replace load-all-then-stringify with paged narrow-select reads
for the weekly and off-host writers, bounding transient heap. Off-host
still includes soft-deleted rows. Covered by a serialize/restore
round-trip test.
- Drop the per-request auth.bearer.success audit insert; the wide event
already records auth_method and user_id, so the polling client no longer
drives a write + pool checkout on every request. Failure-path audit
stays.
- Coach conversation detail: cap the message include to the newest 200 so
a long thread no longer decrypts every message on each open; response
shape unchanged.
- Admin audit-log: floor the scan at the retention horizon so the listing
and its count stop sweeping open-ended over the substring filters.
Add per-bucket HR min/max (migration 0188) threaded through batch ingest
and the series reader; make telegram_chat_id unique (0189) with a
defensive pre-clean; make migration 0187 rerun-safe. Capture a numeric
measurement from a Telegram reminder reply (source=TELEGRAM, strict chat
binding, self-cleaning); blood pressure stays satisfy-only pending a
guided two-number prompt.
Two related changes to the Telegram bot's data path.
One chat ⇄ one account. `User.telegramChatId` is now UNIQUE (migration
0189, defensive pre-clean of any pre-existing duplicate binding). The
inbound webhook resolves the user from the chat id, so a chat shared
across two accounts would route a reply or button tap ambiguously. The
index is over a nullable column, so an unlinked account (null chat id)
never collides — only two accounts binding the same non-null chat id are
rejected.
Numeric measurement capture. A numeric reply to a measurement reminder is
now captured as a Measurement (`source = TELEGRAM`, added to the enum in
0189). On "Erledigt" for a reminder that names a single-value metric, the
bot offers an optional value prompt via force_reply; a numeric reply is
written through the shared `logTelegramMeasurement` helper — same range
guard, canonical unit, cache + rollup refresh as the manual route. The
reply binds to the reminder and user through a TelegramPromptContext keyed
on the prompt message id (the same strict chat/message binding the
mood-note path uses); the userId is never read from the payload. Comma
decimals are tolerated; non-numeric and out-of-range replies get a calm
hint and write nothing; the reply self-cleans on the ~30-min sweep. Blood
pressure (two values) is out of scope for the single-numeric path and
keeps its satisfy-only flow.
The v1.19.0 hourly heart-rate bucket
(`stats:HKQuantityTypeIdentifierHeartRate:<UTC-hour>`) shipped avg-only:
one PULSE row carrying the hour's mean bpm. Add the hour's min/max so a
client can render the intra-hour range without re-uploading raw samples.
- Two nullable columns on `measurements` (`value_min` / `value_max`),
migration 0188. Set only on a well-formed hourly HR bucket row; null for
per-sample readings, per-day cumulative totals, and manual entries.
- Batch ingest threads the spread through the same unit conversion as the
average and persists / overwrites it alongside `value` on the bucket
row, pinning it null everywhere else.
- The series reader returns the spread for `kind=pulse` (null on a
per-sample PULSE row); the OpenAPI Zod contract documents both fields on
the batch entry and the series point. The avg path is unchanged.
A multi-year "All" range read its series through the DAY-only daily
reader, which caps the result at BUCKET_CAP.daily (365) buckets. The
older history was silently dropped — the range collapsed to roughly the
most recent year instead of the whole span.
Add readTieredRollupSeries, which reads the bucket tier matching the
charts own display downsampler (WEEK for 1-2 years, MONTH beyond) and
returns whole-history coverage at that tier. readDailySeries now routes
any window wider than the DAY cap through it, falling finer on a coverage
miss and logging the miss rather than truncating silently. Short and
normal ranges never enter the branch, so the common path is unchanged.
Oura's daily_resilience collection reports a categorical level — limited,
adequate, solid, strong, exceptional. Encode it ordinally (limited=1 …
exceptional=5) into the numeric measurement value so it fits the existing
model with no new categorical column: add the RESILIENCE measurement type
(migration 0186), fetch and map daily_resilience in the Oura sync, and skip
an unknown or missing level rather than coercing it to zero.
RESILIENCE_LEVELS in the Oura client is the single source of truth for the
encoding. The day-keyed externalId keeps a re-sync idempotent so a re-scored
day overwrites in place. Extend the measurement-type label, unit, range,
category, chart-token, and PR-direction maps; add the metric label across all
six locales.
Apple Watch Blood-Oxygen samples OXYGEN_SATURATION through the day and
overnight. The readings map to latest aggregation and sit in neither the
cumulative SUM drain nor the daily-mean drain, so every sample piled up raw
forever. Fold out-of-window per-sample SpO2 rows to one daily-mean stats:
row, soft-delete the raw rows, and preserve the daily min/max on the rollup
DAY bucket via a pre-fold recompute — the lowest sample is the overnight
desaturation nadir, the clinically meaningful figure the mean would hide.
Generalise the PULSE min/max preservation to a min/max-preserving set so
the pre-fold recompute and the skipped post-fold recompute cover both PULSE
and SpO2; resting-HR derivation stays PULSE-only. RESPIRATORY_RATE is left
to the existing daily-mean drain it already belongs to.
Define the go-forward aggregated heart-rate upload shape so clients can
stop posting one Measurement per raw HealthKit sample. A heart-rate row
keyed `stats:HKQuantityTypeIdentifierHeartRate:<bucket-start-hour>`
carries the hour's average bpm as a single PULSE row and rides the
existing `stats:` overwrite path: a re-post within the hour overwrites
the value instead of duplicating, identical to the per-day cumulative
rows. The hour boundary is UTC so the key is timezone-stable.
The batch route rejects a row that targets the heart-rate bucket prefix
with a malformed hour suffix as a per-entry skip
(`malformed_hr_bucket_id`) rather than persisting an un-overwriteable
row; well-formed buckets, per-sample uuid ingest, and the per-day
cumulative overwrite path are all unchanged.
Average-only for now — the measurement row carries a single value
column, so hourly min/max would need a schema change and is left as a
follow-up. Documents all three externalId shapes plus the `updated`
status in the OpenAPI contract.
The dense intra-day retention drain folds out-of-window per-sample PULSE rows
to one daily-mean row to bound the high-frequency ingest. The mean-only fold
erased the two clinically-meaningful daily figures derived from the raw
stream: the resting estimate (20th-percentile of each day's PULSE, the
fallback for users without a native resting series) and the daily min/max held
in the persistent rollup DAY bucket.
Preserve both in the PULSE facet of the fold:
- Recompute the PULSE DAY rollup from the raw rows before the fold, while they
are still live, so the bucket keeps the true intraday min/max/mean; skip the
post-fold recompute that would collapse them.
- For users with no native resting series, mint one derived RESTING_HEART_RATE
row per folded day from the day's 20th percentile, sourced COMPUTED so it
never collides with measured resting rows and reads back as the clean
resting series. Users with a native series are skipped to avoid
double-counting; the probe is memoised per user.
Add a per-day failure boundary so one poisoned day can no longer abort the
whole walk. Surface the derived-resting count through the handler return,
the wide-event annotation, and the worker log.
Capture Withings sleep vitals (HR/respiratory/SpO2/sleep-HRV/score) via the sleep
summary; collapse overlapping BP sources on the cold path for web/iOS parity; add
a STRESS_SCORE source-priority ladder; add a soft-delete-aware measurements index
and cap the all-time cold scan.
STRESS_SCORE has no source-priority ladder, so metricKeyForType returns null
and the SQL source collapse falls to an alphabetical rank-90 tiebreak. It is
safe today because the only producer is the COMPUTED HRV-derived proxy, but
the instant a device-native producer (Oura daily_stress) is enabled both rows
survive and blend nondeterministically.
Add a stress metric key with an OURA-above-COMPUTED default ladder, wire it
through metricKeyForType and RANKED_TYPES, and surface it in the Sources
settings (all six locales). A guard test asserts every RANKED_TYPES member
resolves to a non-null metric key so a future ranked type can't ship unladdered.
Regenerate the OpenAPI for the additive source-priority key.
The cumulative pass lacked the per-day error boundary and adopt-in-place
canonical-noon resolution its sibling drains carry, so the first day whose
local-noon mint collided with the (measured_at, source, sleep_stage) unique
index threw and aborted the entire global walk — leaving years of raw
step/distance rows uncollapsed.
Resolve the canonical daily row by externalId then by the canonical-noon
slot and adopt it in place, retry once on a concurrent-writer P2002, and
route per-day failures through onBucketError so one bad day is stepped over
while every other day still collapses.
One-shot prettier pass so the new format:check gate starts from a clean
baseline. Formatting only — no behaviour change. Generator-owned and
vendored artifacts stay untouched via .prettierignore.
Compliance streaks, the measurement-consolidation day key, and the
correlations weekday helper each carried their own Intl formatter
(en-CA / sv-SE / short-weekday). Delegate them to userDayKey and
wallClockInTz so every day-bucket key stays byte-for-byte aligned and the
locale decision lives in one place.
Add GET /api/measurements/series-batch returning every requested
MeasurementType's daily series in one response, so the dashboard chart
row stops issuing one self-fetch per chart. Extract the rollup-daily read
(per-source collapse, coverage probe + inline fold, live-SQL fallback)
into a shared readDailySeries reader so the single-type route and the
batch route read through the same path with no SQL duplication; the
single route now delegates to it. Cap the batch's per-type reads with
p-limit to protect the shared pool. Sleep stays on its per-night /series
adapter and is not batched.
HealthChart gains an optional preloadedSeries prop: when the dashboard
hands it a covered slice for the daily window it folds those rows instead
of self-fetching, keyed so a range-tab change re-fetches cleanly. Non-
dashboard mounts are unchanged (empty discriminator keeps the cache tuple
byte-identical). Register the endpoint in the OpenAPI table and regen the
spec; add unit + parity coverage for the batch route.
Two additive measurement_type values for the Oura coverage completion:
- SLEEP_SCORE — the headline 0-100 sleep-quality score (unit score),
kept apart from the WHOOP sleep sub-scores so two vendors' headline
numbers never share a bucket.
- BODY_TEMPERATURE_DEVIATION — a signed nightly body-temperature offset
from the personal baseline (celsius). It is a deviation, not an
absolute reading, so it cannot share the body / skin / wrist
temperature buckets.
Migration 0165 extends the enum (additive, no backfill). Register both in
the validation enum, units, plausibility ranges, category overlay, list-UI
label/icon/colour maps, chart-token allowlist, and the PR-direction switch
(both resolve to no record direction). Both are descriptive signals, not
clinical vitals, so they stay out of the doctor-report PDF. Localise the
two list labels across all six locales and regenerate the OpenAPI contract.
Polar Nightly Recharge and Training Load Pro carry two native signals with
no existing home: the HRV-based autonomic-nervous-system charge (distinct
from the 1-6 recovery band that maps to RECOVERY_SCORE) and the device-native
cardio-load strain figure (distinct from WHOOP DAY_STRAIN and the COMPUTED
STRAIN_SCORE). Add both to the canonical enum (migration 0166) and wire them
through the validation enum, canonical units, plausibility ranges, category
cluster, chart-token allowlist, PR-direction resolver, and the measurement-
list label/icon/colour maps, with labels across all six locales. Both are
device-derived composites, so they read no PR direction and stay out of the
clinical vitals PDF, matching the WHOOP-native score classes.
Regenerate the OpenAPI contract for the two new enum members.
Ladder reconciliation ranks every enum source, so the unranked
fall-through needs out-of-enum fixtures now; a stored single-source
ladder resolving one canonical row is pinned as the new contract. The
body-sync test sheds the import its retired weight path used.
Server and first client render now agree everywhere query state used to
leak into the chrome, checkboxes no longer nest inside buttons, delete
confirmations admit the undo they offer, raw step counts drop their
decimals, the truncated unit label wraps instead, streaks know their
singular, the medication header speaks the user's date format, a failed
coach turn keeps the user's message and says what went wrong, settings
status notes follow a language switch, the locale only flips once its
bundle actually loaded, and the last raw api fetches in hooks move onto
the shared client.
A typed apiFetch wrapper owns the envelope unwrap, the ok-check and the
error read for every client call — two hundred call sites across the
app migrate onto it and a lint rule keeps raw api fetches out for good.
The legacy timezone helpers fold into the tz module and test fixtures
drop personal names.
The nightly mean-consolidation upsert targets the externalId arbiter, but
the measurement table carries a second full unique on (user, type,
measured_at, source, sleep_stage) that also spans tombstones. Any row
parked exactly on the canonical local-noon instant made the insert throw
P2002, and because the global pass had no per-bucket fence the first
poisoned day aborted consolidation for every remaining user and type.
writeDay now probes the canonical slot inside the transaction and
resolves the collision deterministically: rows the pass owns (tombstones,
per-sample rows about to fold) shift to the next free second, foreign
live rows keep their instant and the mean row yields instead. The runner
gains an optional per-bucket error fence so one bad day is logged,
counted as daysFailed, and skipped.
- When the sensitive-category encryption preference is on (the default),
encrypt the intent-revealing day-log fields (sexual activity, protection,
pregnancy/progesterone tests, contraception) into an encrypted envelope and
leave the queryable columns null, decrypting on read. The toggle now does
what it says rather than storing those fields in plaintext.
- Make the day-log write a partial merge so a re-log only overwrites the
fields it carries, never nulling untouched ones (no more data loss when the
same day is logged again).
- Make the upsert collision-safe: a manual entry and an imported entry on the
same date adopt one canonical row instead of throwing, and the single POST
returns a clean conflict rather than a 500; one bad date no longer aborts a
whole import.
- Compare the decrypted note, not the ciphertext, so an unchanged re-log is a
true no-op and stops needless re-sync.
- Hoist cycle attribution out of the per-entry loop; map bulk failures to
stable reason codes; audit the import goal change; mask the event name in
discreet reminder pushes; add a one-tap hard-delete purge for all cycle data.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Promote the reproductive HealthKit category types out of the
Apple-Health-import privacy-hold list and route them into cycle day-logs
instead of measurements, and surface cycle data on the three export
surfaces.
Import promotion:
- A new HealthKit -> cycle mapping module is the single source of truth for
how MenstrualFlow, IntermenstrualBleeding, CervicalMucusQuality,
OvulationTestResult, SexualActivity, PregnancyTestResult,
ProgesteroneTestResult, Contraceptive and the symptom category types fold
into CycleDayLog fields / seeded symptom keys. The XML importer
accumulates same-day reproductive samples and writes one canonical
day-log per day with source APPLE_HEALTH and a synthetic per-day
externalId for idempotent re-import. The fold is gated on cycle tracking
being enabled, so a non-cycle export never provisions cycle rows. A
contraceptive method nudges the profile goal toward avoiding pregnancy
only when the goal is still on its default. Basal body temperature stays
a measurement and mirrors onto the day-log. Pregnancy/Lactation status
and the pregnancy-bleeding types stay deferred; the cycle awareness types
remain server-derived.
Export:
- FHIR emits LMP (LOINC 8665-2), average cycle length (64700-8), period
length (64698-4) and the current phase as opt-in reproductive-health
observations grouped into their own document section.
- The doctor PDF gains an opt-in cycle section (recent cycles, average
length and variability, current phase), privacy-default off like mood.
- The full-backup export, the weekly backup worker and the off-host DR
snapshot include the cycle tables, and the admin restore re-creates them;
the day-log note round-trips as ciphertext and is never decrypted onto a
report surface.
Mapping table documented in docs/api/cycle-healthkit-mapping.md. Adds unit
+ integration coverage for the mapping, the import routing, the FHIR
observations and the backup round-trip. OpenAPI regenerated. Six-locale
strings added.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add three MeasurementType values for WHOOP fields a coverage audit found
uncaptured: AVERAGE_HEART_RATE and MAX_HEART_RATE (the cycle score's
whole-day aggregates, kept distinct from spot PULSE / RESTING_HEART_RATE /
WALKING_HEART_RATE_AVERAGE) and SLEEP_DISTURBANCE_COUNT (the per-night
disturbance tally). Units: bpm / bpm / count.
Wiring mirrors the existing per-type surfaces, traced from VO2_MAX and the
v1.11.0 WHOOP-native score classes:
- canonical enum + unit + plausibility range in
src/lib/validations/measurement.ts (every ingest / analytics / rollup
path derives from measurementTypeEnum.options, so the mean-based rollup
tier picks the new types up automatically).
- Prisma enum + migration 0125 (ALTER TYPE ADD VALUE IF NOT EXISTS,
non-transactional, idempotent — same pattern as 0111).
- category overlay (cardiovascular for the HR pair, sleep for the count).
- list-view label key / Lucide icon / chart-colour.
- i18n label key in all six locales.
- chart-token allowlist + orphan-enum stripper.
- PersonalRecord direction (HR pair null — no clean goal axis, like
WALKING_HEART_RATE_AVERAGE; disturbance count MIN, like
BREATHING_DISTURBANCES).
- enum-coverage drift guard (count 60 -> 63, doctor-PDF exclusions for the
device-derived signals) + apple-health HK-counterpart exclusion (these
are WHOOP-only; HealthKit ships no first-class daily-aggregate HR
quantity nor a disturbance tally).
The types render on the measurement list + Insights chart surfaces; like the
v1.11.0 WHOOP-native scores they get no dedicated Insights sub-page (the
sub-page map stays curated) and no LOINC / FHIR mapping (not clinical-grade).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The per-sample cumulative drain strips already-collapsed stats: rows
from the day bucket, so a trailing watch sync that lands after a day's
total was minted leaves the bucket holding only the late samples. The
upsert update branch then replaced the full day total with the partial
late sum, and the contributing samples were already hard-deleted, making
the lost count unrecoverable.
Probe the existing total inside the write transaction and fold the late
sum into it (existing + late) instead of overwriting. The late samples
are genuine additional readings, unlike the legacy-step pass where the
existing total is HealthKit's own aggregate and folding would double-
count. Per-sample rows are still hard-deleted so a re-run cannot
re-accumulate them.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A measurement logged by hand on the iOS app while offline is mirrored
into HealthKit. On pairing a server, adopt-on-pair uploads it as a
MANUAL row while HealthKit background sync independently re-ingests the
same sample as an APPLE_HEALTH row. The two carry different external ids
and sources, so the composite unique index keeps both — doubling the
whole hand-logged history on first pair.
Collapse the pair at ingest: an incoming MANUAL or APPLE_HEALTH row is
dropped as a duplicate when a same-reading row of the opposite source
already exists (same type and value, measuredAt within two seconds).
The match is scoped to that one cross-source pair, so same-source rows
and rows from server-owned sources keep their existing contracts, and
two genuinely distinct readings at the same minute never collapse.
stats:* cumulative rows stay on their per-day overwrite lane.
The collapsed copy returns status "duplicate" with reason
"cross_source_merge" so the client cursor advances unchanged; no
schema, migration, or OpenAPI change.
The Prisma unique-constraint (P2002) narrowing predicate had been
hand-copied into five call sites. Fold them into a single isP2002 in
src/lib/prisma-errors.ts and import it everywhere (the two step-drain
sites keep their local isUniqueConstraintViolation name via an import
alias). Behaviour-identical; one place to chase a future P2002-semantics
change instead of five.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
collapseRollupRowsBySource resolves the per-source rollup rows minted by
the source-aware writer down to one row per bucket, using the user's
source-priority ladder (the same ordering the raw-row picker uses). A
metricKeyForType superset maps every overlapping non-cumulative vital
(RHR, HRV-SDNN, SpO2, body/skin temperature, respiratory rate, weight,
body fat, blood pressure, pulse, VO2max, recovery) plus the cumulative
keys onto a SourcePriorityMetricKey. Unlisted sources fall back to the
highest-count row so a bucket never doubles or goes dark.
Wired into the rollup readers in the following commits.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Prove a WHOOP run and the same Apple-Health run inside the 5-minute clustering
window collapse to the APPLE_HEALTH canonical row (richer GPS + HR record),
and a lone WHOOP run survives when no richer source logged the session. The
measurement-side WHOOP/Apple/Withings oracle already lives in
source-priority-whoop-applehealth.test.ts; this closes the workout axis.
Add the WHOOP integration data layer (migration 0111): seven new
MeasurementType values (HRV_RMSSD, DAY_STRAIN, WORKOUT_STRAIN,
SLEEP_PERFORMANCE, SLEEP_EFFICIENCY, SLEEP_CONSISTENCY, SLEEP_NEED,
ENERGY_EXPENDITURE_KJ), a WHOOP MeasurementSource, the WhoopConnection
and WhoopOAuthState models, and the per-user encrypted BYO-key columns
on User. Day-strain takes DAY_STRAIN (not the COMPUTED STRAIN_SCORE) and
RMSSD takes HRV_RMSSD (not the SDNN HEART_RATE_VARIABILITY) so a native
value never shares a bucket with a derived proxy.
Carry the new types through the completeness walls (units, value ranges,
chart tokens, PR direction, categories, list-meta colours/icons/labels,
HealthKit-mapping exclusions) and feed the cross-source picker: recovery
ladder (WHOOP native above the COMPUTED proxy), WHOOP inserted into the
overlapping vital ladders and the workout ladder, plus skinTemperature
and respiratoryRate metric keys.
Resolve the canonical daily row deterministically: prefer the row already
carrying the target stats: externalId before falling back to the row on the
canonical local-noon instant, with a stable orderBy tiebreaker. Adopting a
sibling and stamping the target externalId onto it could otherwise collide on
the (userId, type, source, externalId) index with the real canonical row.
Re-anchor the adopted row to local-noon only when that slot is free, so an
externalId row at a tz/DST-shifted instant cannot collide on the measured_at
composite during adopt.
Guard the create against a concurrent insert on the same canonical composite:
catch P2002 outside the transaction (a caught error inside leaves the tx in
the aborted state) and retry the fold once, which then resolves and adopts the
winning row instead of creating a duplicate.
Add a real-Postgres case covering a row already on the canonical instant under
the target stats: externalId plus a fresh out-of-window batch: the fold adopts
in place, no P2002, one canonical row with the correctly folded mean.
The dense intra-day retention fold minted its daily-mean stats: row with
an externalId-keyed upsert. On real data a row already occupies the
canonical local-noon instant — a sibling daily stats: row or a per-sample
HRV/PULSE reading that fell on noon — so the upsert lookup missed (the
existing row carries a different externalId) and the INSERT then violated
the (user_id, type, measured_at, source, sleep_stage) unique index with
P2002. The mocked unit tests could not surface it.
Resolve the canonical row by the measured_at composite the collision is on
(sleep_stage IS NULL for these continuous types, matched via the
NULLS NOT DISTINCT index) and adopt it in place — refresh the mean, stamp
the canonical stats: externalId, un-tombstone — or create it when absent.
Exclude the adopted row id from the per-sample soft-delete so a noon-instant
sample becomes the daily row instead of being tombstoned.
Add a testcontainer integration test that seeds the colliding shape (a
pre-existing canonical-instant row plus out-of-window per-sample rows) and
pins no-P2002, coexistence, the noon-sample case, idempotency, and the
retention bound. Update the unit coverage for the adopt-in-place path.
The reminder worker failed to start in the production bundle with
`ReferenceError: Cannot access 'l' before initialization`, taking every
nightly pg-boss job down with it (score jobs + dense-intraday retention).
Root cause is a value-level import cycle between `consolidation-base.ts`
and `drain-per-sample-cumulative.ts`: each imported values from the other.
The bundler merges the tightly-cyclic pair into one module body and hoists
`DRAIN_CUMULATIVE_CUTOFF_HOURS = CONSOLIDATION_GRACE_CUTOFF_HOURS` ahead of
the constant it reads, so the eager read hits a temporal dead zone. It only
surfaced in the production build because dev/test run the modules unbundled.
Hoist the shared leaf — `CONSOLIDATION_GRACE_CUTOFF_HOURS`, the `PerSampleRow`
shape, and the timezone day-math helpers — into a new dependency-free module
`consolidation-tz.ts`. Both sides now import from the leaf, removing the cycle
at its root. Re-export the moved names from their former homes so every
existing import site resolves unchanged.
Daytime heart-rate-variability (SDNN) and heart-rate samples need to keep
their intra-day shape long enough for the stress engine to read them, but
the raw per-sample volume must stay bounded. Add a retention tier that
keeps the last 14 days of raw HRV/HR samples and folds everything older to
one daily-mean row via the proven mean-consolidation drain (same idempotent
stats:<HK>:<day> fold, soft-delete, and rollup recompute).
The two dense-tier types are deliberately EXEMPT from the destructive
daily-mean drain allowlist (HIGH_FREQUENCY_MEAN_TYPES) — that drain would
collapse the intra-day shape and soft-delete the raw rows. A regression test
pins both exclusions so a future edit that adds either type to the drain
allowlist fails loudly.
Migration 0108 adds only the partial index that keeps the retention drain's
discovery and per-user scan cheap; the retention bound lives in code so
tuning it never needs a migration.
Add a server-owned COMPUTED measurement source for rows a server engine
derives rather than a device or a person reports, plus three first-class
0-100 score types — RECOVERY_SCORE, STRESS_SCORE, STRAIN_SCORE (unit
"score", plausibility range 0..100).
COMPUTED is read-only from a client's perspective: it joins the read /
decode enum so the native client can decode the rows it queries and
charts, but every client-facing write surface rejects it exactly like
WITHINGS and IMPORT — the batch ingest allowlist stays {APPLE_HEALTH,
MANUAL} and the single-entry POST 422s a COMPUTED source.
All three score types thread through every type-exhaustiveness site: the
unit + plausibility-range maps, the PR-direction resolver (scores carry
no record direction, like the device events), the category overlay (a
new "scores" cluster), the list-view label / icon / colour maps with
locale strings across all six bundles, and the insight chart-token
allowlist + orphan-enum stripper. They are not clinical vitals, so they
stay out of the doctor-report PDF.
Only the recovery score is computed today; the stress and strain types
are defined now so the engines that compute them later need no schema
change. Migration 0106 is purely additive (ADD VALUE IF NOT EXISTS).
Add mapping tests for the five event classes (verdict resolution,
severity grading, the value-pinned-to-1 contract, unknown-codepoint
fallback, and that a continuous measurement never gets a classification),
the awareness-card gating + device-verdict framing + disclaimer presence,
and the events route narrowing + empty-state contract. Extend the enum,
chart-token and Coach-gate drift guards for the new event types.
Add a discrete EVENT row class to the Measurement model for the
device-flagged health notifications a wearable produces on its own —
irregular-rhythm, high/low heart rate, walking-steadiness and
breathing-disturbance alerts.
Following the sleep-stage / glucose-context nullable-context precedent,
this is an additive nullable column on the existing table, not a new
storage paradigm. Five new MeasurementType values carry the event class
and a new rhythm_classification enum column carries the device's own
verdict / severity (migration 0105, with a CHECK that pins the column to
the event types). An event row stores value 1 and the device verdict;
it never participates in trend, rollup, chart or personal-record
analytics.
For the irregular-rhythm class, HealthLog ingests only the
classification RESULT the device's own certified on-device algorithm
already produced. It never ingests a raw ECG waveform, never
re-classifies, and never derives a verdict of its own.
The Apple Health batch ingest maps the five category identifiers, with
the device's HKCategoryValue codepoint resolved to the stored
classification; an unknown codepoint degrades to a fallback verdict
rather than dropping the event.
Wire seven previously-deferred HealthKit quantity identifiers through
the full pipeline — enum, mapping, plausibility range, category, list-UI
metadata, FHIR coding, and the generic metric-status assessment:
CARDIO_RECOVERY heartRateRecoveryOneMinute bpm
WRIST_TEMPERATURE appleSleepingWristTemperature celsius
FALL_COUNT numberOfTimesFallen count
SIX_MINUTE_WALK_DISTANCE sixMinuteWalkTestDistance m
STAIR_ASCENT_SPEED stairAscentSpeed m/s
STAIR_DESCENT_SPEED stairDescentSpeed m/s
BREATHING_DISTURBANCES appleSleepingBreathingDisturbances count
Each value rides raw on the wire (identity conversion). Fall count is
cumulative (daily SUM); the stair speeds drain to a daily mean; the rest
store as latest per-workout / per-night spot readings. Wrist temperature
and breathing disturbances are flagged privacy-sensitive. The category
sleep-apnea and walking-steadiness EVENT identifiers stay deferred — they
are events, not quantities.
Each new metric gains an Insights sub-page on the shared scaffold with the
generic assessment card, localised across all six locales. Cardio fitness
(VO2 max) — ingested since v1.4.23 but assessment-less — gains its own
/insights/cardio-fitness page so it picks up the assessment instead of
riding as a bare chart-row on the pulse page.
Normal-range anchors are cited where defensible (six-minute walk 400-700 m,
ATS guideline + population reference); the rest defer to the user's own
baseline rather than fabricate a band.
Migration 0104 appends the enum values additively (IF NOT EXISTS, forward-only).
Extend the daily-mean consolidation to cover the high-frequency
gait/mobility metrics that previously mapped to no consolidation set and
piled up at sampling granularity: walking asymmetry, walking
double-support, walking steadiness, and walking heart-rate average. The
correct daily reduction for these is the mean, matching the Health-app
display; all four stay disjoint from the cumulative (sum) set.
Fold the daily-mean drain onto the existing nightly cumulative-drain tick
so new high-frequency spot samples no longer accumulate raw between worker
reboots. The two passes share one concurrency-1 cron, keeping the
maintenance window a single sequential walk. Boot-time discovery stays as
the back-fill for accounts that accumulated raw rows before this shipped.
Recompute the affected day's rollup bucket after the consolidation
commits, the same way the ingest path does, so a rollup-covered read
reflects the single consolidated row rather than a stale mean derived from
the now soft-deleted per-sample rows.
Regular heart-rate / pulse stays raw: it is a discrete clinical reading
and feeds correlation and scatter analytics. No schema change.
