The insights read path served a cold cache by generating inline — up to
ten seconds on first visit — and the nightly warm pass structurally
excluded evening visitors: their visit stamped the cache, the half-four
run skipped it as fresh, and it expired exactly at the next evening's
visit. Stale content now serves immediately with its timestamp while
one background job revalidates, the warm threshold drops to an hour,
forced warming fills both message locales, the analytics caches serve
stale-while-revalidate, the profile endpoint parallelises its reads,
the bug-report status drops a write per call, the umami proxy degrades
to a no-op script instead of an error, the insights route sheds its
heaviest static imports, and the medication page prefetches on intent.
Open follow-up questions now run as a guided in-chat sequence — a quiet
card offers to start, the coach asks one question at a time with
visible progress, answers can be adopted into the matching self-context
field, skipping keeps a question pending and a closing summary says
what was taken over. Proactive nudges learn three new deterministic
signals — weight drifting from its target band, a run of short nights,
an active user falling silent — address the focus the user asked to
watch, and gain per-group toggles and a frequency preference.
A typed apiFetch wrapper owns the envelope unwrap, the ok-check and the
error read for every client call — two hundred call sites across the
app migrate onto it and a lint rule keeps raw api fetches out for good.
The legacy timezone helpers fold into the tz module and test fixtures
drop personal names.
The reminder worker becomes a registry over twelve queue modules cut
along the existing test structure, and the dashboard page sheds its
quick-entry sheets, header and range logic into components — same
behaviour, pinned by the existing suites.
The pregenerate pass now warms the status cards even when the
comprehensive step is budget-blocked, failed or timed out — the gap
that left cards generating on first visit despite the nightly runs —
and the loop gains a per-user timeout so one hanging provider cannot
stall the tick. Schedule edits recompute their rollup days immediately,
the compliance cache keys on the timezone, open doses no longer count
against the nudge trigger, an invite sent against an open registration
still lands in the ledger, the self-report prompt fence uses explicit
data markers, the maintenance crons invalidate the caches they touch,
and a flaky audit assertion polls instead of racing. Six care-focused
achievements join the engine: miss-free streaks, measurement routine,
a complete self-context and a sleep-log run.
Invites move into their own admin section with a real ledger: status
badges, expiry, uses, a redemption history of who joined when (new
redemption table, revocation is now soft so history survives), and an
honest one-time link/QR moment. The assistant self-context grows
structured fields for conditions, allergies and focus — encrypted like
the free text — merges age and gender from the profile into one fenced
prompt block, and asks up to three targeted follow-up questions that
surface as tappable chips above the coach composer. A fourth nudge
trigger gently suggests a check-up when the self-context is stale and
the coach is actually in use.
Users can now tell the assistant about themselves in an encrypted
free-text profile that feeds the coach and briefing prompts as clearly
fenced context, with the coach allowed one targeted follow-up question
when the picture is incomplete. Operators on closed registrations can
mint expiring invite links with a QR code from the admin area; a valid
invite opens registration exactly as many times as it allows and records
who redeemed it. A nightly pass watches three deterministic signals —
low adherence, blood pressure above its target band, a falling recovery
trend — and sends at most one coach nudge per week through the existing
notification channels, gated behind the same opt-outs as every other
assistant surface.
The two-o'clock status crons generated for every account, ignoring both
the per-user coach opt-out and the operator kill switch; they now share
one gated discovery that also skips users the pregenerate pass will
rewarm anyway, and the mood card finally gets the nightly run the other
six always had. Deleted measurements no longer feed any prompt. Locales
without an assistant language fall back to English consistently instead
of German texts with English fallbacks. A rate-limited regenerate stops
claiming success, the dashboard briefing state distinguishes a missing
provider from a pending generation and serves a stale briefing with its
timestamp instead of hiding it, narrative warming actually processes
oldest first, and worker failures reach the error tracker through one
shared reporter while the insight queues gain real retries.
The auto-miss cron stamped every pending row missed a flat day after
its slot even though weekly schedules stay takeable for days, touched
tombstones because it never filtered deleted rows, and a verdict stuck
even after the dose was taken. The cron now derives its cutoff from the
schedule's window reach, skips deleted rows and bumps the sync counter,
and a take clears the auto-miss flag. Editing a schedule tombstones the
open pending rows of the old anchors and reconciles a stale window onto
the new times. The bulk write path attributes takes through the same
band engine as the single-intake routes and accepts forceSlotInstant
per entry. Configured dose windows now reach every compliance call site
through one shared schedule select, and the card window pill takes a
timezone, treats a degenerate window as a point and counts dose times
instead of schedule rows.
A take whose slot the legacy resolver rejected fell back to a standalone
insert that kept the client's scheduledFor, parking a second live row on
an anchor the reminder projector had already minted — the partial unique
index permits that because it spans source. Both intake write routes now
probe the slot source-agnostically and converge onto the existing row
before inserting, and a standalone insert anchors on the take time per
the ad-hoc contract. The compliance rollup counts distinct slots instead
of rows, which also corrects historical duplicates on the next recompute,
and the slot dedup pass runs nightly instead of only at worker boot.
The nightly mean-consolidation upsert targets the externalId arbiter, but
the measurement table carries a second full unique on (user, type,
measured_at, source, sleep_stage) that also spans tombstones. Any row
parked exactly on the canonical local-noon instant made the insert throw
P2002, and because the global pass had no per-bucket fence the first
poisoned day aborted consolidation for every remaining user and type.
writeDay now probes the canonical slot inside the transaction and
resolves the collision deterministically: rows the pass owns (tombstones,
per-sample rows about to fold) shift to the next free second, foreign
live rows keep their instant and the mean row yields instead. The runner
gains an optional per-bucket error fence so one bad day is logged,
counted as daysFailed, and skipped.
A never-acted dose past its 24h cutoff was flipped to skipped=true by the
hourly cron. The compliance engine excludes skipped doses from the
denominator (a deliberate user pause), so a forgotten dose then counted
neither as taken nor missed — silently inflating adherence.
Add a dedicated auto_missed terminal state. The cron now sets auto_missed
(not skipped) on a never-acted pending row, so the engine counts it as a
real miss against the rate, while a user-initiated skip stays excluded.
The two are mutually exclusive on a live row by construction.
Add a CYCLE_FERTILE_SOON reminder that fires two days before the
predicted fertile-window start. Twice-gated and default-OFF: it only
surfaces when CycleProfile.goal is TRYING_TO_CONCEIVE and fertileWindowStart
is populated, so fertile language never reaches the avoid / general / peri
goals. Reuses the existing cycle-reminder cron, discreet mode, clientManaged
suppression, idempotency ledger, and push-attempts plumbing verbatim; the
body carries the non-medical, not-a-contraceptive framing.
Push the cycle-reminder cron's gating into the query instead of loading
every CyclePrediction row each tick: filter on the user's
cycleTrackingEnabled / predictionEnabled relation and restrict to rows
whose nextPeriodStart falls inside a tz-generous [today-grace, today+lead]
string window. The per-tick cost now scales with the in-window cohort, not
the whole table; the exact per-user-timezone window still runs in JS.
Make the period-boundary shortcut transactional and idempotent: wrap the
close-prior + open-new (or stamp-current) cycle mutations in a single
$transaction so two concurrent taps cannot double-close a prior cycle or
compute a length against a stale read, and add withIdempotency so an exact
key replay is a no-op. The boundary day-log now ranks the incoming flow
against any existing same-day entry and never downgrades a richer flow.
Decode sync cursors additively: a live beta v1 token keeps its
measurement / mood / intake watermarks and simply fresh-scans the two
domains added in v2 (cycleDays, cycles) rather than forcing a full
re-init. Correct the stale "no live consumer" comment.
Thread the user's timezone through the Coach cycle snapshot so "today",
day-of-cycle and days-until-period derive from the user's zone — matching
the calendar and insights routes — instead of wall-clock Berlin.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- When the sensitive-category encryption preference is on (the default),
encrypt the intent-revealing day-log fields (sexual activity, protection,
pregnancy/progesterone tests, contraception) into an encrypted envelope and
leave the queryable columns null, decrypting on read. The toggle now does
what it says rather than storing those fields in plaintext.
- Make the day-log write a partial merge so a re-log only overwrites the
fields it carries, never nulling untouched ones (no more data loss when the
same day is logged again).
- Make the upsert collision-safe: a manual entry and an imported entry on the
same date adopt one canonical row instead of throwing, and the single POST
returns a clean conflict rather than a 500; one bad date no longer aborts a
whole import.
- Compare the decrypted note, not the ciphertext, so an unchanged re-log is a
true no-op and stops needless re-sync.
- Hoist cycle attribution out of the per-entry loop; map bulk failures to
stable reason codes; audit the import goal change; mask the event name in
discreet reminder pushes; add a one-tap hard-delete purge for all cycle data.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Promote the reproductive HealthKit category types out of the
Apple-Health-import privacy-hold list and route them into cycle day-logs
instead of measurements, and surface cycle data on the three export
surfaces.
Import promotion:
- A new HealthKit -> cycle mapping module is the single source of truth for
how MenstrualFlow, IntermenstrualBleeding, CervicalMucusQuality,
OvulationTestResult, SexualActivity, PregnancyTestResult,
ProgesteroneTestResult, Contraceptive and the symptom category types fold
into CycleDayLog fields / seeded symptom keys. The XML importer
accumulates same-day reproductive samples and writes one canonical
day-log per day with source APPLE_HEALTH and a synthetic per-day
externalId for idempotent re-import. The fold is gated on cycle tracking
being enabled, so a non-cycle export never provisions cycle rows. A
contraceptive method nudges the profile goal toward avoiding pregnancy
only when the goal is still on its default. Basal body temperature stays
a measurement and mirrors onto the day-log. Pregnancy/Lactation status
and the pregnancy-bleeding types stay deferred; the cycle awareness types
remain server-derived.
Export:
- FHIR emits LMP (LOINC 8665-2), average cycle length (64700-8), period
length (64698-4) and the current phase as opt-in reproductive-health
observations grouped into their own document section.
- The doctor PDF gains an opt-in cycle section (recent cycles, average
length and variability, current phase), privacy-default off like mood.
- The full-backup export, the weekly backup worker and the off-host DR
snapshot include the cycle tables, and the admin restore re-creates them;
the day-log note round-trips as ciphertext and is never decrypted onto a
report surface.
Mapping table documented in docs/api/cycle-healthkit-mapping.md. Adds unit
+ integration coverage for the mapping, the import routing, the FHIR
observations and the backup round-trip. OpenAPI regenerated. Six-locale
strings added.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add two cycle reminders through the existing notification cascade and the
honest self-hosting privacy framing for reproductive-health data.
Notifications:
- New event types CYCLE_PERIOD_SOON (fires ~2 days before the predicted
next-period start, from the cached prediction) and CYCLE_PERIOD_CONFIRM
(a gentle "did your period start?" on/after the predicted start while no
period is logged, within a short grace window). Both default OFF and are
gated again on cycleTrackingEnabled — opt in twice for a sensitive surface.
- A 15-minute pg-boss cron mirrors the mood-reminder worker: it scans users
with cycle tracking enabled + predictionEnabled, computes who is in-window
from the cached forecast, and dispatches one push per event per local day.
Idempotency anchors on the push_attempts ledger (no new table). The queue
is declared, registered in allQueues, scheduled, and wired to a boss.work
handler, with a source-grep guard so it cannot ship unregistered.
- Honours notificationPrefs.cycle.clientManaged (skip the server push when
the device owns local reminders, with a suppressed annotation, mirroring
the medication path) and CycleProfile.discreetNotifications (generic
"HealthLog reminder" body so no cycle event is named on the lock screen).
- Every attempt is recorded in push_attempts via the unchanged dispatcher.
Privacy / disclaimer:
- docs/self-hosting/reproductive-health-data.md — the "no cloud vendor to
subpoena" framing with the honest residual-threat caveat (your own host,
provider, backups, and devices remain part of the surface), citing the
2021 FTC settlement and 2025 verdict as public record only.
- docs/security/cycle-data-threat-model.md — a compact "protects against /
does not protect against" note, plus README pointers.
- CyclePredictionDisclaimer component (reusing the GLP-1 estimate-disclaimer
pattern) for every fertility/prediction surface: descriptive, not a
contraceptive method, not medical advice, never a safe-to-have-unprotected
-sex claim. 6-locale copy under cycle.disclaimer.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add an additive, non-breaking assessment field to GET /api/insights/derived:
{ text, source, updatedAt } | null, keyed to the requested metric id for the
five per-score ids (READINESS, SLEEP_SCORE, RECOVERY_SCORE, STRAIN_SCORE,
STRESS_SCORE). The field is null when the metric is not one of those or when
its status is not "ok". A deterministic text always fills it — composed from
the score's own contributors, so provider-less accounts and the demo are never
empty — and warmer provider prose overrides it once cached, warmed out of band
so the GET never blocks on a model round-trip. de/en, following the v1.12.12
deterministic-narrative model. The batch grid read returns assessment: null;
the detail sheets use the single-metric route.
Introduce a shared buildMetricSignal() helper: a token-lean, signal-rich
descriptor (current, own baseline, signed delta, spread, normal-swing verdict,
normal-range placement, contributors) computed server-side so the model states
the comparison instead of re-deriving it from raw buckets. Thread it into the
generic per-metric assessment snapshot and lead the base prompt from it. Add a
PROMPT_VERSION constant to the base system prompt so prompt changes are
traceable.
Regenerate docs/api/openapi.yaml.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When no AI provider is configured (or before the first AI summary is written),
the period-in-review card stayed empty. Compose a concise, factual, strictly
non-causal summary from the same structured digest the AI prompt is built from
— ranked metric changes vs the prior period, vitals that moved outside their
typical range, and an honest count of statistical associations — and persist it
with a `deterministic` provider marker so a later AI warm replaces it in place.
The read route now warms regardless of provider; the read-never-blocks posture
is unchanged. Fills the no-key demo's retrospective too.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two connect-in-app enhancements for the native client.
return_scheme: GET /api/whoop/connect accepts an optional custom scheme,
threaded through the server-side OAuth-state row so it survives the round
trip without ever riding in the URL or cookie. When a valid scheme is
present, the callback's final redirect targets <scheme>://whoop?whoop=
connected|error&reason=... so an in-app web session auto-completes on its
custom-scheme match; absent or invalid, the web settings redirect is
unchanged. Schemes validate against a strict allowlist (custom app scheme
only, ^[a-z][a-z0-9.+-]*$, http/https/javascript/data/file/vbscript
rejected, pinned to dev.healthlog.app). A rejected scheme falls back to
the web redirect and never reflects an arbitrary value.
Bearer connect ticket: a purely Bearer-authenticated client holds no web
session cookie, so it cannot start the handshake through the cookie-gated
connect route. POST /api/whoop/connect/ticket (Bearer) mints a one-time,
~60s, opaque ticket bound to the user; only its HMAC-SHA256 hash is stored
(the raw value never persists). GET /api/whoop/connect?ticket=<opaque>
accepts the ticket in lieu of a cookie, consumes it atomically (single-use,
reuse rejected), sets the nonce cookie, and 302s to WHOOP exactly as the
cookie path. Expired, consumed, or invalid tickets return a typed 401. The
two combine. The daily WHOOP OAuth-state cleanup also sweeps expired ticket
rows.
The ticket query param is added to the egress redaction denylist so the
opaque value never reaches logs or error reports.
Migration 0124 adds whoop_connect_tickets (id, user_id FK cascade,
token_hash unique, expires_at, consumed_at, created_at) and a nullable
return_scheme column on whoop_oauth_states.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 221b362011d52dda314139e0be7cc195c47d016e)
The native client SPKI-pins the server's TLS leaf certificate. When the
leaf auto-renews (GTS re-issues it roughly every 90 days) the leaf keypair
— and so the SPKI pin — changes, and a client shipped only the old pin
refuses to connect on the next renewal: a silent outage. Nothing
server-side signalled that a client pins the served leaf.
Add a pg-boss monitor that probes the served leaf every 6 hours, computes
its iOS SPKI pin (base64(sha256(DER subjectPublicKeyInfo)) via a raw TLS
socket + X509Certificate), and compares it against the operator's
known-good set in TLS_LEAF_SPKI_PINS (comma-separated for the dual-pin
renewal window). On a served pin outside the set it fails loud: a
tls.pin.leaf_changed wide-event annotation, a system.tls.pin_changed audit
row, and a high-priority SYSTEM_ALERT fanned out to admins through the
existing dispatcher idiom. Transient TLS/network failures annotate
tls.pin.probe_failed and never alarm.
The baseline is an env var, not a persisted last-seen row — a self-learning
baseline would silently adopt the first rotation and suppress the alarm the
pinned client needs; an unset baseline logs the served pin and warns rather
than auto-adopting. The runbook documents extraction, the re-pin + dual-pin
procedure (≥11 days before expiry, ship to TestFlight), and the assumption
that the probed host is the host the client connects to.
Wires the queue into the worker (allQueues + cron + worker), adds the
optional env to the manifest, compose whitelist, and .env example, and
covers the SPKI computation + change detection with a fixture-cert unit
test plus a queue-registration guard.
(cherry picked from commit cdb88cd77b88aec71ad2f21caf2b15c185dfc01e)
Three hardening + performance fixes to the Fitbit/Google Health sync.
Stop soft-delete resurrection. The measurement write probed the existing
row by the `(userId, type, source, externalId)` unique key without excluding
soft-deleted rows, so a reading the user deleted came back on the next hourly
sync (the upsert matched the tombstone and took the update branch — silently
undoing the delete). The write now probes only live rows (`deletedAt: null`)
and treats a tombstone as absent, minting a fresh insert instead. The fresh
insert relies on the partial unique index over `deleted_at IS NULL` to keep
live-row uniqueness; the tombstone sits outside it, and `skipDuplicates` guards
a live-row race.
Batch the backfill writes + collapse the rollup recompute. The write loop
was one upsert per reading (N+1) followed by a per-(type,day) DAY-rollup
recompute, so a multi-year backfill paid thousands of serial round-trips on a
concurrency-1 worker. The existence probe is now a single `findMany` over the
batch; fresh rows go through chunked `createMany`; only already-live rows take a
per-row update for their differing values. On a `fullSync` the per-write rollup
hook is deferred and the touched type-days collapse into one
`recomputeUserRollups(from, to)` pass at the end of the cycle. The incremental
path keeps the inline per-day hook. The `stats:` overwrite contract, the
single-watermark `markSynced`, and the all-403 soft-skip guard are unchanged.
Bound the hourly poll concurrency. The poll synced every connection in one
serial loop, so a single slow Google response stalled the whole cohort. The
cohort now fans out through a bounded `p-limit` pool with per-user error
isolation, extracted into `runFitbitPollCohort` so the contract is unit testable
without exporting worker internals. The accumulators are read after the await to
avoid a compound-assignment lost-update across overlapping pool tasks.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 8369af2471fb8da04a938b5a784433318d81a248)
Extend the Fitbit/Google Health provider with the read path: the
dataPoints.list client half, the metrics-bundle sync, the poll + backfill
jobs, and the connection-management routes. Builds on the existing OAuth +
credentials + token-refresh layer.
- client: paginated `fetchDataPoints` walker over `safeFetch` keyed on
`nextPageToken`, encoding each data type kebab-case in the path and
snake_case in the filter predicate; defensive per-type mappers (finite +
positive guards) for weight, body fat, daily SpO2, daily HRV, daily resting
heart rate, daily respiratory rate, intraday heart rate, sleep-temperature
derivations, and a height profile-seed extractor. `FITBIT_FIELD_MAP` +
`mapping.md` are the source of truth.
- mapping (no new MeasurementType): weight→WEIGHT, body-fat→BODY_FAT,
SpO2→OXYGEN_SATURATION, HRV→HEART_RATE_VARIABILITY (SDNN slot, not
HRV_RMSSD), resting-HR→RESTING_HEART_RATE, respiratory-rate→RESPIRATORY_RATE,
heart-rate→PULSE, sleep-temp→WRIST_TEMPERATURE, height→User.heightCm when
null. Steps/distance/energy/floors/VO2 + sleep + workouts stay for a later
wave.
- sync: `syncUserFitbit` orchestrator with the reauth short-circuit, the
per-data-class 403 soft-skip (the six Restricted bundles grant
independently), the all-soft-skipped success guard, and incremental-vs-full
windows; `upsertFitbitMeasurements` keyed (userId,type,FITBIT,externalId)
with a 24 h overlap for after-the-fact daily re-rolls, recomputing rollups
and invalidating status caches. Google refresh tokens do not rotate, so the
stored refresh token is kept unless a new one is returned.
- jobs: hourly `fitbit-sync` poll (poll-only; Pub/Sub deferred), the
self-converging boot backfill, and the daily OAuth-state sweep — every queue
registered in both `allQueues` and a `boss.work` binding, staggered off the
WHOOP/Withings crons.
- routes: status / sync / disconnect mirroring the WHOOP shape.
Tests cover the mappers, the casing invariant, `fetchDataPoints`, the
orchestrator soft-skip guard, the idempotent upsert, the backfill discovery,
and the queue registration; an integration test pins the upsert idempotency,
the DAY rollup fold, and the Fitbit-vs-Withings per-source no-collapse against
the Postgres testcontainer.
Land the OAuth + credentials half of the Fitbit / Google Health provider,
cloning the WHOOP provider structure and adapting it to the Google identity
flow.
- Client OAuth half (src/lib/fitbit/client.ts): Google authorize-URL builder
(access_type=offline + prompt=consent), authorization-code exchange, token
refresh, and a profile fetch for the connection's external user id. Client
credentials ride in an HTTP Basic-auth header on the token endpoint; the
four launch Restricted read scopes are pinned in FITBIT_OAUTH_SCOPE. All
egress goes through safeFetch.
- Non-rotating refresh tokens: getValidToken refreshes at expiry-minus-five-
minutes, persists the new access token + expiry, and overwrites the stored
refresh token ONLY when the response carries a fresh one — Google keeps the
refresh token stable, unlike WHOOP.
- BYO credentials route (PUT/GET/DELETE /api/fitbit/credentials): per-user
client id/secret stored encrypted on the User columns, built field-by-field
under requireAuth.
- OAuth state + connect + callback routes: a random base64url nonce backed by
a 10-minute FitbitOAuthState ledger row carries the (nonce -> userId)
mapping; the callback validates the cookie/URL state with timingSafeEqual,
consumes the row atomically (replay/expired/cross-user reason tags),
exchanges the code, upserts the encrypted FitbitConnection, clears prior
reauth state, and enqueues the self-converging history backfill.
- Response classifier mirrors WHOOP's HTTP-status buckets; Google signals a
revoked grant with a 401, so there is no invalid_grant body special-case.
Tests cover the authorize URL, Basic-auth token exchange, the no-rotation
refresh branch (token preserved when omitted, overwritten when present), the
classifier status table, and the OAuth-state nonce mint.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add the coach-memory-refresh pg-boss queue: a fire-and-forget enqueue
contract (coach-memory-shared) and a worker pipeline that runs the
conversation-summary refresh then the fact extraction for one long
conversation, each fault-isolated and budget-gated. Register the queue
in allQueues + a boss.work handler in the worker, with a source-grep
inventory guard so the queue can never ship unregistered.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a calm "your week / month in review" summary to the Insights
overview, generated from the period-narrative context (deltas, personal
bands, FDR-controlled associations, coincident days).
- New insight_narratives table (migration 0113): one durable row per
(user, period, locale); generated prose AES-256-GCM at rest following
the coach-message precedent; labels-only provenance stays plaintext.
- generatePeriodNarrative feeds the context into the provider chain with
a tight, descriptive-never-causal prompt; an insufficient context
yields no narrative, a missing provider no-ops, neither fabricates.
- Nightly warm cron on week (Mon) / month (1st) boundaries, budget-gated
per user; the read-only GET enqueues a single-user warm on a cold read.
- Read route is stale-while-revalidate and never blocks on the provider;
the card renders plain text (no markdown), a single provenance
disclosure, and a fixed footprint (no CLS). i18n in all six locales.
Build the WHOOP sync layer on the W2 client, mirroring the Withings
queues. A per-resource sync (recovery/sleep/cycle/workout) pulls since
the last cursor with a 24h overlap on recovery/sleep to absorb WHOOP's
after-the-fact re-scoring, maps via the W2 mappers, and upserts
Measurements with source=WHOOP keyed on (userId,type,source,externalId)
so a re-post overwrites in place. Workouts upsert into Workout with
per-workout strain + HR zones in metadata and kJ->kcal energy.
Token refresh persists BOTH rotated tokens (WHOOP invalidates the prior
access and refresh token on every refresh). A self-converging boot
backfill enqueues one full-history sync per un-backfilled connection and
stamps backfillCompletedAt; the discovery query drops completed accounts,
idempotent across reboots.
Every new queue is registered in the worker's allQueues + cron schedule
(recovery/sleep/workout poll catch-all, cycle poll-only, backfill,
oauth-state sweep). IntegrationStatus gains the whoop key + label.
Tests: rotating-refresh persistence, idempotent measurement upsert,
incremental-window overlap, backfill self-convergence, and a
queue-registration inventory guard.
The recovery / stress / strain nightly finders used distinct + take with no
order, so when more than the cap of users qualified the selected set was
arbitrary across runs. Switch to a grouped query ordered by each user's
newest input first with userId as the stable tiebreak, so the cap takes the
most-recently-active users reproducibly. The strain finder interleaves its
workout and active-energy lists so the merged cap favours recency across both
sources.
- workouts/batch + safeJson: the body cap is a DoS ceiling, not a bound
the worst-case payload always fits under; an extreme GPS-heavy backlog
can exceed it and falls back to smaller batches client-side. Note that
request.text() still buffers the raw body, so the cap bounds the parse
cost, not the initial read.
- .dockerignore: keep LICENSE / LICENSE.md in the image — AGPL-3.0
conveys with the software.
- reminder-worker: the dense walk flag is an operator kill-switch now
that the fold is reworked, not a pending-rework gate.
The v1.10.0.2 kill-switch defaulted the drain OFF after the P2002 fold
collision. With the fold reworked to coexist with the canonical daily row
and integration-tested against real rows, flip the default back ON
(DENSE_INTRADAY_RETENTION_ENABLED=true unless explicitly set to false). The
boot-discovery still defers past the startup storm via the boot-delay so it
never contends for the connection pool on a data-heavy tenant; the nightly
03:50 walk and the per-user handler run again.
The Recovery/Stress/Strain crons fire in the small hours (04:45-04:55
Europe/Berlin, ~03 UTC) but scored the CURRENT UTC day, only a few hours
old: Strain missed the just-completed day's workouts and Stress saw a
near-empty intra-day SDNN set and falsely gated as insufficient. Score the
day that just ended (now - 1 day) across all three engines and their
measuredAt / externalId day stamps.
Resolve the previous-day key in one place by extracting the shared score-row
helpers the three engines each duplicated: scoreDayKey / scoreMeasuredAt /
scoreExternalId, the idempotent upsertScoreRow, and the cohort runScoreBatch
tally loop into src/lib/insights/score-row.ts; and loadBaselineProfile into
the baseline engine (the three byte-identical profile loaders + the partial
strain variant now route through it). The derived route should adopt
loadBaselineProfile too.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wire the two nightly score passes and the dense intra-day retention drain
into the worker. The stress (04:50) and strain (04:55) crons each get a
queue, a schedule, and a single-flight handler; the dense-retention drain
gets a per-user backfill queue with boot discovery (like mean-consolidation)
and folds its steady-state nightly walk onto the existing drain-cumulative
tick, after the daily-mean fold.
A queue not registered in allQueues silently never drains, so a source-grep
guard pins every registration site for all three jobs.
Derive a daily cardio-load proxy from the per-workout heart-rate series
plus active energy. Each workout's HR series scores via Banister's
gender-weighted exponential TRIMP over the fractional heart-rate reserve
(Tanaka age-predicted HRmax, the user's resting HR); the day's TRIMP totals
map to a 0-100 saturating scale. Days with no usable HR series but a
non-trivial active-energy total fall back to an energy-only proxy, with the
path recorded in the result.
Stored as a COMPUTED STRAIN_SCORE row, idempotent per user per day, gated
when neither a usable HR series nor active energy is present.
Standard: Banister 1991; Morton, Fitz-Clarke & Banister 1990 (TRIMP);
Tanaka, Monahan & Seals 2001 (HRmax).
Apple Watch has no electrodermal sensor, so there is no direct stress
measurement to read — what HealthKit does give is discrete SDNN samples
through the day. Derive an honest HRV-through-day PROXY: a suppressed,
below-baseline intra-day SDNN is a recognised marker of higher sympathetic
load. The provenance and label say "HRV-derived proxy" and never imply a
galvanic/EDA reading.
The engine reuses the shared vitals-baseline math (the same SDNN baseline
the wellness index reads), maps the day's below-baseline suppression to a
0-100 score, and stores it as a COMPUTED STRESS_SCORE row — server-owned, a
client can never POST it. It gates and writes nothing below the minimum
intra-day sample floor or without a usable baseline.
Standard: Kim et al. 2018 (HRV and stress meta-analysis); Thayer et al. 2012
(neurovisceral integration). Idempotent per user per day via the nightly job.
Daytime heart-rate-variability (SDNN) and heart-rate samples need to keep
their intra-day shape long enough for the stress engine to read them, but
the raw per-sample volume must stay bounded. Add a retention tier that
keeps the last 14 days of raw HRV/HR samples and folds everything older to
one daily-mean row via the proven mean-consolidation drain (same idempotent
stats:<HK>:<day> fold, soft-delete, and rollup recompute).
The two dense-tier types are deliberately EXEMPT from the destructive
daily-mean drain allowlist (HIGH_FREQUENCY_MEAN_TYPES) — that drain would
collapse the intra-day shape and soft-delete the raw rows. A regression test
pins both exclusions so a future edit that adds either type to the drain
allowlist fails loudly.
Migration 0108 adds only the partial index that keeps the retention drain's
discovery and per-user scan cheap; the retention bound lives in code so
tuning it never needs a migration.
Add a recovery-score engine that turns a user's already-stored nightly
signals into a stored 0-100 score. It reuses the readiness blend — the
resting-heart-rate, HRV (SDNN), sleep and respiratory-rate deviation
from the personal baseline, with the null-redistribution the health
score already uses — so the stored series and the live readiness surface
can never drift. The lineage is the recovery-marker literature (Plews
2013, Buchheit 2014): a higher score reads as more recovered. It is a
descriptive wellness proxy, not a clinical or training-recovery
assessment, and it is only stored when enough components are present —
below that floor no row is written rather than a headline from one
signal.
A nightly job computes each eligible user's score and upserts it as a
COMPUTED RECOVERY_SCORE row, idempotent per user per day on the
externalId day key, so a re-fired tick overwrites in place instead of
duplicating. The score is never computed on a page visit. The queue is
registered in the worker's queue inventory, scheduled, and wired to a
handler; a grep guard pins all four points so it cannot silently never
run.
The on-demand force-warm bounds the comprehensive briefing with a 45 s
timeout and then warms the per-status + generic-metric caches itself. The
timeout could not cancel the underlying generation, so a comprehensive that
resolved after the timeout reached its own per-status eviction and deleted
the rows the warm passes had just written — leaving the user back on a cold
cache.
Thread an AbortSignal into the comprehensive generation and abort it when the
timeout fires; the generation now bails out before its write + evict once the
signal is aborted, so a late resolve can never undo the warm.
Also align the force-warm gating with the route: the route admits the job on
the per-user assessment surface, so warm the per-status + generic-metric
cards whenever that surface is enabled, even when the global briefing switch
is off. Only the comprehensive briefing stays gated on the briefing switch.
Factor the per-status cache-action shape into a single helper so the key is
built in one place rather than hand-assembled across the generators, the
eviction sweep, and the ingest debounce.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 75e0b5a9129893915f7a213ec96253df19341d15)
The on-demand force-warm ran the comprehensive insight first and only ran
the per-status (7) and generic-metric (~30) assessment passes when that
step returned generated or cached. A slow or stalled comprehensive
generation therefore aborted the entire warm — observed in production as a
102 s run ending with assessments_warmed: 0 — leaving every card cold and
the user back on the lazy 30-60 s on-first-click path.
Give the comprehensive step its own bounded budget and treat a timeout,
error, or skip as non-fatal: the per-status and generic-metric passes now
always run afterwards. Each per-card generator resolves its own provider
chain and bounds its own provider call, so running them after a failed
comprehensive costs a cheap chain-resolve when no provider is configured
and is capped per card otherwise. The result reports each section's
outcome independently, and a timed-out comprehensive is reported distinctly
from a failed one so a partial warm is observable.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit f52f8e9c3b32590f3b085d082418ebdd3a923f1d)
Add POST /api/insights/pregenerate so every AI assessment text — the
comprehensive insight, the seven specialised status cards, and every
data-bearing generic metric assessment — can be warmed for the calling
user on demand, in the active locale, instead of waiting for the nightly
cron or a lazy first-visit generation.
The route narrows userId from the session/Bearer, runs the assistant
surface gate, and enqueues a forced full-warm job onto the existing
insight-pregenerate queue, returning immediately. A short per-user bucket
(insights-warm:<userId>, one per three minutes) returns 429 on a tight
loop so the bypassed nightly 20h budget cannot be turned into a provider
cost amplifier.
The worker branches on the payload: a job carrying { userId, force }
runs forceWarmUser for that user, skipping the candidate scan and the
20h budget; a scheduled tick with no payload keeps its existing batch
behaviour. forceWarmUser reuses the existing comprehensive generator and
warm helpers — no new generation logic — and stays safe by construction:
the comprehensive generator no-ops without a provider, the generic warm
pass filters to metrics the user has rows for so an empty metric never
reaches the provider, and every write is idempotent.
The queue name and a generator-free enqueue helper move to
insight-pregenerate-shared.ts, mirroring the insight-status-generate
split, so the route enqueues without pulling the generator tree.
Regenerate the OpenAPI contract for the new route.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit e10ae9bfeca544c7e5aecd9102459d97ddfad6db)
Extend the assessment-card system beyond the seven specialised scopes to
the ~30 HealthKit metric pages, without hand-writing a prompt per metric.
- Add a metric-metadata registry keyed by HealthKit id (display name,
unit, direction, optional normal range, archetype, scope), covering the
data-bearing HealthKit metrics. The seven specialised metrics keep
their own scopes and are excluded from the generic set; an unregistered
id gets no card.
- Add a generic data-driven generator that reuses the existing graded
snapshot, provider runner, stale-while-revalidate cache, and persist
path. One prompt template per archetype (physiological-vital,
activity-fitness, body-composition, mobility-gait,
environmental-exposure) plus a dedicated sleep template, parameterised
by metric metadata. Empty-data metrics return an insufficient marker
with no provider call. Normal ranges use age + sex where the profile
stores them; no ancestry/region ranges.
- Add GET /api/insights/metric-status?metric=<id>: apiHandler-wrapped,
Zod-validated against the closed registry id set (422 on unknown),
read-only with stale-while-revalidate. First insights route with
OpenAPI coverage; regenerate docs/api/openapi.yaml.
- Route the generic metric:<id> scope through the on-demand status queue
worker dispatch.
- Extend the nightly warm pass to the generic metric caches for a user's
data-bearing metrics only (one distinct read gates the set), keeping
de+en warm and never calling a provider for an empty metric.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 41453fbdfee60793c16269b9a9e5f04b6c3ea450)
Extend the daily-mean consolidation to cover the high-frequency
gait/mobility metrics that previously mapped to no consolidation set and
piled up at sampling granularity: walking asymmetry, walking
double-support, walking steadiness, and walking heart-rate average. The
correct daily reduction for these is the mean, matching the Health-app
display; all four stay disjoint from the cumulative (sum) set.
Fold the daily-mean drain onto the existing nightly cumulative-drain tick
so new high-frequency spot samples no longer accumulate raw between worker
reboots. The two passes share one concurrency-1 cron, keeping the
maintenance window a single sequential walk. Boot-time discovery stays as
the back-fill for accounts that accumulated raw rows before this shipped.
Recompute the affected day's rollup bucket after the consolidation
commits, the same way the ingest path does, so a rollup-covered read
reflects the single consolidated row rather than a stale mean derived from
the now soft-deleted per-sample rows.
Regular heart-rate / pulse stays raw: it is a discrete clinical reading
and feeds correlation and scatter analytics. No schema change.
Opening an Insights category page froze the whole UI: the per-metric
status card fired a cold, uncapped LLM fetch on mount with no client
abort, so the request hung up to the server's 60s ceiling and the
pending/resolve re-renders thrashed the tab strip and ate touch events.
Timeouts were never negatively cached, so every visit re-blocked.
Fix the class holistically:
- Bound every status GET on the client. The shared use-insight-status
hook and the inline medication-compliance query now wrap fetch in an
8s AbortController and stop retrying inline; navigation can no longer
await an uncapped round-trip.
- Make the status routes read-only. On a cache miss the generator
enqueues an out-of-band pg-boss job (insight-status-generate) and
returns a preparing envelope; the client polls a bounded interval
until the worker warms the cache. Models the dashboard snapshot's
ready/preparing pattern. The queue is registered in the worker
allQueues loop + a boss.work handler, with a guard test.
- Fix the nightly warm-pass regression: warm every supported locale
(the client reads against the active UI locale, not the persisted
User.locale) and warm on a cached comprehensive outcome too (its
write still evicts the per-status caches). Add a short-TTL negative
cache on the timeout path so a transient provider stall no longer
makes every subsequent visit re-block or re-enqueue.
- Add loading.tsx for the insights segment + the root so any future
RSC await streams a skeleton instead of blocking the transition.
- Stabilise the advisor regenerate callback so a status-query flip
can't re-render the memoised tab strip mid-gesture.
New i18n key insights.assessmentPreparing across all six locales.
The write-path convergence stops new duplicate slot rows, but accounts
that already accrued a pending + taken pair for one dose keep an inflated
expected count, a phantom taken marker, and a suppressed "due now" prompt
until the bad rows are cleared.
Add a boot-time, idempotent cleanup queue (mirroring the existing
backfill queues): discover only the users holding a same-slot duplicate,
then per medication snap every live intake to its canonical slot, keep
the most-actioned row per slot (taken over skipped over pending),
normalise its instant, soft-delete the losers, and recompute the
affected compliance rollups. Rows that resolve to no scheduled slot
(PRN / as-needed) are never collapsed. A second pass is a no-op.
Registered in the worker's queue list with a static guard test, after a
prior incident where a queue was built but never registered and silently
never ran.
A twice-daily medication ended up with two rows for one dose slot: the
projector / reminder worker mints a pending row (source REMINDER) at the
slot's `localHmAsUtc` instant, while a "taken" write from the per-med or
bulk intake route inserted a fresh row (source WEB / API). Because
`source` is part of the `(userId, medicationId, scheduledFor, source)`
unique key — and the written instant could drift by a minute from the
projected one — the two rows coexisted: one taken, one still pending and
overdue. The card then read the slot as taken, counted 100%, and
suppressed the "due now" prompt for a dose the user had not taken.
Resolve every scheduled intake write to the slot's canonical instant
before writing and upsert in place, source-agnostic:
- `resolve-slot-instant.ts` snaps an incoming instant to the nearest
scheduled slot for that day (enumerated through the canonical
recurrence engine, so PRN and cyclic off-weeks resolve to no slot) and
anchors it on the same `localHmAsUtc` the projector and worker use.
Off-slot and as-needed writes resolve to null and keep insert semantics.
- The per-med and bulk routes look up the existing row at the canonical
`(userId, medicationId, scheduledFor)` ignoring source and update it in
place — mirroring the status-toggle route — instead of inserting a
sibling. The bulk per-entry response gains an additive `updated` status.
- The reminder worker's missed-dose mint now skips a slot that already
carries an actioned row (taken or skipped) from any source, not only an
existing pending REMINDER row.
PRN / as-needed writes are unaffected. Regression tests pin the collapse
(exact and one-minute drift), the PRN exemption, and the worker guard.
The warm pass forced the seven per-metric status generators strictly
back to back per user, so a fully configured account paid ~7× one LLM
round-trip serially. Bound the inner pass with a small `p-limit(3)` so
the cards warm a few at a time without saturating the Prisma pool or
fanning out an unbounded number of provider calls. The per-user budget
gate still bounds total cost per night.
The comprehensive generator evicts the seven `insights.<scope>-status`
caches as part of its write so a stale comprehensive insight can never
leave a stale per-metric card behind — but it does not re-fill them.
Before this, the morning after the nightly pre-gen cron ran the
per-metric status caches were empty, so the first `/insights` mount
still ran one live status generation per card.
After a successful comprehensive generation the cron now forces the
seven `*StatusForUser` generators so their caches are warm, not empty,
by the user's first visit. The warm pass only runs on a `generated`
outcome (a no-provider `skipped` has nothing to warm; a `failed` run
means the chain is unhealthy this tick — re-running it seven more times
would only multiply the failure). Each generator resolves the provider
chain internally, so a provider-less account stays cheap. The per-user
budget bucket that already bounds the comprehensive pass also gates the
warm pass — both run behind the same once-per-20h gate.
`runInsightPregenerate` now reports `assessmentsWarmed` and emits an
`insights.pregenerate.run` wide-event with the full per-run tally so the
nightly dashboard can track how many assessments each tick wrote.
Also adds `invalidateStatusInsightsForTypes` + `PER_STATUS_SCOPES` to
the comprehensive generator: a type → scope map (WEIGHT → weight + bmi +
general, BP → blood-pressure + general, PULSE/resting-HR → pulse +
general, everything else → general) so the ingest path can drop only the
scopes a fresh reading dirties.
Pure-history medication detail page, legacy step consolidation, and the
finish of the outbound-fetch migration.
- /medications/[id] becomes a read-only history surface; every setting
moves into a new advanced-settings sheet, and the edit trigger becomes
a two-option picker. The list page is unchanged.
- A pg-boss job folds pre-v1.5.0 granular step rows into one canonical
daily total per user per day and soft-deletes the originals; migration
0087 adds a partial index for the converging discovery scan.
- Every remaining server-side fetch routes through safeFetch (Withings,
Codex OAuth + SSE, bug-report, Umami, GlitchTip, Loki); a new
healthlog/safe-fetch-required ESLint rule guards the boundary.
- Fixes: intake-edit dialog seeded from real values, consolidation skips
a unique-constraint day instead of stranding it, avatar upload reads
the body once under a bounded cap, egress lint rejects absolute URLs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>