Integration syncs (Fitbit, Oura, Polar, WHOOP, Strava, Nightscout) now
mark the per-user analytics cells stale and sweep the workouts cache at
their sync tails — imported rows used to stay invisible to the cached
dashboard, analytics and workout readers until the TTL lapsed. The WHOOP
per-resource webhook/cron driver and the Polar workouts leg carry the
same call, matching the posture Google Health and the workout batch
route already had.
Illness, cycle and encounter write routes hard-evict the analytics
bucket through the new invalidateUserHealthContext helper: Rest Mode,
the cycle context and the Today-rail appointment feed all read from
cached cells under that bucket, and an interactive write must be true on
the very next read.
Every interactive path, every import, MCP and Telegram checked an incoming
measurement against VALUE_RANGES before storing it. The provider sync writers
did not, so a decode slip or a glitched sensor landed a row the application
itself calls impossible, and every mean, median and personal band derived from
it inherited the impossibility. Filtering the read side hid the row. It kept
landing.
The gate is one module reading the domain the app already declares, applied at
two kinds of seam. reconcileExternalMeasurement is the shared identity writer
for Polar, Oura, WHOOP, the Withings sleep leg, the Apple export importer and
the batch endpoint, so gating there means a provider added later inherits the
refusal rather than having to remember a check it never knew about; the new
rejected_range verdict carries no row, which is what makes the compiler force
every caller to handle it. The five writers that hold Prisma themselves gate at
their entry, before the existence probe decides insert versus overwrite, so an
impossible reading can no longer replace a good stored value.
An out-of-band value is dropped. Not clamped, because a clamp invents a reading
nobody took. Not stored behind a flag, because a quarantined row is still a row
and every consumer would have to learn the flag. Not fatal to the batch, because
one bad sample must not strand the run's other readings behind a held watermark.
Absence reads as absence.
A silent drop, though, is indistinguishable from a provider that sent nothing,
so every drop is tallied on the wide event under measurement.range_rejected with
the metric, the provider and the edge it broke. Counts only: the reading itself
is health data and never enters the log line, and the direction plus the
metric's own declared bound is what a diagnosis actually needs. The tally
accumulates rather than replacing, because a provider gates once per resource
leg and the reconciler once per row. One warning per provider lifts the event to
warn so a drop does not hide inside a green line.
A structural guard enumerates all twenty measurement writers with the reason
each is allowed to write, so a new one has to answer the question before it
ships. scripts/diagnose-implausible-measurements.ts reports the rows already
stored, aggregated per metric and source with no value beyond the bucket's own
extremes. It repairs nothing: whether a stored impossible row is a decode slip
or a band that is too narrow is not a call a script gets to make.
Six provider syncs returned a bare count, which narrowed their failures away:
a Nightscout window in which every SGV row was refused by a constraint
returned the same 0 as a quiet night, and an Oura or Polar run whose
collections all failed returned 0 too. The routes then answered 200 with a
count and the settings card drew a green tick over it.
Every sync now reports what it imported AND whether any part of it did not
land, and the route resolves the outcome server-side so the web card and the
native client cannot disagree. Google Health keeps its 502 for a run that
failed and wrote nothing, but a run that failed after writing some resources
is now a partial rather than a discarded result. The labs OCR commit carries
the same resolved outcome.
A sync failure the classifier bucketed as `transient` had no exit. The park
test read `kind === "persistent"` and the persistent streak anchor was only
ever stamped by a persistent failure, so a streak could run for a thousand
attempts with the persistent counter at zero, the state frozen at
`error_transient`, and no ladder it could climb. `transient` is a hypothesis
about retryability, and the classifier defaults every timeout, 429, 5xx and
unrecognised response to it precisely because the wire cannot tell a reboot
from a permanent break. That default is right for one attempt and wrong for a
thousand: an unbroken streak retried hourly for a full day has refuted it by
experiment. Record it as `persistent` from there, which feeds the park ladder
that already exists — 48 h of unbroken failure end to end before an
integration stops attempting.
The counters could not accumulate anyway. `IntegrationStatus` is keyed
(user, integration) while Withings runs four legs on four crons and WHOOP four
resources on four more, and a success cleared the error, every bucket, the
streak anchor and the state whatever leg had recorded them. The ECG tick at
:41 wiped what the sleep tick recorded at :15 — including a reauth park — so
the strike ladder was reset several times an hour and the card painted a green
"connected · 12 minutes ago" over a data class that had stopped arriving. A
success now clears only what it is entitled to clear.
Two columns carry it: `failing_since_at` dates the streak, `failing_leg` names
its owner. The migration backfills the anchor for rows already in an error
state so a connection broken for weeks escalates on its next attempt rather
than restarting its clock.
The collection fetchers degrade an unexpected 200 body to an empty list, so a
Polar-side envelope rename would read as "the user has no data" and stamp a
clean sync while importing nothing. Emit a fixed-shape polar.envelope.drift
breadcrumb (verb, HTTP status, top-level key names only, never values) at each
silent-empty site in the collection fetcher and the exercises fetcher, keeping
the log-and-degrade behavior so a live account stays alive while the gap becomes
visible in the event stream.
Truth up the client docstrings against the verified AccessLink contract: each
vitals list is a bare array or a named wrapper over a server-fixed 28-day window
with no pagination, and exercises is a bare array over the last 30 days of
uploads. Every hourly tick re-reads the full window, so a late-synced device or
a short outage self-heals with no cursor or watermark; history beyond the window
is an upstream cap. No request parameters added.
Restructure the Polar vitals sync from a single Promise.all over the five
collections into a Promise.allSettled over a per-collection spec, each closure
fetching and mapping so a throwing mapper is contained exactly like a failing
fetch. The healthy collections import; a failing one is recorded once as a
partial failure that names it, without stamping success and without rethrowing,
so a single unavailable data type — SpO2 on a watch without the sensor, a
higher-tier cardio-load read an account lacks — stops blanking recharge, sleep,
and activity on every tick.
A 401 is now a per-collection failure classified reauth_required, not a
whole-leg throw; the write-path escape still surfaces unmarked for the poll
boundary to record once. Mirrors the shipped Oura isolation, minus the
token-refresh leg Polar has no path for.
Introduce a recorded-marker primitive on the integration ledger so a provider
that records-then-rethrows a sync failure is not recorded a second time at the
poll-cohort boundary, while an unrecorded escape (an Oura or Polar write-path
throw, or any future path) lands on the ledger exactly once, correctly
classified and redacted by the source that owns it. Without the marker a
boundary-blind recorder would double-count the failure streak and could persist
a Nightscout token-bearing URL into the stored error.
Give the poll-cohort factory a required per-provider failure recorder and wire
one for Oura, Polar, Nightscout, and Strava.
Nightscout now counts only freshly inserted readings instead of insert plus
re-confirm (the inflated count fired the reminder trigger every tick), records a
transient failure and skips the success stamp when some rows fail to write, and
emits the measurement-arrival spine for its inserts. Correct the docstring that
promised a far-history backfill that does not exist: the hourly window imports
about 48 hours and nothing deeper.
Strava now enqueues its history backfill at connect time, mirroring the WHOOP
and Fitbit callbacks, so deep history imports within the hour instead of waiting
for the next worker restart.
Parity suite proves a Polar exercise and an equivalent Strava activity map to a
downstream-indistinguishable Workout row (sport bucket, start instant, duration,
distance, calories, HR, null-vs-zero). Regression guard asserts the exercises
request hits GET /v3/exercises with no userId path segment. Dedup ladder tests
mirror the Strava suite (overwrite-in-place, Apple+Polar twin collapses, Polar
above Strava). Composite test proves a failure in one Polar leg leaves the
other leg's result intact on the shared ledger. Defensive-shape tests cover the
four flagged assumptions: numeric-vs-string id, UTC-offset composition (positive
and negative), bare-array-vs-paged body, and a 403 scope error as a soft error.
Refs #568
Pull the athlete's exercises from the stateless GET /v3/exercises list (last
30 days of uploads, no transaction, no userId in the path) and upsert each into
the Workout table as source POLAR, keyed on (userId, source, externalId) so a
re-fetch or Polar-side re-analysis overwrites in place. A new table-driven
sport map normalises the raw Polar sport labels into the canonical
WorkoutSportType bucket at ingest, mirroring the Strava/WHOOP mappers; the raw
labels and training load ride metadata for provenance.
The exercise leg rides the existing hourly polar-sync cohort tick alongside the
vitals leg, with independent attribution on the shared polar ledger: each leg
is attempted and records its own failure, and neither can mask the other's
result. Polar takes its place in the workout source ladder between Withings and
Strava. No migration and no OpenAPI change: MeasurementSource.POLAR and the
full workouts source enum already exist.
Refs #568
Merge the eight reviewed audit fixes that were cut a week ago and never
landed. Main has moved on since, so each fix was re-checked against the
current tree and each of the five conflicts resolved by understanding
both sides.
Kept, still open on main:
- Proxy early exits now carry the transport security header subset via a
shared helper. Reconciled with main's /api/health worker-only carve-out.
- Batch measurement ingest bounds the entry instant so a future-dated row
can no longer win "latest reading" or poison the rollup buckets.
- Web Push dials through the pinned safeFetch dispatcher with
requirePublicHost, closing the last user-controlled-host egress.
- Mood bulk upsert and its tag/factor links run in one transaction, so a
rejected rating rolls the mood row back instead of leaving a tagless row.
- Full backup carries NutrientIntakeDay with its unit and source. Folded
into main's portable-export / disaster-recovery split.
- Live daily-series fallback restores the min/max spread band the tiered
read already emitted.
- Coach stream aborts its in-flight reader on unmount and resets send state
on a conversation switch.
- Manual water writes share the day-bound helper, nutrient add clamps after
the increment, and the thiamin cap drops to a real supplement ceiling.
- Compliance point window widens symmetrically instead of wrapping overnight.
- Oura, Polar and Fitbit token helpers classify an empty or non-JSON 2xx
body as a handled transient error.
- WebAuthn relying-party fallback to localhost is gated to development.
- Chart-token strip list is derived from the allowlist and pinned by a test.
- Regression tests pin the snapshot-cache tenant key, the GlitchTip
path-secret redaction and the Withings credentials delete.
Superseded by main, tests re-pointed at the current shape:
- The intake CSV import was rewritten on main to run through a background
job with a single-active-job admission gate. The future-date bound is
re-applied at request validation and the per-user limiter is kept for
parity with the bulk twin; the old synchronous-write test is replaced.
The German prompt-injection test asserted only inside `if (decision.refuse)`,
so it stayed green whether or not the filter fired. Un-guard it, drive it from a
case table, and widen the two German patterns to cover the determiners that
walked straight past them ("saemtliche", "die", "deine").
The Oura / Polar / Fitbit token helpers cast a possibly-null `json` to the token
type on a status-only success gate. A 2xx with an empty or non-JSON body handed
callers a null they dereferenced, surfacing an unclassified TypeError instead of
a handled integration error. Narrow explicitly and classify as transient.
The Today digest went stale after logging a dose: nothing invalidated the
`["daily","digest"]` query, and it reads meds from the same server snapshot
the intake routes already hard-evict — so the "dose due" rail item lingered
until a hard reload. Add the digest key to the medication-dependent bundle and
force the inactive digest (unmounted while the user is on the medication card)
to refetch alongside the dashboard snapshot; the server snapshot eviction seam
is reused, so the refetch returns post-write dose state immediately.
Remove the score-ring cluster from the Today hero — it read as uneven and
wasted tile space. The health-score ring is the hero's only ring now; the
score-ring selection contract stays server-side for the native client. Tighten
the hero's vertical rhythm so the worth-a-look rail sits close under the lead
instead of floating below a half-empty column.
Wire the remaining sleep transports into the morning-refresh trigger: Google
Health, Fitbit, Oura, and Polar now enqueue the debounced morning refresh on a
last-night segment landing, matching the Withings / WHOOP / Apple seams — those
sources had no trigger at all. Make the trigger observable: each early-return
gate now annotates a distinct `daily.morning_refresh.skipped` reason (the
not-last-night gate also carries the samples' resolved local dates, today's key
and the tz) so a real landing shows exactly which gate fired.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018V8V9qapgHi7fhgtMJ8SkM
Platform-wide hardening: the failure classes found live this week hunted down
across every integration.
Fitbit carried the same tombstone wedge fixed for Google — probe now matches
soft-deleted rows and the update resurrects them; the comment describing a
partial unique index that never existed is corrected. Sleep segment ids are
stable across re-scoring for Withings (session+segment-start), WHOOP/Polar
(one segment per stage, no positional index) and Oura (run-start keyed), each
with a session-scoped sweep that soft-deletes rows a re-score orphaned — so
legacy and affected nights heal as they are re-read.
One platform rule for deleted rows: source-owned readings resurrect on
re-import (Withings, WHOOP, Oura, Polar, Nightscout, mood webhook, CSV
re-imports, Apple stats day-totals); Apple sample-grain deletions stick per
the paired-client contract, and the mood bulk route's tombstone match is now
a true no-op reported as duplicate. Import counts reconcile against actual
createMany counts.
Google lifecycle: a dead token registers on the cycle's hard-fail ledger, so
an expired connection parks instead of stamping success hourly; backfill and
sleep-repair markers stamp only after a clean pass (pg-boss retries are now
real); failed measurement writes hold the watermark; no-op updates are
skipped; mapping.md tells the current truth.
Scale: doctor-report reads only consumed columns and enabled sections;
exports and backups keyset-paginate; achievements tally via SQL buckets;
series charts day-bucket dense kinds beyond 90 days; BP pairing is a linear
two-pointer; distinct-type scans run server-side; the graded-series cold
fallback is bounded; the fitbit defer tracker loops instead of spreading.
Date-only daily scalars from Oura and Polar were stored at UTC midnight
(`${day}T00:00:00.000Z`). On read these rows re-bucket through
`userDayKey(measuredAt, userTz)`, so for any west-of-UTC user the day
shifted back one: a June 1 SpO2 or readiness row filed under May 31 and
fell out of alignment with the HR/sleep rows that carry true instants.
Extend the existing `canonicalDailyTimestamp` helper to accept an optional
timezone and anchor a date-only day at local noon (noon UTC when no zone
is supplied) — a full twelve hours inside the calendar day, so it
round-trips through `userDayKey` to the same day for every zone within
plus/minus twelve hours. Route every Oura day-keyed mapper (via the shared
`dayAnchor` fallback) and all five Polar daily mappers through it; real
wake/measurement instants stay untouched. Fitbit already anchored at noon
and is now consistent; Whoop and the Withings cursor anchors were verified
not to be measurement anchors.
One-shot prettier pass so the new format:check gate starts from a clean
baseline. Formatting only — no behaviour change. Generator-owned and
vendored artifacts stay untouched via .prettierignore.
Polar exposes only per-stage duration totals, so its mapper already
reconstructs a contiguous CORE-DEEP-REM order exactly like WHOOP and flags
each segment reconstructed. The night reader's honesty allow-list only held
WHOOP, so a Polar-won night was served with reconstructed:false — presenting
a synthetic stage order to web and iOS as if it were the device's measured
timing. Add POLAR to the allow-list so the layout is labelled approximate.
Polar's total_interruption_duration carried the night's awake time but was
dropped, so every Polar night read as fully consolidated (awakeMinutes:null,
awakenings:0) with overstated asleep-vs-in-bed efficiency. Lay it as a
leading AWAKE segment so the asleep stages partition the remaining window and
the reader surfaces real awake time.
WHOOP and Polar independently implemented the same contiguous-timeline walk.
Extract a shared reconstructContiguousSleepTimeline helper under src/lib/sleep
and call it from both; each mapper keeps only its field normalisation. Oura's
measured 5-min hypnogram is a different algorithm and is left untouched.
classifyPolarError now lifts a 400 carrying invalid_grant on the token
endpoint from persistent to reauth_required, mirroring Oura via the shared
isOAuthInvalidGrant helper. A stale or replayed authorization code at connect
time now surfaces the reconnect prompt instead of silently parking after 24 h.
The BYO-credentials DELETE now audit-logs and parks the integration ledger at
disconnected for parity with the dedicated disconnect route, and the PUT
audit-logs the credential write. Refresh the connect-route comment that still
described Polar as env-only OAuth; it has resolved BYO-first since v1.17.1.
Add the invalid_grant classification test plus the previously-missing
callback (happy path, CSRF mismatch, cross-user, off-spec token), connect, and
credentials route tests.
Polar read three collections off the wire and dropped several useful signals
at the mapper. Close the gaps, keeping every value server-authoritative and
canonical-unit:
- Sleep now follows the END-instant convention the canonical night-grouper
depends on. mapSleep reconstructs an ordered, contiguous CORE/DEEP/REM
timeline from the parsed sleep_start_time / sleep_end_time, stamping each
segment at its own end instant with an indexed externalId, plus an IN_BED
envelope and the score at the sleep end. This restores the hypnogram (the
stages no longer collapse onto one midnight-UTC instant) and fixes wake-day
mis-attribution for negative-UTC users. Records without a usable window fall
back to the prior midnight anchor. Synthetic ordering is flagged
reconstructed.
- ans_charge maps to ANS_CHARGE (through-as-is; a baseline-relative deviation
that can run negative).
- distance_from_steps maps to WALKING_RUNNING_DISTANCE (metres), filling the
Polar distance-ladder slot that previously had no producer.
- New fetchers + mappers for Training Load Pro (cardio_load -> CARDIO_LOAD)
and the Elixir SpO2 collection (-> OXYGEN_SATURATION), with Polar added to
the SpO2 source ladder below the established straps.
No strap/profile weight is fetched; egress stays on safeFetch.
Polar and Oura resolved their OAuth client id/secret from the server
environment only, forcing every user on a deploy to share one app. Both
now read per-user BYO keys first, falling back to the shared env app when
unset, mirroring the WHOOP and Fitbit model.
- Add DB-first-then-env resolvers in src/lib/{polar,oura}/credentials.ts,
encrypting the client id/secret at rest on User.
- Repoint the connect, callback, status routes and the Oura refresh path
to the user-scoped resolver.
- Add /api/{polar,oura}/credentials (GET/PUT/DELETE) with Zod-validated
input and userId narrowed from the session.
- Render an optional 'your OAuth app credentials' form on the Polar and
Oura cards; the status read exposes hasOwnCredentials for the saved
placeholder. Existing env-only deploys keep working unchanged.
- Cover both resolvers (DB hit, env fallback, no creds) and the card form
with unit tests; add the new strings across all six locales.
The mobile settings test asserted three integration pills; Polar, Oura and
Nightscout bring the list to six. Remove the unused Oura and Polar field-map
documentation constants and the unused advanced-indices interface the dead-code
gate flagged.
Polar and Oura response classifiers were byte-identical copies of the
status-driven verdict, the typed API error, and the invalid_grant probe.
Extract classifyHttpStatus, an IntegrationApiError base carrying the vendor
label, and a vendor-scoped isOAuthInvalidGrant into a shared module; Polar
and Oura now re-export under their existing names and types so call sites
and instanceof checks are unchanged.
Collapse the triplicated classifyXFailure mapping (Polar, Oura, Nightscout)
onto one toFailureKind helper. Behaviour is identical: same classifications,
same failure kinds, same error types.
Connect a Polar account over AccessLink v3: an authorization-code OAuth
flow (shared app from POLAR_CLIENT_ID / POLAR_CLIENT_SECRET) backed by a
stateless HMAC-signed state token doubling as an httpOnly cookie for CSRF.
The callback verifies the signature, expiry, provider binding, and the
byte-exact cookie match, then encrypts the access token plus member id onto
the user row.
An hourly poll-cohort job maps Nightly Recharge recovery, sleep stages,
and daily activity to canonical measurements tagged source POLAR, idempotent
on (userId, type, source, externalId). The recovery band feeds RECOVERY_SCORE
through the canonical ladder, which now ranks native readiness above the
computed proxy. Settings gains a connect / disconnect card. Tokens are never
logged or returned; every egress goes through safeFetch.
Marc-André Bombeck