Introduce a recorded-marker primitive on the integration ledger so a provider
that records-then-rethrows a sync failure is not recorded a second time at the
poll-cohort boundary, while an unrecorded escape (an Oura or Polar write-path
throw, or any future path) lands on the ledger exactly once, correctly
classified and redacted by the source that owns it. Without the marker a
boundary-blind recorder would double-count the failure streak and could persist
a Nightscout token-bearing URL into the stored error.
Give the poll-cohort factory a required per-provider failure recorder and wire
one for Oura, Polar, Nightscout, and Strava.
Nightscout now counts only freshly inserted readings instead of insert plus
re-confirm (the inflated count fired the reminder trigger every tick), records a
transient failure and skips the success stamp when some rows fail to write, and
emits the measurement-arrival spine for its inserts. Correct the docstring that
promised a far-history backfill that does not exist: the hourly window imports
about 48 hours and nothing deeper.
Strava now enqueues its history backfill at connect time, mirroring the WHOOP
and Fitbit callbacks, so deep history imports within the hour instead of waiting
for the next worker restart.
Parity suite proves a Polar exercise and an equivalent Strava activity map to a
downstream-indistinguishable Workout row (sport bucket, start instant, duration,
distance, calories, HR, null-vs-zero). Regression guard asserts the exercises
request hits GET /v3/exercises with no userId path segment. Dedup ladder tests
mirror the Strava suite (overwrite-in-place, Apple+Polar twin collapses, Polar
above Strava). Composite test proves a failure in one Polar leg leaves the
other leg's result intact on the shared ledger. Defensive-shape tests cover the
four flagged assumptions: numeric-vs-string id, UTC-offset composition (positive
and negative), bare-array-vs-paged body, and a 403 scope error as a soft error.
Refs #568
Pull the athlete's exercises from the stateless GET /v3/exercises list (last
30 days of uploads, no transaction, no userId in the path) and upsert each into
the Workout table as source POLAR, keyed on (userId, source, externalId) so a
re-fetch or Polar-side re-analysis overwrites in place. A new table-driven
sport map normalises the raw Polar sport labels into the canonical
WorkoutSportType bucket at ingest, mirroring the Strava/WHOOP mappers; the raw
labels and training load ride metadata for provenance.
The exercise leg rides the existing hourly polar-sync cohort tick alongside the
vitals leg, with independent attribution on the shared polar ledger: each leg
is attempted and records its own failure, and neither can mask the other's
result. Polar takes its place in the workout source ladder between Withings and
Strava. No migration and no OpenAPI change: MeasurementSource.POLAR and the
full workouts source enum already exist.
Refs #568
Merge the eight reviewed audit fixes that were cut a week ago and never
landed. Main has moved on since, so each fix was re-checked against the
current tree and each of the five conflicts resolved by understanding
both sides.
Kept, still open on main:
- Proxy early exits now carry the transport security header subset via a
shared helper. Reconciled with main's /api/health worker-only carve-out.
- Batch measurement ingest bounds the entry instant so a future-dated row
can no longer win "latest reading" or poison the rollup buckets.
- Web Push dials through the pinned safeFetch dispatcher with
requirePublicHost, closing the last user-controlled-host egress.
- Mood bulk upsert and its tag/factor links run in one transaction, so a
rejected rating rolls the mood row back instead of leaving a tagless row.
- Full backup carries NutrientIntakeDay with its unit and source. Folded
into main's portable-export / disaster-recovery split.
- Live daily-series fallback restores the min/max spread band the tiered
read already emitted.
- Coach stream aborts its in-flight reader on unmount and resets send state
on a conversation switch.
- Manual water writes share the day-bound helper, nutrient add clamps after
the increment, and the thiamin cap drops to a real supplement ceiling.
- Compliance point window widens symmetrically instead of wrapping overnight.
- Oura, Polar and Fitbit token helpers classify an empty or non-JSON 2xx
body as a handled transient error.
- WebAuthn relying-party fallback to localhost is gated to development.
- Chart-token strip list is derived from the allowlist and pinned by a test.
- Regression tests pin the snapshot-cache tenant key, the GlitchTip
path-secret redaction and the Withings credentials delete.
Superseded by main, tests re-pointed at the current shape:
- The intake CSV import was rewritten on main to run through a background
job with a single-active-job admission gate. The future-date bound is
re-applied at request validation and the per-user limiter is kept for
parity with the bulk twin; the old synchronous-write test is replaced.
The German prompt-injection test asserted only inside `if (decision.refuse)`,
so it stayed green whether or not the filter fired. Un-guard it, drive it from a
case table, and widen the two German patterns to cover the determiners that
walked straight past them ("saemtliche", "die", "deine").
The Oura / Polar / Fitbit token helpers cast a possibly-null `json` to the token
type on a status-only success gate. A 2xx with an empty or non-JSON body handed
callers a null they dereferenced, surfacing an unclassified TypeError instead of
a handled integration error. Narrow explicitly and classify as transient.
The Today digest went stale after logging a dose: nothing invalidated the
`["daily","digest"]` query, and it reads meds from the same server snapshot
the intake routes already hard-evict — so the "dose due" rail item lingered
until a hard reload. Add the digest key to the medication-dependent bundle and
force the inactive digest (unmounted while the user is on the medication card)
to refetch alongside the dashboard snapshot; the server snapshot eviction seam
is reused, so the refetch returns post-write dose state immediately.
Remove the score-ring cluster from the Today hero — it read as uneven and
wasted tile space. The health-score ring is the hero's only ring now; the
score-ring selection contract stays server-side for the native client. Tighten
the hero's vertical rhythm so the worth-a-look rail sits close under the lead
instead of floating below a half-empty column.
Wire the remaining sleep transports into the morning-refresh trigger: Google
Health, Fitbit, Oura, and Polar now enqueue the debounced morning refresh on a
last-night segment landing, matching the Withings / WHOOP / Apple seams — those
sources had no trigger at all. Make the trigger observable: each early-return
gate now annotates a distinct `daily.morning_refresh.skipped` reason (the
not-last-night gate also carries the samples' resolved local dates, today's key
and the tz) so a real landing shows exactly which gate fired.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018V8V9qapgHi7fhgtMJ8SkM
Platform-wide hardening: the failure classes found live this week hunted down
across every integration.
Fitbit carried the same tombstone wedge fixed for Google — probe now matches
soft-deleted rows and the update resurrects them; the comment describing a
partial unique index that never existed is corrected. Sleep segment ids are
stable across re-scoring for Withings (session+segment-start), WHOOP/Polar
(one segment per stage, no positional index) and Oura (run-start keyed), each
with a session-scoped sweep that soft-deletes rows a re-score orphaned — so
legacy and affected nights heal as they are re-read.
One platform rule for deleted rows: source-owned readings resurrect on
re-import (Withings, WHOOP, Oura, Polar, Nightscout, mood webhook, CSV
re-imports, Apple stats day-totals); Apple sample-grain deletions stick per
the paired-client contract, and the mood bulk route's tombstone match is now
a true no-op reported as duplicate. Import counts reconcile against actual
createMany counts.
Google lifecycle: a dead token registers on the cycle's hard-fail ledger, so
an expired connection parks instead of stamping success hourly; backfill and
sleep-repair markers stamp only after a clean pass (pg-boss retries are now
real); failed measurement writes hold the watermark; no-op updates are
skipped; mapping.md tells the current truth.
Scale: doctor-report reads only consumed columns and enabled sections;
exports and backups keyset-paginate; achievements tally via SQL buckets;
series charts day-bucket dense kinds beyond 90 days; BP pairing is a linear
two-pointer; distinct-type scans run server-side; the graded-series cold
fallback is bounded; the fitbit defer tracker loops instead of spreading.
Date-only daily scalars from Oura and Polar were stored at UTC midnight
(`${day}T00:00:00.000Z`). On read these rows re-bucket through
`userDayKey(measuredAt, userTz)`, so for any west-of-UTC user the day
shifted back one: a June 1 SpO2 or readiness row filed under May 31 and
fell out of alignment with the HR/sleep rows that carry true instants.
Extend the existing `canonicalDailyTimestamp` helper to accept an optional
timezone and anchor a date-only day at local noon (noon UTC when no zone
is supplied) — a full twelve hours inside the calendar day, so it
round-trips through `userDayKey` to the same day for every zone within
plus/minus twelve hours. Route every Oura day-keyed mapper (via the shared
`dayAnchor` fallback) and all five Polar daily mappers through it; real
wake/measurement instants stay untouched. Fitbit already anchored at noon
and is now consistent; Whoop and the Withings cursor anchors were verified
not to be measurement anchors.
One-shot prettier pass so the new format:check gate starts from a clean
baseline. Formatting only — no behaviour change. Generator-owned and
vendored artifacts stay untouched via .prettierignore.
Polar exposes only per-stage duration totals, so its mapper already
reconstructs a contiguous CORE-DEEP-REM order exactly like WHOOP and flags
each segment reconstructed. The night reader's honesty allow-list only held
WHOOP, so a Polar-won night was served with reconstructed:false — presenting
a synthetic stage order to web and iOS as if it were the device's measured
timing. Add POLAR to the allow-list so the layout is labelled approximate.
Polar's total_interruption_duration carried the night's awake time but was
dropped, so every Polar night read as fully consolidated (awakeMinutes:null,
awakenings:0) with overstated asleep-vs-in-bed efficiency. Lay it as a
leading AWAKE segment so the asleep stages partition the remaining window and
the reader surfaces real awake time.
WHOOP and Polar independently implemented the same contiguous-timeline walk.
Extract a shared reconstructContiguousSleepTimeline helper under src/lib/sleep
and call it from both; each mapper keeps only its field normalisation. Oura's
measured 5-min hypnogram is a different algorithm and is left untouched.
classifyPolarError now lifts a 400 carrying invalid_grant on the token
endpoint from persistent to reauth_required, mirroring Oura via the shared
isOAuthInvalidGrant helper. A stale or replayed authorization code at connect
time now surfaces the reconnect prompt instead of silently parking after 24 h.
The BYO-credentials DELETE now audit-logs and parks the integration ledger at
disconnected for parity with the dedicated disconnect route, and the PUT
audit-logs the credential write. Refresh the connect-route comment that still
described Polar as env-only OAuth; it has resolved BYO-first since v1.17.1.
Add the invalid_grant classification test plus the previously-missing
callback (happy path, CSRF mismatch, cross-user, off-spec token), connect, and
credentials route tests.
Polar read three collections off the wire and dropped several useful signals
at the mapper. Close the gaps, keeping every value server-authoritative and
canonical-unit:
- Sleep now follows the END-instant convention the canonical night-grouper
depends on. mapSleep reconstructs an ordered, contiguous CORE/DEEP/REM
timeline from the parsed sleep_start_time / sleep_end_time, stamping each
segment at its own end instant with an indexed externalId, plus an IN_BED
envelope and the score at the sleep end. This restores the hypnogram (the
stages no longer collapse onto one midnight-UTC instant) and fixes wake-day
mis-attribution for negative-UTC users. Records without a usable window fall
back to the prior midnight anchor. Synthetic ordering is flagged
reconstructed.
- ans_charge maps to ANS_CHARGE (through-as-is; a baseline-relative deviation
that can run negative).
- distance_from_steps maps to WALKING_RUNNING_DISTANCE (metres), filling the
Polar distance-ladder slot that previously had no producer.
- New fetchers + mappers for Training Load Pro (cardio_load -> CARDIO_LOAD)
and the Elixir SpO2 collection (-> OXYGEN_SATURATION), with Polar added to
the SpO2 source ladder below the established straps.
No strap/profile weight is fetched; egress stays on safeFetch.
Polar and Oura resolved their OAuth client id/secret from the server
environment only, forcing every user on a deploy to share one app. Both
now read per-user BYO keys first, falling back to the shared env app when
unset, mirroring the WHOOP and Fitbit model.
- Add DB-first-then-env resolvers in src/lib/{polar,oura}/credentials.ts,
encrypting the client id/secret at rest on User.
- Repoint the connect, callback, status routes and the Oura refresh path
to the user-scoped resolver.
- Add /api/{polar,oura}/credentials (GET/PUT/DELETE) with Zod-validated
input and userId narrowed from the session.
- Render an optional 'your OAuth app credentials' form on the Polar and
Oura cards; the status read exposes hasOwnCredentials for the saved
placeholder. Existing env-only deploys keep working unchanged.
- Cover both resolvers (DB hit, env fallback, no creds) and the card form
with unit tests; add the new strings across all six locales.
The mobile settings test asserted three integration pills; Polar, Oura and
Nightscout bring the list to six. Remove the unused Oura and Polar field-map
documentation constants and the unused advanced-indices interface the dead-code
gate flagged.
Polar and Oura response classifiers were byte-identical copies of the
status-driven verdict, the typed API error, and the invalid_grant probe.
Extract classifyHttpStatus, an IntegrationApiError base carrying the vendor
label, and a vendor-scoped isOAuthInvalidGrant into a shared module; Polar
and Oura now re-export under their existing names and types so call sites
and instanceof checks are unchanged.
Collapse the triplicated classifyXFailure mapping (Polar, Oura, Nightscout)
onto one toFailureKind helper. Behaviour is identical: same classifications,
same failure kinds, same error types.
Connect a Polar account over AccessLink v3: an authorization-code OAuth
flow (shared app from POLAR_CLIENT_ID / POLAR_CLIENT_SECRET) backed by a
stateless HMAC-signed state token doubling as an httpOnly cookie for CSRF.
The callback verifies the signature, expiry, provider binding, and the
byte-exact cookie match, then encrypts the access token plus member id onto
the user row.
An hourly poll-cohort job maps Nightly Recharge recovery, sleep stages,
and daily activity to canonical measurements tagged source POLAR, idempotent
on (userId, type, source, externalId). The recovery band feeds RECOVERY_SCORE
through the canonical ladder, which now ranks native readiness above the
computed proxy. Settings gains a connect / disconnect card. Tokens are never
logged or returned; every egress goes through safeFetch.
Marc-André Bombeck