Commit Graph
10 Commits
Author SHA1 Message Date
Marc-André Bombeck 7e54b6be81 refactor(integrations): split the provider clients into wire types, endpoints and mappers
Oura, Polar, WHOOP and Strava each keep the transport and endpoints in
`client.ts` and gain `wire-types.ts` (the record shapes) and `map.ts` (the
pure record-to-row mappers, no I/O). Google Health's 1,600-line client
becomes `budget.ts` (slice budget and outcome tally), `request.ts` (error
detail, retry, rate limits), `data-points.ts` (the paginated walker) and a
small `client.ts` for the profile and roll-up reads; its `export * from
"./mappers"` facade is gone and its importers name the module that defines
what they use.

The code moved verbatim; the mapper goldens written before the move and
the provider integration tests pass unchanged.
2026-10-23 13:30:10 +02:00
Marc-André Bombeck bc91a542e2 fix(google-health): count backfill progress only once it is persisted and cap the slice chain 2026-10-11 13:41:53 +02:00
Marc-André Bombeck 46047e57c0 fix(google-health): keep long backfills out of reauth parking and settle overlapping sleep segments
A history backfill carried one access token through walks that outlive it.
After an hour every request answered 401, which classifies as a revoked
grant, so the account was parked at error_reauth with a refresh token that
was still valid, and every later run skipped it until a manual reconnect.
Running beside the hourly sync, the backfill also pushed the account over
Google's per-minute quota, and each 429 failed its whole collection.

- Walks take a token source that renews the access token before it expires
  and once more when Google refuses it mid-walk.
- A 429 is waited out (Retry-After, RetryInfo, else a doubling backoff) and
  the request repeated; it is never a reauth.
- error_reauth only short-circuits a run when the token endpoint refused the
  grant (needsReauth). A connection parked by a data request's 401 refreshes
  on its next run and clears itself when that run completes.
- One run per account at a time: syncs, the backfill and the sleep repair
  share an advisory lock. The hourly poll skips a held account, a manual
  sync answers 409 sync_in_progress, the backfill waits a bounded time.
- A backfill attempt records each collection it finishes cleanly
  (backfill_progress) and the next attempt resumes with the rest.
- The incremental watermark is stamped with the cycle's start, not its end.

Overlapping sleep segments that share a stage and an end share the natural
key (type, measuredAt, source, sleepStage). A re-score that moved one row
onto a sibling's slot threw P2002 and failed the sync on every run.

- A batch keeps one reading per natural key, chosen the same way each time
  (longer segment, then smaller externalId).
- A move that still collides is settled by reconcileExternalMeasurement,
  which retires the stale row, instead of failing the sync.
- Updates run before inserts, and the key-format rescue no longer re-keys a
  row this batch is moving.
- The sleep replace-by-window protects every session's fresh rows, so two
  overlapping sessions no longer tombstone and resurrect each other hourly.

Refs #1194, #1195
2026-10-10 12:59:25 +02:00
Marc-André Bombeck 34f847f12f fix(integrations): anchor synced daily totals at local noon in the user's zone
Withings, Fitbit, Google Health, Polar and Oura wrote each daily total and
daily summary at noon UTC of its date. From UTC+12 on (New Zealand all year,
Fiji, Tonga, Samoa) noon UTC is already the next local day, so every synced
step count, SpO2 or resting-pulse day showed up a day late. The rows now sit
at local noon of their date in the user's zone, and the fetch windows run
through the user's own today.

Fitbit weigh-ins carry a local wall clock without an offset and were read
as UTC; they are read in the user's zone now.

ExternalIds stay on the calendar date, so the next sync finds each row and
moves it in place instead of adding a second one, and the day a row leaves
is refolded. Rows inside each integration's re-read window heal on the next
sync; Fitbit and Google Health re-read the whole history on reconnect.
2026-09-26 15:58:02 +02:00
Marc-André Bombeck 62eb431d66 release: v1.32.0 — hardening and scalability (#580)
* release: prepare v1.32.0

* test: stabilize water capture re-entry
2026-07-22 04:14:15 +02:00
Marc-André Bombeck 9c63d3ce20 chore(release): v1.28.25 — every integration held to the same standard
Platform-wide hardening: the failure classes found live this week hunted down
across every integration.

Fitbit carried the same tombstone wedge fixed for Google — probe now matches
soft-deleted rows and the update resurrects them; the comment describing a
partial unique index that never existed is corrected. Sleep segment ids are
stable across re-scoring for Withings (session+segment-start), WHOOP/Polar
(one segment per stage, no positional index) and Oura (run-start keyed), each
with a session-scoped sweep that soft-deletes rows a re-score orphaned — so
legacy and affected nights heal as they are re-read.

One platform rule for deleted rows: source-owned readings resurrect on
re-import (Withings, WHOOP, Oura, Polar, Nightscout, mood webhook, CSV
re-imports, Apple stats day-totals); Apple sample-grain deletions stick per
the paired-client contract, and the mood bulk route's tombstone match is now
a true no-op reported as duplicate. Import counts reconcile against actual
createMany counts.

Google lifecycle: a dead token registers on the cycle's hard-fail ledger, so
an expired connection parks instead of stamping success hourly; backfill and
sleep-repair markers stamp only after a clean pass (pg-boss retries are now
real); failed measurement writes hold the watermark; no-op updates are
skipped; mapping.md tells the current truth.

Scale: doctor-report reads only consumed columns and enabled sections;
exports and backups keyset-paginate; achievements tally via SQL buckets;
series charts day-bucket dense kinds beyond 90 days; BP pairing is a linear
two-pointer; distinct-type scans run server-side; the graded-series cold
fallback is bounded; the fitbit defer tracker loops instead of spreading.
2026-07-11 01:54:12 +02:00
Marc-André Bombeck 1212ed91d5 chore(release): v1.28.21 — sleep history heals itself; plateau context; sync diagnostics
A one-time background pass re-reads each connected account's Google Health
sleep history so nights stored twice before the sleep fix collapse to their
true total (new sleep_repaired_at marker, migration 0238; watermark-safe,
enqueued once at boot per un-repaired connection).

The sleep night read reports when two writers disagree about the same night —
the served total is unchanged, the disagreement rides along as an annotation
(sourceDiscrepancy on the night DTO) so clients can mark the number instead of
silently picking a side.

A GLP-1 medication whose weight has plateaued on the current dose says so next
to the estimated drug level (efficacy tab + insights) — a factual note from the
existing detector, association only, verb-guard covered.

Google Health diagnostics: the sync log now separates a daily-totals read that
returned nothing from one the importer could not parse
(googleHealth.rollup.emptyResponse vs droppedAll), and the connection test's
structure probe reports the raw response envelope key names when a rollup type
parses to zero — a per-account naming drift becomes visible from one report.
2026-07-10 21:23:00 +02:00
Marc-André Bombeck 295a6a8050 fix(google-health): parse the camelCase v4 payloads and correct the read methods
The v4 client read none of its documented response shapes: the DataPoint
value is a camelCase union keyed by the camelCase type name with camelCase
nested times, proto3 int64 fields arrive as JSON strings, and four fetchers
targeted wrong data types or illegal filter fields. Every HTTP-200 type
parsed zero measurements and four fetchers returned HTTP 400.

- Read every value/time path in camelCase; keep snake_case only inside the
  filter parameter. Pin path/filter/payload-key triples per data type.
- Coerce int64 JSON strings in every numeric extractor.
- weight.weightGrams -> kg, height.heightMeters -> cm,
  distance millimetres -> m, bodyFat.percentage, heartRate.beatsPerMinute.
- Move SpO2, HRV, respiratory rate and VO2 max to their daily-* data types
  (the bare sample types reject a .date filter).
- Move steps, distance, active energy and floors to POST :dailyRollUp with
  windowSizeDays 1 (list returns minute-grain buckets, not daily totals;
  floors has no list method), chunked at 90 civil days, day-keyed on
  civilStartTime.date at the UTC-midday anchor.
- Filter sleep on interval.end_time (the only legal sleep time field) and
  anchor sessions on interval.endTime; filter exercise on
  interval.civil_start_time with an offset-less civil bound in the user's
  zone; read exercise metrics from metricsSummary and take the workout id
  from the DataPoint name.
- Seed height from max(sampleTime) explicitly - list order is descending,
  so last-row-wins picked the oldest sample.
- Rewrite the mapper fixtures to the documented shapes.
2026-07-05 13:58:50 +02:00
Marc-André Bombeck 422a6a7df0 fix(google-health): correct day-keying, timezone, and skin-temperature handling
Tighten the transport so device data lands on the right day and in the
right slot.

- Cumulative daily totals (steps, distance, energy, floors) now key their
  measuredAt and stats:<tag>:<day> external id off the interval's civil
  start date anchored at UTC-midday, not the physical instant. Keying off
  the physical instant put a positive-UTC-offset user's day one calendar
  day early and split the same local day across the Google, Apple, and
  Fitbit sources so neither won the per-day source pick.
- Thread the user's timezone through the sleep, workout, and interval
  parses so an offset-less wall-clock timestamp resolves in the user's
  zone rather than the process zone — the same fix the classic Fitbit
  sleep path already carries.
- Drop skin temperature from the launch set. Google reports a signed
  nightly deviation from baseline, not an absolute reading, so mapping it
  into the absolute wrist-temperature slot stored a delta as a
  temperature and dropped every cold-night negative. It returns once a
  signed-delta model exists.
- Remove the unused daily-rollup read path and field-map, and the
  experimental ECG and irregular-rhythm scopes that had no reader — the
  connect flow now requests exactly the four core scopes it uses.
2026-07-03 14:41:44 +02:00
Marc-André Bombeck fb17a4e9c6 feat(google-health): add the Google Health API transport
Add the transport that reads a user's device data from the Google Health
API (health.googleapis.com/v4): Google OAuth with PKCE and offline
access, the paginated dataPoints reader with a daily-rollup fallback for
the types that lack a list method, the metric/activity/sleep/workout
mappers onto the shared measurement shape, and a response classifier
that separates retryable errors, hard rejects, and the re-consent case.

Access tokens live an hour and refresh tokens are reused, not rotated. A
refresh that comes back invalid_grant raises a distinct reauth-required
signal so the app can prompt a fresh consent rather than fail silently —
the periodic re-consent a Testing-mode Google client needs. The four
core readonly scopes ship by default; the Pixel Watch ECG and
irregular-rhythm scopes stay behind an opt-in flag.

Only the transport is new: the mapped-measurement output, the
upsert/dedup/rollup tail, and the source ladders are the shared ones.
2026-07-03 14:13:16 +02:00