Withings, Fitbit, Google Health, Polar and Oura wrote each daily total and
daily summary at noon UTC of its date. From UTC+12 on (New Zealand all year,
Fiji, Tonga, Samoa) noon UTC is already the next local day, so every synced
step count, SpO2 or resting-pulse day showed up a day late. The rows now sit
at local noon of their date in the user's zone, and the fetch windows run
through the user's own today.
Fitbit weigh-ins carry a local wall clock without an offset and were read
as UTC; they are read in the user's zone now.
ExternalIds stay on the calendar date, so the next sync finds each row and
moves it in place instead of adding a second one, and the day a row leaves
is refolded. Rows inside each integration's re-read window heal on the next
sync; Fitbit and Google Health re-read the whole history on reconnect.
Fitbit's handleCollectionFetchError soft-skipped a 403 but RETHREW any
other hard failure, aborting every endpoint ordered after the bad one in
the same resource and stalling the watermark so the whole integration
stayed dead. Record hard failures on a per-cycle AsyncLocalStorage ledger
and return 0 instead of rethrowing — the sibling endpoints keep fetching,
and the ledger still fails the cycle so success is not stamped and the
next tick refetches. Also wrap the per-metric map fan-out in the same
per-collection catch so one malformed point can't skip its siblings.
The shipped Fitbit integration spoke the Google Health API, whose Restricted
scopes need brand verification plus an annual CASA assessment before leaving
testing mode — a wall most self-hosters cannot clear. Retarget the transport to
the classic api.fitbit.com Web API: a self-hoster registers an app at
dev.fitbit.com in minutes, no CASA.
Forks only the transport; the upsert/dedup/rollup tail, source-priority,
backfill, poll worker, and status ledger are reused unchanged.
- OAuth: www.fitbit.com/oauth2/authorize + api.fitbit.com/oauth2/token, PKCE
(S256), classic self-serve scopes.
- Refresh tokens rotate (one-time use): persist the new token every refresh,
with a defensive keep-existing guard on a malformed reply.
- Per-endpoint date-range fetchers chunked to the API range caps; the backfill
horizon and poll concurrency are tuned for the 150 req/h-per-user budget.
- Verified classic-shape extractors for weight, body fat, SpO2, HRV (dailyRmssd),
resting HR, respiratory rate, steps, distance, active calories, floors, VO2 max
(range midpoint), per-segment sleep stages, and workouts.
- Skips intraday HR (needs a per-app grant; daily resting HR covers it) and skin
temperature (a baseline delta with no honest absolute slot).
Three hardening + performance fixes to the Fitbit/Google Health sync.
Stop soft-delete resurrection. The measurement write probed the existing
row by the `(userId, type, source, externalId)` unique key without excluding
soft-deleted rows, so a reading the user deleted came back on the next hourly
sync (the upsert matched the tombstone and took the update branch — silently
undoing the delete). The write now probes only live rows (`deletedAt: null`)
and treats a tombstone as absent, minting a fresh insert instead. The fresh
insert relies on the partial unique index over `deleted_at IS NULL` to keep
live-row uniqueness; the tombstone sits outside it, and `skipDuplicates` guards
a live-row race.
Batch the backfill writes + collapse the rollup recompute. The write loop
was one upsert per reading (N+1) followed by a per-(type,day) DAY-rollup
recompute, so a multi-year backfill paid thousands of serial round-trips on a
concurrency-1 worker. The existence probe is now a single `findMany` over the
batch; fresh rows go through chunked `createMany`; only already-live rows take a
per-row update for their differing values. On a `fullSync` the per-write rollup
hook is deferred and the touched type-days collapse into one
`recomputeUserRollups(from, to)` pass at the end of the cycle. The incremental
path keeps the inline per-day hook. The `stats:` overwrite contract, the
single-watermark `markSynced`, and the all-403 soft-skip guard are unchanged.
Bound the hourly poll concurrency. The poll synced every connection in one
serial loop, so a single slow Google response stalled the whole cohort. The
cohort now fans out through a bounded `p-limit` pool with per-user error
isolation, extracted into `runFitbitPollCohort` so the contract is unit testable
without exporting worker internals. The accumulators are read after the await to
avoid a compound-assignment lost-update across overlapping pool tasks.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 8369af2471fb8da04a938b5a784433318d81a248)
Snapshot the incremental watermark once per sync cycle and thread the same
lower bound to every resource (metrics, activity, sleep, workout), stamping
lastSyncedAt once at the end. Previously each resource read lastSyncedAt then
stamped now() independently, so the first resource advanced the watermark and
later resources only fetched the last overlap window — silently dropping the
gap after an outage longer than the overlap. The full/backfill path is
unaffected (it passes no lower bound).
Reject an OAuth callback whose token response carries no refresh_token instead
of persisting an empty string. An empty refresh token bricks every future
token refresh; the callback now errors out cleanly so the user re-consents
rather than landing a dead connection.
Model the INTERVAL data types (steps/distance/calories/floors daily totals,
sleep and exercise sessions) on Google Health's interval.start_time /
interval.civil_start_time anchor instead of a sample_time / bare date. The
incremental filter for sleep/exercise previously targeted
sample_time.physical_time, which 400s/empties for interval types and stalls
incremental sync. Daily-summary metrics and spot readings keep their existing
date / sample anchors. Shape mismatches still fail safe (return []/null).
Mark the Google Health (Fitbit & Pixel) settings card as experimental with a
badge and a short note that data coverage is still being verified, in all six
locales.
(cherry picked from commit 3f116802149eebe9144729e2779f7f4405f248f2)
Extend the Fitbit / Google Health provider with the remaining sync
resources, driven by the existing syncUserFitbit orchestrator.
Activity (daily cumulative): steps -> ACTIVITY_STEPS, distance ->
WALKING_RUNNING_DISTANCE, the active-energy portion -> ACTIVE_ENERGY_BURNED,
floors -> FLIGHTS_CLIMBED, and VO2 max -> VO2_MAX. Each daily total mints a
stats:<tag>:<YYYY-MM-DD> externalId so a re-fetched day overwrites in place
rather than duplicating, matching the Apple-Health daily-total contract. The
running totals preserve a legitimate zero (a rest day is real data); VO2 max
stays strictly positive (daily latest-wins).
Sleep: map a session's per-stage segments to per-stage SLEEP_DURATION rows
(minutes; measuredAt = the stage end), harmonised onto the shared SleepStage
enum (light -> CORE, deep -> DEEP, rem -> REM, awake/restless -> AWAKE,
in-bed -> IN_BED, classic asleep -> ASLEEP) so the night-total and hypnogram
readers consume the same shape they already read for WHOOP and Apple.
Workouts: map exercise sessions to Workout rows keyed (userId, FITBIT,
externalId), with sport-type, start/end, duration, energy, distance, and HR.
Cross-source twins stay distinct at ingest; the read-time canonical picker
collapses them via the source ladder (Fitbit already ranked below WHOOP).
All three join the orchestrator's resource list and ride the per-resource
403 soft-skip so an independently-granted Restricted bundle that the user
withheld skips only its own resource. Rollups recompute and status-insight
caches invalidate through the shared upsert. No new MeasurementType, no
migration, no new dependency.
Marc-André Bombeck