Every ingest path in the product can now emit a typed, salience-classified
event into pg-boss, consumed by one worker that claims a durable per-day
reaction marker and fans out to the reaction surfaces.
The recency classifier runs at the seam, before anything is enqueued, so a
mass backfill — a multi-year Apple Health export, a provider re-sync, a
30-day catch-up — emits nothing at all. Non-salient writes are annotated and
dropped.
The worker makes no provider call by construction: it imports no provider
module, writes only markers, and enqueues nothing that is not budget-gated
downstream. Business refusals return a status rather than throwing, so a
refusal is never retried against a ceiling that will not move until the local
day rolls over.
Seams: sleep (one hook, inherited by all seven transports), workouts (the
batch route, the four provider syncs, the export importer), weight and blood
pressure (both measurement routes), labs (manual, OCR commit, document fact
commit).
The provider workout syncs could not previously tell a created row from an
updated one, so a re-sync of a recent session would have re-fired on every
poll; they now recover the distinction from the row the upsert already
returns.
Reaction lines are AES-256-GCM at rest, following the InsightNarrative
precedent, and the column is registered for key rotation. Markers are pruned
after 14 days by a daily maintenance job.
The classic 1.2 sleep log and the activities list emit local wall-clock
ISO strings without an offset. `parseLocalInstant` parsed them with a bare
`new Date(iso)`, which reads an offset-less string in the process zone (UTC
in production). A non-UTC user's sleep-segment `measuredAt` (the segment
end) was therefore shifted by their UTC offset, and a near-midnight segment
could flip onto the wrong wake-day in the night reconstruction.
Resolve offset-less timestamps against the user's stored timezone via a new
`zonedWallClockToUtc` helper (the inverse of `wallClockInTz`, DST-safe via
the same two-pass converge as `startOfLocalDayInTz`). Thread the resolved
zone through the sleep and workout sync paths. Strings that already carry an
offset or `Z` stay absolute and parse as-is; an absent zone keeps the prior
host-local fallback.
The shipped Fitbit integration spoke the Google Health API, whose Restricted
scopes need brand verification plus an annual CASA assessment before leaving
testing mode — a wall most self-hosters cannot clear. Retarget the transport to
the classic api.fitbit.com Web API: a self-hoster registers an app at
dev.fitbit.com in minutes, no CASA.
Forks only the transport; the upsert/dedup/rollup tail, source-priority,
backfill, poll worker, and status ledger are reused unchanged.
- OAuth: www.fitbit.com/oauth2/authorize + api.fitbit.com/oauth2/token, PKCE
(S256), classic self-serve scopes.
- Refresh tokens rotate (one-time use): persist the new token every refresh,
with a defensive keep-existing guard on a malformed reply.
- Per-endpoint date-range fetchers chunked to the API range caps; the backfill
horizon and poll concurrency are tuned for the 150 req/h-per-user budget.
- Verified classic-shape extractors for weight, body fat, SpO2, HRV (dailyRmssd),
resting HR, respiratory rate, steps, distance, active calories, floors, VO2 max
(range midpoint), per-segment sleep stages, and workouts.
- Skips intraday HR (needs a per-app grant; daily resting HR covers it) and skin
temperature (a baseline delta with no honest absolute slot).
Use FITBIT_ACTIVITY_PAGE_SIZE for the sleep and exercise reads instead of a
magic 25, remove the unused FitbitDataTypeKey type, and declare the crosstab
day floors independently so they aren't a duplicate re-export of the influence
floors.
Snapshot the incremental watermark once per sync cycle and thread the same
lower bound to every resource (metrics, activity, sleep, workout), stamping
lastSyncedAt once at the end. Previously each resource read lastSyncedAt then
stamped now() independently, so the first resource advanced the watermark and
later resources only fetched the last overlap window — silently dropping the
gap after an outage longer than the overlap. The full/backfill path is
unaffected (it passes no lower bound).
Reject an OAuth callback whose token response carries no refresh_token instead
of persisting an empty string. An empty refresh token bricks every future
token refresh; the callback now errors out cleanly so the user re-consents
rather than landing a dead connection.
Model the INTERVAL data types (steps/distance/calories/floors daily totals,
sleep and exercise sessions) on Google Health's interval.start_time /
interval.civil_start_time anchor instead of a sample_time / bare date. The
incremental filter for sleep/exercise previously targeted
sample_time.physical_time, which 400s/empties for interval types and stalls
incremental sync. Daily-summary metrics and spot readings keep their existing
date / sample anchors. Shape mismatches still fail safe (return []/null).
Mark the Google Health (Fitbit & Pixel) settings card as experimental with a
badge and a short note that data coverage is still being verified, in all six
locales.
(cherry picked from commit 3f116802149eebe9144729e2779f7f4405f248f2)
Extend the Fitbit / Google Health provider with the remaining sync
resources, driven by the existing syncUserFitbit orchestrator.
Activity (daily cumulative): steps -> ACTIVITY_STEPS, distance ->
WALKING_RUNNING_DISTANCE, the active-energy portion -> ACTIVE_ENERGY_BURNED,
floors -> FLIGHTS_CLIMBED, and VO2 max -> VO2_MAX. Each daily total mints a
stats:<tag>:<YYYY-MM-DD> externalId so a re-fetched day overwrites in place
rather than duplicating, matching the Apple-Health daily-total contract. The
running totals preserve a legitimate zero (a rest day is real data); VO2 max
stays strictly positive (daily latest-wins).
Sleep: map a session's per-stage segments to per-stage SLEEP_DURATION rows
(minutes; measuredAt = the stage end), harmonised onto the shared SleepStage
enum (light -> CORE, deep -> DEEP, rem -> REM, awake/restless -> AWAKE,
in-bed -> IN_BED, classic asleep -> ASLEEP) so the night-total and hypnogram
readers consume the same shape they already read for WHOOP and Apple.
Workouts: map exercise sessions to Workout rows keyed (userId, FITBIT,
externalId), with sport-type, start/end, duration, energy, distance, and HR.
Cross-source twins stay distinct at ingest; the read-time canonical picker
collapses them via the source ladder (Fitbit already ranked below WHOOP).
All three join the orchestrator's resource list and ride the per-resource
403 soft-skip so an independently-granted Restricted bundle that the user
withheld skips only its own resource. Rollups recompute and status-insight
caches invalidate through the shared upsert. No new MeasurementType, no
migration, no new dependency.
Marc-André Bombeck