Every ingest path in the product can now emit a typed, salience-classified
event into pg-boss, consumed by one worker that claims a durable per-day
reaction marker and fans out to the reaction surfaces.
The recency classifier runs at the seam, before anything is enqueued, so a
mass backfill — a multi-year Apple Health export, a provider re-sync, a
30-day catch-up — emits nothing at all. Non-salient writes are annotated and
dropped.
The worker makes no provider call by construction: it imports no provider
module, writes only markers, and enqueues nothing that is not budget-gated
downstream. Business refusals return a status rather than throwing, so a
refusal is never retried against a ceiling that will not move until the local
day rolls over.
Seams: sleep (one hook, inherited by all seven transports), workouts (the
batch route, the four provider syncs, the export importer), weight and blood
pressure (both measurement routes), labs (manual, OCR commit, document fact
commit).
The provider workout syncs could not previously tell a created row from an
updated one, so a re-sync of a recent session would have re-fired on every
poll; they now recover the distinction from the row the upsert already
returns.
Reaction lines are AES-256-GCM at rest, following the InsightNarrative
precedent, and the column is registered for key rotation. Markers are pruned
after 14 days by a daily maintenance job.
Platform-wide hardening: the failure classes found live this week hunted down
across every integration.
Fitbit carried the same tombstone wedge fixed for Google — probe now matches
soft-deleted rows and the update resurrects them; the comment describing a
partial unique index that never existed is corrected. Sleep segment ids are
stable across re-scoring for Withings (session+segment-start), WHOOP/Polar
(one segment per stage, no positional index) and Oura (run-start keyed), each
with a session-scoped sweep that soft-deletes rows a re-score orphaned — so
legacy and affected nights heal as they are re-read.
One platform rule for deleted rows: source-owned readings resurrect on
re-import (Withings, WHOOP, Oura, Polar, Nightscout, mood webhook, CSV
re-imports, Apple stats day-totals); Apple sample-grain deletions stick per
the paired-client contract, and the mood bulk route's tombstone match is now
a true no-op reported as duplicate. Import counts reconcile against actual
createMany counts.
Google lifecycle: a dead token registers on the cycle's hard-fail ledger, so
an expired connection parks instead of stamping success hourly; backfill and
sleep-repair markers stamp only after a clean pass (pg-boss retries are now
real); failed measurement writes hold the watermark; no-op updates are
skipped; mapping.md tells the current truth.
Scale: doctor-report reads only consumed columns and enabled sections;
exports and backups keyset-paginate; achievements tally via SQL buckets;
series charts day-bucket dense kinds beyond 90 days; BP pairing is a linear
two-pointer; distinct-type scans run server-side; the graded-series cold
fallback is bounded; the fitbit defer tracker loops instead of spreading.
The v4 client read none of its documented response shapes: the DataPoint
value is a camelCase union keyed by the camelCase type name with camelCase
nested times, proto3 int64 fields arrive as JSON strings, and four fetchers
targeted wrong data types or illegal filter fields. Every HTTP-200 type
parsed zero measurements and four fetchers returned HTTP 400.
- Read every value/time path in camelCase; keep snake_case only inside the
filter parameter. Pin path/filter/payload-key triples per data type.
- Coerce int64 JSON strings in every numeric extractor.
- weight.weightGrams -> kg, height.heightMeters -> cm,
distance millimetres -> m, bodyFat.percentage, heartRate.beatsPerMinute.
- Move SpO2, HRV, respiratory rate and VO2 max to their daily-* data types
(the bare sample types reject a .date filter).
- Move steps, distance, active energy and floors to POST :dailyRollUp with
windowSizeDays 1 (list returns minute-grain buckets, not daily totals;
floors has no list method), chunked at 90 civil days, day-keyed on
civilStartTime.date at the UTC-midday anchor.
- Filter sleep on interval.end_time (the only legal sleep time field) and
anchor sessions on interval.endTime; filter exercise on
interval.civil_start_time with an offset-less civil bound in the user's
zone; read exercise metrics from metricsSummary and take the workout id
from the DataPoint name.
- Seed height from max(sampleTime) explicitly - list order is descending,
so last-row-wins picked the oldest sample.
- Rewrite the mapper fixtures to the documented shapes.
Tighten the transport so device data lands on the right day and in the
right slot.
- Cumulative daily totals (steps, distance, energy, floors) now key their
measuredAt and stats:<tag>:<day> external id off the interval's civil
start date anchored at UTC-midday, not the physical instant. Keying off
the physical instant put a positive-UTC-offset user's day one calendar
day early and split the same local day across the Google, Apple, and
Fitbit sources so neither won the per-day source pick.
- Thread the user's timezone through the sleep, workout, and interval
parses so an offset-less wall-clock timestamp resolves in the user's
zone rather than the process zone — the same fix the classic Fitbit
sleep path already carries.
- Drop skin temperature from the launch set. Google reports a signed
nightly deviation from baseline, not an absolute reading, so mapping it
into the absolute wrist-temperature slot stored a delta as a
temperature and dropped every cold-night negative. It returns once a
signed-delta model exists.
- Remove the unused daily-rollup read path and field-map, and the
experimental ECG and irregular-rhythm scopes that had no reader — the
connect flow now requests exactly the four core scopes it uses.
Add the transport that reads a user's device data from the Google Health
API (health.googleapis.com/v4): Google OAuth with PKCE and offline
access, the paginated dataPoints reader with a daily-rollup fallback for
the types that lack a list method, the metric/activity/sleep/workout
mappers onto the shared measurement shape, and a response classifier
that separates retryable errors, hard rejects, and the re-consent case.
Access tokens live an hour and refresh tokens are reused, not rotated. A
refresh that comes back invalid_grant raises a distinct reauth-required
signal so the app can prompt a fresh consent rather than fail silently —
the periodic re-consent a Testing-mode Google client needs. The four
core readonly scopes ship by default; the Pixel Watch ECG and
irregular-rhythm scopes stay behind an opt-in flag.
Only the transport is new: the mapped-measurement output, the
upsert/dedup/rollup tail, and the source ladders are the shared ones.
Marc-André Bombeck