Work a request starts and does not wait for runs inside the request's
context, so an annotate({ action }) from it landed on the request's event
whenever it ran before the event was emitted. The measurement tail's
status re-warm logged a hand-entered reading as
insights.status.invalidate.skipped on some requests and as
measurement.create on others.
runDetached starts such work so that its action is dropped for the event
it was detached from; meta, warnings and the auth it reads stay as they
were, and an event the work opens for itself keeps its own action.
fireAndForget now takes a function and starts it detached, and every
trackBackgroundTask handle on a request path is built with runDetached,
which a structural guard holds.
The backfill, the sleep repair, the OAuth-state cleanup and the sync
handlers move from lib/jobs into lib/fitbit/jobs, beside the queue facts
and definitions the registry already spreads. Their tests move with them.
Removing the integration is now the folder, its routes, the registry
spread and the drain's two cases.
The updatedAt backfill guard also walks provider jobs folders, so the
moved backfill stays covered.
`integrations/sync-kit/after-write.ts` is the tail every provider write
ends in: the arrivals go out, the touched days are refolded, the read
caches are marked stale back to the earliest touched reading, and the
touched status cards are re-warmed (through `afterMeasurementMutation`,
shared with the interactive routes); Withings also evicts the dashboard
snapshot. A full-history backfill refolds its whole touched range once
through `afterProviderBackfill`.
Withings, WHOOP, Polar, Nightscout, Google Health, Fitbit and Oura drop
their hand-written tails, and no provider module imports the comprehensive
insight generator any more; a guard holds the provider folders to that.
Differences from the copies: the status cards are re-warmed even when the
fold failed, and the caches are marked stale back to the earliest touched
reading instead of wholesale, so older closed blocks survive a sync.
Several provider unit tests mocked an export the old tails never reached
(their failures were caught and only logged); the mocks now name what the
code calls.
`oauth-kit/client.ts` builds a provider's consent URL, code exchange,
refresh and deauthorization from a declaration of what differs: endpoints
and scope, client authentication (form or HTTP Basic, with or without the
client id beside it), the fields a grant adds, how an answer is classified
and the refusal named, the error type, and where the token sits in the
body. The seven hand-written copies of the redirect URI, authorization URL,
token post, Basic header and PKCE pair are gone; each provider keeps an
`oauth.ts` with its declaration.
Shared behaviour is now uniform: a 2xx without a token object (an empty or
non-JSON body) is a transient `empty_token_body` failure for every
provider. WHOOP, Strava and Google Health used to return null, and
Withings threw an unclassified SyntaxError; the OAuth wire golden records
exactly these lines changing and nothing else. Oura's live/legacy endpoint
fallback, Google's invalid_grant lift, Strava's Fault summary and Fitbit's
errorType are declared per provider.
The five hand-written registrars (about 5,500 lines, one 997-line function)
are replaced by `jobs/registry/`: one `defineJob` entry per queue with its
cron, schedule send options, policy, work options, lock, lane and a handler
loader. `registerJobGroup` creates, schedules, retires and binds each group
in the order the registrars used; the worker boot walks the groups and runs
their boot discovery.
Handlers load on their queue's first job. The facts a declaration needs at
boot (names, crons, concurrency, send options) moved out of the heavy job
modules into light `*-queue.ts` siblings, and their importers point there,
so loading the registry evaluates 288 modules instead of the 1,164 the
registrars pulled in. The formerly inline handlers moved verbatim into
handler modules under `jobs/reminder/`.
Fitbit's queues live in `fitbit/jobs/definitions.ts` and enter the registry
through one spread, so removing the integration later is that folder and
that line.
`tests/integration/queue-registration-parity.test.ts` proves pg-boss holds
exactly what the registrars left (queues, schedules, bindings, byte for
byte) and loads every handler once. The queue tests and the runtime guard
assert on the registry objects instead of reading registrar source.
After the Web API turndown every account still saw a Fitbit card that
can no longer connect. The status entry now publishes
retirement.offered: false once the API is gone for an account with no
Fitbit connection, no Fitbit app of its own and no Fitbit data, and the
connections panel leaves the card out. Accounts that had Fitbit keep it
with the retirement notice.
Google turns the legacy Fitbit Web API off on 30 October 2026 (support
ended on 30 September). From that day the hourly poll, the backfill,
the sleep repair and the token refresh make no call, the connect, sync
and test routes refuse with `provider_retired`, and the connection
reads as not connected to the health read and the watchdog, so the
shutdown pages nobody and asks nobody to reconnect. The connection row
and everything it imported stay.
The status envelope publishes `retirement` on the Fitbit entry (the
day, whether it has come, the successor), and the Fitbit card shows
when the connection stops, how to move to Google Health (same devices,
one more consent, tokens cannot be carried over) and that the data
stays; sync and connect grey out after the day. Both integrations can
run side by side meanwhile: every type Fitbit writes is laddered with
Fitbit just above Google Health, so a day both deliver counts once.
The test setups pin the turndown off, so the Fitbit suites do not turn
red on the day; the turndown's own tests flip it.
Fitbit's `dailyRmssd` and Google Health's daily HRV average are both
RMSSD, but the mappers wrote them into HEART_RATE_VARIABILITY, the SDNN
series. The `hrv` ladder ranks both above Apple Health, so an account
with an Apple Watch and a Fitbit had a series that switched estimator
from day to day. The mappers now write HRV_RMSSD, and the `hrvRmssd`
ladder carries both sources where they sat on the SDNN one.
Migration 0420 moves the rows already written (tombstones included,
with a sync-version bump so the change reaches the phone), drops their
SDNN rollup buckets and the RMSSD buckets of every account whose rows
moved, so the boot-time rollup discovery folds those accounts again,
and deletes the SDNN personal records those rows had set. One DO
block, so the move and its cleanup land together.
Every provider sync and the shared mutation tail refolded its touched
(type, day) buckets one at a time: an aggregate, a transaction and three
queue sends per day. They now hand the whole set to
recomputeBucketsForKeys, which leaves the DAY tier exactly as the per-key
hook did (pinned by a parity test against it) at the cost of one
aggregate per chunk of keys and one coarse job per run of buckets.
Output of `node scripts/strip-planning-tags.mjs --write`: 3,157 comment
lines in 1,308 files lose the work-item identifier at their head. Release
versions and issue references stay; nothing outside comment text
changes, which the script checks per file against a comment-free
printout. healthlog/no-planning-tags is enabled at error level with this
commit, so the tree it lands on is clean.
The WHOOP, Oura and Withings list walks and the Fitbit exercise walk each
kept a page loop that, at its ceiling or on a cursor that did not advance,
returned what it had read and said nothing, and the backfill around it
stamped the history complete. `walkPages` keeps a ceiling but reports a
walk that stops with a page pending, or on a cursor it already followed,
as truncated. A backfill reads that through one verdict: a failure
retries, a truncated collection leaves the marker unset without a retry,
a clean run stamps it. The Fitbit exercise walk may read a hundred pages
during a backfill instead of twelve. A guard keeps page-counting loops out
of the provider directories, with Google Health and Strava listed until
their next change.
Fitbit, WHOOP, Oura, Polar and Withings each kept a private
`parseFloat(n.toFixed(2))`, which mis-rounds halves on the double below
the decimal. `importValue` keeps the same two places, rounds half away
from zero on the printed decimal and returns the nearest double, so a
value prints cleanly. A guard keeps `toFixed` and private `round2`
helpers out of the provider directories; the Google Health mapper is the
one listed exception until its next change.
A sync from a watch or an integration now waits the instance's sync gap
(AppSettings.aiSyncRegenGapHours, else AI_SYNC_REGEN_GAP_HOURS, else 6 h)
before it may regenerate a status card again; a hand entry keeps the hour.
Every caller of afterMeasurementMutation and invalidateStatusInsightsForTypes
names its source, so a new ingest path cannot skip the choice.
Refs #1217
`instant.toISOString().slice(0, 10)` and parsing a key back as UTC
midnight are the idioms behind every day-boundary fix this year: the
evening west of UTC and the small hours east of it land on the
neighbouring day. A new rule, healthlog/no-utc-day-key, refuses them in
src/ outside src/lib/tz and tests, and refuses a disable directive for
itself that is not next-line or gives no reason.
The time zone module now names every legitimate form: userDayKey for a
person's day, dateOnlyKey / dateOnlyAtNoonUtc / dayKeyAsUtcMidnight for
date-only values and day labels, daysBetweenDateKeys / weekdayOfDateKey /
isCalendarDateKey / statedDateKey for key arithmetic and parsing.
Of the 133 sites the rule found, 84 are gone. Most moved onto a named
helper with no change in behaviour; these changed behaviour:
- cycle calendar, insights, Coach snapshot and prediction refresh start
their lookbacks at the first instant of the user's day
- a model-transcribed date like "June 10, 2026" no longer lands a day
early in a process east of UTC (documents and lab scans)
- a custom report range covers exactly the picked days in the profile
zone instead of starting the evening before west of UTC
- date-of-birth pickers cap at the user's today, not the UTC one
- an allergy onset date is stored at noon UTC like other date-only values
- the report's cycle summary, the Coach's usual BP range and the intake
projector's rollup recompute use the user's day
- vaccination dates render as dates instead of the previous evening
49 remain, each with its reason on the line above: 39 UTC by design (15
file name stamps, rollup and object keys, the UTC-day score engines) and
10 marked baseline. A guard test requires a classified reason on every
directive and holds the baseline to a ceiling that only goes down.
A full-history backfill read every heart-rate point of an account into one
array, mapped it into a second, and wrote the lot in one call; the cycle's
deferred-rollup ledger then kept one entry per written reading. On a watch
that records a reading a minute that is over a million points, and the
backfill ran a 1 GB heap out of memory within seconds, restarting the
instance it shares with the app (#1023).
The metric walk now hands each page to its write before requesting the
next, and both the Google Health and Fitbit ledgers collapse to one key per
touched type-day as they record. The heart-rate walk's page ceiling rises
from 1000 to 10000: at one point a minute the old ceiling is under two
years, a longer history always ended truncated, and the backfill retried
the whole walk on every attempt and every boot without completing.
The seven integration clients now read their redirect URI and the app URL
through envValue/envOr instead of a local ?.trim() ||. The empty-env guard
drops its pending entry and floors its walk, and proves its matcher still
fires on a real client with the old read put back.
The compose file passes every whitelisted variable through, so an unset
redirect URI reaches the process as an empty string. Withings then sent
redirect_uri= to the provider, and Fitbit and Google Health refused to start
the connection as not configured although NEXT_PUBLIC_APP_URL was set. Every
integration now treats an empty or blank value as unset and derives the
callback from the app URL.
Withings, Fitbit, Google Health, Polar and Oura wrote each daily total and
daily summary at noon UTC of its date. From UTC+12 on (New Zealand all year,
Fiji, Tonga, Samoa) noon UTC is already the next local day, so every synced
step count, SpO2 or resting-pulse day showed up a day late. The rows now sit
at local noon of their date in the user's zone, and the fetch windows run
through the user's own today.
Fitbit weigh-ins carry a local wall clock without an offset and were read
as UTC; they are read in the user's zone now.
ExternalIds stay on the calendar date, so the next sync finds each row and
moves it in place instead of adding a second one, and the day a row leaves
is refolded. Rows inside each integration's re-read window heal on the next
sync; Fitbit and Google Health re-read the whole history on reconnect.
The v1.38.0 notice that warns when the configured OAuth callback origin
differs from the origin the browser is on never rendered in the published
image, and the copyable callback address in the redirect guide showed a
bare path. `integrationCallbackUrl()` read `process.env.NEXT_PUBLIC_APP_URL`
inside a "use client" module. Next.js inlines NEXT_PUBLIC_* into client
bundles at build time, and the GHCR image is built without that variable
(only the version, build-sha and built-at args reach the build), so the
base compiled to an empty string, `new URL("/api/<provider>/callback")`
threw, and the notice returned null on every card for every self-hoster.
The value now comes from the server at request time. A new map
(`src/lib/integrations/callback-urls.ts`) calls each provider's own
redirect-URI function (the four private `getRedirectUri()` helpers are
exported under provider-specific names; Polar, Oura and Strava already
exported theirs), so the page shows exactly what the handshake registers,
explicit `*_REDIRECT_URI` override included. The settings route resolves
the map and passes it as a prop through `IntegrationsSection` and
`ConnectionsPanel` to every card. Fitbit and Google Health validate their
URI and throw on a bad configuration; that surfaces as `null`, the notice
skips the comparison and the guide falls back to the fixed path.
Tests: a map test proving the explicit override wins and the derived form
is used otherwise, a component test rendering the notice from a passed
value (mismatch renders, match does not), and a structural guard that
fails when any "use client" file under src/components/settings/integrations
reads NEXT_PUBLIC_APP_URL.
Integration syncs (Fitbit, Oura, Polar, WHOOP, Strava, Nightscout) now
mark the per-user analytics cells stale and sweep the workouts cache at
their sync tails — imported rows used to stay invisible to the cached
dashboard, analytics and workout readers until the TTL lapsed. The WHOOP
per-resource webhook/cron driver and the Polar workouts leg carry the
same call, matching the posture Google Health and the workout batch
route already had.
Illness, cycle and encounter write routes hard-evict the analytics
bucket through the new invalidateUserHealthContext helper: Rest Mode,
the cycle context and the Today-rail appointment feed all read from
cached cells under that bucket, and an interactive write must be true on
the very next read.
Every interactive path, every import, MCP and Telegram checked an incoming
measurement against VALUE_RANGES before storing it. The provider sync writers
did not, so a decode slip or a glitched sensor landed a row the application
itself calls impossible, and every mean, median and personal band derived from
it inherited the impossibility. Filtering the read side hid the row. It kept
landing.
The gate is one module reading the domain the app already declares, applied at
two kinds of seam. reconcileExternalMeasurement is the shared identity writer
for Polar, Oura, WHOOP, the Withings sleep leg, the Apple export importer and
the batch endpoint, so gating there means a provider added later inherits the
refusal rather than having to remember a check it never knew about; the new
rejected_range verdict carries no row, which is what makes the compiler force
every caller to handle it. The five writers that hold Prisma themselves gate at
their entry, before the existence probe decides insert versus overwrite, so an
impossible reading can no longer replace a good stored value.
An out-of-band value is dropped. Not clamped, because a clamp invents a reading
nobody took. Not stored behind a flag, because a quarantined row is still a row
and every consumer would have to learn the flag. Not fatal to the batch, because
one bad sample must not strand the run's other readings behind a held watermark.
Absence reads as absence.
A silent drop, though, is indistinguishable from a provider that sent nothing,
so every drop is tallied on the wide event under measurement.range_rejected with
the metric, the provider and the edge it broke. Counts only: the reading itself
is health data and never enters the log line, and the direction plus the
metric's own declared bound is what a diagnosis actually needs. The tally
accumulates rather than replacing, because a provider gates once per resource
leg and the reconciler once per row. One warning per provider lifts the event to
warn so a drop does not hide inside a green line.
A structural guard enumerates all twenty measurement writers with the reason
each is allowed to write, so a new one has to answer the question before it
ships. scripts/diagnose-implausible-measurements.ts reports the rows already
stored, aggregated per metric and source with no value beyond the bucket's own
extremes. It repairs nothing: whether a stored impossible row is a decode slip
or a band that is too narrow is not a call a script gets to make.
Mirrors the Google sleep suites name for name: the mapper keeps its externalIds
across a re-score that re-classifies and renumbers segments, the sweep stays
bounded to the night's window and never touches the fresh keep-set, a null
window issues no delete at all, and a cleanup failure never fails the sync.
The end-to-end fixture runs one night stored under the old keys through
`syncUserSleep` and pins the ordering the heal depends on — sweep first, then
the rescue re-keying the tombstoned rows in place, no parallel insert — plus a
second pass that writes nothing at all.
Also covers the repair's verdict gates (bounded 365-day window, marker stamped
only on a clean run, watermark untouched, dead token and reauth park returning
without a stamp), the unchanged-value skip including the re-classified segment
it must not swallow, and both Retry-After paths.
Fitbit re-scores a night after the fact, and the per-segment externalId carried
a positional index plus the stage label. Both moved on a re-score: renumbered
segments minted fresh keys, the previous scoring's rows stayed live because
nothing swept them, and the night total counted both copies from then on.
Key the segment on the stable session anchor plus its own start instant
(`<logId>:sleep:<segment-start>`), and add the replace-by-window sweep every
other segment-writing provider already has. The two ship together with the
natural-key rescue that has been sitting in the Fitbit upsert unused: the sweep
tombstones the old-keyed rows first, so the rescue re-keys those very rows in
place instead of inserting a parallel copy or losing the insert to
`skipDuplicates` against the natural unique index.
A one-shot repair queue re-reads the bounded sleep history once per connection
so no user has to do anything; it stamps `sleepRepairedAt` (migration 0270) only
on a clean run, never touches the sync watermark, and reports the rows it
removed. Nights older than the 365-day backfill horizon keep their old keys and
whatever duplicates they already hold — they can no longer gain new ones.
Alongside: `fitbitGet` now honours a `Retry-After` inside a 120 s cap by waiting
once and retrying once, and fails fast above it so a long hold cannot starve the
rest of the poll cohort or outlive a job's expiration. The measurement upsert
skips a write whose payload is unchanged, which takes the hourly overlap off the
delta feed for every Fitbit resource.
New sync-dead-token suite mirrors the Google Health parity test: the three
getValidToken null-return paths each register on the hard-fail ledger, a
successful refresh leaves it empty, and a dead-token cycle returns
{ imported: 0, failed: true } without stamping markSynced or recordSyncSuccess.
Extends the upsert suite with the two write-catch labels (measurements:create,
measurements:update) and the sync orchestrator suite with a write-label
watermark hold. Rewrites the backfill suite to mirror google-health-backfill:
a clean run stamps, a failed verdict throws without stamping, a parked
connection returns without running the sync.
A revoked or expired Fitbit refresh token used to fail the cycle's status
ledger and then stamp the same cycle a success: the connection flipped back
to connected, the watermark advanced, and after a reconnect the 24h overlap
could no longer reach the missed days. getValidToken's three null-return
paths (credentials missing, refresh failure, vanished connection) now register
on the per-cycle hard-fail ledger, so the cycle's verdict fails, the park at
error_reauth holds, and lastSyncedAt does not move. The two measurement write
catches do the same, so fetched-but-unwritten rows are re-fetched next tick
instead of aging out of the overlap.
syncUserFitbit now returns { imported, failed }; the manual-sync route, the
poll cohort adapter, and the backfill consume the new shape. The backfill
stamps backfillCompletedAt only after a clean full-history walk: a parked
connection returns without stamping, and a failed verdict throws so pg-boss
retries on the already-configured retry budget.
Merge the eight reviewed audit fixes that were cut a week ago and never
landed. Main has moved on since, so each fix was re-checked against the
current tree and each of the five conflicts resolved by understanding
both sides.
Kept, still open on main:
- Proxy early exits now carry the transport security header subset via a
shared helper. Reconciled with main's /api/health worker-only carve-out.
- Batch measurement ingest bounds the entry instant so a future-dated row
can no longer win "latest reading" or poison the rollup buckets.
- Web Push dials through the pinned safeFetch dispatcher with
requirePublicHost, closing the last user-controlled-host egress.
- Mood bulk upsert and its tag/factor links run in one transaction, so a
rejected rating rolls the mood row back instead of leaving a tagless row.
- Full backup carries NutrientIntakeDay with its unit and source. Folded
into main's portable-export / disaster-recovery split.
- Live daily-series fallback restores the min/max spread band the tiered
read already emitted.
- Coach stream aborts its in-flight reader on unmount and resets send state
on a conversation switch.
- Manual water writes share the day-bound helper, nutrient add clamps after
the increment, and the thiamin cap drops to a real supplement ceiling.
- Compliance point window widens symmetrically instead of wrapping overnight.
- Oura, Polar and Fitbit token helpers classify an empty or non-JSON 2xx
body as a handled transient error.
- WebAuthn relying-party fallback to localhost is gated to development.
- Chart-token strip list is derived from the allowlist and pinned by a test.
- Regression tests pin the snapshot-cache tenant key, the GlitchTip
path-secret redaction and the Withings credentials delete.
Superseded by main, tests re-pointed at the current shape:
- The intake CSV import was rewritten on main to run through a background
job with a single-active-job admission gate. The future-date bound is
re-applied at request validation and the per-user limiter is kept for
parity with the bulk twin; the old synchronous-write test is replaced.
The German prompt-injection test asserted only inside `if (decision.refuse)`,
so it stayed green whether or not the filter fired. Un-guard it, drive it from a
case table, and widen the two German patterns to cover the determiners that
walked straight past them ("saemtliche", "die", "deine").
The Oura / Polar / Fitbit token helpers cast a possibly-null `json` to the token
type on a status-only success gate. A 2xx with an empty or non-JSON body handed
callers a null they dereferenced, surfacing an unclassified TypeError instead of
a handled integration error. Narrow explicitly and classify as transient.
Every ingest path in the product can now emit a typed, salience-classified
event into pg-boss, consumed by one worker that claims a durable per-day
reaction marker and fans out to the reaction surfaces.
The recency classifier runs at the seam, before anything is enqueued, so a
mass backfill — a multi-year Apple Health export, a provider re-sync, a
30-day catch-up — emits nothing at all. Non-salient writes are annotated and
dropped.
The worker makes no provider call by construction: it imports no provider
module, writes only markers, and enqueues nothing that is not budget-gated
downstream. Business refusals return a status rather than throwing, so a
refusal is never retried against a ceiling that will not move until the local
day rolls over.
Seams: sleep (one hook, inherited by all seven transports), workouts (the
batch route, the four provider syncs, the export importer), weight and blood
pressure (both measurement routes), labs (manual, OCR commit, document fact
commit).
The provider workout syncs could not previously tell a created row from an
updated one, so a re-sync of a recent session would have re-fired on every
poll; they now recover the distinction from the row the upsert already
returns.
Reaction lines are AES-256-GCM at rest, following the InsightNarrative
precedent, and the column is registered for key rotation. Markers are pruned
after 14 days by a daily maintenance job.
The Today digest went stale after logging a dose: nothing invalidated the
`["daily","digest"]` query, and it reads meds from the same server snapshot
the intake routes already hard-evict — so the "dose due" rail item lingered
until a hard reload. Add the digest key to the medication-dependent bundle and
force the inactive digest (unmounted while the user is on the medication card)
to refetch alongside the dashboard snapshot; the server snapshot eviction seam
is reused, so the refetch returns post-write dose state immediately.
Remove the score-ring cluster from the Today hero — it read as uneven and
wasted tile space. The health-score ring is the hero's only ring now; the
score-ring selection contract stays server-side for the native client. Tighten
the hero's vertical rhythm so the worth-a-look rail sits close under the lead
instead of floating below a half-empty column.
Wire the remaining sleep transports into the morning-refresh trigger: Google
Health, Fitbit, Oura, and Polar now enqueue the debounced morning refresh on a
last-night segment landing, matching the Withings / WHOOP / Apple seams — those
sources had no trigger at all. Make the trigger observable: each early-return
gate now annotates a distinct `daily.morning_refresh.skipped` reason (the
not-last-night gate also carries the samples' resolved local dates, today's key
and the tz) so a real landing shows exactly which gate fired.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018V8V9qapgHi7fhgtMJ8SkM
Single-fix release mirroring the Google Health natural-key rescue from
v1.28.28 into the Fitbit sync. The measurements natural-key unique index
(user, type, measured_at, source, sleep_stage — NULLS NOT DISTINCT,
migration 0055) covers tombstoned rows, so a re-import carrying NEW
externalIds for previously-removed rows collided on the natural key and
skipDuplicates dropped the insert silently. Every planned create now gets
a second chunked probe by natural key with no deletedAt filter; a match —
live or tombstoned — is updated in place with the fresh value, the new
externalId, and deletedAt: null. Probe failure notes the hard-failure
ledger so the watermark holds and the next tick retries. No other code
path touched.
Platform-wide hardening: the failure classes found live this week hunted down
across every integration.
Fitbit carried the same tombstone wedge fixed for Google — probe now matches
soft-deleted rows and the update resurrects them; the comment describing a
partial unique index that never existed is corrected. Sleep segment ids are
stable across re-scoring for Withings (session+segment-start), WHOOP/Polar
(one segment per stage, no positional index) and Oura (run-start keyed), each
with a session-scoped sweep that soft-deletes rows a re-score orphaned — so
legacy and affected nights heal as they are re-read.
One platform rule for deleted rows: source-owned readings resurrect on
re-import (Withings, WHOOP, Oura, Polar, Nightscout, mood webhook, CSV
re-imports, Apple stats day-totals); Apple sample-grain deletions stick per
the paired-client contract, and the mood bulk route's tombstone match is now
a true no-op reported as duplicate. Import counts reconcile against actual
createMany counts.
Google lifecycle: a dead token registers on the cycle's hard-fail ledger, so
an expired connection parks instead of stamping success hourly; backfill and
sleep-repair markers stamp only after a clean pass (pg-boss retries are now
real); failed measurement writes hold the watermark; no-op updates are
skipped; mapping.md tells the current truth.
Scale: doctor-report reads only consumed columns and enabled sections;
exports and backups keyset-paginate; achievements tally via SQL buckets;
series charts day-bucket dense kinds beyond 90 days; BP pairing is a linear
two-pointer; distinct-type scans run server-side; the graded-series cold
fallback is bounded; the fitbit defer tracker loops instead of spreading.
Fitbit's handleCollectionFetchError soft-skipped a 403 but RETHREW any
other hard failure, aborting every endpoint ordered after the bad one in
the same resource and stalling the watermark so the whole integration
stayed dead. Record hard failures on a per-cycle AsyncLocalStorage ledger
and return 0 instead of rethrowing — the sibling endpoints keep fetching,
and the ledger still fails the cycle so success is not stamped and the
next tick refetches. Also wrap the per-metric map fan-out in the same
per-collection catch so one malformed point can't skip its siblings.
Reconciles the v1.26.1 Fitbit setup-guide fix with the Google Health
release: google-health.md stays the full runbook (with a pointer to the
classic fitbit.md guide), the doc-parity guard now asserts the Google
Health page against the google-health client's endpoints and scopes,
and the changelog carries both sections.
The Fitbit integration connects through the classic Fitbit Web API, so a
self-hoster registers a Fitbit developer app at dev.fitbit.com — but the
setup guide walked users through creating a Google Cloud OAuth client,
whose client id the Fitbit sign-in rejects (unauthorized_client). Rewrite
the guide for the dev.fitbit.com flow (app type, HTTPS callback, scopes,
PKCE), leave the old page as a pointer, and add a test that keeps the
documented scopes and endpoints in sync with the client so the guide
cannot drift from the code again. Flags the September 2026 sunset of the
classic Fitbit Web API.
Drop the unreachable text-only-model reason from the OCR capability
enum and its DTO type. Replace the literal NUL delimiter in the Fitbit
dedup map keys with a named constant so text tooling stops treating the
file as binary (behaviour unchanged). Correct the Coach tool-executor
docblock, which overstated cross-tool snapshot-cache sharing.
Fitbit, WHOOP, and Oura all issue one-time-use refresh tokens. The token
persistence did an unguarded read → refresh → write, so two overlapping
syncs for one user both read the same stored refresh token and both POST
it. The provider honours only the first; the loser's refresh comes back 401,
gets classified as reauth, and the connection is parked at `error_reauth`
spuriously even though the peer rotated the token successfully.
Add a shared `persistRotatedToken` helper that writes the rotated pair with
a compare-and-swap on the stored ciphertext (matched verbatim, since AES-GCM
re-encryption of the same plaintext differs byte-for-byte). When the
conditional update touches zero rows a peer already won the race; re-read the
row and reuse the peer's freshly rotated access token instead of erroring.
Wire it into all three providers — Fitbit and WHOOP `getValidToken`
(proactive expiry refresh, keyed by connection id) and Oura `storeOuraTokens`
(reactive 401 refresh, keyed by user id). Both racing callers now end with a
usable token and neither drops the connection.
The classic 1.2 sleep log and the activities list emit local wall-clock
ISO strings without an offset. `parseLocalInstant` parsed them with a bare
`new Date(iso)`, which reads an offset-less string in the process zone (UTC
in production). A non-UTC user's sleep-segment `measuredAt` (the segment
end) was therefore shifted by their UTC offset, and a near-midnight segment
could flip onto the wrong wake-day in the night reconstruction.
Resolve offset-less timestamps against the user's stored timezone via a new
`zonedWallClockToUtc` helper (the inverse of `wallClockInTz`, DST-safe via
the same two-pass converge as `startOfLocalDayInTz`). Thread the resolved
zone through the sleep and workout sync paths. Strings that already carry an
offset or `Z` stay absolute and parse as-is; an absent zone keeps the prior
host-local fallback.
The shipped Fitbit integration spoke the Google Health API, whose Restricted
scopes need brand verification plus an annual CASA assessment before leaving
testing mode — a wall most self-hosters cannot clear. Retarget the transport to
the classic api.fitbit.com Web API: a self-hoster registers an app at
dev.fitbit.com in minutes, no CASA.
Forks only the transport; the upsert/dedup/rollup tail, source-priority,
backfill, poll worker, and status ledger are reused unchanged.
- OAuth: www.fitbit.com/oauth2/authorize + api.fitbit.com/oauth2/token, PKCE
(S256), classic self-serve scopes.
- Refresh tokens rotate (one-time use): persist the new token every refresh,
with a defensive keep-existing guard on a malformed reply.
- Per-endpoint date-range fetchers chunked to the API range caps; the backfill
horizon and poll concurrency are tuned for the 150 req/h-per-user budget.
- Verified classic-shape extractors for weight, body fat, SpO2, HRV (dailyRmssd),
resting HR, respiratory rate, steps, distance, active calories, floors, VO2 max
(range midpoint), per-segment sleep stages, and workouts.
- Skips intraday HR (needs a per-app grant; daily resting HR covers it) and skin
temperature (a baseline delta with no honest absolute slot).
One-shot prettier pass so the new format:check gate starts from a clean
baseline. Formatting only — no behaviour change. Generator-owned and
vendored artifacts stay untouched via .prettierignore.
Add a reminder-satisfy pg-boss queue (mirrors pr-detection) that every
measurement and lab ingest path enqueues — manual create, batch, and the
Withings / Polar / Oura / Nightscout / Fitbit / WHOOP syncs. A reminder
now auto-resolves the moment a matching reading lands instead of waiting
up to 15 minutes. The 15-minute cron stays as the idempotent safety net;
the forward-only guard lets the eventful hook and the cron converge
without double-stamping.
The worker honours the module toggle (a reminder for a disabled module
produces no engine activity) and runs the same satisfaction primitives
the cron uses.
The Google sleep payload carries a real per-stage segment series, but the
mapper summed each stage's segments into one row stamped on the stage's
last end. The hypnogram then reconstructed all of a stage's time as a
single block ending at the night's right edge — the same stacked-bar
artefact as WHOOP, despite the source carrying true timing.
Emit one row per segment with measuredAt at that segment's end and an
indexed externalId so the several segments of one stage stay distinct
under the dedup key. The timeline is measured (real onsets), so these
rows are not flagged reconstructed.
Close the launch-blocking gap where the ConsentReceipt infrastructure was
built but never enforced as a precondition: a direct API caller with a valid
token could forward a health snapshot to a server-managed external LLM with no
receipt on file. Consent was enforced client-side only.
Add a consent guard (src/lib/ai/consent-guard.ts) and call it before the first
external-provider call on the server-managed path. The gate fires only when the
resolved chain could egress via the operator's global key (admin-openai); a
user's own BYOK key (openai/anthropic), their own ChatGPT OAuth account, and
the self-hosted local provider stay ungated — that egress is the user's own act
and the settings flow is its consent. The check fails closed: a BYOK-primary
chain with an admin-openai fallback still requires a receipt, since the runner
may cascade to the operator key. The surface maps to the consent kind (coach ->
ai_coach, insights -> ai_insights_only; master ai_full satisfies either).
- Interactive routes (insights/chat, insights/generate POST) throw
ConsentRequiredError; api-handler renders a 403 with
meta.errorCode = "consent.ai.required", mirroring assistant.disabled.* so
the client can render an inline grant-consent notice.
- Off-request pipelines fail closed quietly: comprehensive generation returns a
typed skipped:no-consent outcome, and the per-metric status family
(runStatusCompletion, gating every status card + the period-narrative warm +
the off-budget Coach memory workers) surfaces the no-key fallback instead of
egressing.
Also rate-limit POST /api/fitbit/sync (the one Fitbit route without a limiter):
a 5/60s baseline plus a tighter 1/hour bucket on the expensive fullSync walk,
keyed by user id, matching the sibling Fitbit routes.
Harden the Fitbit OAuth redirect_uri: assert it is an absolute https origin (or
http on localhost), lands on /api/fitbit/callback, and stays same-origin with
NEXT_PUBLIC_APP_URL — defence-in-depth behind Google's registered-URI check so
a Host-coerced or misconfigured app URL cannot redirect the authorization code
off-origin.
(cherry picked from commit 46b5fc2cea1b6e80bee0445bfdacf5790442c67d)
Three hardening + performance fixes to the Fitbit/Google Health sync.
Stop soft-delete resurrection. The measurement write probed the existing
row by the `(userId, type, source, externalId)` unique key without excluding
soft-deleted rows, so a reading the user deleted came back on the next hourly
sync (the upsert matched the tombstone and took the update branch — silently
undoing the delete). The write now probes only live rows (`deletedAt: null`)
and treats a tombstone as absent, minting a fresh insert instead. The fresh
insert relies on the partial unique index over `deleted_at IS NULL` to keep
live-row uniqueness; the tombstone sits outside it, and `skipDuplicates` guards
a live-row race.
Batch the backfill writes + collapse the rollup recompute. The write loop
was one upsert per reading (N+1) followed by a per-(type,day) DAY-rollup
recompute, so a multi-year backfill paid thousands of serial round-trips on a
concurrency-1 worker. The existence probe is now a single `findMany` over the
batch; fresh rows go through chunked `createMany`; only already-live rows take a
per-row update for their differing values. On a `fullSync` the per-write rollup
hook is deferred and the touched type-days collapse into one
`recomputeUserRollups(from, to)` pass at the end of the cycle. The incremental
path keeps the inline per-day hook. The `stats:` overwrite contract, the
single-watermark `markSynced`, and the all-403 soft-skip guard are unchanged.
Bound the hourly poll concurrency. The poll synced every connection in one
serial loop, so a single slow Google response stalled the whole cohort. The
cohort now fans out through a bounded `p-limit` pool with per-user error
isolation, extracted into `runFitbitPollCohort` so the contract is unit testable
without exporting worker internals. The accumulators are read after the await to
avoid a compound-assignment lost-update across overlapping pool tasks.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 8369af2471fb8da04a938b5a784433318d81a248)
Use FITBIT_ACTIVITY_PAGE_SIZE for the sleep and exercise reads instead of a
magic 25, remove the unused FitbitDataTypeKey type, and declare the crosstab
day floors independently so they aren't a duplicate re-export of the influence
floors.
Snapshot the incremental watermark once per sync cycle and thread the same
lower bound to every resource (metrics, activity, sleep, workout), stamping
lastSyncedAt once at the end. Previously each resource read lastSyncedAt then
stamped now() independently, so the first resource advanced the watermark and
later resources only fetched the last overlap window — silently dropping the
gap after an outage longer than the overlap. The full/backfill path is
unaffected (it passes no lower bound).
Reject an OAuth callback whose token response carries no refresh_token instead
of persisting an empty string. An empty refresh token bricks every future
token refresh; the callback now errors out cleanly so the user re-consents
rather than landing a dead connection.
Model the INTERVAL data types (steps/distance/calories/floors daily totals,
sleep and exercise sessions) on Google Health's interval.start_time /
interval.civil_start_time anchor instead of a sample_time / bare date. The
incremental filter for sleep/exercise previously targeted
sample_time.physical_time, which 400s/empties for interval types and stalls
incremental sync. Daily-summary metrics and spot readings keep their existing
date / sample anchors. Shape mismatches still fail safe (return []/null).
Mark the Google Health (Fitbit & Pixel) settings card as experimental with a
badge and a short note that data coverage is still being verified, in all six
locales.
(cherry picked from commit 3f116802149eebe9144729e2779f7f4405f248f2)
Marc-André Bombeck