`instant.toISOString().slice(0, 10)` and parsing a key back as UTC
midnight are the idioms behind every day-boundary fix this year: the
evening west of UTC and the small hours east of it land on the
neighbouring day. A new rule, healthlog/no-utc-day-key, refuses them in
src/ outside src/lib/tz and tests, and refuses a disable directive for
itself that is not next-line or gives no reason.
The time zone module now names every legitimate form: userDayKey for a
person's day, dateOnlyKey / dateOnlyAtNoonUtc / dayKeyAsUtcMidnight for
date-only values and day labels, daysBetweenDateKeys / weekdayOfDateKey /
isCalendarDateKey / statedDateKey for key arithmetic and parsing.
Of the 133 sites the rule found, 84 are gone. Most moved onto a named
helper with no change in behaviour; these changed behaviour:
- cycle calendar, insights, Coach snapshot and prediction refresh start
their lookbacks at the first instant of the user's day
- a model-transcribed date like "June 10, 2026" no longer lands a day
early in a process east of UTC (documents and lab scans)
- a custom report range covers exactly the picked days in the profile
zone instead of starting the evening before west of UTC
- date-of-birth pickers cap at the user's today, not the UTC one
- an allergy onset date is stored at noon UTC like other date-only values
- the report's cycle summary, the Coach's usual BP range and the intake
projector's rollup recompute use the user's day
- vaccination dates render as dates instead of the previous evening
49 remain, each with its reason on the line above: 39 UTC by design (15
file name stamps, rollup and object keys, the UTC-day score engines) and
10 marked baseline. A guard test requires a classified reason on every
directive and holds the baseline to a ceiling that only goes down.
The seven integration clients now read their redirect URI and the app URL
through envValue/envOr instead of a local ?.trim() ||. The empty-env guard
drops its pending entry and floors its walk, and proves its matcher still
fires on a real client with the old read put back.
The compose file passes every whitelisted variable through, so an unset
redirect URI reaches the process as an empty string. Withings then sent
redirect_uri= to the provider, and Fitbit and Google Health refused to start
the connection as not configured although NEXT_PUBLIC_APP_URL was set. Every
integration now treats an empty or blank value as unset and derives the
callback from the app URL.
Withings, Fitbit, Google Health, Polar and Oura wrote each daily total and
daily summary at noon UTC of its date. From UTC+12 on (New Zealand all year,
Fiji, Tonga, Samoa) noon UTC is already the next local day, so every synced
step count, SpO2 or resting-pulse day showed up a day late. The rows now sit
at local noon of their date in the user's zone, and the fetch windows run
through the user's own today.
Fitbit weigh-ins carry a local wall clock without an offset and were read
as UTC; they are read in the user's zone now.
ExternalIds stay on the calendar date, so the next sync finds each row and
moves it in place instead of adding a second one, and the day a row leaves
is refolded. Rows inside each integration's re-read window heal on the next
sync; Fitbit and Google Health re-read the whole history on reconnect.
The v1.38.0 notice that warns when the configured OAuth callback origin
differs from the origin the browser is on never rendered in the published
image, and the copyable callback address in the redirect guide showed a
bare path. `integrationCallbackUrl()` read `process.env.NEXT_PUBLIC_APP_URL`
inside a "use client" module. Next.js inlines NEXT_PUBLIC_* into client
bundles at build time, and the GHCR image is built without that variable
(only the version, build-sha and built-at args reach the build), so the
base compiled to an empty string, `new URL("/api/<provider>/callback")`
threw, and the notice returned null on every card for every self-hoster.
The value now comes from the server at request time. A new map
(`src/lib/integrations/callback-urls.ts`) calls each provider's own
redirect-URI function (the four private `getRedirectUri()` helpers are
exported under provider-specific names; Polar, Oura and Strava already
exported theirs), so the page shows exactly what the handshake registers,
explicit `*_REDIRECT_URI` override included. The settings route resolves
the map and passes it as a prop through `IntegrationsSection` and
`ConnectionsPanel` to every card. Fitbit and Google Health validate their
URI and throw on a bad configuration; that surfaces as `null`, the notice
skips the comparison and the guide falls back to the fixed path.
Tests: a map test proving the explicit override wins and the derived form
is used otherwise, a component test rendering the notice from a passed
value (mismatch renders, match does not), and a structural guard that
fails when any "use client" file under src/components/settings/integrations
reads NEXT_PUBLIC_APP_URL.
Fitbit re-scores a night after the fact, and the per-segment externalId carried
a positional index plus the stage label. Both moved on a re-score: renumbered
segments minted fresh keys, the previous scoring's rows stayed live because
nothing swept them, and the night total counted both copies from then on.
Key the segment on the stable session anchor plus its own start instant
(`<logId>:sleep:<segment-start>`), and add the replace-by-window sweep every
other segment-writing provider already has. The two ship together with the
natural-key rescue that has been sitting in the Fitbit upsert unused: the sweep
tombstones the old-keyed rows first, so the rescue re-keys those very rows in
place instead of inserting a parallel copy or losing the insert to
`skipDuplicates` against the natural unique index.
A one-shot repair queue re-reads the bounded sleep history once per connection
so no user has to do anything; it stamps `sleepRepairedAt` (migration 0270) only
on a clean run, never touches the sync watermark, and reports the rows it
removed. Nights older than the 365-day backfill horizon keep their old keys and
whatever duplicates they already hold — they can no longer gain new ones.
Alongside: `fitbitGet` now honours a `Retry-After` inside a 120 s cap by waiting
once and retrying once, and fails fast above it so a long hold cannot starve the
rest of the poll cohort or outlive a job's expiration. The measurement upsert
skips a write whose payload is unchanged, which takes the hourly overlap off the
delta feed for every Fitbit resource.
The German prompt-injection test asserted only inside `if (decision.refuse)`,
so it stayed green whether or not the filter fired. Un-guard it, drive it from a
case table, and widen the two German patterns to cover the determiners that
walked straight past them ("saemtliche", "die", "deine").
The Oura / Polar / Fitbit token helpers cast a possibly-null `json` to the token
type on a status-only success gate. A 2xx with an empty or non-JSON body handed
callers a null they dereferenced, surfacing an unclassified TypeError instead of
a handled integration error. Narrow explicitly and classify as transient.
Platform-wide hardening: the failure classes found live this week hunted down
across every integration.
Fitbit carried the same tombstone wedge fixed for Google — probe now matches
soft-deleted rows and the update resurrects them; the comment describing a
partial unique index that never existed is corrected. Sleep segment ids are
stable across re-scoring for Withings (session+segment-start), WHOOP/Polar
(one segment per stage, no positional index) and Oura (run-start keyed), each
with a session-scoped sweep that soft-deletes rows a re-score orphaned — so
legacy and affected nights heal as they are re-read.
One platform rule for deleted rows: source-owned readings resurrect on
re-import (Withings, WHOOP, Oura, Polar, Nightscout, mood webhook, CSV
re-imports, Apple stats day-totals); Apple sample-grain deletions stick per
the paired-client contract, and the mood bulk route's tombstone match is now
a true no-op reported as duplicate. Import counts reconcile against actual
createMany counts.
Google lifecycle: a dead token registers on the cycle's hard-fail ledger, so
an expired connection parks instead of stamping success hourly; backfill and
sleep-repair markers stamp only after a clean pass (pg-boss retries are now
real); failed measurement writes hold the watermark; no-op updates are
skipped; mapping.md tells the current truth.
Scale: doctor-report reads only consumed columns and enabled sections;
exports and backups keyset-paginate; achievements tally via SQL buckets;
series charts day-bucket dense kinds beyond 90 days; BP pairing is a linear
two-pointer; distinct-type scans run server-side; the graded-series cold
fallback is bounded; the fitbit defer tracker loops instead of spreading.
The classic 1.2 sleep log and the activities list emit local wall-clock
ISO strings without an offset. `parseLocalInstant` parsed them with a bare
`new Date(iso)`, which reads an offset-less string in the process zone (UTC
in production). A non-UTC user's sleep-segment `measuredAt` (the segment
end) was therefore shifted by their UTC offset, and a near-midnight segment
could flip onto the wrong wake-day in the night reconstruction.
Resolve offset-less timestamps against the user's stored timezone via a new
`zonedWallClockToUtc` helper (the inverse of `wallClockInTz`, DST-safe via
the same two-pass converge as `startOfLocalDayInTz`). Thread the resolved
zone through the sleep and workout sync paths. Strings that already carry an
offset or `Z` stay absolute and parse as-is; an absent zone keeps the prior
host-local fallback.
The shipped Fitbit integration spoke the Google Health API, whose Restricted
scopes need brand verification plus an annual CASA assessment before leaving
testing mode — a wall most self-hosters cannot clear. Retarget the transport to
the classic api.fitbit.com Web API: a self-hoster registers an app at
dev.fitbit.com in minutes, no CASA.
Forks only the transport; the upsert/dedup/rollup tail, source-priority,
backfill, poll worker, and status ledger are reused unchanged.
- OAuth: www.fitbit.com/oauth2/authorize + api.fitbit.com/oauth2/token, PKCE
(S256), classic self-serve scopes.
- Refresh tokens rotate (one-time use): persist the new token every refresh,
with a defensive keep-existing guard on a malformed reply.
- Per-endpoint date-range fetchers chunked to the API range caps; the backfill
horizon and poll concurrency are tuned for the 150 req/h-per-user budget.
- Verified classic-shape extractors for weight, body fat, SpO2, HRV (dailyRmssd),
resting HR, respiratory rate, steps, distance, active calories, floors, VO2 max
(range midpoint), per-segment sleep stages, and workouts.
- Skips intraday HR (needs a per-app grant; daily resting HR covers it) and skin
temperature (a baseline delta with no honest absolute slot).
One-shot prettier pass so the new format:check gate starts from a clean
baseline. Formatting only — no behaviour change. Generator-owned and
vendored artifacts stay untouched via .prettierignore.
The Google sleep payload carries a real per-stage segment series, but the
mapper summed each stage's segments into one row stamped on the stage's
last end. The hypnogram then reconstructed all of a stage's time as a
single block ending at the night's right edge — the same stacked-bar
artefact as WHOOP, despite the source carrying true timing.
Emit one row per segment with measuredAt at that segment's end and an
indexed externalId so the several segments of one stage stay distinct
under the dedup key. The timeline is measured (real onsets), so these
rows are not flagged reconstructed.
Close the launch-blocking gap where the ConsentReceipt infrastructure was
built but never enforced as a precondition: a direct API caller with a valid
token could forward a health snapshot to a server-managed external LLM with no
receipt on file. Consent was enforced client-side only.
Add a consent guard (src/lib/ai/consent-guard.ts) and call it before the first
external-provider call on the server-managed path. The gate fires only when the
resolved chain could egress via the operator's global key (admin-openai); a
user's own BYOK key (openai/anthropic), their own ChatGPT OAuth account, and
the self-hosted local provider stay ungated — that egress is the user's own act
and the settings flow is its consent. The check fails closed: a BYOK-primary
chain with an admin-openai fallback still requires a receipt, since the runner
may cascade to the operator key. The surface maps to the consent kind (coach ->
ai_coach, insights -> ai_insights_only; master ai_full satisfies either).
- Interactive routes (insights/chat, insights/generate POST) throw
ConsentRequiredError; api-handler renders a 403 with
meta.errorCode = "consent.ai.required", mirroring assistant.disabled.* so
the client can render an inline grant-consent notice.
- Off-request pipelines fail closed quietly: comprehensive generation returns a
typed skipped:no-consent outcome, and the per-metric status family
(runStatusCompletion, gating every status card + the period-narrative warm +
the off-budget Coach memory workers) surfaces the no-key fallback instead of
egressing.
Also rate-limit POST /api/fitbit/sync (the one Fitbit route without a limiter):
a 5/60s baseline plus a tighter 1/hour bucket on the expensive fullSync walk,
keyed by user id, matching the sibling Fitbit routes.
Harden the Fitbit OAuth redirect_uri: assert it is an absolute https origin (or
http on localhost), lands on /api/fitbit/callback, and stays same-origin with
NEXT_PUBLIC_APP_URL — defence-in-depth behind Google's registered-URI check so
a Host-coerced or misconfigured app URL cannot redirect the authorization code
off-origin.
(cherry picked from commit 46b5fc2cea1b6e80bee0445bfdacf5790442c67d)
Use FITBIT_ACTIVITY_PAGE_SIZE for the sleep and exercise reads instead of a
magic 25, remove the unused FitbitDataTypeKey type, and declare the crosstab
day floors independently so they aren't a duplicate re-export of the influence
floors.
Snapshot the incremental watermark once per sync cycle and thread the same
lower bound to every resource (metrics, activity, sleep, workout), stamping
lastSyncedAt once at the end. Previously each resource read lastSyncedAt then
stamped now() independently, so the first resource advanced the watermark and
later resources only fetched the last overlap window — silently dropping the
gap after an outage longer than the overlap. The full/backfill path is
unaffected (it passes no lower bound).
Reject an OAuth callback whose token response carries no refresh_token instead
of persisting an empty string. An empty refresh token bricks every future
token refresh; the callback now errors out cleanly so the user re-consents
rather than landing a dead connection.
Model the INTERVAL data types (steps/distance/calories/floors daily totals,
sleep and exercise sessions) on Google Health's interval.start_time /
interval.civil_start_time anchor instead of a sample_time / bare date. The
incremental filter for sleep/exercise previously targeted
sample_time.physical_time, which 400s/empties for interval types and stalls
incremental sync. Daily-summary metrics and spot readings keep their existing
date / sample anchors. Shape mismatches still fail safe (return []/null).
Mark the Google Health (Fitbit & Pixel) settings card as experimental with a
badge and a short note that data coverage is still being verified, in all six
locales.
(cherry picked from commit 3f116802149eebe9144729e2779f7f4405f248f2)
Extend the Fitbit / Google Health provider with the remaining sync
resources, driven by the existing syncUserFitbit orchestrator.
Activity (daily cumulative): steps -> ACTIVITY_STEPS, distance ->
WALKING_RUNNING_DISTANCE, the active-energy portion -> ACTIVE_ENERGY_BURNED,
floors -> FLIGHTS_CLIMBED, and VO2 max -> VO2_MAX. Each daily total mints a
stats:<tag>:<YYYY-MM-DD> externalId so a re-fetched day overwrites in place
rather than duplicating, matching the Apple-Health daily-total contract. The
running totals preserve a legitimate zero (a rest day is real data); VO2 max
stays strictly positive (daily latest-wins).
Sleep: map a session's per-stage segments to per-stage SLEEP_DURATION rows
(minutes; measuredAt = the stage end), harmonised onto the shared SleepStage
enum (light -> CORE, deep -> DEEP, rem -> REM, awake/restless -> AWAKE,
in-bed -> IN_BED, classic asleep -> ASLEEP) so the night-total and hypnogram
readers consume the same shape they already read for WHOOP and Apple.
Workouts: map exercise sessions to Workout rows keyed (userId, FITBIT,
externalId), with sport-type, start/end, duration, energy, distance, and HR.
Cross-source twins stay distinct at ingest; the read-time canonical picker
collapses them via the source ladder (Fitbit already ranked below WHOOP).
All three join the orchestrator's resource list and ride the per-resource
403 soft-skip so an independently-granted Restricted bundle that the user
withheld skips only its own resource. Rollups recompute and status-insight
caches invalidate through the shared upsert. No new MeasurementType, no
migration, no new dependency.
Extend the Fitbit/Google Health provider with the read path: the
dataPoints.list client half, the metrics-bundle sync, the poll + backfill
jobs, and the connection-management routes. Builds on the existing OAuth +
credentials + token-refresh layer.
- client: paginated `fetchDataPoints` walker over `safeFetch` keyed on
`nextPageToken`, encoding each data type kebab-case in the path and
snake_case in the filter predicate; defensive per-type mappers (finite +
positive guards) for weight, body fat, daily SpO2, daily HRV, daily resting
heart rate, daily respiratory rate, intraday heart rate, sleep-temperature
derivations, and a height profile-seed extractor. `FITBIT_FIELD_MAP` +
`mapping.md` are the source of truth.
- mapping (no new MeasurementType): weight→WEIGHT, body-fat→BODY_FAT,
SpO2→OXYGEN_SATURATION, HRV→HEART_RATE_VARIABILITY (SDNN slot, not
HRV_RMSSD), resting-HR→RESTING_HEART_RATE, respiratory-rate→RESPIRATORY_RATE,
heart-rate→PULSE, sleep-temp→WRIST_TEMPERATURE, height→User.heightCm when
null. Steps/distance/energy/floors/VO2 + sleep + workouts stay for a later
wave.
- sync: `syncUserFitbit` orchestrator with the reauth short-circuit, the
per-data-class 403 soft-skip (the six Restricted bundles grant
independently), the all-soft-skipped success guard, and incremental-vs-full
windows; `upsertFitbitMeasurements` keyed (userId,type,FITBIT,externalId)
with a 24 h overlap for after-the-fact daily re-rolls, recomputing rollups
and invalidating status caches. Google refresh tokens do not rotate, so the
stored refresh token is kept unless a new one is returned.
- jobs: hourly `fitbit-sync` poll (poll-only; Pub/Sub deferred), the
self-converging boot backfill, and the daily OAuth-state sweep — every queue
registered in both `allQueues` and a `boss.work` binding, staggered off the
WHOOP/Withings crons.
- routes: status / sync / disconnect mirroring the WHOOP shape.
Tests cover the mappers, the casing invariant, `fetchDataPoints`, the
orchestrator soft-skip guard, the idempotent upsert, the backfill discovery,
and the queue registration; an integration test pins the upsert idempotency,
the DAY rollup fold, and the Fitbit-vs-Withings per-source no-collapse against
the Postgres testcontainer.
Land the OAuth + credentials half of the Fitbit / Google Health provider,
cloning the WHOOP provider structure and adapting it to the Google identity
flow.
- Client OAuth half (src/lib/fitbit/client.ts): Google authorize-URL builder
(access_type=offline + prompt=consent), authorization-code exchange, token
refresh, and a profile fetch for the connection's external user id. Client
credentials ride in an HTTP Basic-auth header on the token endpoint; the
four launch Restricted read scopes are pinned in FITBIT_OAUTH_SCOPE. All
egress goes through safeFetch.
- Non-rotating refresh tokens: getValidToken refreshes at expiry-minus-five-
minutes, persists the new access token + expiry, and overwrites the stored
refresh token ONLY when the response carries a fresh one — Google keeps the
refresh token stable, unlike WHOOP.
- BYO credentials route (PUT/GET/DELETE /api/fitbit/credentials): per-user
client id/secret stored encrypted on the User columns, built field-by-field
under requireAuth.
- OAuth state + connect + callback routes: a random base64url nonce backed by
a 10-minute FitbitOAuthState ledger row carries the (nonce -> userId)
mapping; the callback validates the cookie/URL state with timingSafeEqual,
consumes the row atomically (replay/expired/cross-user reason tags),
exchanges the code, upserts the encrypted FitbitConnection, clears prior
reauth state, and enqueues the self-converging history backfill.
- Response classifier mirrors WHOOP's HTTP-status buckets; Google signals a
revoked grant with a 401, so there is no invalid_grant body special-case.
Tests cover the authorize URL, Basic-auth token exchange, the no-rotation
refresh branch (token preserved when omitted, overwritten when present), the
classifier status table, and the OAuth-state nonce mint.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Marc-André Bombeck