feat(oura): backfill the full history in resumable slices

Every Oura collection, workouts and heart rate included, is now read
back to the start of the account's history by a backfill that runs
through the shared integration lane in five-minute slices. Each
collection walks from the day the backfill began toward the past one
window at a time (ninety days, thirty for sleep, seven for heart rate)
and the progress note on the user row records after every window where
it got to, so a slice cut short by its budget, the lane or a rate limit
resumes there. A 429 ends the slice and the next one waits for the
Retry-After Oura named. Activity and sleep are walked first to learn
where history starts; the rest stop a week before it.

Heart rate streams a page at a time and keeps only the days it touched,
so memory stays flat however long the history is. A collection the
grant does not cover is a scope note, not a failure; the first slice of
the next chain asks again, and a connect clears the completion stamp so
a newly granted scope is read without walking anything twice.

The two columns arrive empty, so every existing connection is offered
the backfill at the next boot without reconnecting. Before anything
else the backfill retires the WALKING_RUNNING_DISTANCE rows earlier
syncs wrote from the equivalent walking distance and refolds their days;
an account that kept such rows after disconnecting is offered too.
This commit is contained in:
Marc-André Bombeck committed 2026-10-21 19:53:57 +02:00
1 parent 7585994697
commit bce9ad1c25
21 files changed
+1462 -10

No files matched your search

@@ -0,0 +1,13 @@
-- Where the Oura history backfill has got, and when it finished.
--
-- The backfill reads every Oura collection's history in slices through the
-- shared integration backfill lane. After each window it writes, the progress
-- note records where the collection's next window ends, so a slice that stops
-- on its budget or a rate limit resumes there instead of from the newest day.
-- The completion stamp drops the account from boot discovery.
--
-- Both nullable and unset for every existing row: every current Oura
-- connection is offered the backfill at the next boot, with no reconnect.
ALTER TABLE "users" ADD COLUMN IF NOT EXISTS "oura_backfill_progress" JSONB;
ALTER TABLE "users" ADD COLUMN IF NOT EXISTS "oura_backfill_completed_at" TIMESTAMP(3);
+15 -5
View File
@@ -277,22 +277,32 @@ model User {
// v1.17.0 — Oura OAuth credentials (F4, per-user, encrypted at app level).
// Mirrors the WHOOP credential columns; OAuth access/refresh tokens stored
// encrypted.
ouraAccessTokenEncrypted String? @map("oura_access_token_encrypted")
ouraRefreshTokenEncrypted String? @map("oura_refresh_token_encrypted")
ouraAccessTokenEncrypted String? @map("oura_access_token_encrypted")
ouraRefreshTokenEncrypted String? @map("oura_refresh_token_encrypted")
// v1.17.1 — Oura BYO-app client credentials (W1c, per-user, encrypted at app
// level). Mirrors the WHOOP/Fitbit `clientId`/`clientSecret` columns 1:1 so a
// self-hoster pastes their own Oura Cloud app's client id/secret into Settings
// before the OAuth connect flow runs. Stored encrypted as a string via
// `src/lib/crypto.ts`, matching every other `*Encrypted` column.
ouraClientIdEncrypted String? @map("oura_client_id_encrypted")
ouraClientSecretEncrypted String? @map("oura_client_secret_encrypted")
ouraClientIdEncrypted String? @map("oura_client_id_encrypted")
ouraClientSecretEncrypted String? @map("oura_client_secret_encrypted")
/// v1.42.3 — Oura collections the current grant does not cover, as a JSON
/// string array of collection names (`spo2`, `vo2max`, `cardio_age`,
/// `resilience`). Written by the sync when a collection answers 401/403 while
/// its siblings succeed with the same token, which is a scope the grant never
/// carried, not a dead token. The Oura card reads it to ask for one
/// reconnect. Cleared on connect, disconnect and credential removal.
ouraScopeDenied Json? @map("oura_scope_denied")
ouraScopeDenied Json? @map("oura_scope_denied")
/// v1.44 — how far the Oura history backfill has got: per collection, the
/// day its next window ends on, and whether it is done or refused for a
/// scope; plus where the account's history starts once learned. Written
/// after every window so a slice resumes where the last one stopped (see
/// `src/lib/oura/backfill-progress.ts`); kept after completion so a
/// reconnect reads only what the old grant refused.
ouraBackfillProgress Json? @map("oura_backfill_progress")
/// v1.44 — the Oura history backfill finished (self-converging discovery
/// predicate `oura_backfill_completed_at IS NULL`). Cleared on connect.
ouraBackfillCompletedAt DateTime? @map("oura_backfill_completed_at")
// v1.28.x — Strava OAuth credentials (per-user BYO-app, encrypted at app
// level). Strava caps every newly-created API app at single-player mode