tom/HealthLog

0 Fork 0

mirror of https://github.com/MBombeck/HealthLog.git synced 2026-11-10 05:12:14 +00:00

Files

T

Marc-André Bombeck 2d03eba587 fix(google-health): resume a backfill inside a collection and run it in short slices

A backfill noted progress only once a whole collection was read to the end.
A heart-rate history of years that one attempt could not finish (its budget,
a rate limit it could not wait out, a single failed request among thousands
of pages) started the next attempt from the newest page again, finished
nothing, and failed with "marker not stamped" on every attempt.

- Every spot-reading collection read page by page now records, after each
  page it writes, the oldest sample it has written (backfillProgress.cursors).
  The next attempt asks Google only for samples at or beyond that edge. The
  edge is held back once a page's write fails, so no rows are skipped.
- The backfill runs in slices of five minutes or 500 pages. A slice that
  finished a collection or moved an edge continues with the next one; only a
  slice that moved nothing fails, and its error names each unfinished
  collection and why the slice stopped. The slice ceiling is derived from the
  collections and the page ceilings of the resumable ones.
- A rate limit Google will not lift within a minute ends the slice instead of
  failing the walk, and the next slice starts five minutes later. Server
  errors, timeouts and dropped connections are retried three times.
- The heart-rate stream is read last, so the light collections finish in the
  first slice.
- The hourly sync waits up to eight minutes for a running slice instead of
  skipping the account for as long as the import runs.
- Each slice annotates googleHealth.backfill.slice with pages, elapsed time,
  stop reason, per-collection status and edges, and reports progress on the
  job.progress line.

2026-10-15 11:26:49 +02:00

migrations

fix(withings): keep the sync inside the account's API plan

2026-10-14 18:53:58 +02:00

README.md

docs: add navigational READMEs to key code directories

2026-06-05 15:26:34 +02:00

schema.prisma

fix(google-health): resume a backfill inside a collection and run it in short slices

2026-10-15 11:26:49 +02:00

README.md

`prisma/`

The database layer. HealthLog runs PostgreSQL 16 via Prisma 7. This directory holds the single schema file and the forward-only migration history; the generated client is emitted elsewhere.

schema.prisma — 61 models, one file. cuid() primary keys, snake_case columns via @map(...), encrypted columns mostly suffixed *Encrypted (search the file; there are 24 encrypted column references). The generator client block emits to ../src/generated/prisma (imported as @/generated/prisma).
migrations/ — numbered, forward-only SQL migrations (0001_… upward). migration_lock.toml pins the provider to postgresql.

Migration workflow

# author a migration from a schema edit
pnpm dlx prisma migrate dev --name <short_snake_case_name>

# regenerate the client after any schema change
pnpm dlx prisma generate

Number a new migration with the next free prefix. The production entrypoint runs prisma migrate deploy on boot, so every migration must apply cleanly forward — there is no down-migration path. Always include the migration file in the same change as the schema edit.

Conventions

Encryption is fail-closed: *Encrypted columns are written through @/lib/crypto (AES-256-GCM, versioned key ids); never store plaintext in an encrypted column. Key rotation is scripts/rotate-encryption-key.ts + docs/ops/encryption-key-rotation.md.
The generated client under src/generated/prisma is build output — do not edit it.
Schema-touching changes that affect the API surface must regenerate docs/api/openapi.yaml (see ../CLAUDE.md).

Related: ../src/lib/README.md (db.ts, rollups/), ../docs/ops/ (migrations + backup runbooks).

README.md

prisma/

Contents

Migration workflow

Conventions

`prisma/`