tom/HealthLog

0 Fork 0

mirror of https://github.com/MBombeck/HealthLog.git synced 2026-11-08 12:32:43 +00:00

Files

T

Marc-André Bombeck a4feabec81 chore(release): v1.28.30 — the daily briefing stops silently skipping days

The recurring no-briefing day had a chain of causes, closed end to end:

Every failure path in the nightly pregenerate + the comprehensive generator
now annotates a queryable cause (several classes — features-error,
payload-too-large, aborted, generator-failed — previously bumped a counter
with zero trace). A failed nightly warm enqueues one bounded retry ~45
minutes later (singleton per user, existing budgets apply), so a transient
provider hiccup heals before morning. A cached payload that carries NO
briefing no longer satisfies the 24h POST short-circuit nor the
unchanged-content hash gate — a once-stripped briefing could previously
block its own regeneration indefinitely. The card surfaces the omission
reason persistently (grounding strip, generation failure, not attempted)
with a forcing retry.

Documentation corrected against code truth: Withings credentials are
per-user (the documented env path never existed; the dead example entries
and manifest rows are gone, and WITHINGS_WEBHOOK_SECRET is now actually
whitelisted in compose — stock deploys could never deliver it); the classic
Fitbit portal no longer accepts new registrations; the backup-restore
example no longer mints a fresh encryption key and uses pnpm dlx tsx; the
TLS guide describes the real client model (two CA-level SPKI pins,
any-match, leaf renewals need no build); scaling/admin references fixed;
the operator-shared subscription provider is documented.

2026-07-12 11:08:59 +02:00

4.7 KiB

Raw Blame History

Off-host backup & restore

HealthLog ships with an optional daily off-host backup that ships every user's JSON dump, encrypted with AES-256-GCM under a SEPARATE key (BACKUP_ENCRYPTION_KEY), to any S3-compatible bucket — Cloudflare R2, AWS S3, Backblaze B2, MinIO, etc.

The backup runs at 02:30 Europe/Berlin every day from the worker container (queue data-backup-offhost). Object key layout:

<bucket>/YYYY-MM-DD/user-<userId>.json.enc

Wire format (binary)

magic   = "HLBK"           (4 bytes, ASCII)
version = 0x01             (1 byte)
iv      = 12 random bytes  (AES-GCM nonce)
authTag = 16 bytes         (AES-GCM tag)
ciphertext = N bytes       (AES-256-GCM, key = BACKUP_ENCRYPTION_KEY)
plaintext  = JSON dump (UTF-8)

Required env vars

Var	Required	Notes
`BACKUP_ENCRYPTION_KEY`	yes	64 hex chars or 32-byte base64. Different from `ENCRYPTION_KEY`.
`BACKUP_S3_ENDPOINT`	yes	e.g. `https://<account>.r2.cloudflarestorage.com`
`BACKUP_S3_BUCKET`	yes
`BACKUP_S3_ACCESS_KEY`	yes
`BACKUP_S3_SECRET_KEY`	yes
`BACKUP_S3_REGION`	no	defaults to `auto` (Cloudflare R2)
`BACKUP_RETENTION_DAYS`	no	defaults to `30`

Bucket lifecycle (recommended)

The worker prunes objects older than BACKUP_RETENTION_DAYS, but the storage provider's lifecycle rule is the canonical safety net:

Filter: "" (all objects)
Action: Expire after 30 days

For Cloudflare R2 add this from the bucket's Settings → Lifecycle.

Smoke test

After deploying, hit POST /api/admin/backup/test (admin-only). It performs a 1-byte PUT + GET round-trip and returns:

{
  "endpoint": "https://...r2.cloudflarestorage.com",
  "bucket": "healthlog-backups",
  "region": "auto",
  "putLatencyMs": 142,
  "getLatencyMs": 38,
  "ok": true
}

The credentials are never returned.

Restore

Pick a key (e.g. 2026-05-08/user-clx123.json.enc) from the bucket and run the restore script with the same backup credentials and encryption key the backup was written under — a freshly generated BACKUP_ENCRYPTION_KEY cannot decrypt any existing object:

BACKUP_S3_ENDPOINT=https://...r2.cloudflarestorage.com       \
BACKUP_S3_BUCKET=healthlog-backups                           \
BACKUP_S3_ACCESS_KEY=...                                     \
BACKUP_S3_SECRET_KEY=...                                     \
BACKUP_S3_REGION=auto                                        \
BACKUP_ENCRYPTION_KEY=<the key the backup was written under> \
pnpm dlx tsx scripts/restore-backup.ts \
  2026-05-08/user-clx123.json.enc \
  /tmp/restored.json

The production standalone image strips tsx, so a bare pnpm tsx scripts/... fails inside the container — always invoke one-shot scripts via pnpm dlx tsx.

The script downloads the object, decrypts it, and writes the JSON dump to disk. Importing the JSON back into a HealthLog instance is left to the operator (use prisma db seed or a custom script).

Monthly restore drill (automatic)

Since v1.16.4 a pg-boss job (data-restore-drill, cron 11 4 1 * * — 04:11 on the 1st of each month) exercises the read path end-to-end: fetch the most recent backup object from the bucket, decrypt it under the current BACKUP_ENCRYPTION_KEY, JSON-parse it, and sanity-check the payload shape. It performs no database restore — it validates the artefact, not the import path.

Outcomes:

Success — record counts, object age, and sizes land in the wide-event meta (job.restore_drill).
Stale chain — the newest object is older than 3 days: the nightly uploader has stalled (or the lifecycle rule is too aggressive). The drill pages via the worker error reporter (stderr + GlitchTip).
Failure — empty bucket, fetch error, decryption failure (wrong or rotated key), malformed JSON: pages the same way. A decryption failure right after a BACKUP_ENCRYPTION_KEY change means the new key cannot read the existing objects — re-encrypt or accept that pre-rotation backups are only readable with the retired key.
Not configured — deployments without the BACKUP_S3_* vars skip silently (wide-event warning only).

The drill needs no IAM grant beyond the uploader's existing GetObject + ListBucket.

4.7 KiB Raw Blame History