Commit Graph
7 Commits
Author SHA1 Message Date
Marc-André Bombeck f8b0a64d69 fix(backups): preserve canonical recovery integrity 2026-07-20 11:44:29 +02:00
Marc-André Bombeck 3fcae204f7 perf(backup): keyset-page the intake/mood/cycle reads in the off-host DR dump
Measurements were already paged, but the medication-intake, mood, and
cycle tables in the same per-user payload were still unbounded full-table
findMany reads — peak heap tracked the largest tenant's history. Give
them the same id-keyset page walk (full row projection, so payload bytes
are unchanged) so at most one page is resident per round-trip.
2026-07-08 23:45:04 +02:00
Marc-André Bombeck 84683f4ce9 perf: bound heavy-tenant memory on feature, consolidation, backup and auth paths
Cap and page the unbounded reads that let a multi-year tenant pull its
whole measurement history into memory on hot paths.

- extractFeatures: read newest-first with a 6000-row cap (mirrors the
  Coach snapshot) and reverse to ascending, so a 365-400 day window no
  longer materialises hundreds of thousands of PULSE / glucose rows per
  Coach turn and nightly briefing. Full-history extremes still come from
  readAllTimeExtremes, so the cap never relabels an extreme.
- Add a (userId, measuredAt) index on measurements (migration 0210) for
  the type-less range scans that extractFeatures and the consolidation
  passes drive; the existing (userId, type, measuredAt) cannot serve them
  as a prefix.
- runConsolidation: keyset-paginate the per-(user, type) source scan on
  (measuredAt, id) instead of one unbounded findMany, removing the giant
  single result-set allocation that drove the boot storm. Output is
  identical; covered by a paging unit test.
- Stagger the boot-time backfill enqueues with increasing startAfter
  offsets so the heavy full-history loads no longer all drain in parallel
  onto one tenant at startup. Non-boot callers keep their immediate send.
- Backups: replace load-all-then-stringify with paged narrow-select reads
  for the weekly and off-host writers, bounding transient heap. Off-host
  still includes soft-deleted rows. Covered by a serialize/restore
  round-trip test.
- Drop the per-request auth.bearer.success audit insert; the wide event
  already records auth_method and user_id, so the polling client no longer
  drives a write + pool checkout on every request. Failure-path audit
  stays.
- Coach conversation detail: cap the message include to the newest 200 so
  a long thread no longer decrypts every message on each open; response
  shape unchanged.
- Admin audit-log: floor the scan at the retention horizon so the listing
  and its count stop sweeping open-ended over the substring filters.
2026-06-28 03:07:15 +02:00
Marc-André Bombeck 410c62c292 refactor(jobs,dashboard): a thin worker registry and a composed dashboard
The reminder worker becomes a registry over twelve queue modules cut
along the existing test structure, and the dashboard page sheds its
quick-entry sheets, header and range logic into components — same
behaviour, pinned by the existing suites.
2026-06-11 00:10:01 +02:00
Marc-André BombeckandClaude Opus 4.8 e984ada5f1 feat(cycle): import Apple Health reproductive data into cycle, add cycle export
Promote the reproductive HealthKit category types out of the
Apple-Health-import privacy-hold list and route them into cycle day-logs
instead of measurements, and surface cycle data on the three export
surfaces.

Import promotion:
- A new HealthKit -> cycle mapping module is the single source of truth for
  how MenstrualFlow, IntermenstrualBleeding, CervicalMucusQuality,
  OvulationTestResult, SexualActivity, PregnancyTestResult,
  ProgesteroneTestResult, Contraceptive and the symptom category types fold
  into CycleDayLog fields / seeded symptom keys. The XML importer
  accumulates same-day reproductive samples and writes one canonical
  day-log per day with source APPLE_HEALTH and a synthetic per-day
  externalId for idempotent re-import. The fold is gated on cycle tracking
  being enabled, so a non-cycle export never provisions cycle rows. A
  contraceptive method nudges the profile goal toward avoiding pregnancy
  only when the goal is still on its default. Basal body temperature stays
  a measurement and mirrors onto the day-log. Pregnancy/Lactation status
  and the pregnancy-bleeding types stay deferred; the cycle awareness types
  remain server-derived.

Export:
- FHIR emits LMP (LOINC 8665-2), average cycle length (64700-8), period
  length (64698-4) and the current phase as opt-in reproductive-health
  observations grouped into their own document section.
- The doctor PDF gains an opt-in cycle section (recent cycles, average
  length and variability, current phase), privacy-default off like mood.
- The full-backup export, the weekly backup worker and the off-host DR
  snapshot include the cycle tables, and the admin restore re-creates them;
  the day-log note round-trips as ciphertext and is never decrypted onto a
  report surface.

Mapping table documented in docs/api/cycle-healthkit-mapping.md. Adds unit
+ integration coverage for the mapping, the import routing, the FHIR
observations and the backup round-trip. OpenAPI regenerated. Six-locale
strings added.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 10:16:54 +02:00
Marc-André Bombeck 9b6e237f5f chore(release): v1.4.42 — Knip enforcing, queryKey factory closed, iOS Workouts dedup, Withings off-response classification
Six touch-disjoint waves landed on develop before this release:

- W1-KNIP — exports/types tiers flipped to enforcing-mode on main;
  35 unused exports + 52 unused types triaged to 0/0; shadcn surface
  + zod-infer contract types scoped via knip.json ignore-block.
- W2-ZOD-MULTI-ISSUE — `returnAllZodIssues` shared helper +
  `/api/dashboard/widgets` 422 now returns every Zod issue + worker
  audit-ledger breadcrumb. 41 sibling routes documented for v1.4.43.
- W3-QUERYKEY-LONGTAIL — 40 settings/medications/admin/hooks files
  migrated to the queryKeys factory; 9 new factory entries; ESLint
  rule + test-guard scope extended in lockstep.
- W4-ARCH-HYGIENE — BERLIN_DAY_FORMATTER consolidated across nine
  sites (seven status helpers + bucket-series + bp-in-target);
  Suspense double-comment consolidated; doctor-report sanitiser regex
  rewritten to escape-form so diffs are reviewable; pr-detection-
  worker soft-delete filter; offhost-backup DR-intent comment.
- W5-IOS-WORKOUTS — `dedupeWorkoutBatch` write-time cross-source
  dedup helper for the v1.5 iOS HealthKit observer ingest; dashboard
  tile-strip placeholder parity with the live TrendCard chrome.
- W6-WITHINGS-OFF + ENV-CHECK — Withings off-response classifier
  (transient/reauth/persistent) + WithingsApiError typed throw + the
  `pnpm check-env` CLI. APNs `allOrNone: true` closes the AP-2
  silent-disable gap the wave was conceived to close.

No migration, no env-var change, no API contract break for iOS v0.5.4.
The new `/api/dashboard/widgets` 422 envelope is additively extended.

pnpm test --run green at 4815 passing / 1 skipped.
pnpm typecheck, pnpm lint, pnpm knip (no --include flag) green.
2026-05-21 19:33:07 +02:00
Marc-André BombeckandMarc-André Bombeck d7778c1085 feat(v1.4): multi-tenant prep — off-host backup, encryption-key versioning, worker/web split, native-Bearer UA gating (#135)
* feat(crypto): G2 — encryption-key versioning + rotation CLI

Adds backwards-compatible support for ENCRYPTION_KEYS (JSON map of
{ "id": "<key>" }) plus ENCRYPTION_ACTIVE_KEY_ID. Ciphertext format is now
"<keyId>.<base64payload>"; legacy bare-base64 ciphertext still decrypts
under the synthetic "v1" id derived from the existing ENCRYPTION_KEY.

scripts/rotate-encryption-key.ts re-encrypts every encrypted column on
User / WithingsConnection / AppSettings with the active key. Idempotent —
rows already on the active id are skipped.

Why: enables zero-downtime rotation when a key is suspected leaked,
without forcing every encrypted row through a single big-bang rewrite.

Co-Authored-By: Marc-André Bombeck <noreply@anthropic.com>

* feat(auth): G4 — native-Bearer UA gating + rotating refresh tokens

Web callers (UA contains 'Mozilla/') keep the legacy 90d Bearer when they
explicitly opt in via X-Client-Type: native. Native callers (HealthLog-iOS,
HealthLog-iPad, n8n, Health-Connect, unrecognised UAs) now receive a 24h
access token plus a 60d rotating refresh token in the login response.

New /api/auth/refresh endpoint atomically validates + rotates the refresh
token (one-time-use), revokes the previously paired access token, and
audit-logs every refresh + revoke. Reuse of an already-consumed refresh
token revokes the entire family for the user/device — defence against a
stolen refresh token.

Migration 0025_refresh_tokens adds the table; the only DDL is CREATE
TABLE so it is forward-compatible. Down migration documented at
docs/ops/migrations/0025-down.sql.

Why: the V3 audit (still-V2-A) flagged that 90d tokens are too long for
native callers when refresh-token rotation is now table-stakes for mobile
apps. Web flows are unchanged so existing browser sessions keep working.

Co-Authored-By: Marc-André Bombeck <noreply@anthropic.com>

* feat(infra): G3 — web/worker container split via HEALTHLOG_PROCESS_TYPE

Adds a tiny startup gate (src/lib/process-type.ts) so the same Docker image
can run as web-only, worker-only, or 'all' (default — single-container,
backwards compatible). assertSubsystemEnabled() refuses to boot the
reminder worker when HEALTHLOG_PROCESS_TYPE=web, preventing accidental
double-cron when an operator splits the deployment.

docker-compose.yml ships the optional app-worker service block commented
out so 'docker compose up' keeps working out of the box; uncommenting it
+ flipping app to HEALTHLOG_PROCESS_TYPE=web is the only change to scale
HTTP horizontally. Healthchecks are isolated per service so neither
startup deadlocks the other; both wait on db: service_healthy.

docs/self-hosting/scaling.md walks through when + how to split.

Co-Authored-By: Marc-André Bombeck <noreply@anthropic.com>

* feat(jobs): G1 — daily off-host encrypted backup to S3-compatible storage

New pg-boss queue 'data-backup-offhost' runs at 02:30 Europe/Berlin, dumps
each user's measurements + medications + intake events + mood entries,
encrypts the JSON under a SEPARATE key (BACKUP_ENCRYPTION_KEY, isolated
from the application ENCRYPTION_KEY), and uploads to any S3-compatible
target via @aws-sdk/client-s3.

Wire format documented in docs/ops/backup-restore.md:
  magic "HLBK" || version 0x01 || iv(12) || authTag(16) || ciphertext

The worker prunes objects older than BACKUP_RETENTION_DAYS (default 30);
operators are encouraged to also configure a bucket-level lifecycle rule.

Admin endpoint POST /api/admin/backup/test issues a 1-byte PUT+GET
roundtrip and returns endpoint + bucket + per-call latency. Credentials
are never returned.

scripts/restore-backup.ts downloads + decrypts a single object so
operators can verify the chain end-to-end without standing up a separate
restore harness.

Why: GDPR Art. 32 + V3 audit "data durability" — without an off-host copy
the only line of defence against a Postgres volume loss is the in-DB
DataBackup model on the same host. Separate-key isolation means a leak of
the application encryption key does not also expose every user's archive.

Co-Authored-By: Marc-André Bombeck <noreply@anthropic.com>

* fix(multi-tenant): address review findings — rotation coverage, hard decrypt errors, web/worker gates

Multi-agent review on PR #135 caught several CRITICAL + HIGH issues that
would have shipped silent data corruption or weakened the threat model:

Encryption + rotation
- scripts/rotate-encryption-key.ts: stop selecting `bugReportTokenEncrypted`
  (column doesn't exist in the schema; the script crashed on the AppSettings
  step). Add the three encrypted columns it was missing —
  NotificationChannel.config, PushSubscription.p256dh, PushSubscription.auth
  — so rotating to a new key actually re-encrypts every encrypted row.
  Without this, dropping the old key id from ENCRYPTION_KEYS would leave
  notification channels and web-push subscriptions permanently
  undecryptable.
- src/lib/crypto.ts: refuse to silently fall back to the active key when a
  legacy bare-base64 row is decrypted but no v1 entry is configured. Throws
  a clear "Restore ENCRYPTION_KEY (or add a v1 entry to ENCRYPTION_KEYS)
  and run scripts/rotate-encryption-key.ts before removing it" so an
  operator who pulled the v1 too early notices immediately instead of
  hitting opaque GCM tag errors.
- docs/ops/encryption-key-rotation.md: full step-by-step + rollback note +
  troubleshooting section. Spells out that rotation is a forward-only
  operation and what to do if the rollback path is needed.

Web / worker split (G3)
- src/proxy.ts: refuse HTTP traffic with a 503 +
  X-HealthLog-Process-Type: worker header when HEALTHLOG_PROCESS_TYPE=worker.
  The worker container (which still runs `node server.js` for now) no
  longer accepts requests, so a misrouted client gets a clear hint instead
  of a partially-served response.
- src/instrumentation.ts: short-circuit before importing the reminder
  worker when HEALTHLOG_PROCESS_TYPE=web — avoids the noisy
  "[instrumentation] CRITICAL" stderr line we emitted on every web boot.
- /api/version added to PUBLIC_PATHS so the compose healthcheck doesn't
  silently 302-redirect through the auth gate (the prior wget treated the
  login page as a 200 success and would never notice an unhealthy backend).

Refresh-token rotation (G4)
- src/lib/auth/refresh-token.ts: reuse-detection now revokes ALL active
  refresh tokens for the user, not only those scoped to the same deviceId.
  An attacker who rotated without an X-Device-Id header would otherwise
  keep their family alive on the no-device branch. Defence-in-depth.
- revokeRefreshToken() also revokes the paired access token so a leaked
  access token cannot outlive the refresh-token sibling that the caller
  just killed (logout-on-device).

Off-host backup (G1)
- src/lib/jobs/offhost-backup.ts: drop the worker-side DeleteObject path.
  Retention is the bucket's lifecycle-policy job; the worker keeps a
  PutObject + GetObject grant only. A compromised worker can no longer
  wipe the backup history. Per-user errors are now surfaced as Wide
  Event warnings + a structured failures[] array on the report so an
  operator can tell which user failed and why.
- scripts/restore-backup.ts: schema-validate the decrypted payload before
  writing to disk, plus an optional --user-id=<id> flag that refuses to
  proceed unless the dump's userId matches. Prevents silent
  cross-tenant restores when an operator points the script at the wrong
  S3 key.

Idempotency
- src/lib/idempotency.ts: the leak-guard already blocked `hlk_`
  (access-token prefix); also block `hlr_` so a future caller wrapping
  the new /api/auth/refresh endpoint with withIdempotency() can't
  accidentally cache a freshly-issued refresh token.

Migration / schema
- prisma/migrations/0025_refresh_tokens/migration.sql: wrap in a single
  transaction; switch to CREATE { TABLE | INDEX } IF NOT EXISTS so a
  partial-deploy retry is a no-op instead of "relation already exists";
  guard the FK ADD with information_schema lookup for the same reason.
- Schema replaces the single-column `@@index([deviceId])` with a
  composite `@@index([userId, deviceId])` (every lookup also filters on
  userId; the old index would never have been used). Comment on
  accessTokenHash corrected to HMAC-SHA-256.

Container / ops
- Dockerfile: install tzdata + ENV TZ=Europe/Berlin so the daily
  pg-boss schedule (`30 2 * * *` Europe/Berlin) actually fires at 02:30
  Berlin instead of silently falling back to UTC on the alpine base.
- docker-compose.yml: tighten the healthcheck to `wget --spider` (matches
  the Dockerfile HEALTHCHECK style, doesn't pipe the body), pass through
  BACKUP_RETENTION_DAYS so a custom retention env reaches the container.
- .env.example: document every new env var added in this PR
  (HEALTHLOG_PROCESS_TYPE, ENCRYPTION_KEYS, ENCRYPTION_ACTIVE_KEY_ID,
  BACKUP_S3_*, BACKUP_ENCRYPTION_KEY, BACKUP_RETENTION_DAYS) with
  comments and the all-or-none semantics for the backup block.

491 tests still green, typecheck clean.

* chore: retrigger CI

---------

Co-authored-by: Marc-André Bombeck <noreply@anthropic.com>
2026-05-08 14:33:39 +02:00