The 36-hour grace window was a module-local constant inside
reminder-worker.ts while the admin route and the CLI both defaulted
to undefined. Lift the constant onto the helper module so the worker,
the admin route, and the CLI all import one source of truth. The
worker continues to pass the constant through to the helper; the
admin route and the CLI keep their existing default-undefined
behaviour (drain everything the operator points at) so a one-shot
pass still collapses every row regardless of recency.
WIP toward v1.4.35. Lands the persistent rollup foundation from the
v1.4.34 cache-aggregation research §9. Reads still served by the
v1.4.34.1 live-SQL aggregator — the parity-safe read-swap
(count/min/max/mean + dailyByType from buckets, slope/r2/sd stay on
live aggregation) ships in a follow-up commit so v1.4.35 is the
single release that flips both foundation + read-swap.
Schema:
- new MeasurementRollup model with composite PK
(userId, type, granularity, bucketStart) and a descending
bucketStart index. RollupGranularity enum (DAY/WEEK/MONTH/YEAR).
- Prisma migration 0067_v1434_measurement_rollups is additive:
CREATE TYPE / CREATE TABLE IF NOT EXISTS / CREATE INDEX IF NOT
EXISTS / ADD CONSTRAINT FK. No ALTER on existing tables.
- db-compat schema-bootstrap path mirrored.
Populator (src/lib/measurements/rollups.ts):
- recomputeBucketsForMeasurement: sync DAY; WEEK/MONTH/YEAR via
pg-boss (singleton-keyed).
- recomputeUserRollups: bounded full-range, used by Apple Health
import + backfill.
- ensureUserRollupsFresh: cheap stale-detection + bounded DAY-90d
recompute. Error-swallowing.
- SQL via $queryRawUnsafe + strict enum whitelist for date_trunc
unit (Postgres 42803 forbids binding the unit literal).
Write hooks on every measurement-mutating endpoint; each wrapped
in try/catch so a populator hiccup never fails the write.
Apple Health import worker recomputes the full range once at
completion (per-row hooks would multiply cost).
Backfill (scripts/backfill-rollups.ts): idempotent, single-user
serial, single-digit minutes for a 100k-row account.
Tests: +14 unit, +3 integration. setup.ts.truncateAllTables
extends to the new table.
Read-swap follow-up:
- comprehensive-aggregator + analytics/route + summaries-slice
each call ensureUserRollupsFresh as a read side-effect (warms
the table). The response stays on the live-SQL aggregator so
byte-shape parity with v1.4.34.5 is preserved.
- Next commit lands the partial read-swap: count/min/max/mean
and dailyByType from buckets (math-identical compose), while
slope/r2/sd stay on indexed live aggregation. Together they
ship as v1.4.35.
Pre-shipping state — do NOT tag v1.4.35 from this commit alone.
Add `src/lib/http/cache-headers.ts` exporting two presets and one
applier. `NO_STORE_BUT_BFCACHE` (`private, max-age=0, must-revalidate`)
replaces the framework's stock `no-store, must-revalidate` on
authenticated page responses; Chromium treats the latter as a hard
bfcache breaker, while the former keeps shared caches out, forces
revalidation on every navigation, and stays bfcache-eligible.
`SHORT_LIVED_PUBLIC` covers static-ish one-hour public payloads.
`applyAuthedHeaders(res)` stamps the bfcache-friendly directive onto
a Response in-place for route handlers that opt into the same posture.
Also add `scripts/print-bundle-report.mjs` (wired earlier as the
`bundle-report` npm script). Turbopack dropped the webpack "First
Load JS" summary line from `next build`; the script reads back the
`.next/analyze/client.json` payload that `@next/bundle-analyzer`
writes when `pnpm analyze` runs and prints a sorted table of the
largest client chunks (parsed, gzip, stat sizes plus totals). Output
restores an at-a-glance signal for client-bundle regressions on
release rounds without re-running Lighthouse.
v1.4.30 PR push failed CI on first attempt because the two new
MeasurementType enums weren't reflected in `docs/api/openapi.yaml`.
A separate `chore(openapi): regenerate the public spec` commit
was needed before the checks turned green. This hook catches the
drift at commit time locally so the contributor sees the
regeneration prompt before pushing.
New `.githooks/pre-commit`:
- Skips when no relevant files (`src/lib/openapi/`,
`src/lib/validations/`, `src/app/api/`, the generator script
itself) are staged.
- Runs `pnpm openapi:check` quietly; exits fast on match.
- On drift, runs `pnpm openapi:generate`, re-stages the
regenerated `docs/api/openapi.yaml`, and continues the commit.
- Skips in CI so the existing `pnpm openapi:check` job in
`.github/workflows/security.yml` stays the server-side
authoritative gate.
`scripts/install-hooks.sh` flips `git config core.hooksPath` to
`.githooks/` — idempotent, one-shot per fresh clone. AGENTS.md
adds the install command to the contributor workflow section.
R-E H-5. Up to v0.4.1.1 the iOS app stored mood-entry prose as
`tags: ["note:<text>"]` because the server had no dedicated column.
The workaround contaminated the tags axis with prose the Coach
evidence shelf misread as taxonomic labels.
Schema (migration 0063, additive):
- mood_entries.note (TEXT, nullable)
Validation + routes:
- createMoodEntrySchema gains optional note (max 500 chars)
- updateMoodEntrySchema accepts note + null to clear
- POST /api/mood-entries persists note alongside tags
- PUT /api/mood-entries/[id] honours partial note updates
- POST /api/mood-entries/bulk carries note through the bulk path
Backfill script at scripts/backfill-mood-note-column.ts lifts the
legacy note:<text> tag entries into the new column. Idempotent
dry-run mode is the default; --confirm commits. Helper logic lives
in src/lib/mood/note-backfill.ts for tight unit coverage.
iOS contract: additive. Old iOS payloads (no note field) round-trip
unchanged. New iOS code paths can read or write note without
touching tags.
Idempotent maintenance tool that collapses pre-Option-A per-sample
APPLE_HEALTH cumulative rows (steps, active energy, flights climbed,
walking + running distance, time in daylight) into one row per day
per type, keyed by the locked dailyStatsExternalId shape. Per
user × type × calendar-day (anchored to User.timezone): SUM the values,
UPSERT a daily row at canonical midday-local (matching the Withings
activity-sync convention), DELETE the per-sample rows.
Three entry points share src/lib/measurements/drain-per-sample-cumulative.ts:
- Library helper for unit + integration tests
- CLI at scripts/drain-per-sample-cumulative.ts with --user / --all
+ --confirm flags (default dry-run)
- Admin endpoint POST /api/admin/drain-per-sample-cumulative gated
by requireAdmin() with dryRun defaulting to true
Operator runs the drain once after the iOS TestFlight build adopts
HealthKitStatisticsService.swift. Re-running on a fully-collapsed
account reports zero buckets touched.
Tests cover bucketing semantics, the canonical-timestamp math across
DST + half-hour zones, idempotent re-runs, dry-run preview, spot-metric
exclusion, and already-collapsed-row skip via real-Postgres.
When `MAXMIND_LICENSE_KEY` is unset the script still drops the `.empty`
marker and exits 0 — that path is the deliberate maintainer override and
the runtime resilience layer in `src/lib/geo.ts` carries it. When the
key IS set, every failure mode now bubbles up:
- A curl error (401 wrong-key, 403 throttle, 5xx upstream, network
drop) returns the curl exit code instead of swallowing it into a
fallback `.empty` marker. The diagnostic names the most common
causes so the operator can act without re-reading the script.
- A tar extraction failure or a missing MMDB inside the tarball
returns 1 with a matching message.
- A final assertion at end-of-script confirms both
`GeoLite2-City.mmdb` and `GeoLite2-ASN.mmdb` ended up non-empty in
the output directory; otherwise exit 1.
Smoke-tested locally: no-key invocation still exits 0 with the `.empty`
marker; a deliberately bad key now exits 22 with the curl 401 surfaced.
The previous resilience pass switched curl errors into a graceful
fallback path but the script still ran `find` against `dirname
$tmp_tarball` (`/tmp` on Linux runners). GitHub Actions runners
populate `/tmp` with `systemd-private-*` directories that are mode
700; `find` walks them, returns exit code 1 on permission-denied,
and under `set -e` aborts the script before our fallback branches
can fire.
Switch each edition to a per-call `mktemp -d` working directory. The
tarball downloads + extracts + finds all happen inside an isolated
directory the script owns. `find` never crosses into shared `/tmp`
state, so permission-denied stops being a class of failure.
Also drop the post-fetch cleanup `find` for date-stamped extraction
dirs — those now live inside `$work_dir` which gets `rm -rf`'d in
every branch.
Verified locally with both a populated download and a bogus key on
a Linux container; both paths exit 0, the bogus-key path writes
`.empty`, the populated path skips the marker.
Build on the previous fetch-script softening (674c6b5b) by hardening
the download path itself, so a transient MaxMind-side issue never
takes the release pipeline down. The runtime gate stays the only
place that surfaces `offline geo unavailable` to the maintainer.
`fetch_edition` now uses `curl --fail-with-body` and captures the
exit code instead of letting `set -e` abort the script. Any non-zero
result — 401 EULA-pending, 403 throttled, 5xx server hiccup, 404
edition-removed, network timeout — flips into the same fallback shape
as the missing-key path: write the `.empty` marker, log a clear
reason on stderr, and return 0 so the second edition still gets its
chance. Tarball-extract and MMDB-locate failures route through the
same fallback. After both fetches, the marker state is summarised:
marker present means at least one edition fell back and the runtime
resolver will use ipwho.is; marker absent means both MMDBs landed
cleanly. The pre-fetch `rm -f "$OUT_DIR/.empty"` from the previous
commit still seeds the clean state; the per-edition fallback re-adds
the marker on demand.
`set -euo pipefail` stays, but every recoverable path either captures
the exit code (`|| curl_exit=$?`) or guards with `if !`, so the
strict-mode abort never reaches a fallback branch.
This unblocks PR #169, where the MAXMIND_LICENSE_KEY secret is set
but MaxMind's account-side GeoLite2 EULA acceptance has not
propagated yet and the download endpoint returns 401. The build no
longer crashes on `curl: (22) The requested URL returned error: 401`.
Replace the hard exit 1 in the `Fetch GeoLite2 databases` step with a
`::warning::` line plus a `.empty` marker in `assets/geolite2/`. The
Dockerfile COPY still has a non-empty source, the build continues, and
the runtime detection layer (next commit) picks up the marker on first
geo lookup and emits a one-shot admin notification. The maintainer is
no longer forced to wire `MAXMIND_LICENSE_KEY` before every release.
`scripts/fetch-geolite2.sh` mirrors the same behaviour: when the key is
unset it drops the marker and exits 0 cleanly; when the key is present
it clears any stale marker before downloading so a freshly populated
directory is not misread as the fallback state.
Rewrites src/lib/geo.ts so lookupIpLocation reads from the bundled
MaxMind GeoLite2-City MMDB first and falls back to the existing
ipwho.is path only for IPs the offline DB cannot resolve. Adds a
sibling lookupIpAsn that returns { asn, carrier } from the
GeoLite2-ASN MMDB so the admin login-overview can grow a carrier
chip alongside the auth-provider chip.
The MMDB files are too large to vendor in git. scripts/fetch-geolite2.sh
downloads them at build time using a MaxMind license key; the
Dockerfile copies assets/geolite2/ into /opt/geolite2/. When the
license key is absent or the files are missing the image still
builds and the resolver falls back to ipwho.is (v1.4.26 behaviour).
GeoLite2 is CC BY-SA 4.0 — attribution lives on the new public
/about page and in docs/audit/v1427-summary.md.
The Security & Quality workflow ran the OpenAPI drift check with
`continue-on-error: true` since v1.4.23 while the Zod registry caught
up with the hand-maintained legacy spec. Coverage is now complete and
the v1.5 iOS Swift codegen needs the contract pinned, so the gate
flips to hard-fail: any route signature change without a regenerated
`docs/api/openapi.yaml` blocks the merge.
The CI error message points at `pnpm openapi:generate` and the
registry under src/lib/openapi/ so the fix is self-evident.
scripts/i18n/build-locale.py walks messages/en.json and emits the four
AI-initial locale bundles (FR / ES / IT / PL) via a layered translation
strategy: full-path overrides for highly contextual strings, a
common-vocab lookup keyed on the lowercase EN value, placeholder
protection that splits {interpolation} tokens and <tag> markers out
before translation and re-glues them verbatim afterwards, and a
fallback to the English value for long-tail prose the script can't
cover. Re-running the script on a fresh en.json keeps the four
locale bundles in parity automatically — handy when new keys land
between releases.
The phase report (.planning/phase-W9e-v1425-translations-report.md)
captures the per-locale vocabulary calls (Apple-Health / Withings
register checks for each language), the system-prompt routing
decision (EN body + REPLY LANGUAGE footer rather than four native
~500-line bodies, to preserve every safety contract verbatim), and
the deferred polish for v1.4.26 (hand-review prose for the highest-
traffic surfaces, native FR/ES/IT/PL prefs strings in
buildPrefsPrefix, locale-native date formats).
Wires the OpenAPI 3.1 generator scaffolding ahead of the iOS-touched
route registrations. `zod-openapi` (samchungy variant, picked in W1
stream-3) reads Zod v4's native `.meta()` so schemas can stay in their
original validation files; `yaml@2` with `sortMapEntries: true` keeps
the emitted spec stable across re-runs.
`pnpm openapi:generate` overwrites `docs/api/openapi.yaml` from the
registry; `pnpm openapi:check` exits non-zero on drift. The legacy
hand-maintained spec is preserved at `docs/api/openapi-v1422-legacy.yaml`
so the iOS contract reference doesn't disappear during the
incremental migration.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Long-overdue prettier sweep; prettier had drifted vs. checked-in code
across ~130 files (mostly long-argument line-break wrapping in TS and
table-padding in markdown). Pure formatting — no semantic changes —
and required to clear the v1.4.6 release gate's `pnpm format:check`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(crypto): G2 — encryption-key versioning + rotation CLI
Adds backwards-compatible support for ENCRYPTION_KEYS (JSON map of
{ "id": "<key>" }) plus ENCRYPTION_ACTIVE_KEY_ID. Ciphertext format is now
"<keyId>.<base64payload>"; legacy bare-base64 ciphertext still decrypts
under the synthetic "v1" id derived from the existing ENCRYPTION_KEY.
scripts/rotate-encryption-key.ts re-encrypts every encrypted column on
User / WithingsConnection / AppSettings with the active key. Idempotent —
rows already on the active id are skipped.
Why: enables zero-downtime rotation when a key is suspected leaked,
without forcing every encrypted row through a single big-bang rewrite.
Co-Authored-By: Marc-André Bombeck <noreply@anthropic.com>
* feat(auth): G4 — native-Bearer UA gating + rotating refresh tokens
Web callers (UA contains 'Mozilla/') keep the legacy 90d Bearer when they
explicitly opt in via X-Client-Type: native. Native callers (HealthLog-iOS,
HealthLog-iPad, n8n, Health-Connect, unrecognised UAs) now receive a 24h
access token plus a 60d rotating refresh token in the login response.
New /api/auth/refresh endpoint atomically validates + rotates the refresh
token (one-time-use), revokes the previously paired access token, and
audit-logs every refresh + revoke. Reuse of an already-consumed refresh
token revokes the entire family for the user/device — defence against a
stolen refresh token.
Migration 0025_refresh_tokens adds the table; the only DDL is CREATE
TABLE so it is forward-compatible. Down migration documented at
docs/ops/migrations/0025-down.sql.
Why: the V3 audit (still-V2-A) flagged that 90d tokens are too long for
native callers when refresh-token rotation is now table-stakes for mobile
apps. Web flows are unchanged so existing browser sessions keep working.
Co-Authored-By: Marc-André Bombeck <noreply@anthropic.com>
* feat(infra): G3 — web/worker container split via HEALTHLOG_PROCESS_TYPE
Adds a tiny startup gate (src/lib/process-type.ts) so the same Docker image
can run as web-only, worker-only, or 'all' (default — single-container,
backwards compatible). assertSubsystemEnabled() refuses to boot the
reminder worker when HEALTHLOG_PROCESS_TYPE=web, preventing accidental
double-cron when an operator splits the deployment.
docker-compose.yml ships the optional app-worker service block commented
out so 'docker compose up' keeps working out of the box; uncommenting it
+ flipping app to HEALTHLOG_PROCESS_TYPE=web is the only change to scale
HTTP horizontally. Healthchecks are isolated per service so neither
startup deadlocks the other; both wait on db: service_healthy.
docs/self-hosting/scaling.md walks through when + how to split.
Co-Authored-By: Marc-André Bombeck <noreply@anthropic.com>
* feat(jobs): G1 — daily off-host encrypted backup to S3-compatible storage
New pg-boss queue 'data-backup-offhost' runs at 02:30 Europe/Berlin, dumps
each user's measurements + medications + intake events + mood entries,
encrypts the JSON under a SEPARATE key (BACKUP_ENCRYPTION_KEY, isolated
from the application ENCRYPTION_KEY), and uploads to any S3-compatible
target via @aws-sdk/client-s3.
Wire format documented in docs/ops/backup-restore.md:
magic "HLBK" || version 0x01 || iv(12) || authTag(16) || ciphertext
The worker prunes objects older than BACKUP_RETENTION_DAYS (default 30);
operators are encouraged to also configure a bucket-level lifecycle rule.
Admin endpoint POST /api/admin/backup/test issues a 1-byte PUT+GET
roundtrip and returns endpoint + bucket + per-call latency. Credentials
are never returned.
scripts/restore-backup.ts downloads + decrypts a single object so
operators can verify the chain end-to-end without standing up a separate
restore harness.
Why: GDPR Art. 32 + V3 audit "data durability" — without an off-host copy
the only line of defence against a Postgres volume loss is the in-DB
DataBackup model on the same host. Separate-key isolation means a leak of
the application encryption key does not also expose every user's archive.
Co-Authored-By: Marc-André Bombeck <noreply@anthropic.com>
* fix(multi-tenant): address review findings — rotation coverage, hard decrypt errors, web/worker gates
Multi-agent review on PR #135 caught several CRITICAL + HIGH issues that
would have shipped silent data corruption or weakened the threat model:
Encryption + rotation
- scripts/rotate-encryption-key.ts: stop selecting `bugReportTokenEncrypted`
(column doesn't exist in the schema; the script crashed on the AppSettings
step). Add the three encrypted columns it was missing —
NotificationChannel.config, PushSubscription.p256dh, PushSubscription.auth
— so rotating to a new key actually re-encrypts every encrypted row.
Without this, dropping the old key id from ENCRYPTION_KEYS would leave
notification channels and web-push subscriptions permanently
undecryptable.
- src/lib/crypto.ts: refuse to silently fall back to the active key when a
legacy bare-base64 row is decrypted but no v1 entry is configured. Throws
a clear "Restore ENCRYPTION_KEY (or add a v1 entry to ENCRYPTION_KEYS)
and run scripts/rotate-encryption-key.ts before removing it" so an
operator who pulled the v1 too early notices immediately instead of
hitting opaque GCM tag errors.
- docs/ops/encryption-key-rotation.md: full step-by-step + rollback note +
troubleshooting section. Spells out that rotation is a forward-only
operation and what to do if the rollback path is needed.
Web / worker split (G3)
- src/proxy.ts: refuse HTTP traffic with a 503 +
X-HealthLog-Process-Type: worker header when HEALTHLOG_PROCESS_TYPE=worker.
The worker container (which still runs `node server.js` for now) no
longer accepts requests, so a misrouted client gets a clear hint instead
of a partially-served response.
- src/instrumentation.ts: short-circuit before importing the reminder
worker when HEALTHLOG_PROCESS_TYPE=web — avoids the noisy
"[instrumentation] CRITICAL" stderr line we emitted on every web boot.
- /api/version added to PUBLIC_PATHS so the compose healthcheck doesn't
silently 302-redirect through the auth gate (the prior wget treated the
login page as a 200 success and would never notice an unhealthy backend).
Refresh-token rotation (G4)
- src/lib/auth/refresh-token.ts: reuse-detection now revokes ALL active
refresh tokens for the user, not only those scoped to the same deviceId.
An attacker who rotated without an X-Device-Id header would otherwise
keep their family alive on the no-device branch. Defence-in-depth.
- revokeRefreshToken() also revokes the paired access token so a leaked
access token cannot outlive the refresh-token sibling that the caller
just killed (logout-on-device).
Off-host backup (G1)
- src/lib/jobs/offhost-backup.ts: drop the worker-side DeleteObject path.
Retention is the bucket's lifecycle-policy job; the worker keeps a
PutObject + GetObject grant only. A compromised worker can no longer
wipe the backup history. Per-user errors are now surfaced as Wide
Event warnings + a structured failures[] array on the report so an
operator can tell which user failed and why.
- scripts/restore-backup.ts: schema-validate the decrypted payload before
writing to disk, plus an optional --user-id=<id> flag that refuses to
proceed unless the dump's userId matches. Prevents silent
cross-tenant restores when an operator points the script at the wrong
S3 key.
Idempotency
- src/lib/idempotency.ts: the leak-guard already blocked `hlk_`
(access-token prefix); also block `hlr_` so a future caller wrapping
the new /api/auth/refresh endpoint with withIdempotency() can't
accidentally cache a freshly-issued refresh token.
Migration / schema
- prisma/migrations/0025_refresh_tokens/migration.sql: wrap in a single
transaction; switch to CREATE { TABLE | INDEX } IF NOT EXISTS so a
partial-deploy retry is a no-op instead of "relation already exists";
guard the FK ADD with information_schema lookup for the same reason.
- Schema replaces the single-column `@@index([deviceId])` with a
composite `@@index([userId, deviceId])` (every lookup also filters on
userId; the old index would never have been used). Comment on
accessTokenHash corrected to HMAC-SHA-256.
Container / ops
- Dockerfile: install tzdata + ENV TZ=Europe/Berlin so the daily
pg-boss schedule (`30 2 * * *` Europe/Berlin) actually fires at 02:30
Berlin instead of silently falling back to UTC on the alpine base.
- docker-compose.yml: tighten the healthcheck to `wget --spider` (matches
the Dockerfile HEALTHCHECK style, doesn't pipe the body), pass through
BACKUP_RETENTION_DAYS so a custom retention env reaches the container.
- .env.example: document every new env var added in this PR
(HEALTHLOG_PROCESS_TYPE, ENCRYPTION_KEYS, ENCRYPTION_ACTIVE_KEY_ID,
BACKUP_S3_*, BACKUP_ENCRYPTION_KEY, BACKUP_RETENTION_DAYS) with
comments and the all-or-none semantics for the backup block.
491 tests still green, typecheck clean.
* chore: retrigger CI
---------
Co-authored-by: Marc-André Bombeck <noreply@anthropic.com>
HealthLog — personal health-tracking PWA with weight, blood pressure,
pulse, mood tracking, medication compliance, Withings integration,
doctor report PDF export, and Dracula-themed UI.
