Forwarding headers were believed from any connection on a private address.
Behind Docker Desktop, OrbStack and docker-proxy every connection to a
published port arrives from the gateway, a private address, so a caller
reaching the port directly could again choose the client address that rate
limits, the audit log and sign-in notices record. TRUSTED_PROXY_CIDRS
(comma-separated CIDRs) now names the proxies: when set, only those peers may
report a client address. The private-range default stays when it is unset,
the boot log states the rule in force in one line, and the reverse-proxy
guide says what to set for a proxy on the same host, on a Docker network, or
on a public address (which, unlisted, puts all clients in one bucket).
An invalid TRUST_PROXY_HOPS threw on every request that resolved the client
address, so every route answered 500, the health probe included. It now
resolves no address, warns once, and is named at boot.
The client-IP resolver trusted one forwarding hop by default and read the
rightmost X-Forwarded-For entry. Next fills that header with the socket's
address only when the request carries none, so a caller reaching the
published port directly could send its own and choose a fresh address per
request: every per-IP limit (sign-in, second factor, passkey, share unlock,
webhooks) became per-request, and sessions, the audit log and sign-in
notices recorded the chosen address.
The server now writes each connection's own peer address into an internal
header from Node's http.server.request.start diagnostics channel, installed
at boot and overwriting anything a caller sent under that name. The resolver
reads forwarding headers only when that peer is loopback, private,
unique-local or CGNAT, where a reverse proxy sits; a public peer is its own
client address, and a log line says once that its forwarding headers were
ignored. With TRUST_PROXY_HOPS=0 the peer is used instead of no address, so
a directly exposed host no longer shares one bucket across all anonymous
callers. Proxies on the same host, Docker network or tailnet need no change.
auditLog resolves login locations through lib/geo.ts, and the resolver
imported the notification dispatcher for the offline-database notice. The
dispatcher's collaborators write audit rows, so audit, geo, the dispatcher,
channel state, delivery identity, managed-profile lifecycle, the OAuth
connect target and the grants formed one strongly connected cluster.
The notice now lives in lib/notifications/offline-geo-notice.ts. The
resolver keeps deciding when it runs (first public-IP lookup of a process,
again after a runtime database fetch) through a slot the notice module
fills at boot from instrumentation.ts. Runtime import cycles fall from 13
to 7.
The server reads messages/<locale>.json from disk on first use. A missing
or corrupt file threw, and since the matchers iterate every locale, one bad
non-English catalog turned every request that touched them into a 500.
That locale is now served the English catalog, and the failure is logged
once as i18n.catalog.fallback with the locale and the reason. English is
the fallback itself: a process whose messages/en.json is missing or does
not parse refuses to start, with the file and the reason in the message.
The image build checks the catalogs twice with
scripts/verify-message-catalogs.mjs: in the builder against the locale
list in the source, and in the runner stage against the copies the image
carries.
The profile section in docs/contributing/benchmarks.md says how to run
scripts/bench/boot-profile.ts, what the profile of the v1.44 runtime
found (the worker graph as one 753 ms block, a live heap of about
165 MB of which the query compiler holds 50 and the catalogs 33) and what
changed. scripts/bench/results/v1.45.0-*.md hold the before and after
numbers per area, from back-to-back runs on the same machine. The size
baselines take the lowered numbers.
The event-loop monitor starts first in register(), so loading the
database client and the worker graph is inside its view, and a stall line
now carries the process uptime, the startup phase, the boot step running
at the time and the oldest work in flight. Each boot step is timed, and
the boot ends with one boot.steps line.
The five registrars and the lifecycle helpers are imported one at a time
after boss.start(), with a turn of the event loop in between, instead of
all at once when the worker module loads. The longest stall is then the
largest registrar graph rather than the sum of all five.
The server answers before pg-boss is bound, and a producer rebinds in
the background after losing its connection. A request in either window
read the boss as null and skipped its enqueue: OAuth callbacks lost the
first sync and the history backfill, measurement and mood writes lost
their rollup recompute, WHOOP webhook deliveries were dropped.
whenBossReady() waits for the boss while a start is under way (boot or
reconnect), bounded at 60 s and logged when it gives up, and answers
null at once when nothing is coming. enqueueWhenBossReady() sends at
once through a bound boss and otherwise defers the send to the
background, so the request is not held for the boot.
Migration 0434 turns the route opt-in on for every account that already
holds a stored workout track, so its tracks keep arriving after the
upgrade; only accounts without one start off.
Workouts stored before this release with a track but no elevation get
their metres climbed written into Workout.elevationM by a one-time
background pass. It runs detached at worker boot, reads one track at a
time with a pause between batches of 100, and keeps its position in
app_settings, so a restart resumes and a finished pass ends at once.
The "starting" health reason is bounded: after five minutes the probe
says plain degraded and the process logs one line, and the time the
background role took to come up is logged either way. The compose
healthcheck's ten-minute start period already keeps those probes from
counting; its comment now says so.
POST /api/measurements/batch ran one interactive reconciler transaction
per fresh sample: a savepoint, three advisory locks, two candidate reads
and the insert, about ten round trips. A 500-entry upload of new samples
was 500 of those, the 2 to 15 second outliers. Fresh samples now land
through INSERT ... ON CONFLICT DO NOTHING under the same per-account
advisory lock. Both measurement identities are full unique indexes, so
the statement inserts exactly what the reconciler would have inserted,
and every skipped row still goes through the reconciler for its own
verdict. Only rows whose external and natural keys each occur once in
the batch are buffered, and the buffer is flushed before any other row
is reconciled, so the order of effects stays the sequential one. The
contract and every per-entry status are unchanged; a golden test runs
each batch through both paths against the same table.
/api/health took seconds after a start because register() awaited the
whole pg-boss worker start (queues, schedules, boot discovery) before
the server took a request. The worker, or a web process's producer, now
starts in the background after the blocking boot checks; the probe
answers at once with reason "starting" until it is up, and gives the
database two seconds before calling it not answering.
The getting-started requirements now say a host needs 3 GB of RAM, 4 GB
recommended, and drop the advice to lower APP_MEMORY_LIMIT to 1g on a
2 GB host. The compose and .env example comments say the same.
At boot the app reads its cgroup memory limit (v2, then v1) and logs one
line when it is below MEMORY_FLOOR_MB. The support report adds the
container limit and a note below the floor, and Admin > System status
shows "below the supported floor" next to the existing low-heap flag.
The app keeps running either way.
runtime.heap.sampled records heap, old space, RSS, external memory and
the work in flight every five minutes, every minute at or above 70 % of
the heap limit. At 85 % runtime.heap.pressure lists the oldest requests
and jobs in flight and the routes most of them share, once, re-armed
only after the heap falls below 70 % and thirty minutes pass. A
heap-limit crash leaves no other record of what the process was doing.
HEALTHLOG_HEAP_SNAPSHOT=1 adds one heap snapshot at that moment into
HEALTHLOG_DIAG_DIR (0700 directory, 0600 file), off by default.
docs/ops/heap-diagnostics.md warns that a snapshot holds decrypted
health data and keys and says how to delete it.
Stored times are UTC in timestamp columns without a zone, and Postgres
reads now(), CURRENT_TIMESTAMP and offset-free timestamptz parameters
through the session time zone. On a database whose default is not UTC
every expiry check ran off by the offset. The Prisma pool, pg-boss, the
lock and the maintenance connections now send TimeZone=UTC as a startup
option, or SET it on connect when the startup options are disabled for
a pooler. A boot check reads SHOW TimeZone and warns on anything else.
Every configured key id now has a canary in encryption_key_canaries,
checked at boot. A key that cannot open its canary, or existing data
written before the canary existed, puts the process in a refusal state:
every API route answers 503 encryption.key_mismatch, /api/health reports
reason encryption_key_mismatch, the pages render one explanation, and
neither the queue producer nor the worker starts. The process stays up
so the reason is visible instead of hiding in a crash loop.
The admin Encryption page gains a "Back up your encryption key" step:
key id and fingerprint (never the key), where the key lives per
platform, a check of a pasted copy that is compared and discarded, and
a confirmation bound to the active key id and fingerprint. The overview
gets a tile and admins get a dashboard banner while it is due.
HEALTHLOG_PLATFORM picks the platform the step opens on. Adds the
TrueNAS self-hosting page and key backup sections in the docs.
The fail-closed crypto loader throws a stack trace on a missing/short
ENCRYPTION_KEY rather than a sentence an operator can act on, and
check-env is a dev/CI tool the image operator never runs. Add a boot-time
readiness summary (instrumentation register) that prints a clear line per
core secret — with the exact openssl fix on a bad one — before anything
throws, plus a one-glance status of the optional channels and the
Secure-cookie transport hint. Never logs secret values; never throws.
One-shot prettier pass so the new format:check gate starts from a clean
baseline. Formatting only — no behaviour change. Generator-owned and
vendored artifacts stay untouched via .prettierignore.
Production showed multi-second windows where every response — routes,
RSC payloads, static assets — waited together while handler durations
stayed under half a second: the process stalled, and nothing
route-shaped could name it. A self-rescheduling tick now records how
late it fires; a tick past the stall threshold emits its own
event_loop_stall background event, and every wide event carries the
recent worst lag so the first request finishing after a stall is
annotated with it. Measured via a timeout chain rather than
perf_hooks.monitorEventLoopDelay — the histogram's sampling timer races
a sibling read-and-reset interval in the same timers phase and the
stall sample can vanish into the reset window.
* feat(crypto): G2 — encryption-key versioning + rotation CLI
Adds backwards-compatible support for ENCRYPTION_KEYS (JSON map of
{ "id": "<key>" }) plus ENCRYPTION_ACTIVE_KEY_ID. Ciphertext format is now
"<keyId>.<base64payload>"; legacy bare-base64 ciphertext still decrypts
under the synthetic "v1" id derived from the existing ENCRYPTION_KEY.
scripts/rotate-encryption-key.ts re-encrypts every encrypted column on
User / WithingsConnection / AppSettings with the active key. Idempotent —
rows already on the active id are skipped.
Why: enables zero-downtime rotation when a key is suspected leaked,
without forcing every encrypted row through a single big-bang rewrite.
Co-Authored-By: Marc-André Bombeck <noreply@anthropic.com>
* feat(auth): G4 — native-Bearer UA gating + rotating refresh tokens
Web callers (UA contains 'Mozilla/') keep the legacy 90d Bearer when they
explicitly opt in via X-Client-Type: native. Native callers (HealthLog-iOS,
HealthLog-iPad, n8n, Health-Connect, unrecognised UAs) now receive a 24h
access token plus a 60d rotating refresh token in the login response.
New /api/auth/refresh endpoint atomically validates + rotates the refresh
token (one-time-use), revokes the previously paired access token, and
audit-logs every refresh + revoke. Reuse of an already-consumed refresh
token revokes the entire family for the user/device — defence against a
stolen refresh token.
Migration 0025_refresh_tokens adds the table; the only DDL is CREATE
TABLE so it is forward-compatible. Down migration documented at
docs/ops/migrations/0025-down.sql.
Why: the V3 audit (still-V2-A) flagged that 90d tokens are too long for
native callers when refresh-token rotation is now table-stakes for mobile
apps. Web flows are unchanged so existing browser sessions keep working.
Co-Authored-By: Marc-André Bombeck <noreply@anthropic.com>
* feat(infra): G3 — web/worker container split via HEALTHLOG_PROCESS_TYPE
Adds a tiny startup gate (src/lib/process-type.ts) so the same Docker image
can run as web-only, worker-only, or 'all' (default — single-container,
backwards compatible). assertSubsystemEnabled() refuses to boot the
reminder worker when HEALTHLOG_PROCESS_TYPE=web, preventing accidental
double-cron when an operator splits the deployment.
docker-compose.yml ships the optional app-worker service block commented
out so 'docker compose up' keeps working out of the box; uncommenting it
+ flipping app to HEALTHLOG_PROCESS_TYPE=web is the only change to scale
HTTP horizontally. Healthchecks are isolated per service so neither
startup deadlocks the other; both wait on db: service_healthy.
docs/self-hosting/scaling.md walks through when + how to split.
Co-Authored-By: Marc-André Bombeck <noreply@anthropic.com>
* feat(jobs): G1 — daily off-host encrypted backup to S3-compatible storage
New pg-boss queue 'data-backup-offhost' runs at 02:30 Europe/Berlin, dumps
each user's measurements + medications + intake events + mood entries,
encrypts the JSON under a SEPARATE key (BACKUP_ENCRYPTION_KEY, isolated
from the application ENCRYPTION_KEY), and uploads to any S3-compatible
target via @aws-sdk/client-s3.
Wire format documented in docs/ops/backup-restore.md:
magic "HLBK" || version 0x01 || iv(12) || authTag(16) || ciphertext
The worker prunes objects older than BACKUP_RETENTION_DAYS (default 30);
operators are encouraged to also configure a bucket-level lifecycle rule.
Admin endpoint POST /api/admin/backup/test issues a 1-byte PUT+GET
roundtrip and returns endpoint + bucket + per-call latency. Credentials
are never returned.
scripts/restore-backup.ts downloads + decrypts a single object so
operators can verify the chain end-to-end without standing up a separate
restore harness.
Why: GDPR Art. 32 + V3 audit "data durability" — without an off-host copy
the only line of defence against a Postgres volume loss is the in-DB
DataBackup model on the same host. Separate-key isolation means a leak of
the application encryption key does not also expose every user's archive.
Co-Authored-By: Marc-André Bombeck <noreply@anthropic.com>
* fix(multi-tenant): address review findings — rotation coverage, hard decrypt errors, web/worker gates
Multi-agent review on PR #135 caught several CRITICAL + HIGH issues that
would have shipped silent data corruption or weakened the threat model:
Encryption + rotation
- scripts/rotate-encryption-key.ts: stop selecting `bugReportTokenEncrypted`
(column doesn't exist in the schema; the script crashed on the AppSettings
step). Add the three encrypted columns it was missing —
NotificationChannel.config, PushSubscription.p256dh, PushSubscription.auth
— so rotating to a new key actually re-encrypts every encrypted row.
Without this, dropping the old key id from ENCRYPTION_KEYS would leave
notification channels and web-push subscriptions permanently
undecryptable.
- src/lib/crypto.ts: refuse to silently fall back to the active key when a
legacy bare-base64 row is decrypted but no v1 entry is configured. Throws
a clear "Restore ENCRYPTION_KEY (or add a v1 entry to ENCRYPTION_KEYS)
and run scripts/rotate-encryption-key.ts before removing it" so an
operator who pulled the v1 too early notices immediately instead of
hitting opaque GCM tag errors.
- docs/ops/encryption-key-rotation.md: full step-by-step + rollback note +
troubleshooting section. Spells out that rotation is a forward-only
operation and what to do if the rollback path is needed.
Web / worker split (G3)
- src/proxy.ts: refuse HTTP traffic with a 503 +
X-HealthLog-Process-Type: worker header when HEALTHLOG_PROCESS_TYPE=worker.
The worker container (which still runs `node server.js` for now) no
longer accepts requests, so a misrouted client gets a clear hint instead
of a partially-served response.
- src/instrumentation.ts: short-circuit before importing the reminder
worker when HEALTHLOG_PROCESS_TYPE=web — avoids the noisy
"[instrumentation] CRITICAL" stderr line we emitted on every web boot.
- /api/version added to PUBLIC_PATHS so the compose healthcheck doesn't
silently 302-redirect through the auth gate (the prior wget treated the
login page as a 200 success and would never notice an unhealthy backend).
Refresh-token rotation (G4)
- src/lib/auth/refresh-token.ts: reuse-detection now revokes ALL active
refresh tokens for the user, not only those scoped to the same deviceId.
An attacker who rotated without an X-Device-Id header would otherwise
keep their family alive on the no-device branch. Defence-in-depth.
- revokeRefreshToken() also revokes the paired access token so a leaked
access token cannot outlive the refresh-token sibling that the caller
just killed (logout-on-device).
Off-host backup (G1)
- src/lib/jobs/offhost-backup.ts: drop the worker-side DeleteObject path.
Retention is the bucket's lifecycle-policy job; the worker keeps a
PutObject + GetObject grant only. A compromised worker can no longer
wipe the backup history. Per-user errors are now surfaced as Wide
Event warnings + a structured failures[] array on the report so an
operator can tell which user failed and why.
- scripts/restore-backup.ts: schema-validate the decrypted payload before
writing to disk, plus an optional --user-id=<id> flag that refuses to
proceed unless the dump's userId matches. Prevents silent
cross-tenant restores when an operator points the script at the wrong
S3 key.
Idempotency
- src/lib/idempotency.ts: the leak-guard already blocked `hlk_`
(access-token prefix); also block `hlr_` so a future caller wrapping
the new /api/auth/refresh endpoint with withIdempotency() can't
accidentally cache a freshly-issued refresh token.
Migration / schema
- prisma/migrations/0025_refresh_tokens/migration.sql: wrap in a single
transaction; switch to CREATE { TABLE | INDEX } IF NOT EXISTS so a
partial-deploy retry is a no-op instead of "relation already exists";
guard the FK ADD with information_schema lookup for the same reason.
- Schema replaces the single-column `@@index([deviceId])` with a
composite `@@index([userId, deviceId])` (every lookup also filters on
userId; the old index would never have been used). Comment on
accessTokenHash corrected to HMAC-SHA-256.
Container / ops
- Dockerfile: install tzdata + ENV TZ=Europe/Berlin so the daily
pg-boss schedule (`30 2 * * *` Europe/Berlin) actually fires at 02:30
Berlin instead of silently falling back to UTC on the alpine base.
- docker-compose.yml: tighten the healthcheck to `wget --spider` (matches
the Dockerfile HEALTHCHECK style, doesn't pipe the body), pass through
BACKUP_RETENTION_DAYS so a custom retention env reaches the container.
- .env.example: document every new env var added in this PR
(HEALTHLOG_PROCESS_TYPE, ENCRYPTION_KEYS, ENCRYPTION_ACTIVE_KEY_ID,
BACKUP_S3_*, BACKUP_ENCRYPTION_KEY, BACKUP_RETENTION_DAYS) with
comments and the all-or-none semantics for the backup block.
491 tests still green, typecheck clean.
* chore: retrigger CI
---------
Co-authored-by: Marc-André Bombeck <noreply@anthropic.com>
HealthLog — personal health-tracking PWA with weight, blood pressure,
pulse, mood tracking, medication compliance, Withings integration,
doctor report PDF export, and Dracula-themed UI.
Marc-André Bombeck