The admission drain ran each full-history import with no wide event in
scope, so every external call, warning and annotation the provider walk
recorded was dropped and a failing backfill left only its error message.
Each admitted run now opens a job.integration_backfill event with its kind,
identity, job id and slice, emitted when the run settles. The Health Connect
import worker gets the same treatment.
The external call list on one event is bounded so a long walk cannot hold
thousands of entries: failed and annotated calls keep the larger share,
the rest is counted in external_calls_omitted.
A wide event carries one action and the last annotation wins, so a hop
that failed before the next provider answered, one failed account among
six, or a caught error the code carried on past was either a meta key on
someone else's info line or nothing at all. `emitSignal` writes one extra
line under a stable action and level, joined to the surrounding event by
request and trace id; `logCaught` / `caughtAs` are its warn-level shape
for a catch that degrades on purpose.
Every failed pg-boss run now writes `job.run.failed` at error with the
queue and how many runs of it in a row have failed, counted from pg-boss's
own rows (`readConsecutiveRunFailures`). `reportJobRun` takes a fan-out
pass's whole candidate list: an all-failed run is `job.run.failed`, a
partial one `job.run.partial` at warn, and the run reports `all_failed` in
its facts so a completed night with nothing done still counts toward the
streak. The repeated-failure alert keys on that one action and number.
Retention is per queue now. `failureReaderRetention` refuses a queue
retention under 96 hours (the 72-hour reader window plus a day), and a
guard refuses any `deleteAfterSeconds` that is neither that helper nor a
literal at or above the floor.
A refusal the request was meant to get (any error carrying a 4xx status)
is recorded at warn instead of error. Expired-token 401s were most of the
error lines and hid the real failures between them.
A background job opens one wide event for its whole run, and the per-request
memo cached the operator switches, module availability, module preferences,
provider presence and consent kinds on it. The nightly status passes loop
over every candidate inside that one event, so a switch turned off mid-run
did not stop the remaining users, and the wire re-check read the same cached
answer.
memoizePerRequest takes a freshInBackground option; every input that decides
whether AI work may run sets it, so a background caller reads them on every
call while a request still reads them once.
A 502 from a channel test reports the user's relay or mail setup, not a fault on this server, yet the status alone raised every press to an error-level event. The event builder gains a ceiling for the status-derived level, and the test routes cap it at warn on the 502 path. The 500 for an unnamed failure stays at error, and explicit elevations are unaffected.
The notifications guide notes that a reverse proxy which replaces 5xx pages hides the detail, so the card falls back to Test failed.
Refs #947
The mark named its action unconditionally, so a job that had already
named its own (the low-stock underflow branch) or a settings route
whose test then failed at the relay lost that name on the wide event.
Set the action only when the event has none, and always land the
channel and origin meta pair, through a small hasAction() on the
builder. Note the parse-time floor on the Nightscout page, which shares
the grammar.
Refs #947
addWarning pushed the raw string while setError, setHttp and
addExternalCall all scrub through redactSecrets. A warning routinely
carries an outbound error string, and for several integrations that
string is the request URL with a credential in it, so a secret could
reach stdout and the log store unredacted. It now scrubs on the way in
like the other entry points, and a test asserts a licence-key-bearing
warning is masked.
- add an import-free contract module both bundles can name, holding the
header names, the parse and the verdict
- add the server fence: header read, wide-event stamp, and the refusal
- add RecordSessionChangedError as a third sibling on the existing
sharing-error arm, so the 409 needs no new serialisation branch
Adds AuditLog.actorUserId (migration 0294) and threads it centrally, out
of the wide-event context the acting-account resolver already stamps. No
call site passes an actor, so no route can forget one.
NULL keeps meaning "acted for themselves" — the reading every existing
row carries. The threading fills the column only when a request resolved
an acting account AND the row is being filed under that account, so a
self-action, a refusal, and a delegate's row about themselves all stay
NULL rather than acquiring an id that would rewrite what the history says.
Delegated reads coalesce to one row per owner, delegate and day, through
a partial unique index and an insert that expects to lose the race. The
alternative is a row per request on the busiest table in the schema and
an activity view made of identical lines.
The column carries no foreign key, against the grain of user_id beside
it: SET NULL there erases an account from its own history, while here it
would turn "the delegate opened this record" into "the owner opened this
record". An id that no longer resolves reads as a deleted account; an id
that silently became NULL reads as a lie.
One resolver, both transports, on every request. The cookie path reads
the carrier off the session row it has already loaded; the Bearer path
reads the `X-HealthLog-Account` selector header, because stamping switch
state onto a long-lived token would make the credential itself ambient
authority. Two carriers converge on one function for the same reason
`resolveBearerToken` lives in one file for two wires.
Three declared modes. `requireRecordAuth("read" | "write")` honours a
valid switch and substitutes the owner as the data scope while keeping
the delegate as the actor. `requireActorAuth()` always serves the
caller's own rows and keeps working under a switch, which is what the
switcher and the banner need.
Bare `requireAuth()` refuses to run under a switch at all, with a stable
`sharing.not_permitted` code. It does not fall back to the caller's own
account, and that arm is the point of the whole design: a delegate who
believes they are reading the owner's record and is handed their own is
the same data mixing pointed the other way. A route nobody has reviewed
is inaccessible while switched, never wrongly accessible.
The grant is loaded from Postgres on the request that uses it. Nothing
memoises the verdict, so a revocation lands on the delegate's next
request rather than their next login.
Refusals do not enumerate. The denial takes no constructor arguments and
carries a fixed message, and neither branch looks the named account up:
an account that does not exist and one that granted nothing produce the
same empty row from the same query, so the two answers are identical in
bytes and in timing rather than by careful wording. The reason reaches
the audit row and the wide event and nothing else.
`getSession()` still answers who is calling. That is what leaves
`requireAdmin`, `requireCookieAuth` and `requireFreshMfa` structurally
out of a switch's reach; the row's carrier rides out beside the session
id as a value to be checked, never as a decision already made.
The selector header is a Bearer carrier only. Sent over the cookie
transport, or to an actor surface, it is refused rather than ignored. A
client that names an account on a route which never reads one has a bug,
and a correct response to a wrong expectation hides that bug until it
arrives as a support case about data that looks wrong.
Five test files gain the session shape their fixtures always claimed to
have; they were building an auth context without one.
Three leaks of the same class - data leaving the process unprotected.
addMeta and addExternalCall wrote their values verbatim while setError
and setHttp scrubbed, so the builder's redaction contract held for two
entry points out of four. The finished event is serialised whole to
stdout and the log store, which made meta a trusted sink by accident: a
Telegram request timeout put the bot token there, and the AI provider
chain fed upstream error bodies through it. Both now scrub, at any depth.
Scrubbing cannot help with the second half. A gateway that echoes the
request it rejected echoes the prompt, and on that path the prompt
carries the health snapshot; no pattern separates that from any other
text. Upstream bodies are therefore classified rather than quoted - the
diagnostic question is which rejection occurred and whether it repeats,
which a classification answers without the content.
The idempotency replay cache stored response bodies unencrypted. The
create paths echo their own decrypted payload, so cycle notes, mood text
and allergy reactions sat in cleartext for 24 hours in a column that
lands in every backup; the existing guard missed it because it matches
secret shapes and health data is not secret-shaped. Bodies are encrypted
now, and when encryption is unavailable the response is not cached at
all rather than written in the clear.
chore(release): v1.30.18
Harden the remote MCP endpoint and the in-repo OAuth bridge against the
adversarial review of the milestone branch. The two ship-blockers made the
"read-only, revocable" promise false; the rest tightened the surface.
Audience binding at the resource server: a token whose only grant is the new
`health:read` scope is the MCP read credential. The REST edge now accepts it on
safe (read) methods and refuses it on writes/deletes with 403, so a connector
token can read over `/mcp` and GET routes but can never POST/DELETE the record
despite a "read-only" consent. The `/mcp` resolver is unaffected.
Revocable, theft-detectable refresh tokens: refresh artifacts now anchor to a
persistent connection row. Revoking a connection stops every future refresh and
kills its access tokens; a replayed (already-rotated) refresh is treated as
theft and revokes the whole connection. Connections surface in settings as the
unit a user can disconnect. The old access token is revoked on rotation.
Also: fail closed on the OAuth/`/mcp` surface when no canonical origin is
configured and mark the discovery documents non-cacheable; honour the global API
kill-switch on `/mcp`, `/token`, `/authorize`, and `/register`; cap the CIMD and
registration body reads while streaming instead of after; show the verified
client origin on consent; validate CIMD redirect-URI schemes; assert same-origin
on the consent POST; and add a daily prune for expired connector tokens.
New migration 0195 adds the connection anchor and links access tokens to it.
The wide-event type declared loop_p99_ms while the monitor publishes
loop_last_ms — a dashboard built on the declared field would find
nothing; the type now states what is emitted. Ladder reconciliation
collapses duplicate entries first-occurrence-wins, so a duplicate-laden
ladder written through the API can no longer overflow the schema's
eight-entry cap when the settings page echoes the resolved shape back.
Production showed multi-second windows where every response — routes,
RSC payloads, static assets — waited together while handler durations
stayed under half a second: the process stalled, and nothing
route-shaped could name it. A self-rescheduling tick now records how
late it fires; a tick past the stall threshold emits its own
event_loop_stall background event, and every wide event carries the
recent worst lag so the first request finishing after a stall is
annotated with it. Measured via a timeout chain rather than
perf_hooks.monitorEventLoopDelay — the histogram's sampling timer races
a sibling read-and-reset interval in the same timers phase and the
stall sample can vanish into the reset window.
Largest feature delta in the v1.4.x line. Insights expands into seven dedicated metric sub-pages; GLP-1 medication tracking lands end-to-end (drug-level chart with Research Mode acknowledgment, EMA titration ladder, cadence + compliance, side-effect taxonomy, pen/vial inventory); cross-source priority becomes a two-axis resolver; per-user timezone threads through ten analytics surfaces; Withings activity + sleep v2 sync; onboarding rebuilt as a nested-route wizard; Personal Records ship end-to-end; Health Score provenance accordion; Coach runs native first-party prompts across six locales with a 1800-assertion refusal-probe matrix; OpenAPI drift gate hard-fails; multi-arch GHCR image. Migrations 0043 through 0060 are additive and forward-only. PROMPT_VERSION 4.24.0 → 4.25.0 with GROUND RULE 9 (refuses dose recommendations) and GROUND RULE 15 (refuses drug-level estimates with MDR + MDCG 2021-24 cites).
See CHANGELOG.md for the full per-section delta.
* Revert "revert: roll source tree back to v1.4.0 to restore production"
This reverts commit e444a90b10.
* fix(v1.4.2): summaries.WEIGHT + medications array guards + healthcheck IPv4
Three production bugs surfaced after the v1.4.0 deploy stabilized:
1. `TypeError: undefined is not an object (evaluating 'E?.summaries.WEIGHT')`
on /dashboard and /insights for users without weight readings —
the optional chain stopped at `data` when it needed to also cover
`data.summaries`. Fixed: `data?.summaries?.WEIGHT` (one level deeper).
2. `TypeError: y?.filter is not a function` on /medications — the
medications query value was non-array (likely a stale
service-worker response from an older API shape). Replaced with
`Array.isArray(medications) ? medications : []` so the page is
immune to any future shape drift.
3. Healthcheck-induced 503 outage (already in 1.4.1 — included here
so 1.4.2 is the single coherent release). The docker-compose
healthcheck used `localhost` which busybox-wget resolves to IPv6
::1; Next.js standalone listens on IPv4 0.0.0.0 only. Pinned
to 127.0.0.1.
This release also includes everything from v1.4.1 (which never
published a clean GHCR tag): the per-section admin extraction, the
testcontainers integration suite, the Playwright + axe-core E2E
foundation, the moodLog SSRF fix, the central error-redaction in
WideEventBuilder + Glitchtip, the medical-citation drift fixes
(ESH 2023, Saint-Maurice attribution, steps source), and the docs
sync.
Co-Authored-By: Marc-André Bombeck <mbombeck@gmail.com>
Production at healthlog.bombeck.io has been 503-ing since the v1.4.1
deploys started landing on apps-01 (Coolify). The container boots —
Next.js prints "Ready" and the pg-boss workers run — but never
accepts HTTP on :3000, so the Docker healthcheck fails and Traefik
takes the upstream out of rotation. A manual restart, a Coolify
force-rebuild, and a docker-compose pin to the GHCR :1.4.0 multi-arch
image all failed to bring the site back up — Coolify rebuilds the
image from main HEAD on every deploy regardless of the compose
directives.
This commit resets the working tree to commit 21bd46d (v1.4.0
release). Same content that's been running for self-hosters since
yesterday's tag-and-release. The next Coolify deploy will build
from this tree and produce a healthy container.
The v1.4.1 work is NOT lost:
- PRs #144, #145, #137, #146, #147, #148, #149, #150 remain in
git history.
- Their commits are still tagged (`v1.4.1`), still on the GHCR
multi-arch image (`ghcr.io/mbombeck/healthlog:1.4.1`), still in
the GitHub Release notes.
- Self-hosters who have already pulled the v1.4.1 image keep it.
- Local development continues from main HEAD with the v1.4.1
code — the regression only surfaced under the Coolify build
flow.
Re-applying v1.4.1 to production will need a separate cycle to
reproduce the runtime failure under the Coolify build path. That
work is tracked in docs/ops/v141-followup-issues.md (added back
when the tree is reapplied) and the deploy gating in
.github/workflows/e2e.yml will catch this class of bug going
forward.
No DB migration. No env-var change. No API contract change.
Co-Authored-By: Marc-André Bombeck <mbombeck@gmail.com>
The v1.4.1 audit pass turned up one CRITICAL, two HIGH, and four
medical-citation drift findings. This commit addresses everything
that's safe to land in a hardening release; deeper architectural
fixes (idempotency-race transactional rewrite, encryption-key
fallback gate, refresh-token reuse-detection serialisation, moodLog
secret HMAC lookup column) are tracked for v1.4.2 in
docs/ops/v141-followup-issues.md.
Security
--------
CRITICAL — moodLog SSRF
src/lib/validations/moodlog.ts now refines the credentials URL
with isPublicUrl() so an authenticated user cannot store
http://169.254.169.254/ (cloud metadata) or RFC1918 hosts.
src/lib/moodlog/sync.ts re-checks isPublicUrl(baseUrl) at the
fetch site (so legacy rows stored before the credential guard are
also refused) and switches to redirect: "manual" + 3xx → fail so a
public host cannot 302 the request to an internal target with the
user's apiKey on the redirect hop. New unit test
src/lib/validations/__tests__/moodlog.test.ts asserts the guard
rejects 169.254.x.x, 10.x, 172.16.x.x, 192.168.x.x, 127.0.0.1, and
localhost.
HIGH — error-message secret leakage
New src/lib/logging/redact.ts redacts Bearer tokens (incl. our
hlk_/hlr_), Telegram bot URLs (bot<digits>:<token>), and
query-string secrets (?secret=, ?code=, ?token=, ?api_key=). The
redactor is called from WideEventBuilder.setError() — every error
reaching Loki is scrubbed once, centrally — and from the
reportToGlitchtip() call in api-handler.ts so the Glitchtip
incident UI is also protected. New unit suite covers the four
patterns plus over-redaction (Bearer's \\S+ greedily consumes
anything up to the next whitespace, which is the safer side of
the trade-off).
Truthfulness — citation drift fixes (audit found 0 hallucinations,
4 drift items)
- messages/{en,de}.json bpClassificationTitle and avg30dEsc:
"ESC/ESH 2018" → "ESH 2023". User-visible on the dashboard and
doctor-report PDF. The numeric thresholds are unchanged from
2018; the 2023 update is ESH-only since ESC withdrew from
joint authoring.
- src/lib/analytics/classifications.ts: section header,
getBpTargetsByAge() docblock, and inline comment updated to
cite ESH 2023 with the same "numerically unchanged" caveat.
- src/app/api/insights/targets/route.ts:375: source label "WHO"
→ "Saint-Maurice JAMA 2020". The AI-prompt anti-pattern guard
explicitly forbids citing WHO for a step number; this surface
was the last "WHO" mislabel in the codebase.
- Saint-Maurice "mortality plateau 8000-12000" attribution
softened to "continued dose-response benefit through ~12,000
steps/day, not a plateau" in src/lib/medical-citations.ts,
src/lib/analytics/effective-range.ts,
src/lib/analytics/classifications.ts, and both AI prompts. The
original JAMA 2020 paper shows continuing benefit (HR 0.49 at
8k vs 4k, HR 0.35 at 12k); the plateau-shaped finding belongs
to Paluch 2022 Lancet PH (PMID 35247352), not Saint-Maurice.
Quality gates
pnpm typecheck clean
pnpm test 669 / 669 (was 658; +11 from new suites)
pnpm test:integration 10 / 10 (untouched)
pnpm lint 0 errors
HealthLog — personal health-tracking PWA with weight, blood pressure,
pulse, mood tracking, medication compliance, Withings integration,
doctor report PDF export, and Dracula-themed UI.
Marc-André Bombeck