Commit Graph
243 Commits
Author SHA1 Message Date
Marc-André Bombeck 12e422c25e Lighten the client shell and finish four client surfaces
- Load each settings section on its own: the section route imported all
  twenty-one statically. A preload starts the chunk beside the record
  gate's read, and the loading state paints the settings card shell.
  /settings/[section] goes from 611 to 403 KB gz.
- Take Zod out of the shell every route shares: the account-access parse
  run on each boot is plain TypeScript now, held to the published schema
  by an equivalence test over good and broken blocks. The canonical unit
  table moves to its own module, so the display transform no longer
  pulls Zod in either. Layout entry 199 to 136 KB gz; /insights 369 to
  304, /medications 428 to 365, /labs 400 to 337.
- Open a saved Coach conversation at its last answer: an instant jump
  instead of the smooth scroll that lost the end, then follow late
  layout (charts, tables) for a few seconds until the reader scrolls.
- Show the document archive as a month grid of preview tiles, two
  across on a phone and three or four on a desktop, with the month
  headings kept.
- Add the admin card for the measurement table maintenance: a status
  read on the maintenance route, one calm confirmation, and the run's
  state while it is queued, running, finished or stopped.
- Re-base the bundle budget: /insights at 304, /settings/[section]
  watched at 403, aggregate cap 4335 with the reason written down.
2026-10-08 11:25:11 +02:00
Marc-André Bombeck 5c23f9ed63 Merge AI operations and error-log discipline into release/v1.42.0 2026-10-08 09:06:58 +02:00
Marc-André Bombeck 27342ee734 fix(ai): name the real cause of failed briefings and stop retrying what cannot heal
The nightly briefing warm completed every night and read "all-failed",
because one run event held every account's annotations and the last
account won. Each candidate now runs in its own wide event and the run
reports every outcome through `reportJobRun`, so ok, dead sign-in,
withheld and failed accounts show side by side.

A dead Codex refresh token threw a bare error with no status, so the chain
read it as a network failure: the health ledger recorded hard_failed, no
"reconnect" hint appeared, and every night tried the same token. The
refresh now throws `CodexRefreshError` with a 401 for a dead grant
(invalid_grant / invalid_refresh_token on a 400 included), as do a missing
or undecryptable stored credential. The ledger records auth_failed, the
chain marks the primary credential expired, and the nightly pass reports
the account as auth_failed without queuing a retry.

A briefing the outbound screen withheld (a causal claim) failed outright,
and the 45-minute provider retry then asked the same question eight times
a night. The generator now makes one bounded repair that names the broken
rule and keeps the repaired payload only if it passes the same parse,
grounding and screen; a withheld outcome queues no retry.

A provider that keeps failing is paused: from five consecutive hard
failures the skip window starts at 30 minutes and doubles per further
failure, capped at a day. A paused provider is walked last and tried again
when the window lifts; one success resumes it. The operator card shows how
many accounts it is paused for and until when. After a 5xx from a
non-canonical OpenAI-compatible endpoint the client asks the free model
listing, and a configured model the endpoint no longer offers is named in
the error, on the hop line and on the card.

Also:
- The tiered context's live day band read `measured_at AT TIME ZONE tz`,
  which treats the stored UTC as local time; a far-from-UTC reader's
  evening readings landed on the next day. It converts from UTC first now.
- Google Health intraday heart rate re-reads two hours per sync instead of
  a day; daily summaries and sleep keep the day (#1023).
- The first filtered daily read that answers 400 and triggers the
  snake_case retry is recorded as a note, not an error (#1023).
- The single derived-score read goes through the derived cache with
  stale-while-revalidate, like the batch beside it.
2026-10-08 09:06:09 +02:00
Marc-André Bombeck 900a5c4f1f perf: stop keeping folded samples and re-reading duplicates
The nightly folds soft-deleted every raw Apple Health sample they folded
into an hourly or daily stats row and kept it for 75 days. Nothing ever
read those rows as deletions; they only held a re-upload of the sample
off, and they were most of the measurements table and its indexes.

Ingest now refuses such a re-upload directly: a raw sample of a dense or
mean type, old enough to have been folded, whose hour or day a live
stats row covers, is a duplicate with reason folded_window. The batch
route and the Apple Health export import apply the same rule. With that
in place the dense retention and the daily-mean consolidation delete the
raw rows outright, and a new compaction-tombstone-purge job removes the
backlog: 5,000 rows a batch, one account at a time under the restore
lock, a short pause between batches, resumable because the predicate is
idempotent, queued at every boot and continuing itself while it makes
progress. A person's own deletions are never in its predicate. The sync
feed leaves compaction tombstones out and a restore does not write them
back; all three classify through the same function.

The batch route answers an exact re-sent sample without a transaction:
one read over the batch's sample ids, live or deleted, decides what the
reconciler would have decided, at the same point in the loop, so every
status, reason, counter and the cross-source merge stay as they were. A
batch of duplicates only no longer queues PR detection, a reminder sweep
or the detection audit row. Caches are evicted only for a foreground or
manual sync or a batch that wrote a hand-entered reading, and marked
stale otherwise; the targets cache follows the same split.

Also:
- POST /api/admin/maintenance/measurements queues VACUUM (ANALYZE) and a
  REINDEX INDEX CONCURRENTLY per index, largest first, on a connection of
  its own; it refuses while the purge runs, drops invalid leftover copies
  and stops at the first failed rebuild. Runbook in docs/ops.
- The pure volume queues keep their finished jobs for 96 hours instead
  of seven days, never less than the failure window plus a day.
- work_mem is set per app connection from DATABASE_WORK_MEM (16MB), also
  when the statement timeout is disabled.
- The PR-detection fallback tick re-scans only accounts whose readings
  or workouts changed since the previous tick.
- Type discovery for the status refill walks the live-rows index as a
  loose index scan instead of grouping every live row.
- The host metrics chart polls every five minutes, like its sampler.
- The batch route records per-type HealthKit arrivals.
2026-10-08 08:41:25 +02:00
Marc-André Bombeck e406b9c7ab fix(coach): calm the answer column, its replies and its focus marks
The running status line now carries its seconds right after the text, in
the line's own muted tone, instead of at the far edge of the column. The
live count and every finished round read "4 s", the same spacing as the
settled summary, with a no-break space so a round never wraps between
number and unit.

Reply pills (follow-ups, clarifying choices, memory and plan decisions)
sit directly under the answer, and the action row is the last thing in
the column. Beside a pointer the row stays invisible until hover or
focus; above the pills it left an empty band between an answer and its
replies. Hovering or focusing the message still reveals the row without
moving anything.

Keyboard focus in the Coach panel is a solid ring in the neutral
text-field focus tone, never the purple ring: the pills, Undo and Change
merge it over the button primitive, and the hand-built controls carry it
directly. A structural test keeps the purple ring out of the panel.

A disabled native select now looks disabled, like Input and Select, so a
thinking depth locked by the operator no longer reads as a live choice.
In Settings, thinking depth takes the shape of Tone and Reply length
beside it; a memory entry's Forget is an icon like its edit; the
operator's highest-depth label sits on the switches' left edge.
2026-10-06 21:24:24 +02:00
Marc-André Bombeck c7d659eb11 fix(coach): ask the capability before deferring a memory refresh, and pin test fixtures to the new quiet time 2026-10-06 21:02:41 +02:00
Marc-André Bombeck c95dcb34db feat(settings): set how hard the Coach thinks, cap it as an operator, edit its memory
Thinking depth (Off, Low, Medium, High) sits under the model in the
Coach's quick settings and in Settings → Coach. It renders the resolved
block from the account payload: levels above the operator's cap are
disabled and say so, the field locks with one sentence when the
operator switched reasoning off, and Off reads Minimal where the
provider cannot stop thinking. The provider's own reasoning select
leaves the quick settings and says the Coach has its own.

The operator gets a switch to allow deeper thinking and a select for
the highest depth. The memory list shows where each entry came from,
lets it be edited in place, and is linked from the quick settings with
its count. The deep-linked data section no longer draws a ring.
2026-10-06 20:47:27 +02:00
Marc-André Bombeck f18e6ed34b fix(backup): notice a backup run the process died under, and take its account last
A weekly or nightly backup job whose process died (a restart, a
container out of memory) stayed active until its two- or four-hour
expiry, and its retry walked the accounts in the same order: the account
that killed it has the oldest copy, so it led again, and every account
after it went without a copy for as long as that record stayed too
large.

Both passes now send with a five-minute pg-boss heartbeat, so a dead run
fails within minutes and its retry picks it up. Each pass records when it
started and finished every account (backup_pass_attempts, migration
0373); an account whose last attempt never finished goes behind all the
others. The backups page shows when the run in progress started and names
an account a run died under, for the weekly and the off-host card, and a
heartbeat timeout reads as what it means.
2026-10-04 15:56:56 +02:00
Marc-André Bombeck 8ac3fddc3b Carry visit and vaccination free text readable in a portable backup
A practitioner's note, a visit's reason, outcome and body site and a
vaccination's note travelled as stored ciphertext in a portable file, the
same defect the cycle and mood sections had. They now go through the
shared sealed-text helpers: readable in a portable file, sealed under the
receiving host's key on restore, ciphertext kept only where this host
opens it and otherwise reported as visitCiphertext or
vaccinationCiphertext. The practitioner address and phone ciphertext of a
disaster-recovery file gets the same check, since the key preflight now
leaves these sections to it.
2026-10-04 10:19:31 +02:00
Marc-André Bombeck 43a53255c9 Carry custom mood labels readable in a portable backup
The labels of the mood tags and mood categories an account created
travelled as the stored ciphertext in a portable file, so a restore onto
a host with another key refused the file or wrote labels nobody could
open. They now follow the cycle note: readable in a portable file, sealed
under the receiving host's key on restore, ciphertext kept only where
this host opens it and otherwise reported as moodLabelCiphertext.

The sealing helpers move to one module both sections share.
2026-10-04 08:31:11 +02:00
Marc-André Bombeck 8121935642 Carry cycle free text readable in a portable backup
A portable file carried the day-log note, the sensitive-category envelope
and custom symptom labels as the stored ciphertext, so a restore onto a
host with another key either refused the file or wrote values nobody
could open. The portable file now carries them readable and the restore
seals them under the receiving host's key; a disaster-recovery file keeps
the ciphertext.

A file written before this still restores: each ciphertext value is
opened with this host's keys first, and one that does not open is kept
out of the row and reported in the skip summary as cycleCiphertext. The
key preflight leaves the cycle sections to that per-value check.
2026-10-04 08:17:34 +02:00
Marc-André Bombeck d6571c4e9c feat(symptoms): add person-defined symptoms and timestamped occurrences
A symptom of one's own ("aura", "headache") is a definition with an
encrypted name and an icon from the cycle custom-symptom allowlist; each
occurrence carries a time, a 0-10 intensity (the pain score's scale), an
optional encrypted note and an optional link to one of the record's open
illness episodes. Both ride the illness module: every route answers
illness.disabled when it is off.

- Migration 0369 (idempotent): symptom_definitions and symptom_events,
  intensity CHECK 0-10, episode link ON DELETE SET NULL.
- /api/symptoms/definitions{,/[id]} and /api/symptoms/events{,/[id]}:
  eight active definitions per record (a hidden one frees its slot,
  showing it again re-checks), hide or purge, events create/list/edit/
  delete, a new link must name an open episode of the same record
  (422 symptoms.episode.notOpen). Error group symptoms.*.
- Sharing: reads at READ, logging an occurrence at WRITE, defining and
  editing at MANAGE; C1/C3/C4 audit details; frozen lists, the handler
  matrix and the activity verbs updated.
- Both encrypted columns registered and rotated; both models in the wipe
  plan; both carried at both backup ends with occurrences nested in their
  definition, plaintext in a portable file and ciphertext in a recovery
  file, and a link to an episode the file lacks dropped and named
  (symptomEpisodeReference).
- OpenAPI regenerated.
2026-10-03 21:13:45 +02:00
Marc-André Bombeck 308470f229 feat(encryption): back up the key, and refuse to serve when it cannot open the database
Every configured key id now has a canary in encryption_key_canaries,
checked at boot. A key that cannot open its canary, or existing data
written before the canary existed, puts the process in a refusal state:
every API route answers 503 encryption.key_mismatch, /api/health reports
reason encryption_key_mismatch, the pages render one explanation, and
neither the queue producer nor the worker starts. The process stays up
so the reason is visible instead of hiding in a crash loop.

The admin Encryption page gains a "Back up your encryption key" step:
key id and fingerprint (never the key), where the key lives per
platform, a check of a pasted copy that is compared and discarded, and
a confirmation bound to the active key id and fingerprint. The overview
gets a tile and admins get a dashboard banner while it is due.

HEALTHLOG_PLATFORM picks the platform the step opens on. Adds the
TrueNAS self-hosting page and key backup sections in the docs.
2026-10-03 18:06:28 +02:00
Marc-André Bombeck 7abaa25c14 feat(backup): export the account settings and restore them onto the account
Both backup purposes now carry an accountSettings section. The restore writes
it to the account being restored inside the same transaction, names every
refused value under the new accountSetting skip catalogue, and re-warms the
stored texts after the transaction when a unit preference changed, the way the
unit settings routes do. A file without the section leaves the account row as
it is, which is how every older backup restores. The instance settings
restore stays its own opt-in.
2026-10-01 13:04:41 +02:00
Marc-André Bombeck 537ac3c7a1 fix(settings): stop settings and admin cards widening the page on a phone
Three cards put a full sentence into the SettingsCardHeader status slot
(mood reminder, low-stock warning, the app-log refresh button), which
does not wrap: the row grew to 1000px and /settings/notifications
panned sideways at every width. The sentence moves into the card body,
and the slot is capped at the row so a long status wraps instead.

Also on a phone: the provider-chain add control wraps, the report
picker's standard-template action drops under its sentence instead of
running past the card, and the dashboard layout rows put their switches
and arrows on a second line so the tile name is no longer cut to five
characters.
2026-10-01 09:44:43 +02:00
Marc-André Bombeck a2fceb5585 fix(backups): answer the restore preview from counts kept with the copy
The restore dialog asked GET /api/admin/backups/{id}/summary for the
contents of a copy, and the route read the whole stored copy to answer:
every piece opened, every measurement parsed, the rest validated. On an
account of 1.8 million readings on slow hardware that took about two
minutes. The dialog stopped waiting after 15 s and said the contents
could not be read, and its automatic retry made the server read the
whole copy a second time.

The copy passes through the process once anyway while it is written, so
the preview is worked out there and kept on the row (DataBackup.preview,
migration 0364): the summarizeBackup counts, the schema version, and
the inner key ids with the sections they sit in and one sample value
each. Every creation path goes through storeBackupBlob (the weekly and
manual pass, and an upload), so every new copy carries one. The column
is plain JSON: counts and dates only, and the samples are ciphertext
that stands for what is inside the copy, which a key rotation must not
re-seal.

The route answers from it without opening the copy. Both verdicts are
taken again on every read, from what the preview keeps: whether this
release restores the schema version, and whether this server holds the
keys the copy needs. The pieces are still counted against the row, so
a copy that lost one is refused as the restore and the download refuse
it. A preview names the copy it describes and is ignored once the copy
changes; converting a single stored value into pieces carries it over.

A copy written before this has no preview. It is read whole once, as
before, and the result is kept on the row, so it is slow only the first
time; concurrent requests for the same copy share the one read.

Only a file that fails the schema answers 422 "Backup payload failed
schema validation". Read refusals keep their codes; anything else is
the server failing and answers 500.

The dialog no longer retries the preview, waits up to 55 s, and when it
stops waiting on an older copy it says the copy is still being read and
to open the dialog again in a few minutes.

Refs #1031
2026-09-30 22:51:02 +02:00
Marc-André Bombeck 93b6c086b4 fix(restore): let a large restore outlast the request statement timeout
A restore of an account with 1.9 million readings, most of them deleted
by the nightly consolidation and still awaiting the purge, was cancelled
in its clearing step: `DELETE FROM measurements` ran past the 60 s
statement timeout every pooled connection carries, and the whole restore
rolled back with a message that named no cause (#1031).

Most of that statement was not the delete. Two columns point into
`measurements` with ON DELETE SET NULL and had no index, so Postgres
scanned `personal_records` and `ecg_recordings` once per deleted reading:
38 of 40 s on a seeded account with 300 personal records. Migration 0362
indexes both, which also speeds up the tombstone purge and account
deletion.

The restore transaction now sets its own statement and idle-in-transaction
limits to the transaction budget with SET LOCAL, and the budget counts
the rows the account holds as well as the rows the file carries. A
rolled-back restore names what the database refused it over (timeout,
lock, constraint, storage, connection, other) on the job row, the
console, the audit row and the wide event.
2026-09-27 11:24:41 +02:00
Marc-André Bombeck f57b6bf979 fix(admin): one card that fails to render no longer takes the page down
An admin page stacks independent cards, and a card that threw while
rendering replaced the whole page, so a field missing from the off-host
card's payload hid the snapshot table beside it. Every card in an admin
section now sits in its own boundary and fails as an error card with a
retry that remounts it; the rest of the page keeps working.
2026-09-27 03:19:39 +02:00
Marc-André Bombeck 66b1aef40c Merge branch 'fix/encrypt-free-text-and-local-ai' into release/v1.39.3
# Conflicts:
#	src/app/api/import/csv/route.ts
2026-09-26 21:50:05 +02:00
Marc-André Bombeck 4409529c89 Merge branch 'fix/backup-rotation-and-deletion' into release/v1.39.3
# Conflicts:
#	docs/api/openapi.yaml
#	src/lib/jobs/apple-health-import-worker.ts
2026-09-26 21:48:20 +02:00
Marc-André Bombeck c86f23c2c0 fix(security): keep ALLOW_LOCAL_AI_PRIVATE_HOSTS=true for the operator's own AI, deprecated
Making `=true` grant nothing would stop a single-user self-host's local
model on a patch update. It is deprecated for one release cycle instead,
in a narrowed and safe form: it still grants a private host, but only to
the operator's own configurations (the instance-wide admin provider and
AI settings saved on an admin account), and only through the pinned
operator-approved dispatcher, so redirects stay forbidden and metadata,
link-local and unspecified addresses stay refused. A non-admin account's
own base URL needs an exact origin in AI_PRIVATE_ORIGINS.

Provider resolution passes the owning account's role to the client
(`operatorTrusted`); the save route and the connection test apply the
same rule to the account being configured. The boot summary now says the
setting is deprecated and names the replacement, and the admin AI
settings page shows the exact AI_PRIVATE_ORIGINS line for the private
origins saved on the instance (every account's Local and gateway URL and
the admin base URL, scheme, host and port only), so the operator can copy
it and drop the old setting.
2026-09-26 20:59:10 +02:00
Marc-André Bombeck 03ccb21227 Merge branch 'fix/step-up-and-session-revocation' into release/v1.39.3
# Conflicts:
#	messages/de.json
#	messages/en.json
#	messages/es.json
#	messages/fr.json
#	messages/it.json
#	messages/ko.json
#	messages/pl.json
2026-09-26 20:40:59 +02:00
Marc-André Bombeck 1c7785c23b fix(backup): take off-host copies out with the account, and say which keys backups still need
Deleting an account, deleting a managed profile, "Delete all data" and the
admin wipe now write an off-host deletion request in the same transaction
as the deletion. A new job (offhost-backup-purge, kicked straight away and
run nightly as a backstop) lists the bucket once, deletes every
<date>/user-<id>.json.enc object of each pending account and removes the
request only when nothing of the account is left. A refused delete keeps
the request with the bucket's answer and is retried; the off-host card
shows how many deletions are outstanding and why. A copy the nightly run
was writing while the account was deleted is taken back after the upload.

The off-host card also reads the bucket's lifecycle rule and says plainly
when there is none (every nightly copy kept forever) or when the
credential may not read it.

Every stored backup now records the key ids its content was written
under, and every off-host upload records the same per key. The admin
encryption view lists, per key, the stored copies that need it with the
oldest date, and when the last off-host copy needing it expires. The
"safe to drop" badge now requires that no backup still needs a retired
key, instead of only that every row was rotated.

Migration 0358 adds data_backups.inner_key_ids, the key-use table and the
deletion requests.
2026-09-26 19:54:40 +02:00
Marc-André Bombeck 0fc83cb0a1 fix(auth): ask for a fresh proof before the record or a credential to it leaves
Exporting the whole record, making a clinician share link, minting a token,
and the admin backup download, upload and restore, data wipe and password
reset ran on a live session alone. The plaintext full backup did, while the
encrypted one beside it asked for a second factor. Whoever held a stolen
session could take the record, or leave behind a credential that outlives
the session.

These now go through one gate, requireRecentProof (assertRecentCookieProof
for the routes that resolve their caller themselves). A browser session
passes after a sign-in or re-proof within five minutes: a second factor on
an account that has one, otherwise the password or a passkey. The refusal is
401 auth.reproof.required with meta.methods, and the web opens the re-proof
dialog, posts the proof to the new POST /api/auth/reproof and retries. A
password proof stamps the new Session.reproofAt (migration 0359), never
mfaVerifiedAt, so it cannot stand in for a second factor anywhere.

On Bearer, the full backup asks for an X-Step-Up elevation; the encrypted
export asks for a second-factor elevation on an account that has one. Share
links and connector tokens keep taking the token alone, because the shipped
app calls them without an elevation and reads a 401 there as a dead
session; the guard test names them.

Key rotation keeps its stricter second-factor gate.
2026-09-26 19:54:38 +02:00
Marc-André Bombeck 405c996b6a fix(backup): refuse a restore that needs a dropped key or reaches into another account
A backup carries every encrypted column as the stored ciphertext, and a
restore writes it back verbatim. Key rotation re-seals the stored copy's
envelope but cannot reach inside it, so a copy taken before a rotation
still needs the retired key. Without it the restore went through and
left rows no reader can open.

The restore, the restore preview, the admin upload and the monthly
restore drill now collect the key ids of every inner ciphertext value,
hold them against ENCRYPTION_KEYS and decrypt one value per key. A
missing key, or an id configured with different key material, is
refused before anything is deleted, with the key id named. The restore
status and the preview say which key to put back.

The restore also checks, inside its transaction, that it added no
reference from the account's rows to rows another account owns. The
check is derived from the schema's foreign keys rather than written per
section, so the tag and category upserts keyed on a global key are
covered along with every link table. The illness parent update is
scoped to the owner as well.

The off-host listing now follows every page: the first page held only
the oldest dates, so the drill could read an old object and report the
chain as stale.
2026-09-26 19:22:03 +02:00
Marc-André Bombeck afcb37e76c feat(config): let the operator turn off the breach and update checks
Two requests left the server without anything being connected: the
HaveIBeenPwned range query on every password set (registration, change,
admin reset) and the daily GitHub releases request behind the About page
and the admin version tile. Neither could be turned off, which made
"nothing leaves" untrue for an instance that wanted it to be.

PASSWORD_BREACH_CHECK_DISABLED and UPDATE_CHECK_DISABLED (1, true, yes or
on) stop each request. Both stay on by default. The breach check keeps
its fail-open contract: switched off, a password is simply not looked up.
The update route answers status "unknown" with reason "disabled" and
the admin tile says the check is off. The password routes reach the
check only through the switch, and a test holds that import line.

Both are on the compose whitelist, in the env manifest under a new
"Outbound requests" group with IP_GEO_LOOKUP_DISABLED, and documented in
both env examples and the getting-started guide.
2026-09-26 15:33:34 +02:00
Marc-André Bombeck 47e9f7e81c feat(backup): run a restore as a background job with progress
Restoring a large account was one request: 94 s for 1.25 million readings.
A reverse proxy that cuts a request at 60 s (nginx's default) showed the
operator an error for a restore that went on to finish on the server.

The restore request now checks the typed confirmation, that the backup
exists and that its stored copy opens, admits one job for the account and
answers 202 with its id. The `backup-restore` queue runs the restore and
records its phase, counts and outcome on a `backup_restore_jobs` row
(migration 0347), which `GET /api/admin/backups/restores` and
`GET /api/admin/backups/restores/{id}` read back.

What a restore checks and writes is unchanged. The body moved from the
route into `restoreBackup` (src/lib/export/restore-backup.ts): every
refusal still happens before the first delete and the replacement is still
one transaction, so a failure part-way leaves the account as it was. While
it runs the account keeps showing its current data, and other writes are
neither refused nor held, as before.

- One restore per account: a partial unique index admits one queued or
  running job, and a second request gets 409 `backup.restore.active` naming
  the running job. A per-account advisory lock inside the transaction holds
  even if a worker thought gone is still writing.
- Time budget: the job's expiry is two hours and a file whose transaction
  limit would run past three quarters of it is refused before anything is
  deleted (`jobDeadline`, beside `jobBudget`).
- Resuming: the job writes a heartbeat every two seconds. A boot sweep
  queues a job whose worker stopped again from the start, at most twice,
  then fails it as interrupted. The admission check releases an abandoned
  job so it never holds the account's slot.
- The staged file is the stored `DataBackup` row, already encrypted at
  rest. It stays whatever the outcome, so a retry needs no new upload. The
  job records a digest of the copy and refuses one that was replaced while
  it waited.
- Failures carry a stable code and a sentence that says what to do.
- The console follows the job: phase and counts while it runs, the outcome
  after, and after a reload it finds the job again from the server. A poll
  that fails on the network is never shown as a failed restore. A web
  process separate from the worker evicts the account's caches when it
  first reports the restore finished. The success line goes through the
  outcome module, so an empty file is not drawn as a tick.

The model is registered in the backup plan (not carried) and the wipe plan
(wiped with the account); the queue in the maintenance registrar, the
subsystem surface and the job-fact allowlist; the new code in the error
catalogue; the routes in the OpenAPI contract. The tests that pin restore
semantics now drive the real route and the real job through one helper, and
`backup-restore-job.test.ts` pins the asynchronous contract itself.
2026-09-25 07:39:06 +02:00
Marc-André Bombeck 237acc0002 fix(backup): export, upload and restore a large account as streams
An account of 1.25 million measurements could not export its data or be
restored from any backup (#1031). Measured in a 1 GB container:

- The full export and the passphrase export built the whole payload and
  its JSON string in memory; the app ran out of heap and restarted.
- The weekly backup's JSON is 662 MB, longer than any string V8 can hold,
  so `unpackBackupBlob` threw on every read. Restore, download and the
  restore preview all answered "could not be decrypted".
- Upload accepted 10 MB, a fraction of a large export.

Every reader and writer of a backup file now streams it:

- `scanBackupJson` reads a backup's bytes once, keeps every section but
  the named bulk arrays, and hands their elements out one at a time.
  `readStreamedBackup` uses it twice: a first pass that validates each
  measurement against the element schema and keeps the rest of the file,
  and a second pass that hands the measurements to the writer in batches.
  Stored copies and off-host objects are authenticated whole on their
  compressed ciphertext before any plaintext is read; only the
  decompression is streamed.
- Restore writes the measurements with one `INSERT … FROM unnest` per
  batch. Prisma caches a query plan per `createMany` row count, and one
  1 000-row plan kept about 145 MB alive, which alone put the restore past
  the heap limit. The transaction budget grows with the measurement count.
  The awards and ECG sections are handed the ids they reference that were
  written, not every written id.
- The export routes write the file into the response as it is produced.
  The passphrase export spools its ciphertext to a temporary file because
  the HLX1 format puts the tag first; the format is unchanged, so existing
  archives open as before.
- Upload accepts the raw body, JSON or gzip, up to the 512 MB request
  ceiling, checks it as it arrives and stores it in the same pass; the admin
  page compresses the file in the browser before sending it. Multipart stays
  for small files.
- Download streams the stored JSON; the preview counts measurements from
  the stream. The monthly restore drill and `scripts/restore-backup.ts`
  read the off-host object the same way; the script can write `.json.gz`.

On the seeded account in a 1 GB container: export 25 s at 389 MB peak,
passphrase export 25 s, both uploads 19 s, restore of the weekly backup
94 s at 554 MB peak, and export, upload and restore round-trip to exactly
the same rows. The encrypted-column guard now also recognises the raw SQL
statement that writes `DataBackup.data`.
2026-09-25 02:05:19 +02:00
Marc-André Bombeck 676bb2eff7 fix(ai): refresh the account's AI answer after every change to its inputs
Every AI surface reads its capability from the ai block on
/api/auth/me, but setting up or removing a provider, signing in to or out
of Codex, saving the fallback chain, the central-Codex opt-in, the
local-OCR switch, a minted consent receipt, the admin's server key and
central Codex, and a guardian's save inside a managed record evicted only
their own reads. The surfaces stayed on the old answer until a reload.

All of these writes, and the ones that already refreshed the payload
(consent, modules, Hide Coach, the operator's switches), now evict one
bundle, aiInputDependentKeys. A structural test holds every client write
to those endpoints to it.
2026-09-24 12:11:14 +02:00
Marc-André Bombeck 3d85445e50 feat(settings): explain operator switches and keep the Coach memory reachable
Settings, AI says which AI features the server's operator switched off and
shows the one setup hint while no provider is configured. The consent
withdrawal says what it deletes and what stays, and a document-reading
consent on file gets its own withdrawal. Regenerate follows the briefing
capability and the connection test names an operator refusal.

The Coach memory (facts and stored conversations) is never gated on the
Coach: Settings, Coach always shows it, and Settings, AI shows it while
the Coach page is out of reach, whenever rows exist.

Onboarding offers AI only where it can be set up here: the done-screen
panel and the getting-started row go away while the operator has AI off
or inside somebody else's record.
2026-09-24 11:05:54 +02:00
Marc-André Bombeck a7888a9011 feat(ai): replace the Correlations switch with Reading documents
The operator switches become master, Coach, Daily Briefing, Status
notes and Reading documents. Correlations goes: it gated statistics,
not anything a model writes. Reading documents covers the document
vault, lab scans and medication extraction.

The Coach module's operator availability now follows the Coach
switch, so there is one place to turn the Coach off server-wide. The
migration folds a stored Coach availability of false into the switch,
and gives everyone who hid the Coach the AI analysis opt-out, since
hiding the Coach no longer stops nightly briefing and status notes.

A switch read that fails now fails closed. The insights module is
described as AI analysis in every locale, and the switch guard now
keeps both retired switches out and freezes the callers of the old
operator-only gate.
2026-09-24 07:36:47 +02:00
Marc-André Bombeck 32eaf7a29e fix(admin): show each channel's cause in the notification test toast
The admin test response carried a code and status per channel that nothing rendered. The toast now lists each failing channel with the translated cause and the HTTP status or SMTP code. The relay reply is no longer part of that response, and the ntfy error no longer echoes the topic, which the sender treats as a secret.

Refs #947
2026-09-13 14:50:33 +02:00
Marc-André Bombeck 4460047910 Merge the coach cap counting only the tokens the operator paid for
Background insights can no longer use up the day before somebody asks a question; a fifth of the daily limit is held back for what a person does by hand.
2026-09-11 14:18:21 +02:00
Marc-André Bombeck b565edc48d docs(ai): drop planning vocabulary from the ledger comments
CLAUDE.md forbids planning shorthand in committed artefacts, and this branch
introduced 74 lines of it across source, tests, a schema field and two
migrations. Someone reading the migration SQL in a year has no way to resolve a
wave letter or a round number, and a comment that points at something outside
the repository names no reason at all.

Three passes. A version followed by a parenthetical keeps the version and loses
the parenthetical, because the version says since when something holds. A
sentence that leaned on the vocabulary to carry its meaning is rewritten to
state the reason directly: the arrival reservation now says it was refused over
money the operator never spent, rather than naming the defect a wave existed to
remove. Three test titles carried the marker in the name itself and are renamed
to what they pin.

Comments only. No behaviour, no renames beyond those three titles.
2026-09-11 12:37:54 +02:00
Marc-André Bombeck c66bcbc295 feat(admin): the AI card shows the day's spend split by who paid
The provider-health route answered a spendToday payload that nothing rendered,
which is the pattern the two-ended rule exists to stop. It is also the screen
the operator opens when a surface refuses on budget, and the one mixed number
could not answer the question that raises: a day at 1.24 million tokens is
alarming until the card says that 151 200 of them were the instance's own key
and the rest ran on the users' own plans. The operator ceiling is enforced
against the second figure. Copy lands in all seven locales.
2026-09-11 11:11:17 +02:00
Marc-André Bombeck 0290554d36 test(e2e): an invite link registers from a cookie-less window
Every layer of the invite chain had unit coverage and the chain was broken
anyway: the proxy test proved the edge admits the landing, the page test proved
it redirects onto the registration form, and neither could see that the
redirect arrives in an RSC flight payload after the shell has already flushed.
The operator's path — mint an invitation, open the URL in a private window,
reach a usable form — was exercised by no spec at any level.

The new spec mints a real invitation in the admin UI and opens it twice: once
in a genuinely cookie-less context, asserting both where the visitor lands and
that they never passed through the sign-in page, and once in the admin's own
window, asserting the explaining panel and that the admin's session survives. A
third case pins the server half through the wire. Against `origin/main` all
three fail — the anonymous visitor lands on `/auth/login`, the panel does not
exist, and the registration POST answers 201 over a live session.

The minted URL gains a test id so the spec reads it from its own element
rather than from the first paragraph in the ticket.
2026-09-11 09:40:26 +02:00
Marc-André Bombeck d37993822e fix(admin): keep the off-host freshness badges readable in both themes
The destructive tone as 12px badge text on its own wash measures 4.08:1 in
the light theme and 3.99:1 in the dark one; the muted foreground on the
muted wash 3.6:1 in the dark one. The badge takes the shape the status rows
on the same console already use: the token paints a small indicator and the
wording stays in the foreground colour. The word states the condition, so
nothing hangs on the colour alone.
2026-09-10 16:00:38 +02:00
Marc-André Bombeck a7faae8ded refactor(admin): read the off-host card from evidence, sort the worst row first
Four smaller things around the same card.

The configured flag came from the environment of whichever process serves the
request. A hand-rolled split that gives BACKUP_S3_* to the worker and not to
the web process would have shown an operator "nothing leaves this host" over a
ledger full of fresh rows. The ledger is queried either way now, and a row in
it outranks an environment read: a row is proof an upload happened.

Rows sorted by username put the one account that needs attention wherever the
alphabet leaves it, which on a hundred-account cohort is the crowd problem the
card exists to end, moved from a count into a list. Worst verdict first,
alphabetical within a verdict, decided server-side so the tests and the
accessibility fixture see the order the operator does.

The card's docblock claimed the worker holds PutObject and GetObject. The
runbook is right and the multipart path needs AbortMultipartUpload too, so the
prose no longer documents a grant that would break the uploader.

And the byte formatter moved to lib. The off-host card was importing it out of
a thousand-line client component, which costs nothing while both cards render
on one route and becomes a cost the moment either moves.
2026-09-10 10:32:33 +02:00
Marc-André Bombeck 9d18b8fd25 fix(admin): let the error-reporting badge follow reports, not a parseable URL
The badge read the enabled flag plus a DSN that parses. Neither says anything
about the target accepting a report: a wrong public key, a wrong project id and
a host that stopped answering all parse, and every one of them showed green
with a line naming the host. That is the false reassurance an operator finds
out about when they go looking for a crash that was never reported.

Nothing in the tree recorded an outcome, so there was nothing to read. The
sender now stamps one next to the DSN it used — instant on a success, a short
classification on a failure. Never the response body: that is text from a
remote host and has no business in this database or on this page.

The badge is green only while a report has left inside the last fortnight,
amber while a target is set and nothing has come back, and it names the reason
underneath. The clock lives with the ledger, so the verdict is resolved in the
route and the card paints what it is handed. A failure that a later success
superseded is not shown at all. Pressing Test records like any other send, so
the button proves the target on demand.
2026-09-10 10:28:57 +02:00
Marc-André Bombeck 25f0310596 feat(backups): tell an empty ledger apart from an empty bucket
On the release that adds the ledger the table is empty, so on a host that has
been uploading nightly for months every account read Never, under a sentence
saying no copy had ever reached the bucket. That sentence is untrue on exactly
the morning an operator opens the new card for the first time, and it is a
claim the host cannot make: what is empty is the record, not the bucket.

The run now records every account it walks, not only the ones an object came
of. A walk that produced nothing moves the attempt instant and leaves the
success columns alone, so the two cases separate: no row at all is no record
yet, a row without a success is never. Never now means what the word says —
a run looked at this account and has never put anything there.

The card names the last check on the never row, so the state is a fact with a
timestamp rather than an assertion. Runbook and all seven bundles carry the
same two sentences. The integration file proves both arms against real
Postgres, including that a failed run leaves the previous good copy on the
row.
2026-09-10 10:15:31 +02:00
Marc-André Bombeck 34d647a884 test(backups): prove the off-host ledger end to end and the card's empty state
The unit tests either side of this mock Prisma, so neither proves the
migration applies or that the writer and the reader agree on a shape. The
integration file runs the real nightly job against real Postgres with an
in-memory bucket, then reads the row back and asks the console's own
classifier what it says: fresh after a run, one row across two runs, stale
once it ages past two nightly periods, gone when the account is.

The component test pins the case an operator most needs stated: a host with
no bucket configured says so, rather than painting an empty list that reads
as "no problems". The accessibility scan's admin-backups response gains one
row in each state, because a card that paints only its header proves nothing
about the rows anybody reads.

Two small corrections ride along, both from writing the tests. The freshness
badge composes its classes through `cn` like every other surface, and the
error-reporting line waits for the settings to arrive instead of asserting
"nothing is sent" while the answer is still loading.
2026-09-10 09:34:33 +02:00
Marc-André Bombeck 532483e782 fix(admin): say where error reports go, and stop calling a silent target configured
There is no default error-reporting DSN in this tree and there should not be:
a self-hoster's crash reports must not leave their host because nobody told
them not to. That part was already right, so nothing here adds a default.

What was wrong is what the console said about it. The GlitchTip card showed a
green "Configured" badge as soon as a DSN existed, whether or not the switch
that actually sends anything was on, and it never said where reports go. An
operator reading that badge has been told their errors are being collected
somewhere, which may be true of neither half.

The badge now tracks reports actually leaving, and one line under the switch
states the case: the host they go to, or that they stay here, or that tracking
is on with no target. The host only — the public key in the DSN answers a
different question. The deploy runbook says the same thing for a
maintainer-operated instance.
2026-09-10 09:28:12 +02:00
Marc-André Bombeck 1c10bbbde8 feat(admin): say per account how fresh the off-host copy is
The nightly off-host job reports counts — uploaded, failed, total — and a
count cannot name the account it is failing for. A cohort where one object
has been refused every night for a month reads as "99 uploaded, 1 failed"
each time, which looks like weather rather than like an account with no
disaster-recovery copy at all.

The worker now writes a ledger row when an object lands (account, instant,
size), and Admin → Backups gains a second card that rates every account
against the nightly schedule: fresh inside one run, due when one run
produced nothing, stale when two did not, never when no object has ever
landed. On a host without BACKUP_S3_* the card says off-host backup is not
configured and stops there.

The verdicts are computed from the ledger, never from a bucket listing: the
worker's grant is deliberately PutObject + GetObject, and an admin page must
not put the operator's credentials on its render path to state a number the
database already holds.
2026-09-10 09:27:58 +02:00
Marc-André Bombeck 17671d9dc1 Merge the AA-contrast status rows on the admin page 2026-09-10 07:41:22 +02:00
Marc-André Bombeck 9e1f64e733 fix(admin): keep the worker-status row readable with no worker present
The admin snapshot paints its worker row in `--destructive` whenever the
process serving the page is not the worker: a split deployment running
`HEALTHLOG_PROCESS_TYPE=web`, or a worker that stopped answering. The token
is AA on a card (4.85:1 in the light theme) but the stat tile lays a
`bg-muted/50` wash over that card, and axe measured the tinted wording at
3.97:1 there, under the 4.5:1 AA floor. `e2e/a11y.spec.ts` went red in
exactly that state and green in every other one, which is why the console
could ship it.

The status grids now name a tone instead of spelling a class at every call
site. Success and warning keep their tint, both clear AA on the tile.
Destructive takes the shape the other status rows already use, the
medication detail row and the invite chips: the token paints a small
indicator and the wording stays in `text-foreground`. The word itself
("Stopped", "Error") states the condition, so nothing hangs on the colour
alone. The database, Umami and GlitchTip rows sit on the same surface with
the same tint and move with it.

The success-affordance guard's two pinned entries for the status grids went
stale in the process: the tint they carried now lives once in the tone
table, so the pins follow it there.

Checked against a standalone build on a throwaway database: the whole a11y
spec green in both themes and both viewports with `HEALTHLOG_PROCESS_TYPE`
at `web` (the failing state) and at `all`. jsdom cannot measure a contrast
ratio, so the new unit test pins the token choice for both arms instead.
2026-09-10 06:56:50 +02:00
Marc-André Bombeck 343d6397a0 test(e2e): say what the backup journey really touches, and pin it to promises
The file claimed it stayed inside its own account. "Backup now" enqueues the
pass for every account on the instance, so the header says that now, and says
what is confined instead: the restore's transaction, and the instance-wide
settings the dialog's opt-in leaves alone — which the journey asserts is clear
before it confirms, because a default that flipped would hand this file the
power to rewrite the document size cap for every spec sharing the server.

The run's own answer is asserted where it happens. Without it a missing worker
cost two minutes of polling and then blamed the backup pass for a 503 about
something else; the failure now names the endpoint in two seconds.

The rest are the assertions that pinned incidentals rather than promises: the
Argon2 cost comes from the constant the writer exports, so a legitimate cost
bump is not a red spec; the document is found by id rather than by position;
the poll compares the row against the row it saw before the click rather than
against the runner's clock; the non-admin bounce is pinned by pathname; the
empty-state CTA has its own test id, so nothing has to pick between two
buttons that share one; the deliberately corrupted copy is deleted when the
control is done with it instead of sitting in the console's list; and the
fixture orders by id after the timestamp so a tie cannot present as a change.
2026-09-10 05:48:18 +02:00
Marc-André Bombeck 6b366b4f27 fix(backup): make the console restore leave instance settings alone
A disaster-recovery snapshot carries the singleton `app_settings` row —
registration, the MFA requirement, the default locale and timezone, module
availability, the notification and AI configuration, the document size cap and
quota — and the restore wrote it back as a side effect of restoring one
account. An operator putting a single record back on Wednesday reconfigured the
whole host from Monday's file, silently, with nothing on screen saying so.

Rebuilding a host from a snapshot is a real case, so the write stays: it is now
`restoreInstanceSettings: true` on the request and a checkbox beside the typed
gate, off unless the operator ticks it. Restoring an account and reconfiguring
the installation are two decisions, so they are two answers.

The 422 confirmation sentence and the dialog copy drop the instance-wide claim;
the opt-in carries it instead, in all seven catalogues. Integration coverage
runs both arms: without the flag the settings row keeps its own values, with it
the file's values land.
2026-09-10 05:16:59 +02:00
Marc-André Bombeck 5cb5c7df44 test(e2e): drive backup and restore end to end through the settings surfaces
One journey across the whole path: an account writes readings in three
domains and files a PDF, downloads its own record as the passphrase-sealed
archive, an admin snapshots it from the backups console, and the snapshot is
restored over an account that has since lost a reading and a dose and gained
one the snapshot never saw.

The assertions address what the documentation promises rather than what the
implementation happens to do. The archive is checked byte by byte against the
HLX1 header in src/lib/export/passphrase-archive.ts — magic, version, KDF id
and the Argon2 cost the file carries — because "a file arrived" is equally
true of plaintext JSON under a new extension. The restored reading and dose
are checked by their ORIGINAL ids, because a restore that re-created them
under fresh ones would have written a different record that reads the same.
The reading created after the snapshot is asserted GONE: the runbook says the
restore replaces the account's data tables, and replacing is not merging.

Two refusing controls. A stored copy with one flipped ciphertext byte is
refused with the route's decrypt failure and leaves the record untouched — the
envelope is AES-256-GCM and its tag covers every byte, so the flip is caught
before the transaction opens. A delegate holding whole-record READ inside the
account gets the not-covered panel instead of the export card, never reaches
the console, and is refused 403 by the list, the manual run and the restore,
the last of them naming the real snapshot so the refusal cannot be mistaken
for a missing row.

Waiting is on the job's own evidence: the console's endpoint is polled until
this account's stored copy is newer than the click, so nothing in the file
guesses at a queue. The journey has its own account, its own delegate and its
own cookie jars; only the reset between repetitions and the deliberate
corruption reach Postgres directly, because a browser cannot express either.

The backups console gains data-testid hooks and a data-backup-id on each row
so the spec addresses controls rather than translated labels.
2026-09-10 04:53:03 +02:00
Marc-André Bombeck 5556478691 Make uploads and every other write work over plain HTTP
crypto.randomUUID() is [SecureContext] in the Web Cryptography IDL, so a
page served over plain HTTP never gets it. Measured in Chromium against
http://<lan-ip>: isSecureContext false, randomUUID and subtle undefined,
getRandomValues still a function.

The document vault minted its queue id and Idempotency-Key that way, inside
the file-input change handler. On a LAN self-host the call threw before a
queue row existed, so picking a file produced no upload, no error and no
trace — the picker simply looked inert, and the only way through was to
reach the app over HTTPS. The client transport has minted a default
Idempotency-Key the same way since v1.37.18, which put every apiPost and
apiPatch on the same footing: on such a host nothing wrote at all.

Add randomId(), which prefers the platform call and otherwise assembles a
v4 UUID from getRandomValues — the same CSPRNG, the same 122 bits, no
secure context required — and route every browser-side call site through
it. Nothing in an upload needs one: the bytes travel over multipart XHR and
every cryptographic operation happens on the server. Passkeys, web push and
home-screen install still need HTTPS; browsers withhold those outright.

A structural guard fails a direct crypto.randomUUID() anywhere outside the
two server-side files that may keep it, so the class cannot return one call
site at a time.

Also point the CSV and JSON import cards straight at the import guide. They
resolved through a redirect under the integrations tree, which is the wrong
place for a guide that is not an integration runbook. The suite that
covered those links pinned the old spelling rather than the guarantee; it
now asserts an absolute docs-site URL, no internal path, and new-tab
hygiene.
2026-09-05 16:22:39 +02:00
Marc-André Bombeck 7dbec3bc4d feat(admin): name the status a failing provider came back with
The provider-health ledger has recorded the HTTP status of every failure
since v1.11.0 and nothing has ever read it. The operator card could say
three of five chains were failing without saying whose problem that was,
which is the one thing the operator needs: a 401 is a credential only they
can replace, a 429 or a 503 is the provider's afternoon and waiting is the
right move.

The fold carries the status of the failure whose instant the row already
shows, not an aggregate over the column. A MAX of two HTTP statuses is
arithmetic on a label, and a status describing a different failure than
the one on screen would send the operator to the wrong place. The column
joins the grouping key instead, and the fold picks the group belonging to
the newest failure.

Blank where there was nothing to record: a connection that never completed
has no status, and a chain that has since recovered had its status cleared
along with the rest of its failure state.
2026-09-03 06:05:58 +02:00