Commit Graph
248 Commits
Author SHA1 Message Date
Marc-André Bombeck 16024397c8 feat(observability): log heap samples and a one-time pressure report
runtime.heap.sampled records heap, old space, RSS, external memory and
the work in flight every five minutes, every minute at or above 70 % of
the heap limit. At 85 % runtime.heap.pressure lists the oldest requests
and jobs in flight and the routes most of them share, once, re-armed
only after the heap falls below 70 % and thirty minutes pass. A
heap-limit crash leaves no other record of what the process was doing.

HEALTHLOG_HEAP_SNAPSHOT=1 adds one heap snapshot at that moment into
HEALTHLOG_DIAG_DIR (0700 directory, 0600 file), off by default.
docs/ops/heap-diagnostics.md warns that a snapshot holds decrypted
health data and keys and says how to delete it.
2026-10-19 16:00:53 +02:00
Marc-André Bombeck b838ed4403 feat(dawarich): fetch daily movement values from a Dawarich instance
Connect, status, places, settings, test, sync and disconnect replace their
stubs. A nightly job (03:20) and the connect backfill read the timeline,
visits and residency endpoints through safeFetch and store one row of daily
values per local day: distances, places visited, minutes away from the home
place and the sealed country with its travel-day verdict. A private instance
needs its exact origin in DAWARICH_PRIVATE_ORIGINS.

The connection joins the shared ledger and the connection health model,
watched for errors only. The day carries a movement block, the model
projection drops the travel country, the two movement channels join the
correlation matrix under the environment module, travel days become trip
suggestions, and the days travel in the environment backup section.
2026-10-16 01:20:53 +02:00
Marc-André Bombeck ae8b614dd0 feat(insights): choose the status regeneration gap by ingest source
A sync from a watch or an integration now waits the instance's sync gap
(AppSettings.aiSyncRegenGapHours, else AI_SYNC_REGEN_GAP_HOURS, else 6 h)
before it may regenerate a status card again; a hand entry keeps the hour.
Every caller of afterMeasurementMutation and invalidateStatusInsightsForTypes
names its source, so a new ingest path cannot skip the choice.

Refs #1217
2026-10-15 22:49:00 +02:00
Marc-André Bombeck 6e72de1c35 feat(db): add the v1.43 schema and drop the readable location columns
Migrations 0402 to 0410: AI usage levels, biomarker aliases and merges,
medication ingredients (with the supplement label run kind), life
circumstance fact kinds, AI sections and AUDIT-C, Dawarich context, the
pulse day-mean method, intraday hour buckets, the readable location drop
and five relation link tables. Every new column carries its reader or a
pending-consumer marker.

0409 refuses to run while any home, location period or stored day still
holds a readable location, and names the counts. Locations are read from
the sealed copy only; the encryption backfill no longer handles them, and
the demo seed writes sealed locations only.

The new tables are backed up and wiped, the new sealed columns are in the
rotation registry with labels of their own, the AI usage level and the
life-circumstance confirmations travel with the backup, and the wellbeing
score reads the mental-health screeners only.
2026-10-15 21:48:57 +02:00
Marc-André Bombeck 89ccb71a12 chore(demo): show one row of six tiles with the weight chart below it 2026-10-15 18:49:51 +02:00
Marc-André Bombeck 9839bde4f6 chore(demo): let the dental check-up fall due on the seed day 2026-10-15 18:46:19 +02:00
Marc-André Bombeck eab4d644f7 feat(ops): log pending migrations before migrate deploy and document a dump before every update 2026-10-15 15:12:57 +02:00
Marc-André Bombeck 4a49784464 chore(demo): file the demo medications under categories, let the evening supplement record itself, and store two lab readings converted from SI units 2026-10-15 11:11:57 +02:00
Marc-André Bombeck 369775e039 chore(deps): move dotenv to 18
The breaking changes in 18 do not reach this repository: nothing preloads
dotenv with -r, nothing reads a .env.vault, and both entry points in use
(the default export in prisma.config.ts and the side-effect dotenv/config
import in the one-shot scripts) keep their behaviour. dotenv/config stays
quiet by default.

What does change is the load message, which now goes to stderr. Prisma
reads prisma.config.ts on every migrate deploy at container start, so
both loads there pass quiet: true rather than adding two stderr lines to
each boot. The image's /opt/prisma-cli install, which the config resolves
dotenv from, moves to the same 18.0.6.
2026-10-10 01:08:57 +02:00
Marc-André Bombeck 891c85a82b chore(demo): seed readiness and health-score history and a start for magnesium
The day view reads readiness from the nightly COMPUTED recovery row and
the health score from the record of each local day; neither job reaches
back, so a fresh seed showed both for today at most. Mint the rows the
jobs would have written over the window, and give magnesium the start
date its running line needs.
2026-10-09 18:30:29 +02:00
Marc-André Bombeck 1199e009b2 chore(demo): carry ragweed well into October in the seeded pollen 2026-10-09 18:30:29 +02:00
Marc-André Bombeck 145d138c66 chore(demo): seed a multi-year history for the timeline and day view
The demo account carried 90 days, which leaves the timeline's All zoom
and the day view with nothing to show before last quarter. The seed now
writes about four and a half years before that window: blood pressure in
bursts with a silent quarter and a thin one, weekly weight, phone steps,
watch resting heart rate, HRV and sleep from the third year, yearly
blood panels with HbA1c, lipids and hemoglobin, a resolved chronic
condition and three infections, ramipril with a stated start, a dose
change and a sick-day pause, a finished antibiotic course, flu shots and
a Tdap booster, visits, life events at day, month and year precision, a
sealed home location with a holiday period, and the stored weather, air
quality and pollen for every day. The timeline module is switched on.

All of it is invented and relative to the seed run; rows are written in
chunks so the seed stays quick.
2026-10-09 18:30:29 +02:00
Marc-André Bombeck 7c6510eba9 fix(db): let a PgBouncer host turn the session startup options off
The statement timeout and, since v1.42, work_mem reach Postgres as the
libpq options startup parameter, which PgBouncer refuses unless it is in
ignore_startup_parameters. A host behind it could drop the string by
setting the timeout to 0; work_mem is now always sent, so that no longer
worked. DATABASE_SESSION_OPTIONS_DISABLED sends no session settings,
documented in the example env, the env manifest, the compose whitelist
and the scaling guide.
2026-10-08 22:37:57 +02:00
Marc-André Bombeck c79fc6cc7b feat: lay down the day view and timeline contract
Land every file the day view, the timeline and life events share before
any of them is built, so the server, the day view and the timeline can be
built side by side without editing the same file (#613).

Schema and migration 0387: LifeEvent with a short category list, calendar
start and end dates, a day/month/year precision, encrypted title and note,
a soft-delete tombstone and an index for the per-user live list. Every new
column is annotated for the column-reader sweep. Life events travel in both
backup purposes with their two ends in one file, are wiped with the
account, and are registered for key rotation.

src/lib/day/contract.ts carries the constants and wire types for
GET /api/day/{date}, /api/day/index, /api/timeline, /api/timeline/readiness
and the life-event routes, free of Zod so the navigation chrome can import
it; the schemas beside it feed the routes and the published contract. The
Coach and MCP tool name get_day is reserved there, and life events are
listed as the one section no model ever sees.

The opt-in timeline module, its navigation entry, the day-section and
timeline-lane surfaces, the day and timeline query keys (evicted by every
record data bundle), and stub routes that answer 501 behind their
admissions. The four cross-section reads declare the whole record for now;
the life-event list is a profile read, and life events join the profile
sharing domain and its consent copy in all seven locales. The day,
timeline and lifeEvents message namespaces carry their base copy as
release-pending keys.

The aggregate bundle cap moves to 4400: the branch already measured 4365
against 4335, and the new copy adds 10 KB.
2026-10-08 13:03:37 +02:00
Marc-André Bombeck aca497d9ac fix(insights): correlate residuals, not the calendar
Two daily series that both follow the season (temperature and blood
pressure, daylight and sleep) correlated strongly with nothing between
them. In a seeded simulation of such a pair the engine surfaced a
finding in 58 % of 400-day runs and 30 % of 180-day runs, and a real
next-day effect under the season came back with the wrong sign.

Every daily pair now goes through seasonallyAdjustedPearson: a linear
trend and one annual harmonic are removed from both series (trend only
under 120 paired days), the residuals are correlated, and the p-value
uses the Pyper-Peterman effective sample size. Shrinkage and the
phrasing tier read the effective size too. BH-FDR and the effect floor
are unchanged. Pairs too short to fit the basis are not tested.

The weather channels are tested once over the mean of the day before
and the day itself against the same day's outcome (lagDays 0), which
pairs the overnight low with the night it shaped now that sleep is
keyed on the wake day. Discovered pairs carry an additive nEff, the
same-day templates are wired, and the matrix skips the weather read
when the environment module is off.

scripts/compare-correlation-findings.ts counts findings per account
before and after, read-only, for the release-candidate dry run.
2026-10-08 08:32:15 +02:00
Marc-André Bombeck e63be67b5a feat: lay down the shared v1.42 contract
Land every file the v1.42 work shares before any of it is built, so the
parts built side by side never edit the same file.

Schema and migrations 0379-0386: the HEALTH_CONNECT measurement source,
ImportJob.kind, the air-quality columns and the per-user switch on the
environment tables, sealed environment locations with nullable plaintext,
managed-profile handover links, the per-type HealthKit arrival ledger, lab
unit-conversion provenance, and user-defined vaccines. Every new column is
annotated for the column-reader sweep.

HEALTH_CONNECT reaches every exhaustive list: the source labels, the
read enum (not the write allowlist), the source-priority ladder cap, now
the number of sources rather than a fixed eight, and the write-path smoke
test. RMSSD gets its own priority class and a 300 ms ceiling. The batch
syncTrigger gains `manual`, with its label on the Apple Health card.

Queues for the Health Connect import, the compaction-tombstone purge and
the measurement-table maintenance are registered with stub handlers;
host-metric sampling moves to every five minutes. Environment and work_mem
variables reach the compose whitelist, the production example and the
manifest. The proxy admits the handover link and redirects it at the edge,
and the Health Connect upload bypasses the proxy body limit.

The new routes answer 501 behind their final authentication and are
published in the contract; backup, wipe and key rotation classify the new
models and columns; the seven message bundles carry the base copy. Guards
that list release-pending keys and columns fail once an entry is wired, so
those lists empty themselves before the tag.

Also re-anchor a login-throttle fixture to the wall clock, where its
expiry is compared, and re-base the aggregate bundle cap on a measured
12 KB of new locale strings.
2026-10-08 07:56:59 +02:00
Marc-André Bombeck 210becb28b feat(coach): lay down the reasoning, trail and memory contract
Add the shared contract the reasoning release builds on, with no change
to how a turn runs today.

- Schema: operator reasoning controls on app_settings (0376), the
  encrypted trail text on coach_messages (0377), and source,
  last_used_at and source_message_id on coach_facts (0378).
- Provider contract: CompletionParams.reasoning and onReasoning,
  CompletionResult.reasoning and providerState, AiMessage.providerState.
- Reasoning levels, thinking budgets and the background-job table in
  src/lib/ai/reasoning/levels.ts; coachPrefsJson.reasoning, absent
  meaning medium, salvaged field by field like every other pref.
- Stream: activity, memoryNote and planProposal frames, done.stop and
  done.withheldResults, result.interim, a compare chart kind, eight
  table refs, three more clarification kinds with an assumed choice,
  the change_assumption chip, and assumptions, activity, stop,
  memoryNote and planProposal on the provenance. Zod mirror, type
  parity and the native-client key rule cover every new frame.
- Chat request: memoryDecision and planDecision, parsed and not yet
  acted on.
- Stub contracts for the memory functions and the activity recorder,
  inert until their implementations land.
- Backup, restore and rotation carry the new columns; a fact's message
  reference is resolved against the restored messages like its
  conversation reference.
- Message keys for the trail, memory, plans, assumptions, replies,
  interim tables, comparison charts and reasoning settings in all seven
  locales, each reachable from a catalog and checked both ways.
2026-10-06 18:39:50 +02:00
Marc-André Bombeck 130ace74ee fix(crypto): keep updatedAt when key rotation re-seals a value
Key rotation rewrote every ciphertext column through a plain Prisma update,
in the in-app corpus walk and in the rotation script alike, so a rotation
stamped the run time on every encrypted row: the Coach panel then listed
old conversations as recent, and every client re-pulled measurements and
mood entries whose sync cursor had moved. Both now read each row's
updatedAt and write it back, matching on it, so a row edited since the
read is left for the next run instead of being stamped. The models that
carry the column are listed beside the registry and checked against the
schema, and the updatedAt guard now covers the rotation writes.
2026-10-05 13:07:22 +02:00
Marc-André Bombeck 7d42cf7122 fix(coach): keep updatedAt through storage-only rewrites
The title encryption backfill moved each Coach conversation's title into
ciphertext with a plain Prisma update, which stamps updatedAt. The Coach
panel orders and groups conversations by that column, so threads untouched
for months showed up under Today. The memory summary refresh did the same
in the background.

Storage-only rewrites now carry the row's own updatedAt and match on it, so
a turn that lands meanwhile wins and the row is retried on the next pass:
conversation titles, practitioner contacts, measurement and mood notes
(where updatedAt is the sync cursor), inventory notes, and the mood-note
script. The summary refresh writes through raw SQL. A guard freezes both
rules: every backfill write on a model with @updatedAt names the column or
is listed with the reason the stamp is wanted, and every conversation
write outside the person's own actions keeps it.
2026-10-05 12:40:55 +02:00
Marc-André Bombeck adb0e7c3b1 fix(demo): seal the Coach titles the way the app writes them
A readable title is moved to title_encrypted by the boot backfill, and
that update also sets updated_at to the moment it ran, so every seeded
thread sorted under "Today" a minute after the restart. The seed now
writes the ciphertext directly.
2026-10-05 11:39:15 +02:00
Marc-André Bombeck 25d486f956 fix(demo): end glucose at yesterday and steady the sleep average
The vitals check holds a day's glucose mean against the band of full
days, so a part day seeded up to the current hour read "below your
range". Spot readings and the CGM stream now end with yesterday.

The aggregate sleep walk drifted to almost nine hours, against which the
~7.5 h per-stage nights read as an hour short. A tighter walk keeps the
trailing average near the nights it is compared with.
2026-10-05 11:28:40 +02:00
Marc-André Bombeck 2b5d22e93d fix(demo): seed the surfaces the start page and Coach now show
The demo seed predates the Today overview, the grouped Coach history and
the procedure filter on documents, and it carried a few defects that a
fresh seed made visible:

- Store every Date as UTC. node-postgres serialises a Date in the host's
  zone and the cast to timestamp drops the offset, so a seed run from a
  machine on Berlin time wrote every instant two hours ahead.
- Keep today's later glucose readings, CGM samples and the evening mood
  entry out of the future.
- Close the seasonal allergy and its flare in the spring. Any open
  episode puts the account in Rest Mode, and the open one read
  "Rest mode, day 401" on the start page.
- Pin today's vitals to the median of the prior four weeks, so the
  Today overview does not report a random tail value as out of range.
- Confirm the key-backup step for the key the demo runs with, which
  removes the backup banner every visitor saw above the dashboard.
- Add two practitioners, a knee arthroscopy with its operative report
  and discharge letter linked (small generated PDFs), its follow-up and
  a check-up booked for tomorrow.
- Replace the single Coach exchange with five conversations from today
  back over a few weeks, the newest with a full answer and its steps.
- Drop the layout field `heroVisible`, which the layout type no longer
  has.
2026-10-05 11:23:28 +02:00
Marc-André Bombeck d07f5796d7 Seed the demo Coach answer with valid metric tokens, steps and token meta 2026-10-04 20:02:49 +02:00
Marc-André Bombeck db9618ee7b Merge remote-tracking branch 'origin/main' into wip/v140-H 2026-10-04 00:43:10 +02:00
Marc-André Bombeck 69bae8bdb4 Add ENCRYPTION_KEY_CHECK=warn as an escape hatch for the boot key check
At a first start there is no recorded key yet, so an operator facing a
refusal they are certain is false has no row to delete. With
ENCRYPTION_KEY_CHECK=warn (default enforce) the check still runs and logs
its full outcome, /api/health names a mismatch as a warning field without
failing, requests are served, and no key is recorded unless every key id
passes.

The variable is on the compose whitelist, in both example files, the env
manifest, the Unraid and Portainer templates and the published-env guard.
The log block, the mismatch page and the self-hosting pages offer it as the
last resort after the three real fixes, and say that it disables a safety
check.
2026-10-03 23:46:40 +02:00
Marc-André Bombeck 2c295bbea8 Probe the oldest stored values before recording an encryption key
The boot key check recorded whatever key was configured whenever its probe
found nothing to try. It only read string columns, so an install whose
ciphertext sat in Bytes columns (notes, documents, ECG, symptoms) read as
empty; a probe that ran out of time did the same; and rows written by a
process that served under a wrong key could open on the next boot and get
that key recorded as the right one.

The probe now reads every registered column, string and Bytes, in all four
stored layouts, and tries the oldest values first, ordered by the row's
updatedAt (else createdAt, else primary key). A key is recorded only when
the oldest value that opens is preceded by at most one that does not, or
when no column holds a single value under the key id. A time-out, an
unreadable table, oversized values or mixed results are inconclusive: the
process serves, records nothing and checks again at the next start.

When the first insert loses a race, the stored record is read back and must
open under this process's key, else the outcome is a mismatch.

A rotation that walked every column without errors removes the record of
each previous key id that no longer holds any value. The refusal log block
and the self-hosting pages explain how to reset the check for a key id.
2026-10-03 23:29:55 +02:00
Marc-André Bombeck 1494409209 Merge remote-tracking branch 'origin/main' into wip/v140-H
# Conflicts:
#	messages/de.json
#	messages/en.json
#	messages/es.json
#	messages/fr.json
#	messages/it.json
#	messages/ko.json
#	messages/pl.json
#	src/__tests__/sharing-surface-guard.test.ts
#	src/__tests__/v137-wave-zero-fixtures.test.ts
#	tests/integration/sharing-manage-handler-matrix.test.ts
2026-10-03 22:52:49 +02:00
Marc-André Bombeck a14174fe4d Let a medication have several courses
A medication taken now and then (#1024) keeps one row and gains course
rows: calendar spans, both ends inclusive, an optional encrypted note, no
schedule or dose of their own. Courses never overlap and every course but
the latest has ended, so the latest course is always the one the
medication's startsOn/endsOn project. src/lib/medications/courses.ts is
the only writer of those two columns for a medication with courses; it
writes under a per-medication advisory lock and re-projects in the same
transaction, so reminders, the next-due line and intakeActionable keep
reading the columns and stay right.

- Migration 0368 adds medication_courses and backfills one course per
  medication window (an end-only window starts on the creation day, or
  on its end day when that lies earlier); a medication without a window
  stays course-less. Idempotent, data-preserving.
- GET/POST /api/medications/{id}/courses and PATCH/DELETE
  .../courses/{courseId} (owner-only); create seeds the first course
  from a window; the medication PUT edits the latest course and refuses
  a window that would break the rules (medications.course.*).
- List, detail, create and update carry courses, courseCount,
  previousCourseEndedOn and canStartCourse, resolved on the server.
- Compliance expands over the span of all courses and drops the slots
  between them; every compliance read selects the courses. A rolling
  cadence is not anchored by an intake from a previous course.
- The card names the last course's end and the running course's number;
  the detail Lifecycle group lists the courses with start, edit and
  delete. The Coach compliance block, the MCP resource, the doctor
  report window and table, the clinician view and FHIR (one
  MedicationStatement per course with effectivePeriod and status) read
  the courses.
- Backup carries courses per medication and the restore writes them and
  re-projects; an older file derives the one course its window
  describes. Wipe plan, encrypted columns, rotation and OpenAPI follow.
2026-10-03 22:39:25 +02:00
Marc-André Bombeck ef0028c020 Merge remote-tracking branch 'origin/main' into wip/v140-I
# Conflicts:
#	docs/api/openapi.yaml
2026-10-03 22:16:29 +02:00
Marc-André Bombeck 1564e7c396 Add the sick note document kind
A certificate of incapacity for work (in Germany the
Arbeitsunfähigkeitsbescheinigung) is now its own document kind,
SICK_NOTE, instead of Other. The enum value rides migration 0370.

Uploads, edits, the kind filter, bulk edits, the import script and the
backup format accept it; the vault files it with a thermometer glyph
between prescriptions and referrals; the filing suggestion can propose
it and names the words a sick note carries. The extraction's own kind
field keeps its three values.
2026-10-03 21:40:01 +02:00
Marc-André Bombeck 4cb8118802 Let a person name their own medication categories
A medication can be filed under a category the person named, next to the
built-in ones (#1041). The labels live in medication_category_labels,
encrypted at rest, capped at 20 per account; a medication filed under one
carries its custom:<uuid> key in the category column.

- GET/POST /api/medications/categories (POST idempotent, 30/min per actor,
  admitted at MANAGE like the cycle symptom vocabulary) and owner-only
  PATCH/DELETE /api/medications/categories/{key}. DELETE moves every
  medication filed under the category to OTHER in the same transaction
  and answers movedCount.
- category is widened to a built-in value or custom:<uuid> on the wire,
  the create and update bodies refuse a key that is not the caller's
  (422 medications.category.unknown), and list, detail, the insights
  medication tiles and the MCP medications resource carry categoryLabel.
  A key with no owned label reads as OTHER and is counted on the event.
- The wizard's class step lists the person's categories under the
  built-in rows with an inline add row; the list page gains a category
  filter (session only) and a manage sheet for rename, hide and delete.
- Backup carries customMedicationCategories (decrypted labels) and the
  restore writes them before the medications; wipe plan, encrypted
  column registry, rotation script, sharing guards and OpenAPI follow.
2026-10-03 21:38:03 +02:00
Marc-André Bombeck d6571c4e9c feat(symptoms): add person-defined symptoms and timestamped occurrences
A symptom of one's own ("aura", "headache") is a definition with an
encrypted name and an icon from the cycle custom-symptom allowlist; each
occurrence carries a time, a 0-10 intensity (the pain score's scale), an
optional encrypted note and an optional link to one of the record's open
illness episodes. Both ride the illness module: every route answers
illness.disabled when it is off.

- Migration 0369 (idempotent): symptom_definitions and symptom_events,
  intensity CHECK 0-10, episode link ON DELETE SET NULL.
- /api/symptoms/definitions{,/[id]} and /api/symptoms/events{,/[id]}:
  eight active definitions per record (a hidden one frees its slot,
  showing it again re-checks), hide or purge, events create/list/edit/
  delete, a new link must name an open episode of the same record
  (422 symptoms.episode.notOpen). Error group symptoms.*.
- Sharing: reads at READ, logging an occurrence at WRITE, defining and
  editing at MANAGE; C1/C3/C4 audit details; frozen lists, the handler
  matrix and the activity verbs updated.
- Both encrypted columns registered and rotated; both models in the wipe
  plan; both carried at both backup ends with occurrences nested in their
  definition, plaintext in a portable file and ciphertext in a recovery
  file, and a link to an episode the file lacks dropped and named
  (symptomEpisodeReference).
- OpenAPI regenerated.
2026-10-03 21:13:45 +02:00
Marc-André Bombeck 37793298a8 Merge remote-tracking branch 'origin/main' into feat/key-backup-and-transport
# Conflicts:
#	docs/api/openapi.yaml
2026-10-03 19:22:01 +02:00
Marc-André Bombeck 0acf3da575 feat(documents): read documents and lab scans in the background
"Read with AI" and the lab scan used to run their model calls inside the
request. At the default response time a read of three calls outlasted the
100 s a proxy in front of the server commonly allows, and the person saw a
cut connection for a read the server was still doing.

Both now run as a background run (`DocumentAiRun`, migration 0366):

- `POST /api/documents/inbound/{id}/index` with `Prefer: respond-async`
  answers 202 with a run id once every quick refusal has passed. Without
  the header it answers exactly as before, which the iPhone app relies on.
  A second request while a read of the same document is queued or running
  attaches to that run.
- `POST /api/labs/ocr/extract` always answers 202. Only the web client
  calls it.
- `GET /api/ai-runs/{id}` serves the run: the same body the synchronous
  route answers with, or the failure code and status it would have given.
  Scoped to the caller; another account's run is a 404.

The worker (`document-ai-run`, a long queue with a one-hour expiry, one
job per run under `lockedPass`) claims the run, asks the capability again
for the record and re-checks the wire for the provider it picks, sets the
run's deadline from the person's AI response time and refuses a read that
cannot fit the job's expiry. Sync route and worker run one body per kind,
so the two paths cannot drift. Budget is reserved at enqueue and settled
once by whoever ends the run.

Input and result are sealed with labelled AES-256-GCM, registered for key
rotation; the input is dropped when the run ends. A five-minute reaper
fails a run no worker took within fifteen minutes (and hands its
reservation back), fails a run past its deadline, and deletes a finished
run after an hour. Runs are outside backups and inside the data wipe.

The web client polls through the query-key factory and says, calmly, that
the read continues in the background, and when no worker has picked it up.
2026-10-03 18:18:56 +02:00
Marc-André Bombeck 308470f229 feat(encryption): back up the key, and refuse to serve when it cannot open the database
Every configured key id now has a canary in encryption_key_canaries,
checked at boot. A key that cannot open its canary, or existing data
written before the canary existed, puts the process in a refusal state:
every API route answers 503 encryption.key_mismatch, /api/health reports
reason encryption_key_mismatch, the pages render one explanation, and
neither the queue producer nor the worker starts. The process stays up
so the reason is visible instead of hiding in a crash loop.

The admin Encryption page gains a "Back up your encryption key" step:
key id and fingerprint (never the key), where the key lives per
platform, a check of a pasted copy that is compared and discarded, and
a confirmation bound to the active key id and fingerprint. The overview
gets a tile and admins get a dashboard banner while it is due.

HEALTHLOG_PLATFORM picks the platform the step opens on. Adds the
TrueNAS self-hosting page and key backup sections in the docs.
2026-10-03 18:06:28 +02:00
Marc-André Bombeck c2ab9cda11 test(coach): put the dialog scenarios to a real model nightly
The live evaluation now captures the tool calls a generation makes, with
their arguments, and hands them to the judge. runScenarioLive puts a
scenario to the model in the chat turn's own context (system prompt,
tool-mode and dialog addenda, DATA INVENTORY, EARLIER TABLES, transcript)
over the real tool catalogue, answers each call with a stand-in result,
and reads the reply through the turn's clarification and chip parsers.
runScenarioJudge grades each scenario a few times with the same
evaluateScenario the deterministic suite uses and reports a pass rate per
scenario; the server-only scenarios and checks are left out.

Like the existing judge it runs only with COACH_EVAL_API_KEY, from the
nightly workflow's non-blocking step, and never throws.
2026-09-27 13:07:25 +02:00
Marc-André Bombeck 78fc22d0f8 Merge branch 'coach/wpF-contracts' into release/v1.39.4 2026-09-27 11:10:37 +02:00
Marc-André Bombeck a8b3e5d7f4 feat(coach): store a turn's result tables encrypted beside the message
A new nullable column, coach_messages.results_encrypted, holds the tables
of values a turn read as AES-GCM ciphertext; only their metadata rides the
plaintext provenance. readMessageResults serves them to the owner, each
table whole or withheld with a reason. The provenance restore learns the
steps, method, table metadata, chips, clarification and the forced-answer
marker, entry by entry against the wire schema.

The column is registered for key rotation, carried through both backup
formats (ciphertext in a disaster-recovery file, readable JSON in a
portable one) and goes with its row on conversation delete and wipe.
2026-09-27 10:36:27 +02:00
Marc-André Bombeck b8e78a78b3 chore(openapi): regenerate for the practitioner backup ciphertext fields
The practitioner entry of the backup payload gains the optional
locationEncrypted and phoneEncrypted fields. The rotation script names each
new column once, at its call, so a dropped call is noticed rather than
covered by a comment.
2026-09-27 10:21:48 +02:00
Marc-André Bombeck 5eb7956bad fix(privacy): encrypt workout GPS tracks and practitioner contact details at rest
The GPS track of an outdoor workout was stored readable, and its first and
last points are usually the person's front door. The phone number and address
in the address book of doctors and practices were readable too, while the
note beside them was already encrypted.

Migration 0361 adds workout_routes.geometry_encrypted (the geometry loses
its NOT NULL) and practitioners.phone_encrypted / location_encrypted, plus
partial indexes on the backfill's discovery predicates. A track is sealed as
one binary AES-256-GCM value bound to its purpose; the contact fields use the
same Bytes codec as the note. The practitioner's name, practice and specialty
stay readable, because the picker searches and sorts on them.

Every writer stores ciphertext only: workout batch ingest, practitioner
create and edit, and both restore formats. Readers prefer the ciphertext and
fall back to the old column for rows not yet migrated: the workout detail
(map, splits, GPX), the activity insight's climb figure, the practitioner
DTO behind the address book, visits and vaccinations. The wire shape does not
change. An edit's audit row now names a changed phone number or address
without recording either value.

free-text-encryption-backfill covers the three columns in the same pass,
finding a route through its workout. All three are registered for key
rotation (registry, CLI script, and the in-app rotation). A disaster-recovery
backup carries the contact fields as ciphertext only; a portable export
carries them readable. Workout routes are still not part of a backup, as
before. The old columns are marked for removal in a later release.
2026-09-27 09:59:34 +02:00
Marc-André Bombeck ed05772a8e test(e2e): run the delegated-writes journey and fail on silent skips
The delegated-writes spec checked for the invitation form's level control
right after page load, before the form had painted, found nothing and
skipped every test on every run, CI included.

- Give the journey its own account pair, seeded and reset in global setup,
  so it no longer invites the same pair as the account-sharing journey
  (a second live grant between two accounts is refused with 409).
- Wait for the form through its data-slot and fail when the WRITE level
  control is missing instead of skipping.
- Revoke the journey's grant in afterAll.
- Keep it out of the mobile project in the config rather than through a
  runtime skip.

Add two checks so a skip cannot hide a spec again:

- e2e-conditional-skip-guard.test.ts requires every conditional
  test.skip/test.fixme in e2e/ to carry a reason and refuses a bare
  test.skip().
- scripts/check-e2e-skipped-specs.mjs reads the Playwright JSON reports of
  both shards in the e2e gate job and fails when a spec file skipped all of
  its tests across every project, apart from an allowlist with reasons.
2026-09-27 04:48:08 +02:00
Marc-André Bombeck 66b1aef40c Merge branch 'fix/encrypt-free-text-and-local-ai' into release/v1.39.3
# Conflicts:
#	src/app/api/import/csv/route.ts
2026-09-26 21:50:05 +02:00
Marc-André Bombeck 4409529c89 Merge branch 'fix/backup-rotation-and-deletion' into release/v1.39.3
# Conflicts:
#	docs/api/openapi.yaml
#	src/lib/jobs/apple-health-import-worker.ts
2026-09-26 21:48:20 +02:00
Marc-André Bombeck c86f23c2c0 fix(security): keep ALLOW_LOCAL_AI_PRIVATE_HOSTS=true for the operator's own AI, deprecated
Making `=true` grant nothing would stop a single-user self-host's local
model on a patch update. It is deprecated for one release cycle instead,
in a narrowed and safe form: it still grants a private host, but only to
the operator's own configurations (the instance-wide admin provider and
AI settings saved on an admin account), and only through the pinned
operator-approved dispatcher, so redirects stay forbidden and metadata,
link-local and unspecified addresses stay refused. A non-admin account's
own base URL needs an exact origin in AI_PRIVATE_ORIGINS.

Provider resolution passes the owning account's role to the client
(`operatorTrusted`); the save route and the connection test apply the
same rule to the account being configured. The boot summary now says the
setting is deprecated and names the replacement, and the admin AI
settings page shows the exact AI_PRIVATE_ORIGINS line for the private
origins saved on the instance (every account's Local and gateway URL and
the admin base URL, scheme, host and port only), so the operator can copy
it and drop the old setting.
2026-09-26 20:59:10 +02:00
Marc-André Bombeck 849748c4cd fix(backup): bind new off-host objects to their key and account, and cap what an uploaded backup may inflate to
Off-host objects are now written as envelope version 4: the header names
the off-host key by fingerprint, and the object's key in the bucket
(<date>/user-<id>.json.enc) is authenticated as GCM associated data, so
a copy moved to another account or another night no longer opens.
Objects of versions 1 to 3 still read. BACKUP_ENCRYPTION_PREVIOUS_KEYS
(comma-separated, on the compose whitelist and in the env manifest)
keeps retired off-host keys readable, which is the rotation path the
off-host key did not have.

The admin backup upload counted the bytes that arrived and none after
the gunzip, so a small gzip could expand without limit into the JSON
scanner and take the app down. The inflated bytes are now held to 100
times the compressed bytes read (at least 256 MB, at most 4 GB) and a
file past that is refused with 413 before anything is stored.
2026-09-26 20:18:55 +02:00
Marc-André Bombeck 1a674cf4b0 fix(security): grant private AI endpoints by exact origin, dialled pinned
ALLOW_LOCAL_AI_PRIVATE_HOSTS=true let any user of the instance point the
Local provider or the OpenAI-compatible gateway at any private address
the server could reach, the metadata range included, and a granted host
was dialled with no connect-time pin at all.

AI_PRIVATE_ORIGINS grants a private AI endpoint with the exact-origin
grammar NOTIFICATION_PRIVATE_ORIGINS and NIGHTSCOUT_PRIVATE_ORIGINS use.
A granted call goes through the pinned operator-approved dispatcher:
redirects forbidden, and metadata, link-local and unspecified answers
dropped at dial time even for a granted name. Everything else stays on
the public pin. The Local client (buffered and streaming) and the
gateway and admin-key tags of the OpenAI client all take their policy
from one function.

The host-list form of ALLOW_LOCAL_AI_PRIVATE_HOSTS keeps working with the
same pinning; a never-grantable literal or a host:port entry is refused
with a one-time warning. Its "true" form grants nothing any more and is
reported in the boot readiness summary with the replacement.

A failed lab-report scan no longer puts the endpoint's response body into
the wide event; status and model stay. The response to the caller never
carried it.
2026-09-26 19:46:11 +02:00
Marc-André Bombeck f7b3ce6ee0 fix(privacy): encrypt Coach conversation titles and custom-metric notes at rest
The title of a Coach conversation is the first 80 characters of the
opening message, and the note on a custom-metric reading is free text;
both were stored readable while every other note was encrypted.

Migration 0357 adds coach_conversations.title_encrypted and
custom_metric_entries.note_encrypted (the title loses its NOT NULL) plus
partial indexes on the backfill's discovery predicate. Every writer now
stores ciphertext only: conversation create, proactive nudge, reminder
surfacing, rename, entry create and edit, and both restore formats.
Readers prefer the ciphertext and fall back to the old column for rows
not yet migrated; a ciphertext that does not open throws instead of
falling back. The history title search decrypts in memory, since SQL can
no longer match the title.

free-text-encryption-backfill is a boot-time, idempotent pg-boss job
(staggered past the startup storm, daily discovery tick) that seals each
remaining readable value and nulls it in one transaction per row. A pass
that sees a row twice stops with an error instead of looping.

Both columns are registered for key rotation (registry, CLI script, and
the in-app rotation that walks the registry). A disaster-recovery backup
carries ciphertext only, sealing a not-yet-migrated row on the way out;
a portable export carries readable text. The old columns are marked for
removal in a later release.
2026-09-26 19:46:01 +02:00
Marc-André Bombeck 026e8fc1ed Merge branch 'fix/compose-env-and-claims' into release/v1.39.3 2026-09-26 17:00:46 +02:00
Marc-André Bombeck bbf6a0aeb9 build: name the OCR language packages literally in the staging script 2026-09-26 15:45:28 +02:00
Marc-André Bombeck 2c88acc5f7 fix(labs): serve the in-browser OCR engine from this origin
Local lab OCR reads a report photo on the device with tesseract.js, whose
defaults load the worker, the WebAssembly core and the language data
from cdn.jsdelivr.net. The production CSP allows workers and fetches only
from this origin, so the engine could not load; where it did load (dev),
the browser of anyone scanning a report contacted a third-party CDN.

The worker, the three LSTM core builds and the deu and eng best_int
language data (about 17 MB) are now copied from pinned packages into
public/tesseract/ at prebuild and predev, and createWorker is pointed at
them with the worker started from its URL rather than a blob. A test
holds the options against the staged file list.
2026-09-26 15:37:13 +02:00