Commit Graph
94 Commits
Author SHA1 Message Date
Marc-André Bombeck 77ace106b3 fix(http): refuse state changes a browser sends from another site
The session cookie is SameSite=Lax, and SameSite compares the registrable
domain, so a page on a sibling subdomain of the instance could send a
CORS-simple POST with the person's cookie attached. safeJson accepted any
Content-Type containing "application/json", so a text/plain body that
mentions it in a parameter, which needs no preflight, was parsed as the
person's own request: readings added, entries deleted, integrations
disconnected, a delegate invited within the re-proof window.

apiHandler now refuses a POST, PUT, PATCH or DELETE before authentication
when the browser marks it Sec-Fetch-Site same-site or cross-site, or, from
a browser that sends no fetch metadata, when its Origin names a host other
than the one the request went to or APP_URL. A request with an
Authorization header, or with neither header (the phone app, scripts,
webhook senders), is unaffected, as are the webhook and URL-credential
routes that authenticate with their own secret. The JSON media type is
compared exactly, with parameters allowed, in safeJson and in every route
that branched on a substring of it.
2026-10-25 13:52:04 +01:00
Marc-André Bombeck 474fc08649 fix(documents): bound the lab staging an index request continues into
The index route without Prefer: respond-async, which the iPhone app
calls, continues into the lab staging with no run schedule. The staging
then planned a long report's text as up to twenty reads with a retry
each: forty model calls inside one HTTP request, the same class the
extract route was bounded against.

The staging now plans a missing schedule as one request does
(SYNCHRONOUS_READ_SCHEDULE, at most three reads): a longer report has
its text shared out between them, and what does not fit is left out and
reported as extractionIncomplete on the document. The request bound
moves into ocr-plan beside the schedule it bounds, so the extract run
and the staging share one definition. Background runs and the automatic
index job keep planning against their own deadline.
2026-10-25 02:52:10 +01:00
Marc-André Bombeck d6c2760415 Merge remote-tracking branch 'origin/release/v1.46.0' into fix/review-meds-docs
# Conflicts:
#	src/__tests__/baselines/source-function-lines.json
#	src/__tests__/baselines/version-stamp-comments.json
#	src/__tests__/sharing-surface-guard.test.ts
2026-10-25 02:45:23 +01:00
Marc-André Bombeck c9481b672c fix(documents): bound an extraction that runs inside the request
Without Prefer: respond-async the extract route ran the read with no
schedule, so a twenty-page report read a page at a time, with a retry
each, could make forty model calls in one HTTP request. The iPhone app
sends no Prefer header and reads the document detail from the answer,
so the request keeps its 200 and its body.

An extraction without a run deadline now plans at most three reads (six
calls): a longer document has its pages or text shared out between
them, and what does not fit is reported as extractionIncomplete on the
document. The background run keeps reading it a page at a time.

Refs #1257
2026-10-25 01:01:13 +02:00
Marc-André Bombeck b0ea3b1654 fix(ai): charge what failed hops spent when a later hop answers
A chain whose first provider reasoned without answering and whose second
one answered billed only the answer: result.tokensUsed is the answering
call's alone, and the failed hop's spend, already reported on its
FallbackHop, was dropped by every caller that reconciles the day's ledger.

The raw runners now report tokensSpent on success too, the sum the failed
hops before the answer reported. The briefing, status, Coach (no-tools path
and every tool-loop round) and medication-extract reconciles charge it on
top of the answer. On failure, the Coach turn ledger's close keeps what the
failed chain reported spending instead of refunding everything, and the
medication-extract and document-chat routes charge it too, as the status
and briefing paths already did. Document indexing resolves a single provider
and already charged its failed call.

Refs #1279
2026-10-25 00:33:18 +02:00
Marc-André Bombeck 52fa5a06b1 feat(documents): let the picker bring back a deleted document
The document picker showed a document deleted in HealthLog as "Deleted
in HealthLog" and would not let it be ticked, so a person who deleted an
imported document by mistake met a dead end exactly where they would look
for it again.

Such a row can now be ticked, and the import sends revive: true for it.
A document still in Recently deleted is restored through the shared
restore, nothing is fetched, and the row reads "Restored from Recently
deleted". One the purge already removed is fetched and stored again, and
only once it is stored does the ledger forget its key, so a failed fetch
leaves the deletion standing. Without revive the route answers "deleted"
as before, and the token upload has no such switch: a Paperless workflow
or the import script re-sending a deleted document is still refused,
whether it is in Recently deleted or long purged.

The search names the document a deleted key still points at (documentId,
null once purged), so the row can say "In Recently deleted" rather than
"Deleted in HealthLog". The outcome gains "restored", sent only in answer
to revive. The row logic moves to picker-rows.ts.

Refs #1275
2026-10-24 20:55:19 +02:00
Marc-André Bombeck 4752115d7d feat(documents): add Recently deleted for deleted documents
Deleting a document only stamps a tombstone and the purge removes it
after 30 days, but the only way back was the undo toast, gone a few
seconds after the delete. Recently deleted lists the deleted documents
of the record with type, filing date, where an import came from, when
each was deleted and how long it has left, and offers Restore, Restore
all and Delete now.

GET /api/documents/inbound/deleted lists them (owner-only, capped at 200
with the list-cap meta). POST /api/documents/inbound/{id}/purge removes
one deleted document for good through the same function the nightly
purge runs, so its import keys and aliases still move into the ledger
and an importer re-sending it is still answered "deleted"; a live
document answers 409 documents.inbound.notDeleted. The usage read gains
deletedCount, 0 under a grant. The restore route and bulk restore
share one restore, which now also clears ledger entries for the
document's own keys.

The vault header shows a quiet "Recently deleted (n)" link only while
something is in it, and the delete toasts and confirmations say that
deleted documents stay there for 30 days. The purge's queue facts and
grace window move to document-purge-queue.ts so the worker registry
still loads none of the work.

Refs #1275
2026-10-24 20:51:35 +02:00
Marc-André Bombeck a1ee2ac003 fix(documents): keep document reads inside their job's expiry
Since the lab staging reads a long report in parts, one auto-index could
make the transcription plus up to twenty reads of two calls each at the
person's response timeout, hours past pg-boss's fifteen-minute default,
which failed the job and retried it beside the reads still running. The
staging now takes the run's deadline as a schedule and plans only the
reads that fit before it. The index job is long, with an expiry above the
calls that run before the plan can say no, a stop on jobDeadline and a
lock per document; the document run hands the same staging its own
deadline, which moves the run's expiry as the reads are planned. The
document summary, one call and its reasoning repeat, gets an expiry above
that worst case.

ocrReadCallMs in the read core is the one per-call timeout both deadlines
divide by. The expiry guard now names every importer of the document and
lab read entry points with the queues that reach it.

The runtime guard's planted cases stripped every file's comments again on
each rescan, which took over a second alone and timed out under a full
parallel run; the stripped text is now memoised.

Refs #1274
2026-10-24 11:28:57 +02:00
Marc-André Bombeck 5a7bd95c7b fix(documents): read a long document through the lab scan's read core
Extracting lab values from a stored document still sent the whole text
in one call. A multi-page report produced more JSON than one call's
output ceiling, the answer was cut mid-fact, the parse failed twice, and
the read came back as 422 "Couldn't read the document. Try a clearer
copy." The lab scan had already learned to read the same report a page
or a part at a time, but each surface carried its own copy of the read,
so the document path never got it. It also ran under the shared 60 s
timeout, dropped every fact past 120 (or every fact when one did not
match the schema), and charged a flat reservation.

The read now lives once, in lib/documents/ocr-read.ts and ocr-plan.ts:
the per-read limits, the corrective retry, the salvage of a cut
answer's complete rows, the wire re-check and budget reservation before
each read, and the planning of a PDF a page at a time and text in parts
at line boundaries. The lab scan and the document extraction (the
extract route, its background run and the lab auto-staging) both read
through it, and the background run plans its reads on the same deadline
as a lab scan. The per-read budget moves to a shared metered budget.

The document path validates the answer fact by fact, keeps the lab
scan's row cap (400), and says why nothing came back: 422
documents.inbound.outputLimit when the reads stopped at the output
limit, 429 when the budget refused them, extractFailed only for answers
that were not the schema. A run whose reads were cut or not reached
reports `incomplete` on its result.

A guard holds the class: the output ceiling is set only in the core,
both surfaces read through runOcrReads, both runs share the deadline,
and a lab read and a document read go out under the same limits.

Refs #1257
2026-10-24 10:34:31 +02:00
Marc-André Bombeck 91a5464aa3 refactor(documents): move the vault upload reading and the list read out of their route 2026-10-23 15:10:26 +02:00
Marc-André Bombeck 5b49fa6680 refactor(api): read route bodies through parseJsonBody
132 route handlers spelled out safeJson, safeParse and the 422 refusal by
hand in one of six shapes. Each block becomes one parseJsonBody call with
the same size cap, wide-event action, issue count, error code, message and
status it had, so every response is the one it was. The comments inside
the replaced refusal blocks go with them.

A ratchet (route-body-parse-ratchet) counts the route files that still call
safeJson themselves; the count may only fall.
2026-10-23 11:12:39 +02:00
Marc-André Bombeck 5185edca74 fix(rate-limit): fail the shared write ceiling open and skip reads over POST
The shared write bucket threw on any SQL error, so a lock on rate_limits
answered every authenticated write with 500. It is a loop cutter, not an
access check: when the bucket cannot be read the write now goes ahead with
a warn signal. The login and other auth-surface limits call the limiter
directly and still fail closed. requireAuth now passes readOverPost to the
meter, the draft-only document assist declares it, and the three chat
routes, which meter their turns with buckets of their own, declare that
instead of being counted twice.
2026-10-20 16:09:28 +02:00
Marc-André Bombeck 8db7c71b0e fix(api): one helper for list pages, exact cut signal, total order
listPage (src/lib/http/list-query.ts) reads one row past the limit and
cuts it back, on top of the existing list-cap signal rather than beside
it. Every capped list now gets its page there, and every read passed to
it orders by something ending in the row id, so rows that share a sort
value cannot repeat or vanish at a page boundary.

- practitioners, vaccinations and encounters were cut at their take
  with no signal; they now answer meta.truncated and accept offset, and
  their web readers walk the pages instead of stopping at the first
- symptoms/events reported hasMore on a page that was exactly full; it
  is now true only when a row past the page exists
- the document list's hand-rolled limit + 1 goes through the helper
- security-activity answers an out-of-range limit with the 422 its
  published schema already promised instead of clamping it

list-page-guard binds every apiListSuccess route to listPage, forbids
hand-cut lists and length-derived hasMore, and requires the id
tiebreaker in every listPage read.
2026-10-20 13:29:14 +02:00
Marc-André Bombeck fea1d1e0b5 chore: strip planning tags from comments and enable the rule
Output of `node scripts/strip-planning-tags.mjs --write`: 3,157 comment
lines in 1,308 files lose the work-item identifier at their head. Release
versions and issue references stay; nothing outside comment text
changes, which the script checks per file against a comment-free
printout. healthlog/no-planning-tags is enabled at error level with this
commit, so the tree it lands on is clean.
2026-10-20 09:08:40 +02:00
Marc-André Bombeck b4bc039d16 feat(links): close the relation matrix and publish related[] on six details
The link service gains medication and lab result sources and the five new
tables, each readable from both ends. POST/DELETE /api/medications/{id}/links
and /api/labs/{id}/links replace their 501 stubs; the visit links route also
files medications and doses, the dose links route its visit. A scan commit
with a documentId files each written result against its report.

GET on a lab result, visit, document, condition, medication and dose now
carries related[]: stored links, the condition a result shares a visit with,
and the medication whose efficacy target is the result's marker, each with
its web path. A kind is listed only when its module is on and the caller's
grant covers its section. relation-matrix.test.ts holds every one of the
fifteen pairs to a table, a reader or a written exception.
2026-10-15 23:57:34 +02:00
Marc-André Bombeck 74fa4e10b6 Merge branch 'v143/labs' into release/v1.43.0 2026-10-15 23:06:51 +02:00
Marc-André Bombeck 81be73fca8 feat(labs): merge markers, keep aliases and read lab series per marker
POST /api/biomarkers/{id}/merge folds markers into one in a transaction,
converting readings where the registry knows the units and refusing the
whole merge where it does not; a dry run answers the same result and
writes nothing. Aliases appear on the marker and are edited through the
marker's PUT. Scan and document review rows name the matched marker or
offer up to three, and a chosen marker is remembered as an alias. The
doctor report, FHIR and MCP group readings by marker. Aliases and the
merge ledger travel through backup and restore.
2026-10-15 22:22:40 +02:00
Marc-André Bombeck 2fd62cc115 feat(api): carry the request id in meta.requestId on every error envelope 2026-10-15 22:05:06 +02:00
Marc-André Bombeck dcb7a0bc09 fix(ai): count the daily AI budget on the account's own day
The token ledger keyed every day on UTC while the daily briefing and the
other per-day generations run on the account's local day, so east of UTC
the first generations after local midnight were refused against the
previous day's spent ceiling. The budget day is now the calendar day in
the account's time zone; each request still resolves it once and books
its reservation and reconcile on the same row. The medication extraction
reads relative phrases against the same local day, the operator's spend
card shows the operator's own day, and the limit copy says the budget
resets at midnight in the person's time zone.
2026-10-11 13:09:12 +02:00
Marc-André Bombeck 12a8ab7c63 feat(documents): open a PDF or image original in a new browser tab
The document sheet gains an Open in new tab action for the types the
browser renders itself (PDF, JPEG, PNG, WebP, GIF), so a long report
gets the browser's own zoom and search instead of the small preview. It
opens the object URL the sheet already fetched through the app transport:
a plain navigation to the serve route carries no record-session
assertion and would be refused inside a shared record. Every other type
keeps the download only, and the serve route's matrix now pins SVG and
HTML to attachment as well.
2026-10-11 12:37:57 +02:00
Marc-André Bombeck 7340cd3ab7 feat(documents): run the vault's months side by side by default
An account that never picked an arrangement now sees the months flowing
across the width. The layout PUT stores only the fields the person chose,
so an untouched field keeps following the default while a saved choice
stays.
2026-10-09 08:50:20 +02:00
Marc-André Bombeck 4273af231e feat(documents): choose how the vault is laid out, and confirm every delete
The wrench beside Upload opens Settings, Appearance, Documents, the way
the illness journal offers its display options: preview cards or a
compact list, and months stacked as blocks or flowing across the full
width with the month name inline above the document that opens it. The
choice is stored per account in `users.documents_layout_json` behind
GET/PUT /api/documents/inbound/layout (preserve-when-absent, module
gated, the read delegable like the rest of the vault), carried in the
account-settings backup and cleared by the data wipe.

Deleting from the vault now always asks first. The bulk bar's Delete and
a focused card's Delete key both go through one confirmation on the page,
and the detail sheet's Delete opens its own; before, all three deleted on
the spot with only an undo toast behind them.

The selection bar is two tight lines: the count with an X at its right
edge, then one action row with Delete alone at the right in the
destructive style. Below 1024 px the verbs are icon-only at the 44 px tap
floor, so the row stays one line at 360 px. The bar carries the
selection-bar slot the Coach launcher already steps aside for.
2026-10-08 16:45:06 +02:00
Marc-André Bombeck 0219a85e92 feat(documents): filter the archive by procedure or surgery
The vault's filter row gains a procedure facet beside the condition
facet. It lists the record's procedures and surgeries that hold at least
one live document, headed the way the visits page heads them (the
reason, else the practice, else the kind) with the body site and date
beneath, and narrows the list through the existing `encounterId` filter.
Same control, placement and clear behaviour as the condition facet; the
choice rides the URL as `?encounter=`, and an upload dropped into the
filtered view is filed against the procedure.

The usage read publishes the choices as `linkedProcedures` (additive),
decrypting only the label columns, and returns none when a sharing
grant does not cover the visits section.

On a phone an active facet now gives up label width before the search
field does, and the search keeps room for its icon and padding.
2026-10-04 21:59:22 +02:00
Marc-André Bombeck bb58b40b2a Run fact extraction in the background, re-checking under a row lock
The extract route accepts `Prefer: respond-async` for all three inputs
(text, stored, vision) and queues a DOCUMENT_EXTRACT run (added to
migration 0370). The run's result is the count of staged facts and the
document status; the facts themselves are read from the document.
Without the header the route answers with the document detail as
before; both paths run one shared body.

A background read can take minutes, and a review the person finishes in
the meantime must not have its approved facts replaced. Staging now
locks the document row and asks again whether it may stage, refusing
with the route's own alreadyConfirmed / alreadyPartlyConfirmed codes.
The synchronous path gets the same re-check.

The vault's stored-text extract uses the background path and shows the
shared waiting line in place of its hint while the read runs.
2026-10-03 21:14:47 +02:00
Marc-André Bombeck eda5776886 Run document summaries and filing suggestions in the background
The summary and suggest routes accept `Prefer: respond-async`: after the
same quick refusals they answer 202 with a run id, and the background
document AI worker runs the read under two new run kinds
(DOCUMENT_SUMMARY, DOCUMENT_SUGGEST, migration 0370). Without the header
both routes answer synchronously as before; both paths run one shared
body, so they cannot drift.

The worker asks the capability for the record and re-checks the wire for
the provider it picks before every read. A summary that is to be stored
marks the document PENDING while queued, READY when it lands, and
UNAVAILABLE when the run fails or the write does not land. A second
request attaches to a live run only when it would produce the same thing.

The vault asks for both in the background and shows the same calm
waiting line the index uses, worded so a transient read never promises
that leaving the page keeps it. The egress guard and the route inventory
now hold every run kind to its dispatcher re-check and its capability.
2026-10-03 20:51:57 +02:00
Marc-André Bombeck 0acf3da575 feat(documents): read documents and lab scans in the background
"Read with AI" and the lab scan used to run their model calls inside the
request. At the default response time a read of three calls outlasted the
100 s a proxy in front of the server commonly allows, and the person saw a
cut connection for a read the server was still doing.

Both now run as a background run (`DocumentAiRun`, migration 0366):

- `POST /api/documents/inbound/{id}/index` with `Prefer: respond-async`
  answers 202 with a run id once every quick refusal has passed. Without
  the header it answers exactly as before, which the iPhone app relies on.
  A second request while a read of the same document is queued or running
  attaches to that run.
- `POST /api/labs/ocr/extract` always answers 202. Only the web client
  calls it.
- `GET /api/ai-runs/{id}` serves the run: the same body the synchronous
  route answers with, or the failure code and status it would have given.
  Scoped to the caller; another account's run is a 404.

The worker (`document-ai-run`, a long queue with a one-hour expiry, one
job per run under `lockedPass`) claims the run, asks the capability again
for the record and re-checks the wire for the provider it picks, sets the
run's deadline from the person's AI response time and refuses a read that
cannot fit the job's expiry. Sync route and worker run one body per kind,
so the two paths cannot drift. Budget is reserved at enqueue and settled
once by whoever ends the run.

Input and result are sealed with labelled AES-256-GCM, registered for key
rotation; the input is dropped when the run ends. A five-minute reaper
fails a run no worker took within fifteen minutes (and hands its
reservation back), fails a run past its deadline, and deletes a finished
run after an hour. Runs are outside backups and inside the data wipe.

The web client polls through the query-key factory and says, calmly, that
the read continues in the background, and when no worker has picked it up.
2026-10-03 18:18:56 +02:00
Marc-André Bombeck 838629276c fix(documents): store stated dates at noon UTC
A report's stated date, a document's filing date and the date a committed
lab reading or condition carries were written as UTC midnight, which is
the previous evening west of UTC: a reading dated the 1st showed on the
31st. Date-only values are now stored at noon UTC through one helper in
the time zone module, the convention the lab scan and the cycle engine
already use. Rows written at midnight still read back to the same date.
A document uploaded without a date files under the uploader's own day
rather than the UTC one.
2026-10-01 09:33:50 +02:00
Marc-André Bombeck 1eac05b454 fix(documents): key imports by source instance, keep a token on its origin, and harden the import path
- Source keys carry the normalised origin of the Paperless-ngx or Papra
  they came from (source_instance on the document, the alias and the
  purge ledger; unique NULLS NOT DISTINCT). A key stored without one
  (v1.39.2 imports) matches any instance of its system, and a lookup
  without one matches any stored instance. The upload route, the lookup
  route and the import script accept or send it; backups carry it.
- Saving a connection reuses the stored token only for the same origin;
  a new origin needs the token again (422 tokenRequired).
- Document ids are checked against what each system issues (Paperless
  digits, Papra [A-Za-z0-9_-], never . or ..) before any URL is built;
  the Papra organization id likewise.
- Paperless API 9 filters dates on created__gte / created__lte, in the
  picker and the script. A search page the local re-check empties reads
  on, up to five source pages. Tags are read past the first 250.
- The import's key pre-check uses the lookup allowance, a failed fetch
  no longer charges the upload allowance, and one import per key runs at
  a time under an advisory lock, so a double click downloads once.
- Reading and deleting one's own connection works with the documents
  module off and the operator list unset.
2026-09-26 14:20:17 +02:00
Marc-André Bombeck 92f2cc729f fix(documents): name the document-AI capability in each route that can queue it, and catalogue the picker's error codes
The ingest path takes the capability from its caller, so the upload and
the import route each read getAiCapability("documentAi") in their own
source, where the AI route inventory classifies them.
2026-09-26 13:18:48 +02:00
Marc-André Bombeck fdc60f69c2 feat(documents): search and import from Paperless-ngx and Papra on request
A person stores one connection per system (base address, Papra
organization id, API token encrypted at rest) and can search it by
document name with tag and date filters, then import picked documents
one per request through the same ingest path as an upload: source-key
answer before any download, content dedup, deleted stays deleted, quota,
provenance, and the person's own AI reading setting. An import can link
the document to the visit, condition or vaccination it was picked for.

The operator lists the instances in DOCUMENT_SOURCE_ORIGINS; unset, the
feature is off. Only listed origins are contacted, through safeFetch with
the origin pinned, redirects refused, timeouts and byte caps. The routes
are cookie-only and owner-only. The token is registered for rotation,
removed by Delete all data, and not carried in backups.
2026-09-26 13:02:36 +02:00
Marc-André Bombeck 89b18e001e refactor(documents): move the upload's store path into one ingest function
The upload route keeps authentication, its rate buckets, multipart parsing
and the response shape; the source-key answer, size and type checks,
pre-link narrowing, content-hash dedup with alias memory, the quota-gated
insert, the audit row and the background jobs move to
src/lib/documents/ingest.ts unchanged, so a second way in can store
through the same rules.
2026-09-26 12:53:32 +02:00
Marc-André Bombeck 499b7dd9a7 fix(documents): keep deferred imports local and meter every re-send
A document uploaded with aiRead=defer is now indexed locally by any
automatic path, because the indexer reads the marker from the row rather
than from the job. Index all for search leaves such documents out and no
longer counts them. Only an explicit Read with AI or Generate summary ends
the hold.

Uploads that send a body keep their slot of the hourly limit, duplicates
included; handing it back let a token send the same bytes under id after
id for free. One document takes at most 20 further source ids, beyond
which the upload answers 409 documents.inbound.sourceAliasLimit. The
query-string key check is metered on the lookup allowance it shares with
GET /api/documents/inbound/source.

Refs #1038
2026-09-26 00:38:26 +02:00
Marc-André Bombeck 10a1c3ca2e fix(documents): keep every import key deleted, answer re-sends cheaply
A keyed upload that matched a stored document by its bytes never recorded
its key, so deleting that document let the next import run store it again.
Such keys are now kept as aliases of the document, the source-key check
reads them, and the purge carries them into the ledger with the row's own.

A source key may ride the query string and is then answered before the
body is read or the hourly allowance charged; an upload answered as a
duplicate hands its slot back. GET /api/documents/inbound/source answers
whether a key is held, under the same documents:write scope, so an
importer can skip a download.

aiRead=defer is stored on the row. The summary catch-up, the summary job
and lab auto-staging skip such documents until an explicit Read with AI
or Generate summary clears the marker; backup and restore carry it.

A documents:write caller no longer sees the quota figures on a full vault.

Refs #1038
2026-09-25 22:35:11 +02:00
Marc-André Bombeck 55a32ff7d1 feat(documents): add a document upload token for imports from other systems
A narrow `documents:write` token, minted cookie-only at
`POST /api/tokens/documents` and accepted by the vault upload route alone.
A scoped caller draws on its own hourly bucket per token
(`DOCUMENT_UPLOAD_LIMIT_PER_HOUR`, default 120) and gets a receipt
`{ id, duplicate }` instead of the stored row.

Uploads take optional `sourceSystem` / `sourceId` fields. The source key is
unique per account across live and deleted documents, and the tombstone
purge carries it to a small ledger, so re-sending a document the owner
deleted answers 200 `deleted: true` and stores nothing. `aiRead=defer`
indexes locally and skips the summary and lab staging.

The detail sheet shows where an imported document came from; backup and
restore carry both fields; Delete All Data clears the ledger.

Refs #1038, #938
2026-09-25 21:48:04 +02:00
Marc-André Bombeck b480ac4543 feat(documents): link a document to visits and vaccinations from its own sheet
A document could show its visits read-only and its vaccinations not at
all, so a childhood record covering a dozen doses had to be attached
from a dozen dose forms. The document's detail sheet now lists the
visits and vaccinations it is filed against and, for someone who may
manage the vault, links and unlinks them in one place, with the records
dated near the document on top. Both ends write the same link tables
through the link service, so the dose's form and the document's sheet
cannot disagree about what is filed where.

The document detail response carries the linked doses, named by
catalogue slug or the person's own wording. It is null rather than
empty for a grant that opens the vault but not the health background,
so a shared page never claims to record no doses. The update route
takes a replace-set of vaccination ids, up to 100 so one record can
cover a whole primary schedule; ownership is re-checked, a foreign id
answers 404 and the change lands in the audit log beside the other link
fields. Linking from the dose suggestion now refreshes the sheet.

Refs #1024, #1005
2026-09-24 09:39:27 +02:00
Marc-André Bombeck bd8ea9d86b feat(ai): gate document, lab and medication extraction on their capabilities
Reading a stored document (suggest, summary, extract, index, chat,
reindex), scanning a lab report and turning medication text into a
schedule now answer under documentAi, labsOcr and medicationExtract. The
operator's master switch and the new reading-documents switch stop the
work and the egress; a module off for the record, a delegate inside
somebody else's record, and a missing receipt refuse with the capability
envelope. Existing no-provider codes keep their status.

The document pick and the labs pick re-check the capability for the
provider they picked, immediately before anything is sent, so a job
enqueued before a switch flipped finds no provider and falls back to the
provider-free text layer. Consent follows the document rule for all
three: any provider that leaves the machine needs an ai_extraction or
ai_full receipt, answered for the provider actually picked.

The fenced Coach chat and attachments answer under coach and documentAi.
Uploads are always stored and indexed; the background summary is only
enqueued while documentAi is open. The capability probes report the
capability beside the provider answer and keep a read offered when only
the receipt is missing. medications/extract keeps its 503 for no provider
and adds ai.provider.none, and its budget refusal gains the
coach.budget.exceeded code.
2026-09-24 08:44:57 +02:00
Marc-André Bombeck 18ed57b867 fix(ai): every surface keeps a ceiling on the day's mixed total
The owner split left an operator-funded chain with no ceiling at all. Its arm
compared operator_tokens, and that counter returns to roughly zero on every
reconcile the user's own plan settled: on the 2026-09-11 evidence the shared
key answered 500 and codex served, so the counter never grew, every job
reservation was admitted forever, and a runaway client loop could write an
unbounded row. A pre-existing concurrency contract caught it from the other
side, two reservations that must not both pass one cap both passing.

reserveBudget now refuses when the owner's counter is over its cap OR the day's
mixed total is over the abuse ceiling, and a background surface gets its share
of that ceiling too, so background spend is bounded on the total and not only
under a counter the operator's key never fills. The surface is a required
argument, so the type checker enumerated every reservation site.

The hop guard learns the same lesson: an operator-funded fallback hop is now
rationed at the calling surface's ceiling rather than the full operator cap. A
job admitted under the job share could previously spend the whole 200 000 once
its chain fell back onto the shared key, which is the original defect reached
by the fallback path.

The job share applies only where the operator pays for the chain. Multiplying
whichever ceiling the chain resolved halved background generation for a
self-hoster on a local model, who costs the operator nothing.

Refusals now name the ceiling they hit and whose it was. A refusal used to emit
the day's mixed total and nothing else, which on an operator refusal is not the
counter that tripped.
2026-09-11 11:11:01 +02:00
Marc-André Bombeck d3196ee69e fix(ai): background generation cannot spend the interactive day
The automatic generators — status and reference cards, the briefing, the
arrival reaction, the nudge, document summaries and the index backfill —
reserved against the same ceiling as the chat. On the operator's account
that meant 150-330 ledger rows a day, 172 metric generations before
06:42Z, and a ceiling already spent when he opened the chat.

Every background surface now reserves through
`resolveDailyCapFor("job", chain)`: half the cost owner's ceiling, all
job surfaces together. The interactive routes keep the whole ceiling, so
a day of background work can no longer take the user's own away.
2026-09-11 10:09:34 +02:00
Marc-André Bombeck badef28e96 feat(ai): the daily ledger records which tokens the operator paid for
`coach_usage` counted one number per (user, day). A turn served by the
user's own ChatGPT plan after the operator's shared key answered 500 was
therefore charged to the operator-cost cap, and the operator's chat was
locked out every morning by background work he pays for himself.

The ledger gains `operator_tokens`: the share of `total_tokens` served by
an operator-funded provider. `reserveBudget` books a reservation to the
owner of the chain's primary and `reconcileSpend` settles it against the
provider that actually served, both inside the same statements as the
total so concurrent requests can never see one counter without the other.
Every call site names its cost owner and its serving hop; the argument is
required, so a new surface cannot inherit the old blind accounting.
2026-09-11 09:42:14 +02:00
Marc-André Bombeck 9ce0bd5027 fix(api): name the refused fields on every shape rejection
Fifty-nine 422s raised off a Zod `safeParse` answered with one opaque
sentence and nothing a client could map onto a field, while the other
166 sent `details.issues` — the multi-issue envelope that exists so a
caller with three bad fields fixes all three in one round-trip. Twelve
of the fifty-nine put a dotted machine token in `error` where the
sentence belongs, so `error` was sometimes text to show a person and
sometimes a token that must not be shown, with nothing in the payload
to tell them apart.

Every one of them now answers through `apiValidationError`, which keeps
the existing `error` string and the existing `meta` byte for byte and
adds the issue list beside them. The dotted tokens keep their place in
`error` for the clients that already match on them and are published in
`meta.errorCode` as well, which is the field a machine code belongs in
and the one a new client should read.

A guard reads the route tree per refusal site rather than per file —
per file is what let this grow, since a route that used the helper once
looked compliant to a `grep -l` sweep — and asserts it still finds the
sites it judges, so a matcher that stops matching fails instead of
reporting a clean zero.
2026-09-10 07:19:55 +02:00
Marc-André Bombeck 08100b6207 fix(api): put Retry-After and the rate-limit triple on every 429
A refused client had nothing to back off against. `rateLimitHeaders()`
emitted `X-RateLimit-Remaining` and an ISO `X-RateLimit-Reset`, no
`Retry-After` and no cap, and only about 70 of the 199 route files that
consult a limiter attached even those two. The rest answered with a bare
prose envelope, including every endpoint an offline replay queue hits:
the four batch ingests, `/api/sync/changes` and `/api/auth/refresh`. A
first-run Apple Health import trips the 60-per-minute batch ceiling
reliably, and the only strategy left to the client was a blind backoff.

Threading headers through 129 handlers is 129 edits the next new route
forgets, so the verdict is carried instead: `checkRateLimit` records it
in a request-scoped cell and `apiHandler` dresses any 429 that leaves
without headers. A route that already built its own is left alone —
`Retry-After` is the presence test, and it comes from the same builder.

`Retry-After` is whole seconds rounding UP (RFC 9110 §10.2.3), floored
at zero. `X-RateLimit-Limit` is new and needs the cap, so the limiter
result carries it. `X-RateLimit-Reset` keeps its ISO form: it is the
wrong convention for the name, but it is on the wire today and changing
it is a decision of its own.

The cell lives in its own leaf module because 177 test files replace
`@/lib/rate-limit` wholesale with a mock factory, and `apiHandler` must
not depend on anything those factories stand in for.
2026-09-10 03:55:46 +02:00
Marc-André Bombeck ac44b35867 Claim reminder slots before egress, classify the bulk race, fold the vault probes
- The measurement-reminder tick claims its slot in the record-event
  ledger BEFORE any provider egress (the documented claim-first pattern
  from reminder-dedup.ts); a crash between dispatch and the nextDueAt
  advance can no longer double-send.
- mood-entries/bulk classifies a lost P2002 race as the duplicate it is
  (with the winning row id) instead of a skip carrying a raw Prisma
  message — the row IS stored, the racing twin wrote it. The workouts
  batch already re-derives per-entry status from RETURNING.
- The vault list's content-index and thumbnail probes ride the same
  Promise.all as the link loads: four independent grouped queries, no
  serialized round-trips on a 4s-polled list.
2026-08-14 06:06:43 +02:00
Marc-André Bombeck 893db887e9 Record why a document's index attempt failed and say so
A content-index attempt that failed (raster failure, a scan with no
text layer, a provider error) recorded its reason only in the server's
observability stream. In the vault, hasContentIndex false looked
identical for a document nobody ever tried and one that failed three
times, and the detail sheet had nothing to say about either.

Apply the summaryState pattern to the index. Two additive columns on
InboundDocument (migration 0333) record when the last attempt finished
and why it produced no index, as a closed reason set: local-empty,
local-unsupported, decrypt-error, raster-failed, provider-error,
pdf-needs-anthropic, empty-transcription. The writer sits at the choke
point the auto job and the corpus backfill share, and in the manual
index route; a success clears the reason. The provider path's specific
failure now survives the local fallback, since "no text" is less
actionable than "the PDF would not render". rasterizePdf names its
failure class so a build that cannot rasterize at all is distinguished
from a PDF that would not render. New guard in both index paths: an
empty provider transcription never becomes a successful empty index.

The detail sheet renders the reason in plain words (muted, six
locales). The DTO and OpenAPI carry the two fields; the disaster
recovery backup and restore carry them verbatim like their summary
siblings.

Refs #776
2026-08-13 12:40:50 +02:00
Marc-André Bombeck 45dc93932a Settle a timed-out assessment refresh instead of calling it failed
The force regenerate aborts its request at 45 s while the server's
inline generation keeps running and usually writes the fresh briefing
moments later. The old handling fired an error toast at the abort and
re-read the old cache once; with a one hour staleTime and focus
refetch off, nothing ever converged, so the page claimed failure while
holding a finished assessment at arm's length.

A timeout now enters a settling state: the hook remembers the cachedAt
baseline it was showing and polls the read path on the existing
bounded cadence (25 s, up to 10 attempts) until it can prove an
outcome. Cache advanced past the baseline: the fresh payload lands and
the success toast fires. A failure marker newer than the baseline, or
the attempt cap: an honest error. While settling, the strip's button
stays busy and says the assessment is still being prepared; no toast
fires until the outcome is proven. Both poll reasons live in the
nextAdvisorPollInterval seam, the verdict in resolveAdvisorSettle, the
toast mapping in regenerateToastKind; all three are pure and pinned by
unit tests. The contract is documented at both ends (hook and generate
route). New copy in all six locales.

Refs #786
2026-08-13 12:24:46 +02:00
Marc-André Bombeck 86a3d74187 Report how many documents the reindex actually queues
The index-all trigger returned Boolean(jobId) while the client typed
the field as a number, so the toast read "Indexing true document(s)".
The route now answers with the count of live documents still lacking a
content index, computed from the same live-scoped counts the usage
gauge reads, and an honest 0 when no job was created. OpenAPI schema
updated to the integer shape; the hook's type carries a contract note
so the wire and the toast cannot drift apart again. Route test pins
the number, a component test pins that the toast renders the count and
never the word "true".

Refs #776
2026-08-13 12:13:55 +02:00
Marc-André Bombeck fd104dd0ac feat(documents): review and extract lab values on the document itself
The server chain existed end to end (extract stages PENDING facts, per-fact
edit clears the low-confidence gate, confirm is the only write into the
structured stores) but the document detail offered no control over any of
it: it counted observation facts and linked to Labs. A user whose automatic
staging run was skipped had to re-upload the same PDF through Labs > Scan a
report.

- The detail sheet gains a staged-facts block with honest states: pending
  facts open an inline review (approve/reject per fact; a low-confidence
  fact is checked and saved first via the existing per-fact edit); a
  lab-filed, already-read document with nothing staged offers "Extract lab
  values" over its stored text; otherwise nothing renders.
- The extract route gains { mode: "stored" }: the same text-structuring
  pass the automatic worker runs, over the document's own content index.
  No re-upload, no second read of the original; full gauntlet including
  the shared document-AI bucket with charge-at-dispatch honesty; 422
  notIndexed when no stored text exists.
- A manual "Read with AI" / "Read again" now continues into the same
  lab auto-staging the background worker performs; the response reports
  labFactsStaged and the toast names the values waiting for review.
- Unchecked values are rejected and the copy says so; an unverified
  low-confidence fact receives no decision and stays pending rather than
  being silently discarded. Confirmation remains the only write into
  Labs, conditions, and medications; approved facts carry the
  committedRecordId link back to the source document (existing model, no
  schema change).
- The "lab values from this document" link now counts APPROVED facts
  only — it claims values that actually exist in Labs.
- Copy in all six locales; OpenAPI regenerated for the new extract mode
  and the index response field.
2026-08-13 00:33:14 +02:00
Marc-André Bombeck 8e16df6894 feat(documents): make the document-AI rate limit an operator setting with honest accounting
The document-AI actions (read/index, extract, suggest, summarise) shared a
hard-coded 6/hour bucket that lied twice: the 429 said "try again in a few
minutes" while the window is an hour, and several failure paths burned a
slot before any provider was dispatched.

- DOCUMENT_AI_LIMIT_PER_HOUR configures the shared per-user ceiling
  (default 6, clamped 1-1000), wired end to end: compose environment
  whitelist, .env.production.example, env manifest, operator docs.
- INSIGHTS_RATE_LIMIT_PER_HOUR joins the compose whitelist and manifest;
  it was documented but never listed, so an operator value could not reach
  the container.
- The 429 mirrors the bucket's reset instant into meta.retryAt (plus the
  existing X-RateLimit-Reset header); the client derives "try again in
  about N minutes" from it in all six locales, with a plural-aware key.
- A slot is consumed only when a request actually reaches the provider (or
  the local index/echo work): malformed bodies, unreadable originals,
  refused consent, and exhausted daily budgets refund the charged slot via
  a window-scoped refundRateLimit primitive. Consistent across all four
  routes sharing the bucket.
2026-08-13 00:33:14 +02:00
Marc-André Bombeck 4d2f706dad feat(doctor-report): a report can mention a visit, and know why it exists
The VISITS leaf ships in the release that made a visit possible, because a
selectable checkbox that produces nothing is worse than no checkbox. Leaf,
collector and PDF renderer land together, and the leaf carries no LEAF_MODULE
entry — like ALLERGIES beside it — because a visit is core and the whole report
is already gated by doctorReport. The absence is commented at the map so a
later reader does not read it as an oversight.

The rows print the date, the practice with its specialty, the kind, the reason,
the outcome and the conditions the visit was filed against. The two encrypted
columns decrypt at render, fail-soft, and never enter the stored selection. A
condition the person has since deleted is not named: the link row survives a
soft delete, and printing a label they withdrew would put it in front of a
doctor.

No FHIR resource type is added. That deferral is already written at the
Encounter model, and the bundle builder now says so at the point a reader would
look for the missing section: FHIR splits the future from the past across
Appointment and Encounter, which is a serialisation question this release does
not answer.

Two properties are free and worth stating: a stored selection never grows
(unknown ids drop) and a clinician share link's sections are frozen write-once,
so no existing report and no already-shared link gains a section.

The panel also offers a booked visit as the report's reason — one query, one
prefilled window end, one dismissible offer that names its own source and never
applies itself. A window control that moves on its own is what makes an export
panel untrustworthy.
2026-08-08 14:38:26 +02:00
Marc-André Bombeck be9c79974f feat(documents): a document can say which visit it belongs to
Every arm here is the same shape as the one that already exists for
conditions, because the vault's filing model is settled and a second shape
would be the thing to explain rather than the thing to use: encounterIds on
upload and on PATCH with replace-set semantics, linkEncounter / unlinkEncounter
beside the episode pair in the bulk action, and an encounterId list filter.

Both directions ride ONE table. The visit's sheet writes encounter_document_links
from its end and the document reads it from the other, declared as a second
direction in the link service rather than a fourth table. A visit's kind
crosses the wire as the enum constant rather than a rendered name: unlike a
condition label, which is the person's own words, a kind is one of eight closed
values whose human name the reader's own bundle owns.

The suggestion is offered where the document is being looked at, resolved by
the shared rule rather than by this surface: one candidate pre-selects with an
undo, two offer a picker with nothing pre-selected, none offers nothing at all.
It resolves with no AI provider configured, and it renders nothing once the
document already names a visit.

The bulk verb has a caller — the vault's selection bar files a whole selection
against a visit, mirroring the condition verb and hiding itself on an account
with no visits. Nothing here is one-ended.

OpenAPI regenerated in the same commit.
2026-08-08 13:41:42 +02:00
Marc-André Bombeck ba21c3a9bc Put both link families behind one service
Three narrow link tables landed with the visit record, which is a step
away from the one-shared-service direction the polymorphic-linking work
argues for. This bounds that step to a single module: every caller now
speaks one vocabulary — a source, a target kind, a list of target ids —
so replacing the storage underneath later is a change in one file rather
than a sweep across every route that files something.

The document-condition links move behind the same signature, so there
are two families and one call surface. Behaviour is unchanged and the
existing vault tests are the proof; where a test double had to follow
the new query shape, it says why in the diff.

The rules that used to live at each call site now live in the module and
are proven against real Postgres rather than a fake that ignores a
where clause: both ends re-read scoped to the owner before a row is
written, a repeated link is a success that changes nothing, a call is
capped at a hundred targets, and a tombstoned record is not linkable on
either end. Unlinking one deliberately still works after a delete, or a
filing nobody can clear would be left behind.

Reads return a resolved label and date rather than a bare id, so a
client rendering a list never re-derives what the server already knows.
The restore is the one documented exemption and writes the tables
directly: it rebuilds an account rather than filing something in one.
2026-08-08 07:26:14 +02:00