Commit Graph
94 Commits
Author SHA1 Message Date
Marc-André Bombeck ada490c7e9 Merge remote-tracking branch 'origin/main' into fix/v1397-tz 2026-10-01 10:59:51 +02:00
Marc-André Bombeck 5f548d331f fix(labs): say when a scanned PDF was read from its first pages only
A PDF longer than the pages a scan renders was read from its first ten
pages, and the cut was recorded only in the wide event. The person checking
the extracted rows had no way to know that values on later pages were never
read.

The rasterizer now returns the PDF's page count, the extract response
carries pageCoverage { read, total } when the cap applied, and the review
screen says how many of the pages were read.
2026-10-01 10:05:35 +02:00
Marc-André Bombeck 7c5ff89def feat(lint): refuse UTC day keys outside the time zone module
`instant.toISOString().slice(0, 10)` and parsing a key back as UTC
midnight are the idioms behind every day-boundary fix this year: the
evening west of UTC and the small hours east of it land on the
neighbouring day. A new rule, healthlog/no-utc-day-key, refuses them in
src/ outside src/lib/tz and tests, and refuses a disable directive for
itself that is not next-line or gives no reason.

The time zone module now names every legitimate form: userDayKey for a
person's day, dateOnlyKey / dateOnlyAtNoonUtc / dayKeyAsUtcMidnight for
date-only values and day labels, daysBetweenDateKeys / weekdayOfDateKey /
isCalendarDateKey / statedDateKey for key arithmetic and parsing.

Of the 133 sites the rule found, 84 are gone. Most moved onto a named
helper with no change in behaviour; these changed behaviour:

- cycle calendar, insights, Coach snapshot and prediction refresh start
  their lookbacks at the first instant of the user's day
- a model-transcribed date like "June 10, 2026" no longer lands a day
  early in a process east of UTC (documents and lab scans)
- a custom report range covers exactly the picked days in the profile
  zone instead of starting the evening before west of UTC
- date-of-birth pickers cap at the user's today, not the UTC one
- an allergy onset date is stored at noon UTC like other date-only values
- the report's cycle summary, the Coach's usual BP range and the intake
  projector's rollup recompute use the user's day
- vaccination dates render as dates instead of the previous evening

49 remain, each with its reason on the line above: 39 UTC by design (15
file name stamps, rollup and object keys, the UTC-day score engines) and
10 marked baseline. A guard test requires a classified reason on every
directive and holds the baseline to a ceiling that only goes down.
2026-10-01 09:59:57 +02:00
Marc-André Bombeck 838629276c fix(documents): store stated dates at noon UTC
A report's stated date, a document's filing date and the date a committed
lab reading or condition carries were written as UTC midnight, which is
the previous evening west of UTC: a reading dated the 1st showed on the
31st. Date-only values are now stored at noon UTC through one helper in
the time zone module, the convention the lab scan and the cycle engine
already use. Rows written at midnight still read back to the same date.
A document uploaded without a date files under the uploader's own day
rather than the UTC one.
2026-10-01 09:33:50 +02:00
Marc-André Bombeck fe53804c4c refactor(coach): split the chat turn into a pipeline under lib/ai/coach/turn
The route keeps the front door: auth, the coach capability, the body
read and schema, the rate limit, the locale and the inbound refusal
screen. Everything after it moves into one module per step:
conversation, context, chain, budget, model, reply-guards, cards,
provenance, persist, sse, errors and pipeline. The capability re-check
at the egress site stays owned by the route and is handed to the
pipeline, so the capability inventory still reads both calls there.

The stream producer now writes through a TurnEmitter created inside
createSseStream and passed to the model step, which is where live
progress frames will come from. Nothing writes to it early yet.

No behaviour change: the golden transcripts pass byte-identical, the
existing route tests pass unmodified, and every annotate, audit and
error-code name appears as often as before.

Guards follow the code. The fence call-site check reads turn/context.ts,
the German-default locale ban covers every file in turn/, and the
fenced module-graph test now asserts that turn/sse.ts and turn/errors.ts
reach neither the tool registry nor the snapshot builder. That test
named lib/ai/coach/tools.ts, which stopped existing when the registry
became a directory, so its negative check could not fail; it points at
tools/index.ts now, and a new sanity check fails if the tool route can
no longer reach the path it names.
2026-09-27 10:14:07 +02:00
Marc-André Bombeck 656542b040 fix(documents): date Papra search rows by their document date only
Papra's date: filter matches the document date, so a row now shows that
date or none, and an undated document falls out of a date range in the
list as it does on the server. Import still files an undated document
under the day it was added.
2026-09-26 15:15:32 +02:00
Marc-André Bombeck 057281998f fix(documents): read Papra as it answers
Papra organization ids are checked against org_ plus 24 characters
before anything is sent. A 403 user.not_in_organization and a 400
server.invalid_request.params read as no such organization instead of a
refused key; the error body is read for its code and never passed on.
The connection test also reads the tags, so a key without tags:read
fails at save (permissionMissing) instead of on the first filter. Name
searches quote each word, so a leading dash is not a negation, and the
date range goes to Papra as date:>= / date:<= with the per-row check
kept as a net.
2026-09-26 14:46:01 +02:00
Marc-André Bombeck c5a63934bd fix(documents): an import of bytes deleted in HealthLog stays deleted under any source key
Keyed uploads and picker imports also check deleted, not yet purged
documents by content and answer deleted, remembering the new key on the
tombstone so the purge carries it into the ledger. A hand upload without
a key still stores a new copy. No content hash of a deleted document is
kept past the purge.
2026-09-26 14:42:13 +02:00
Marc-André Bombeck 1eac05b454 fix(documents): key imports by source instance, keep a token on its origin, and harden the import path
- Source keys carry the normalised origin of the Paperless-ngx or Papra
  they came from (source_instance on the document, the alias and the
  purge ledger; unique NULLS NOT DISTINCT). A key stored without one
  (v1.39.2 imports) matches any instance of its system, and a lookup
  without one matches any stored instance. The upload route, the lookup
  route and the import script accept or send it; backups carry it.
- Saving a connection reuses the stored token only for the same origin;
  a new origin needs the token again (422 tokenRequired).
- Document ids are checked against what each system issues (Paperless
  digits, Papra [A-Za-z0-9_-], never . or ..) before any URL is built;
  the Papra organization id likewise.
- Paperless API 9 filters dates on created__gte / created__lte, in the
  picker and the script. A search page the local re-check empties reads
  on, up to five source pages. Tags are read past the first 250.
- The import's key pre-check uses the lookup allowance, a failed fetch
  no longer charges the upload allowance, and one import per key runs at
  a time under an advisory lock, so a double click downloads once.
- Reading and deleting one's own connection works with the documents
  module off and the operator list unset.
2026-09-26 14:20:17 +02:00
Marc-André Bombeck 92f2cc729f fix(documents): name the document-AI capability in each route that can queue it, and catalogue the picker's error codes
The ingest path takes the capability from its caller, so the upload and
the import route each read getAiCapability("documentAi") in their own
source, where the AI route inventory classifies them.
2026-09-26 13:18:48 +02:00
Marc-André Bombeck cab67b4c06 test(documents): pin the picker's origin allowlist and both source adapters 2026-09-26 13:03:45 +02:00
Marc-André Bombeck fdc60f69c2 feat(documents): search and import from Paperless-ngx and Papra on request
A person stores one connection per system (base address, Papra
organization id, API token encrypted at rest) and can search it by
document name with tag and date filters, then import picked documents
one per request through the same ingest path as an upload: source-key
answer before any download, content dedup, deleted stays deleted, quota,
provenance, and the person's own AI reading setting. An import can link
the document to the visit, condition or vaccination it was picked for.

The operator lists the instances in DOCUMENT_SOURCE_ORIGINS; unset, the
feature is off. Only listed origins are contacted, through safeFetch with
the origin pinned, redirects refused, timeouts and byte caps. The routes
are cookie-only and owner-only. The token is registered for rotation,
removed by Delete all data, and not carried in backups.
2026-09-26 13:02:36 +02:00
Marc-André Bombeck 89b18e001e refactor(documents): move the upload's store path into one ingest function
The upload route keeps authentication, its rate buckets, multipart parsing
and the response shape; the source-key answer, size and type checks,
pre-link narrowing, content-hash dedup with alias memory, the quota-gated
insert, the audit row and the background jobs move to
src/lib/documents/ingest.ts unchanged, so a second way in can store
through the same rules.
2026-09-26 12:53:32 +02:00
Marc-André Bombeck 499b7dd9a7 fix(documents): keep deferred imports local and meter every re-send
A document uploaded with aiRead=defer is now indexed locally by any
automatic path, because the indexer reads the marker from the row rather
than from the job. Index all for search leaves such documents out and no
longer counts them. Only an explicit Read with AI or Generate summary ends
the hold.

Uploads that send a body keep their slot of the hourly limit, duplicates
included; handing it back let a token send the same bytes under id after
id for free. One document takes at most 20 further source ids, beyond
which the upload answers 409 documents.inbound.sourceAliasLimit. The
query-string key check is metered on the lookup allowance it shares with
GET /api/documents/inbound/source.

Refs #1038
2026-09-26 00:38:26 +02:00
Marc-André Bombeck 10a1c3ca2e fix(documents): keep every import key deleted, answer re-sends cheaply
A keyed upload that matched a stored document by its bytes never recorded
its key, so deleting that document let the next import run store it again.
Such keys are now kept as aliases of the document, the source-key check
reads them, and the purge carries them into the ledger with the row's own.

A source key may ride the query string and is then answered before the
body is read or the hourly allowance charged; an upload answered as a
duplicate hands its slot back. GET /api/documents/inbound/source answers
whether a key is held, under the same documents:write scope, so an
importer can skip a download.

aiRead=defer is stored on the row. The summary catch-up, the summary job
and lab auto-staging skip such documents until an explicit Read with AI
or Generate summary clears the marker; backup and restore carry it.

A documents:write caller no longer sees the quota figures on a full vault.

Refs #1038
2026-09-25 22:35:11 +02:00
Marc-André Bombeck 55a32ff7d1 feat(documents): add a document upload token for imports from other systems
A narrow `documents:write` token, minted cookie-only at
`POST /api/tokens/documents` and accepted by the vault upload route alone.
A scoped caller draws on its own hourly bucket per token
(`DOCUMENT_UPLOAD_LIMIT_PER_HOUR`, default 120) and gets a receipt
`{ id, duplicate }` instead of the stored row.

Uploads take optional `sourceSystem` / `sourceId` fields. The source key is
unique per account across live and deleted documents, and the tombstone
purge carries it to a small ledger, so re-sending a document the owner
deleted answers 200 `deleted: true` and stores nothing. `aiRead=defer`
indexes locally and skips the summary and lab staging.

The detail sheet shows where an imported document came from; backup and
restore carry both fields; Delete All Data clears the ledger.

Refs #1038, #938
2026-09-25 21:48:04 +02:00
Marc-André Bombeck e89f41710a Merge branch 'feat/v139-targets-and-links' into release/v1.39.0 2026-09-24 09:53:41 +02:00
Marc-André Bombeck b480ac4543 feat(documents): link a document to visits and vaccinations from its own sheet
A document could show its visits read-only and its vaccinations not at
all, so a childhood record covering a dozen doses had to be attached
from a dozen dose forms. The document's detail sheet now lists the
visits and vaccinations it is filed against and, for someone who may
manage the vault, links and unlinks them in one place, with the records
dated near the document on top. Both ends write the same link tables
through the link service, so the dose's form and the document's sheet
cannot disagree about what is filed where.

The document detail response carries the linked doses, named by
catalogue slug or the person's own wording. It is null rather than
empty for a grant that opens the vault but not the health background,
so a shared page never claims to record no doses. The update route
takes a replace-set of vaccination ids, up to 100 so one record can
cover a whole primary schedule; ownership is re-checked, a foreign id
answers 404 and the change lands in the audit log beside the other link
fields. Linking from the dose suggestion now refreshes the sheet.

Refs #1024, #1005
2026-09-24 09:39:27 +02:00
Marc-André Bombeck ce72fe226e refactor(ai): re-check every chokepoint through the one egress helper
The status chokepoint, the comprehensive briefing, reaction lines and
AI-composed nudges now ask `aiEgressRefusal` about exactly the chain they
are about to send to, instead of each pairing the operator-key check
with a surface receipt lookup of its own. The consent kinds come from the
capability table, so the wire and the published capability answer from
the same rule.

The document summary, its catch-up pass and the index backfill stop on a
closed switch or module before anything is picked, and leave "no
provider" and "consent needed" to the document pick, which re-checks the
wire for the provider it chose. A closed switch still only heals PENDING
to NONE; a withheld pick records UNAVAILABLE. The per-pick document
consent helper is removed, since nothing calls it any more.

The auto-read toggle mints the shared `AI_EXTRACTION_CONSENT_KIND`, and
the web grant helper goes back to minting the master grant only.

Refs #975
2026-09-24 09:29:00 +02:00
Marc-André Bombeck bd8ea9d86b feat(ai): gate document, lab and medication extraction on their capabilities
Reading a stored document (suggest, summary, extract, index, chat,
reindex), scanning a lab report and turning medication text into a
schedule now answer under documentAi, labsOcr and medicationExtract. The
operator's master switch and the new reading-documents switch stop the
work and the egress; a module off for the record, a delegate inside
somebody else's record, and a missing receipt refuse with the capability
envelope. Existing no-provider codes keep their status.

The document pick and the labs pick re-check the capability for the
provider they picked, immediately before anything is sent, so a job
enqueued before a switch flipped finds no provider and falls back to the
provider-free text layer. Consent follows the document rule for all
three: any provider that leaves the machine needs an ai_extraction or
ai_full receipt, answered for the provider actually picked.

The fenced Coach chat and attachments answer under coach and documentAi.
Uploads are always stored and indexed; the background summary is only
enqueued while documentAi is open. The capability probes report the
capability beside the provider answer and keep a read offered when only
the receipt is missing. medications/extract keeps its 503 for no provider
and adds ai.provider.none, and its budget refusal gains the
coach.budget.exceeded code.
2026-09-24 08:44:57 +02:00
Marc-André Bombeck c8a01ac3ed feat(ai): resolve AI capabilities from their layers
A pure resolver turns operator switches, module access, provider-work
authority, provider presence and consent receipts into one state per
capability, reporting the outermost reason. The document provider rank
moves to its own module so the resolver and the document routes read
the same order.
2026-09-24 07:26:33 +02:00
Marc-André Bombeck b565edc48d docs(ai): drop planning vocabulary from the ledger comments
CLAUDE.md forbids planning shorthand in committed artefacts, and this branch
introduced 74 lines of it across source, tests, a schema field and two
migrations. Someone reading the migration SQL in a year has no way to resolve a
wave letter or a round number, and a comment that points at something outside
the repository names no reason at all.

Three passes. A version followed by a parenthetical keeps the version and loses
the parenthetical, because the version says since when something holds. A
sentence that leaned on the vocabulary to carry its meaning is rewritten to
state the reason directly: the arrival reservation now says it was refused over
money the operator never spent, rather than naming the defect a wave existed to
remove. Three test titles carried the marker in the name itself and are renamed
to what they pin.

Comments only. No behaviour, no renames beyond those three titles.
2026-09-11 12:37:54 +02:00
Marc-André Bombeck 18ed57b867 fix(ai): every surface keeps a ceiling on the day's mixed total
The owner split left an operator-funded chain with no ceiling at all. Its arm
compared operator_tokens, and that counter returns to roughly zero on every
reconcile the user's own plan settled: on the 2026-09-11 evidence the shared
key answered 500 and codex served, so the counter never grew, every job
reservation was admitted forever, and a runaway client loop could write an
unbounded row. A pre-existing concurrency contract caught it from the other
side, two reservations that must not both pass one cap both passing.

reserveBudget now refuses when the owner's counter is over its cap OR the day's
mixed total is over the abuse ceiling, and a background surface gets its share
of that ceiling too, so background spend is bounded on the total and not only
under a counter the operator's key never fills. The surface is a required
argument, so the type checker enumerated every reservation site.

The hop guard learns the same lesson: an operator-funded fallback hop is now
rationed at the calling surface's ceiling rather than the full operator cap. A
job admitted under the job share could previously spend the whole 200 000 once
its chain fell back onto the shared key, which is the original defect reached
by the fallback path.

The job share applies only where the operator pays for the chain. Multiplying
whichever ceiling the chain resolved halved background generation for a
self-hoster on a local model, who costs the operator nothing.

Refusals now name the ceiling they hit and whose it was. A refusal used to emit
the day's mixed total and nothing else, which on an operator refusal is not the
counter that tripped.
2026-09-11 11:11:01 +02:00
Marc-André Bombeck d3196ee69e fix(ai): background generation cannot spend the interactive day
The automatic generators — status and reference cards, the briefing, the
arrival reaction, the nudge, document summaries and the index backfill —
reserved against the same ceiling as the chat. On the operator's account
that meant 150-330 ledger rows a day, 172 metric generations before
06:42Z, and a ceiling already spent when he opened the chat.

Every background surface now reserves through
`resolveDailyCapFor("job", chain)`: half the cost owner's ceiling, all
job surfaces together. The interactive routes keep the whole ceiling, so
a day of background work can no longer take the user's own away.
2026-09-11 10:09:34 +02:00
Marc-André Bombeck badef28e96 feat(ai): the daily ledger records which tokens the operator paid for
`coach_usage` counted one number per (user, day). A turn served by the
user's own ChatGPT plan after the operator's shared key answered 500 was
therefore charged to the operator-cost cap, and the operator's chat was
locked out every morning by background work he pays for himself.

The ledger gains `operator_tokens`: the share of `total_tokens` served by
an operator-funded provider. `reserveBudget` books a reservation to the
owner of the chain's primary and `reconcileSpend` settles it against the
provider that actually served, both inside the same statements as the
total so concurrent requests can never see one counter without the other.
Every call site names its cost owner and its serving hop; the argument is
required, so a new surface cannot inherit the old blind accounting.
2026-09-11 09:42:14 +02:00
Marc-André Bombeck 08100b6207 fix(api): put Retry-After and the rate-limit triple on every 429
A refused client had nothing to back off against. `rateLimitHeaders()`
emitted `X-RateLimit-Remaining` and an ISO `X-RateLimit-Reset`, no
`Retry-After` and no cap, and only about 70 of the 199 route files that
consult a limiter attached even those two. The rest answered with a bare
prose envelope, including every endpoint an offline replay queue hits:
the four batch ingests, `/api/sync/changes` and `/api/auth/refresh`. A
first-run Apple Health import trips the 60-per-minute batch ceiling
reliably, and the only strategy left to the client was a blind backoff.

Threading headers through 129 handlers is 129 edits the next new route
forgets, so the verdict is carried instead: `checkRateLimit` records it
in a request-scoped cell and `apiHandler` dresses any 429 that leaves
without headers. A route that already built its own is left alone —
`Retry-After` is the presence test, and it comes from the same builder.

`Retry-After` is whole seconds rounding UP (RFC 9110 §10.2.3), floored
at zero. `X-RateLimit-Limit` is new and needs the cap, so the limiter
result carries it. `X-RateLimit-Reset` keeps its ISO form: it is the
wrong convention for the name, but it is on the wire today and changing
it is a decision of its own.

The cell lives in its own leaf module because 177 test files replace
`@/lib/rate-limit` wholesale with a mock factory, and `apiHandler` must
not depend on anything those factories stand in for.
2026-09-10 03:55:46 +02:00
Marc-André Bombeck 963f61cf0f Drop a reference window whose floor sits above its ceiling
A lab report's reference range reaches the record by four paths. Three of
them validate that the floor does not exceed the ceiling; the document
path did not, and the catalog store writes what it is handed. An
integration run against a real Postgres confirmed the consequence: a
100/30 pair landed on the row and on the freshly minted biomarker, so
every later reading of that analyte inherited a window that cannot be
true and a high value came back reassuring.

Dropping beats swapping. A transposed pair says the transcription is
wrong, not which of the two numbers is, and inventing an order would
state something the report never printed. The verbatim string survives
either way, so nothing the document actually said is lost.

Three points, answering according to what each one is:
- the extraction schema drops the pair, because every field there is
  catch-guarded so one bad value cannot sink a 120-fact document
- the review-screen edit refuses with the wording the lab schemas
  already use, because a person can see both numbers and fix them
- the commit path drops it as the last gate, which is what covers facts
  staged before this rule existed and confirmed afterwards

parseReferenceRange has answered a transposed PRINTED window this way
since it was written, and the commit path already counts a window it
could not read at all. This extends both to the case that was readable
and cannot be true.
2026-08-31 19:38:08 +02:00
aucun6352 f7cc47850a fix(i18n): make the Korean locale actually work end to end
Registering `ko` in the `Locale` union left seven locale-keyed tables
without an entry, so the build could not type-check and the safety
screens, the citation-coverage grader and the reference-range parser
read nothing at all for a Korean reader.

Three failures were not missing keys but wrong assumptions:

- The shared fold decomposes with NFD and strips U+0300-U+036F. Latin
  accents become plain letters, but Hangul decomposes into jamo, which
  are not in that range and survived — so folded Korean text was a jamo
  sequence no precomposed table entry could ever match. It recomposes to
  NFC now; every Latin, Polish and German fold is byte-identical.
- `normaliseLabKey` kept `[a-z0-9]`, which reduced every Korean analyte
  name to the empty string, so the derived index held none of them and a
  Korean cholesterol result reached a FHIR export uncoded.
- The reference-range parser had the hyphen and the dashes but not the
  tilde a Korean lab actually prints (`3.5~5.0`).

The Korean safety banks carry no `\b` next to a Hangul token — `\b` is
an ASCII `\w` boundary and Hangul is not `\w`, so `/\b증량/` matches
nothing — and the clause order is the mirror of the Latin banks, since
Korean closes with the verb. The dose patterns require a target-marking
particle for the same reason the Latin banks require "to"/"by": without
it the permitted restatement "7.5mg을 복용 중이고, 체중은 내려가고
있어요" tripped the lowering pattern.

The Coach and Insights prompts route `ko` down the reviewed English body
plus a Korean reply directive; there is no `safety-contracts.ko.yaml`, so
the native builder has no Korean ground rules to compose from.
2026-08-26 20:00:44 +09:00
Marc-André Bombeck 154b609768 refactor(documents): attach the derived analyte names to their signal class by identity
The derived names extended the analyte class by list index. Reordering the
signal list would have re-pointed them at the units or the report header with
nothing to notice, so the class is a named constant and the check compares
against it.
2026-08-23 14:44:23 +02:00
Marc-André Bombeck b06d7dbd92 fix(documents): recognise a lab report printed outside English and German
The gate that decides whether a transcribed document is worth an extraction
call scored four classes of evidence and needed two. Three of them — the
reference-range label, the analyte names, the report header — were written in
English and German, so a French, Spanish, Italian or Polish report scored at
most one: the units, which are the same everywhere. Auto-staging never fired
for it. The document was lost from the flow silently, because the manual
extract button stays available and nothing reads as broken.

The text is folded once — lower case, accents dropped — and every pattern is
matched against that, which on its own recovers a good deal: "Cholestérol",
"Créatinine" and "Emoglobina" reach the existing stems as soon as the accents
are out of the way. The reference-range labels and report headers of the four
locales are added by hand, because they are prose a lab printed and the app
owns no string for them.

The analyte names ARE derived, from `labs.catalog.<slug>` across every shipped
bundle: those are names the app itself displays, and a new locale should not
need an edit here to be recognised. Names under four folded characters are
dropped — "ALT" is German for "old", and a three-letter alternative in a gate
this cheap would make every German letter a lab report. The acronyms worth
having are in the hand-written half, where each was considered on its own.

Which way this errs, stated at the code: a false negative loses a whole
document from auto-staging, while a false positive costs one extraction call
and stages facts a person then declines — nothing commits either way. So the
vocabulary is widened, and the two-distinct-class rule holds the other side.
The derived names extend the analyte class rather than forming a fifth one, so
a document naming one analyte in two languages still scores one. Tests pin
each addition on a case that fails the gate without it, and pin the prose that
must keep failing it.
2026-08-23 14:28:53 +02:00
Marc-André Bombeck 2dc553057d fix(labs): read a reference range printed in French, Spanish, Italian or Polish
The free-text reference-range parser decided what a printed window meant from
an alternation of German and English words. The app ships six locales, and a
lab prints in the language of the country it stands in, so `jusqu'à 5,0`,
`hasta 5,0`, `fino a 5,0` and `do 5,0` all fell through to `low: null,
high: null`. The string was stored, the reading was then judged against the
biomarker's own band or against nothing at all, and no surface anywhere said
that the lab had printed a window the app could not read.

The vocabulary is TABULATED here, not derived from `messages/<locale>.json`
the way an app-owned word list would be. These words are prose a lab printed
on paper; the app owns no string they could be read out of, and the language
of a report has nothing to do with the language of the UI. So the table is
hand-written, keyed by the shipped `Locale` union so that adding a locale
fails to compile until someone has answered how that language prints a range,
and every locale's words are tried against every input at once. The cost is
written down at the table: a report in a language absent from it stays
text-only.

Three things had to change beside the word lists. Introducing words are now
stripped only on a second pass, because Italian "da", Polish "od" and French
"de" also state a floor on their own and stripping first read "da 3,5" as a
bare number. Input and table are diacritic-folded and their apostrophes
normalised, so a table row written "jusqu'a" meets a report's "jusqu'à" and
the curled apostrophe a word processor emits. And one-letter separators ("a",
"e", "y") are only accepted with whitespace on both sides, so "3.5e5" is not
read as a window. Decimal commas already worked and are pinned by the new
cases.

An unreadable window now has a name. `isUnreadableRange` separates "the report
stated nothing" from "the report stated something nobody could read" — the two
were indistinguishable at every call site, which is what let the French case
pass unnoticed. The manual form already showed the reader that the text was
kept but not read; the two ingest paths now count it into their wide event, so
an unsupported notation surfaces as a number rather than as silence.

The German and English lists are the previous alternation split word for word,
and the existing suite covering them passes unchanged.
2026-08-23 14:16:15 +02:00
Marc-André Bombeck a1ca0dc911 Let a German search find an English letter, and the other way round
Document search does whole-token equality over a blind index, so
'Blutdruck' never found the discharge letter that says 'blood pressure'
and 'hypertension' missed every German Arztbrief. A curated German/English
medical synonym list (vitals, common labs, imaging, specialties, document
words — static, no vendor names) now expands the QUERY tokens before they
are hashed: the stored index, its tokenizer version and the schema are
untouched, the expansion happens ahead of the one-way hash, and recall
simply widens. Typed tokens always survive the expansion cap.
2026-08-14 09:34:54 +02:00
Marc-André Bombeck 6ebd5e8711 Guard the operator's cap on fallback hops and reap dead summary promises
- The provider-chain walker now checks the day's recorded spend against
  OPERATOR_COST_CAP before invoking an operator-funded admin-* entry. A
  request whose primary provider runs on the user's own key reserves
  under the generous user-plan ceiling; a fallback hop (or health-ledger
  reorder) onto the operator's key could then spend past the operator's
  cap unchecked. An exhausted cap skips the candidate like a failed hop.
- summaryState PENDING gets a giver-upper: the detail serialiser
  degrades a PENDING older than an hour to UNAVAILABLE (which offers the
  manual generate action), and an hourly reaper persists the same heal
  following the orphan-ImportJob reconcile precedent — a job dead past
  its retry budget no longer leaves the sheet saying "generating"
  forever.
2026-08-14 05:53:17 +02:00
Marc-André Bombeck 589b35589b Prove the raster-failed chain and name the fax-codec cause
The residual scan from the report is a CCITT Group 4 single-image PDF,
the codec class pdfjs rasterization is known to fail on. Chain checks:
a pdfjs open failure or throw already lands as render-failed with the
error text annotated; the zero-pages shape now annotates too instead
of returning silently. An end-to-end test drives the REAL rasterizer
through prepareVisionInput and pins the rasterFailed mapping; the test
header documents why a tiny in-repo CCITT fixture is deliberately not
checked in (probed against this pdfjs version, it renders the page as
blank instead of throwing, so the fixture cannot deterministically
fail and would be flaky across upgrades). The detail view's
raster-failed line now names scanned fax compression formats as a
known cause, in all six locales.

Refs #776
2026-08-13 13:04:41 +02:00
Marc-André Bombeck 893db887e9 Record why a document's index attempt failed and say so
A content-index attempt that failed (raster failure, a scan with no
text layer, a provider error) recorded its reason only in the server's
observability stream. In the vault, hasContentIndex false looked
identical for a document nobody ever tried and one that failed three
times, and the detail sheet had nothing to say about either.

Apply the summaryState pattern to the index. Two additive columns on
InboundDocument (migration 0333) record when the last attempt finished
and why it produced no index, as a closed reason set: local-empty,
local-unsupported, decrypt-error, raster-failed, provider-error,
pdf-needs-anthropic, empty-transcription. The writer sits at the choke
point the auto job and the corpus backfill share, and in the manual
index route; a success clears the reason. The provider path's specific
failure now survives the local fallback, since "no text" is less
actionable than "the PDF would not render". rasterizePdf names its
failure class so a build that cannot rasterize at all is distinguished
from a PDF that would not render. New guard in both index paths: an
empty provider transcription never becomes a successful empty index.

The detail sheet renders the reason in plain words (muted, six
locales). The DTO and OpenAPI carry the two fields; the disaster
recovery backup and restore carry them verbatim like their summary
siblings.

Refs #776
2026-08-13 12:40:50 +02:00
Marc-André Bombeck 8e16df6894 feat(documents): make the document-AI rate limit an operator setting with honest accounting
The document-AI actions (read/index, extract, suggest, summarise) shared a
hard-coded 6/hour bucket that lied twice: the 429 said "try again in a few
minutes" while the window is an hour, and several failure paths burned a
slot before any provider was dispatched.

- DOCUMENT_AI_LIMIT_PER_HOUR configures the shared per-user ceiling
  (default 6, clamped 1-1000), wired end to end: compose environment
  whitelist, .env.production.example, env manifest, operator docs.
- INSIGHTS_RATE_LIMIT_PER_HOUR joins the compose whitelist and manifest;
  it was documented but never listed, so an operator value could not reach
  the container.
- The 429 mirrors the bucket's reset instant into meta.retryAt (plus the
  existing X-RateLimit-Reset header); the client derives "try again in
  about N minutes" from it in all six locales, with a plural-aware key.
- A slot is consumed only when a request actually reaches the provider (or
  the local index/echo work): malformed bodies, unreadable originals,
  refused consent, and exhausted daily budgets refund the charged slot via
  a window-scoped refundRateLimit primitive. Consistent across all four
  routes sharing the bucket.
2026-08-13 00:33:14 +02:00
Marc-André Bombeck be9c79974f feat(documents): a document can say which visit it belongs to
Every arm here is the same shape as the one that already exists for
conditions, because the vault's filing model is settled and a second shape
would be the thing to explain rather than the thing to use: encounterIds on
upload and on PATCH with replace-set semantics, linkEncounter / unlinkEncounter
beside the episode pair in the bulk action, and an encounterId list filter.

Both directions ride ONE table. The visit's sheet writes encounter_document_links
from its end and the document reads it from the other, declared as a second
direction in the link service rather than a fourth table. A visit's kind
crosses the wire as the enum constant rather than a rendered name: unlike a
condition label, which is the person's own words, a kind is one of eight closed
values whose human name the reader's own bundle owns.

The suggestion is offered where the document is being looked at, resolved by
the shared rule rather than by this surface: one candidate pre-selects with an
undo, two offer a picker with nothing pre-selected, none offers nothing at all.
It resolves with no AI provider configured, and it renders nothing once the
document already names a visit.

The bulk verb has a caller — the vault's selection bar files a whole selection
against a visit, mirroring the condition verb and hiding itself on an account
with no visits. Nothing here is one-ended.

OpenAPI regenerated in the same commit.
2026-08-08 13:41:42 +02:00
Marc-André Bombeck ba21c3a9bc Put both link families behind one service
Three narrow link tables landed with the visit record, which is a step
away from the one-shared-service direction the polymorphic-linking work
argues for. This bounds that step to a single module: every caller now
speaks one vocabulary — a source, a target kind, a list of target ids —
so replacing the storage underneath later is a change in one file rather
than a sweep across every route that files something.

The document-condition links move behind the same signature, so there
are two families and one call surface. Behaviour is unchanged and the
existing vault tests are the proof; where a test double had to follow
the new query shape, it says why in the diff.

The rules that used to live at each call site now live in the module and
are proven against real Postgres rather than a fake that ignores a
where clause: both ends re-read scoped to the owner before a row is
written, a repeated link is a success that changes nothing, a call is
capped at a hundred targets, and a tombstoned record is not linkable on
either end. Unlinking one deliberately still works after a delete, or a
filing nobody can clear would be left behind.

Reads return a resolved label and date rather than a bare id, so a
client rendering a list never re-derives what the server already knows.
The restore is the one documented exemption and writes the tables
directly: it rebuilds an account rather than filing something in one.
2026-08-08 07:26:14 +02:00
Marc-André Bombeck fd2bd67ff3 feat(labs): keep the reference range a report prints beside a value
A lab runs its own method on its own device, so the window a physician
evaluates a value against is the one printed next to it on that report,
not a catalog band averaged over every lab. Reading a report pulled the
value and dropped that range: the extraction asked for it, the commit
handed it to the biomarker resolver, and the resolver only uses it when
it mints a NEW marker. For every analyte the user already had — the
normal case after a first panel — the printed range was read and then
thrown away.

Add `sourceReferenceLow` / `sourceReferenceHigh` / `sourceReferenceText`
to `LabResult` and carry the range from the report to the reading. The
text is stored whether or not numbers can be read out of it, because
"bis 5,0" and "negativ" state a window too; a range that could not be
parsed is kept verbatim, never replaced with invented bounds.

`resolveEffectiveReferenceRange` is the one place the precedence lives:
the source window wins for that single reading, and the catalog stays
the net for readings without one. A printed string with no derivable
bound does not displace the catalog. Every surface that judges a
reading routes through it — the API verdict, the doctor report, the
insights blocks and per-marker assessment, the coach snapshot, the MCP
read and the FHIR `referenceRange` — so a value cannot read in range on
one surface and out on another.

The API keeps `referenceLow` / `referenceHigh` as the window in force,
so a client that reads only those two renders the right range and the
right verdict with no change of its own; the catalog band, the source
window, its origin and a divergence flag ride alongside for clients
that want to show both.

Read side: the chart draws the source window as a second, visually
distinct band with the reading it came from named beneath it, the
reading rows name the printed range and the catalog band where the two
disagree, and the doctor-report PDF prints the report's own string with
a footnote naming the saved range. Entry side: one optional field on
the lab form, folded away until asked for, and the printed range is
editable on the scan review screen.

The range parser covers the notations German and English reports
actually print — two-sided, one-sided, comma or dot decimals, thousands
separators, trailing units — reads units without ever converting them,
and refuses to guess on anything ambiguous.
2026-08-07 21:40:41 +02:00
Marc-André Bombeck c91558f18f feat(analytics): replace health score with reference composite 2026-07-28 21:10:15 +02:00
Marc-André Bombeck 0c7059804b feat(ai): resolve an OpenAI-compatible gateway as its own provider
The chain gains a seventh entry. It builds the OpenAI client against the
base URL the user saved, with the bearer they saved for it and the model
they named, and it reads none of the OpenAI columns — the pinned
api.openai.com arm stays pinned.

Host policy is the Local provider's, unchanged: a public host always, a
private one only where the operator allowlisted it, which is what a LAN
LiteLLM or vLLM needs. Egress keeps riding safeFetch.

The JSON-mode dialect the Local client learns per endpoint moves into a
shared module and now serves both clients, so a gateway that rejects
response_format is detected once and retried without it.

The entry stays out of the default chain: it can only resolve against a
base URL somebody configured.
2026-07-26 22:27:48 +02:00
Marc-André Bombeck 5ad66af565 fix(coach): mark withheld figures with an elision mark plus a per-message notice
Replace the internal "[unverified]" token the grounding guard left in a Coach
reply with the editorial elision mark […], and add a quiet muted caption
under the bubble explaining that some figures could not be checked against the
user's data. The mark holds the slot grammar-safely mid-sentence, before a
surviving unit, and inside a range; the notice carries the meaning. The withheld
count rides the existing provenance envelope (unverifiedFigures) through the
stream, persistence, and reload, and the fenced document-chat path carries a
minimal envelope so its notice survives too. New i18n key across all six locales.

Also fix an adjacent reload gap: provenanceFromJson restored only a subset of the
serialised envelope, so a persisted suggestion card, confirm action card, and the
tool trace were dropped on conversation reload despite the render layer expecting
them to survive. Restore them defensively alongside the new field.
2026-07-24 09:29:20 +02:00
Marc-André Bombeck a69d0493f8 feat(coach): typed grounding tokenizer + normal-form reconciler + narrowed risk/dose banks
Replace the untyped magnitude bag in the Coach numeric verifier with a typed
tokenizer and a normal-form reconciler, and narrow the outbound risk and dose
banks so correct figures and correct refusals stop being blocked while every
fabrication shape still blocks.

Grounding verifier (coach-prose-grounding.ts):
- Classify each prose number before grading — ISO/written dates, clock times,
  bare years, ordinals, list markers, hyphenated time spans, ranges, percents,
  thousands-separated integers, unit-suffixed measurements. Benign types are
  never graded or stripped; ranges decompose into per-endpoint magnitudes.
- Grade against typed authoritative entries (value + kind + aggregation).
  Sign-insensitive, canonical rounding, ratio to percent in percent space, and
  minutes to hours widen the match only for an aggregate, kinded entry; kind
  scoping stops a weight value grounding a blood-pressure claim.
- Thousands/decimal separators follow the reply language.
- Boundary-safe strip so a flagged token never clips a larger grounded number.

Outbound screen (outbound-screen.ts):
- Risk bank requires an asserted figure or a categorical engine result; a bare
  engine or horizon mention (education/refusal) passes. Spelled-out percents
  count as figures. No refusal-context exemption. EN + DE.
- Dose bank keeps a maintenance-continuation restatement, voided by any change
  stem. Taper suffix fixed.

Route + fenced-chat thread the reply locale into the verifier.
2026-07-23 20:07:03 +02:00
Marc-André Bombeck 62eb431d66 release: v1.32.0 — hardening and scalability (#580)
* release: prepare v1.32.0

* test: stabilize water capture re-entry
2026-07-22 04:14:15 +02:00
Marc-André Bombeck 3aa8ccd541 fix: localize document summaries and refine mobile coach 2026-07-20 22:34:56 +02:00
Marc-André Bombeck 3c5afea912 test(documents): format PDF cleanup coverage 2026-07-20 14:30:44 +02:00
Marc-André Bombeck 74a386fcfd chore(deps): update July dependency bundle 2026-07-20 14:18:48 +02:00
Marc-André Bombeck 5d62236412 feat(arrivals): add the data-arrival event spine
Every ingest path in the product can now emit a typed, salience-classified
event into pg-boss, consumed by one worker that claims a durable per-day
reaction marker and fans out to the reaction surfaces.

The recency classifier runs at the seam, before anything is enqueued, so a
mass backfill — a multi-year Apple Health export, a provider re-sync, a
30-day catch-up — emits nothing at all. Non-salient writes are annotated and
dropped.

The worker makes no provider call by construction: it imports no provider
module, writes only markers, and enqueues nothing that is not budget-gated
downstream. Business refusals return a status rather than throwing, so a
refusal is never retried against a ceiling that will not move until the local
day rolls over.

Seams: sleep (one hook, inherited by all seven transports), workouts (the
batch route, the four provider syncs, the export importer), weight and blood
pressure (both measurement routes), labs (manual, OCR commit, document fact
commit).

The provider workout syncs could not previously tell a created row from an
updated one, so a re-sync of a recent session would have re-fired on every
poll; they now recover the distinction from the row the upsert already
returns.

Reaction lines are AES-256-GCM at rest, following the InsightNarrative
precedent, and the column is registered for key rotation. Markers are pruned
after 14 days by a daily maintenance job.
2026-07-19 13:17:34 +02:00
Marc-André Bombeck 0627648bea Merge remote-tracking branch 'origin/work/document-and-model-correctness' into release/v1.30.32 2026-07-19 11:52:41 +02:00
Marc-André Bombeck 8e938a4f45 fix(documents): anchor extracted-fact provenance to the real document span
Every staged fact carried a sourceText the model echoed back, stored verbatim
as 'the exact verbatim span of the document the fact came from'. Nothing
checked that the span existed. A paraphrase, a normalised rendering, or an
outright invention was persisted and shown to the reviewer as a transcription
of their own clinical record — an anchor pointing at nothing, or at a span the
document never contained.

Treat the echo as a lookup key instead. anchorSourceText resolves it against
the extracted text (exact, then whitespace-collapsed and case-folded to absorb
OCR reflow) and stores the span the document actually carries, with its offset.
A quote that matches nothing is dropped and the fact is flagged unanchored;
vision mode has no extracted text at all, so its facts are unanchored by
construction. Rows written before this resolve to unanchored on read — their
quote is preserved but never presented as checked.
2026-07-19 11:06:14 +02:00