Papra organization ids are checked against org_ plus 24 characters
before anything is sent. A 403 user.not_in_organization and a 400
server.invalid_request.params read as no such organization instead of a
refused key; the error body is read for its code and never passed on.
The connection test also reads the tags, so a key without tags:read
fails at save (permissionMissing) instead of on the first filter. Name
searches quote each word, so a leading dash is not a negation, and the
date range goes to Papra as date:>= / date:<= with the per-row check
kept as a net.
Keyed uploads and picker imports also check deleted, not yet purged
documents by content and answer deleted, remembering the new key on the
tombstone so the purge carries it into the ledger. A hand upload without
a key still stores a new copy. No content hash of a deleted document is
kept past the purge.
- Source keys carry the normalised origin of the Paperless-ngx or Papra
they came from (source_instance on the document, the alias and the
purge ledger; unique NULLS NOT DISTINCT). A key stored without one
(v1.39.2 imports) matches any instance of its system, and a lookup
without one matches any stored instance. The upload route, the lookup
route and the import script accept or send it; backups carry it.
- Saving a connection reuses the stored token only for the same origin;
a new origin needs the token again (422 tokenRequired).
- Document ids are checked against what each system issues (Paperless
digits, Papra [A-Za-z0-9_-], never . or ..) before any URL is built;
the Papra organization id likewise.
- Paperless API 9 filters dates on created__gte / created__lte, in the
picker and the script. A search page the local re-check empties reads
on, up to five source pages. Tags are read past the first 250.
- The import's key pre-check uses the lookup allowance, a failed fetch
no longer charges the upload allowance, and one import per key runs at
a time under an advisory lock, so a double click downloads once.
- Reading and deleting one's own connection works with the documents
module off and the operator list unset.
The ingest path takes the capability from its caller, so the upload and
the import route each read getAiCapability("documentAi") in their own
source, where the AI route inventory classifies them.
A person stores one connection per system (base address, Papra
organization id, API token encrypted at rest) and can search it by
document name with tag and date filters, then import picked documents
one per request through the same ingest path as an upload: source-key
answer before any download, content dedup, deleted stays deleted, quota,
provenance, and the person's own AI reading setting. An import can link
the document to the visit, condition or vaccination it was picked for.
The operator lists the instances in DOCUMENT_SOURCE_ORIGINS; unset, the
feature is off. Only listed origins are contacted, through safeFetch with
the origin pinned, redirects refused, timeouts and byte caps. The routes
are cookie-only and owner-only. The token is registered for rotation,
removed by Delete all data, and not carried in backups.
The upload route keeps authentication, its rate buckets, multipart parsing
and the response shape; the source-key answer, size and type checks,
pre-link narrowing, content-hash dedup with alias memory, the quota-gated
insert, the audit row and the background jobs move to
src/lib/documents/ingest.ts unchanged, so a second way in can store
through the same rules.
A document uploaded with aiRead=defer is now indexed locally by any
automatic path, because the indexer reads the marker from the row rather
than from the job. Index all for search leaves such documents out and no
longer counts them. Only an explicit Read with AI or Generate summary ends
the hold.
Uploads that send a body keep their slot of the hourly limit, duplicates
included; handing it back let a token send the same bytes under id after
id for free. One document takes at most 20 further source ids, beyond
which the upload answers 409 documents.inbound.sourceAliasLimit. The
query-string key check is metered on the lookup allowance it shares with
GET /api/documents/inbound/source.
Refs #1038
A keyed upload that matched a stored document by its bytes never recorded
its key, so deleting that document let the next import run store it again.
Such keys are now kept as aliases of the document, the source-key check
reads them, and the purge carries them into the ledger with the row's own.
A source key may ride the query string and is then answered before the
body is read or the hourly allowance charged; an upload answered as a
duplicate hands its slot back. GET /api/documents/inbound/source answers
whether a key is held, under the same documents:write scope, so an
importer can skip a download.
aiRead=defer is stored on the row. The summary catch-up, the summary job
and lab auto-staging skip such documents until an explicit Read with AI
or Generate summary clears the marker; backup and restore carry it.
A documents:write caller no longer sees the quota figures on a full vault.
Refs #1038
A narrow `documents:write` token, minted cookie-only at
`POST /api/tokens/documents` and accepted by the vault upload route alone.
A scoped caller draws on its own hourly bucket per token
(`DOCUMENT_UPLOAD_LIMIT_PER_HOUR`, default 120) and gets a receipt
`{ id, duplicate }` instead of the stored row.
Uploads take optional `sourceSystem` / `sourceId` fields. The source key is
unique per account across live and deleted documents, and the tombstone
purge carries it to a small ledger, so re-sending a document the owner
deleted answers 200 `deleted: true` and stores nothing. `aiRead=defer`
indexes locally and skips the summary and lab staging.
The detail sheet shows where an imported document came from; backup and
restore carry both fields; Delete All Data clears the ledger.
Refs #1038, #938
A document could show its visits read-only and its vaccinations not at
all, so a childhood record covering a dozen doses had to be attached
from a dozen dose forms. The document's detail sheet now lists the
visits and vaccinations it is filed against and, for someone who may
manage the vault, links and unlinks them in one place, with the records
dated near the document on top. Both ends write the same link tables
through the link service, so the dose's form and the document's sheet
cannot disagree about what is filed where.
The document detail response carries the linked doses, named by
catalogue slug or the person's own wording. It is null rather than
empty for a grant that opens the vault but not the health background,
so a shared page never claims to record no doses. The update route
takes a replace-set of vaccination ids, up to 100 so one record can
cover a whole primary schedule; ownership is re-checked, a foreign id
answers 404 and the change lands in the audit log beside the other link
fields. Linking from the dose suggestion now refreshes the sheet.
Refs #1024, #1005
The status chokepoint, the comprehensive briefing, reaction lines and
AI-composed nudges now ask `aiEgressRefusal` about exactly the chain they
are about to send to, instead of each pairing the operator-key check
with a surface receipt lookup of its own. The consent kinds come from the
capability table, so the wire and the published capability answer from
the same rule.
The document summary, its catch-up pass and the index backfill stop on a
closed switch or module before anything is picked, and leave "no
provider" and "consent needed" to the document pick, which re-checks the
wire for the provider it chose. A closed switch still only heals PENDING
to NONE; a withheld pick records UNAVAILABLE. The per-pick document
consent helper is removed, since nothing calls it any more.
The auto-read toggle mints the shared `AI_EXTRACTION_CONSENT_KIND`, and
the web grant helper goes back to minting the master grant only.
Refs #975
Reading a stored document (suggest, summary, extract, index, chat,
reindex), scanning a lab report and turning medication text into a
schedule now answer under documentAi, labsOcr and medicationExtract. The
operator's master switch and the new reading-documents switch stop the
work and the egress; a module off for the record, a delegate inside
somebody else's record, and a missing receipt refuse with the capability
envelope. Existing no-provider codes keep their status.
The document pick and the labs pick re-check the capability for the
provider they picked, immediately before anything is sent, so a job
enqueued before a switch flipped finds no provider and falls back to the
provider-free text layer. Consent follows the document rule for all
three: any provider that leaves the machine needs an ai_extraction or
ai_full receipt, answered for the provider actually picked.
The fenced Coach chat and attachments answer under coach and documentAi.
Uploads are always stored and indexed; the background summary is only
enqueued while documentAi is open. The capability probes report the
capability beside the provider answer and keep a read offered when only
the receipt is missing. medications/extract keeps its 503 for no provider
and adds ai.provider.none, and its budget refusal gains the
coach.budget.exceeded code.
A pure resolver turns operator switches, module access, provider-work
authority, provider presence and consent receipts into one state per
capability, reporting the outermost reason. The document provider rank
moves to its own module so the resolver and the document routes read
the same order.
CLAUDE.md forbids planning shorthand in committed artefacts, and this branch
introduced 74 lines of it across source, tests, a schema field and two
migrations. Someone reading the migration SQL in a year has no way to resolve a
wave letter or a round number, and a comment that points at something outside
the repository names no reason at all.
Three passes. A version followed by a parenthetical keeps the version and loses
the parenthetical, because the version says since when something holds. A
sentence that leaned on the vocabulary to carry its meaning is rewritten to
state the reason directly: the arrival reservation now says it was refused over
money the operator never spent, rather than naming the defect a wave existed to
remove. Three test titles carried the marker in the name itself and are renamed
to what they pin.
Comments only. No behaviour, no renames beyond those three titles.
The owner split left an operator-funded chain with no ceiling at all. Its arm
compared operator_tokens, and that counter returns to roughly zero on every
reconcile the user's own plan settled: on the 2026-09-11 evidence the shared
key answered 500 and codex served, so the counter never grew, every job
reservation was admitted forever, and a runaway client loop could write an
unbounded row. A pre-existing concurrency contract caught it from the other
side, two reservations that must not both pass one cap both passing.
reserveBudget now refuses when the owner's counter is over its cap OR the day's
mixed total is over the abuse ceiling, and a background surface gets its share
of that ceiling too, so background spend is bounded on the total and not only
under a counter the operator's key never fills. The surface is a required
argument, so the type checker enumerated every reservation site.
The hop guard learns the same lesson: an operator-funded fallback hop is now
rationed at the calling surface's ceiling rather than the full operator cap. A
job admitted under the job share could previously spend the whole 200 000 once
its chain fell back onto the shared key, which is the original defect reached
by the fallback path.
The job share applies only where the operator pays for the chain. Multiplying
whichever ceiling the chain resolved halved background generation for a
self-hoster on a local model, who costs the operator nothing.
Refusals now name the ceiling they hit and whose it was. A refusal used to emit
the day's mixed total and nothing else, which on an operator refusal is not the
counter that tripped.
The automatic generators — status and reference cards, the briefing, the
arrival reaction, the nudge, document summaries and the index backfill —
reserved against the same ceiling as the chat. On the operator's account
that meant 150-330 ledger rows a day, 172 metric generations before
06:42Z, and a ceiling already spent when he opened the chat.
Every background surface now reserves through
`resolveDailyCapFor("job", chain)`: half the cost owner's ceiling, all
job surfaces together. The interactive routes keep the whole ceiling, so
a day of background work can no longer take the user's own away.
`coach_usage` counted one number per (user, day). A turn served by the
user's own ChatGPT plan after the operator's shared key answered 500 was
therefore charged to the operator-cost cap, and the operator's chat was
locked out every morning by background work he pays for himself.
The ledger gains `operator_tokens`: the share of `total_tokens` served by
an operator-funded provider. `reserveBudget` books a reservation to the
owner of the chain's primary and `reconcileSpend` settles it against the
provider that actually served, both inside the same statements as the
total so concurrent requests can never see one counter without the other.
Every call site names its cost owner and its serving hop; the argument is
required, so a new surface cannot inherit the old blind accounting.
A refused client had nothing to back off against. `rateLimitHeaders()`
emitted `X-RateLimit-Remaining` and an ISO `X-RateLimit-Reset`, no
`Retry-After` and no cap, and only about 70 of the 199 route files that
consult a limiter attached even those two. The rest answered with a bare
prose envelope, including every endpoint an offline replay queue hits:
the four batch ingests, `/api/sync/changes` and `/api/auth/refresh`. A
first-run Apple Health import trips the 60-per-minute batch ceiling
reliably, and the only strategy left to the client was a blind backoff.
Threading headers through 129 handlers is 129 edits the next new route
forgets, so the verdict is carried instead: `checkRateLimit` records it
in a request-scoped cell and `apiHandler` dresses any 429 that leaves
without headers. A route that already built its own is left alone —
`Retry-After` is the presence test, and it comes from the same builder.
`Retry-After` is whole seconds rounding UP (RFC 9110 §10.2.3), floored
at zero. `X-RateLimit-Limit` is new and needs the cap, so the limiter
result carries it. `X-RateLimit-Reset` keeps its ISO form: it is the
wrong convention for the name, but it is on the wire today and changing
it is a decision of its own.
The cell lives in its own leaf module because 177 test files replace
`@/lib/rate-limit` wholesale with a mock factory, and `apiHandler` must
not depend on anything those factories stand in for.
A lab report's reference range reaches the record by four paths. Three of
them validate that the floor does not exceed the ceiling; the document
path did not, and the catalog store writes what it is handed. An
integration run against a real Postgres confirmed the consequence: a
100/30 pair landed on the row and on the freshly minted biomarker, so
every later reading of that analyte inherited a window that cannot be
true and a high value came back reassuring.
Dropping beats swapping. A transposed pair says the transcription is
wrong, not which of the two numbers is, and inventing an order would
state something the report never printed. The verbatim string survives
either way, so nothing the document actually said is lost.
Three points, answering according to what each one is:
- the extraction schema drops the pair, because every field there is
catch-guarded so one bad value cannot sink a 120-fact document
- the review-screen edit refuses with the wording the lab schemas
already use, because a person can see both numbers and fix them
- the commit path drops it as the last gate, which is what covers facts
staged before this rule existed and confirmed afterwards
parseReferenceRange has answered a transposed PRINTED window this way
since it was written, and the commit path already counts a window it
could not read at all. This extends both to the case that was readable
and cannot be true.
Registering `ko` in the `Locale` union left seven locale-keyed tables
without an entry, so the build could not type-check and the safety
screens, the citation-coverage grader and the reference-range parser
read nothing at all for a Korean reader.
Three failures were not missing keys but wrong assumptions:
- The shared fold decomposes with NFD and strips U+0300-U+036F. Latin
accents become plain letters, but Hangul decomposes into jamo, which
are not in that range and survived — so folded Korean text was a jamo
sequence no precomposed table entry could ever match. It recomposes to
NFC now; every Latin, Polish and German fold is byte-identical.
- `normaliseLabKey` kept `[a-z0-9]`, which reduced every Korean analyte
name to the empty string, so the derived index held none of them and a
Korean cholesterol result reached a FHIR export uncoded.
- The reference-range parser had the hyphen and the dashes but not the
tilde a Korean lab actually prints (`3.5~5.0`).
The Korean safety banks carry no `\b` next to a Hangul token — `\b` is
an ASCII `\w` boundary and Hangul is not `\w`, so `/\b증량/` matches
nothing — and the clause order is the mirror of the Latin banks, since
Korean closes with the verb. The dose patterns require a target-marking
particle for the same reason the Latin banks require "to"/"by": without
it the permitted restatement "7.5mg을 복용 중이고, 체중은 내려가고
있어요" tripped the lowering pattern.
The Coach and Insights prompts route `ko` down the reviewed English body
plus a Korean reply directive; there is no `safety-contracts.ko.yaml`, so
the native builder has no Korean ground rules to compose from.
The derived names extended the analyte class by list index. Reordering the
signal list would have re-pointed them at the units or the report header with
nothing to notice, so the class is a named constant and the check compares
against it.
The gate that decides whether a transcribed document is worth an extraction
call scored four classes of evidence and needed two. Three of them — the
reference-range label, the analyte names, the report header — were written in
English and German, so a French, Spanish, Italian or Polish report scored at
most one: the units, which are the same everywhere. Auto-staging never fired
for it. The document was lost from the flow silently, because the manual
extract button stays available and nothing reads as broken.
The text is folded once — lower case, accents dropped — and every pattern is
matched against that, which on its own recovers a good deal: "Cholestérol",
"Créatinine" and "Emoglobina" reach the existing stems as soon as the accents
are out of the way. The reference-range labels and report headers of the four
locales are added by hand, because they are prose a lab printed and the app
owns no string for them.
The analyte names ARE derived, from `labs.catalog.<slug>` across every shipped
bundle: those are names the app itself displays, and a new locale should not
need an edit here to be recognised. Names under four folded characters are
dropped — "ALT" is German for "old", and a three-letter alternative in a gate
this cheap would make every German letter a lab report. The acronyms worth
having are in the hand-written half, where each was considered on its own.
Which way this errs, stated at the code: a false negative loses a whole
document from auto-staging, while a false positive costs one extraction call
and stages facts a person then declines — nothing commits either way. So the
vocabulary is widened, and the two-distinct-class rule holds the other side.
The derived names extend the analyte class rather than forming a fifth one, so
a document naming one analyte in two languages still scores one. Tests pin
each addition on a case that fails the gate without it, and pin the prose that
must keep failing it.
The free-text reference-range parser decided what a printed window meant from
an alternation of German and English words. The app ships six locales, and a
lab prints in the language of the country it stands in, so `jusqu'à 5,0`,
`hasta 5,0`, `fino a 5,0` and `do 5,0` all fell through to `low: null,
high: null`. The string was stored, the reading was then judged against the
biomarker's own band or against nothing at all, and no surface anywhere said
that the lab had printed a window the app could not read.
The vocabulary is TABULATED here, not derived from `messages/<locale>.json`
the way an app-owned word list would be. These words are prose a lab printed
on paper; the app owns no string they could be read out of, and the language
of a report has nothing to do with the language of the UI. So the table is
hand-written, keyed by the shipped `Locale` union so that adding a locale
fails to compile until someone has answered how that language prints a range,
and every locale's words are tried against every input at once. The cost is
written down at the table: a report in a language absent from it stays
text-only.
Three things had to change beside the word lists. Introducing words are now
stripped only on a second pass, because Italian "da", Polish "od" and French
"de" also state a floor on their own and stripping first read "da 3,5" as a
bare number. Input and table are diacritic-folded and their apostrophes
normalised, so a table row written "jusqu'a" meets a report's "jusqu'à" and
the curled apostrophe a word processor emits. And one-letter separators ("a",
"e", "y") are only accepted with whitespace on both sides, so "3.5e5" is not
read as a window. Decimal commas already worked and are pinned by the new
cases.
An unreadable window now has a name. `isUnreadableRange` separates "the report
stated nothing" from "the report stated something nobody could read" — the two
were indistinguishable at every call site, which is what let the French case
pass unnoticed. The manual form already showed the reader that the text was
kept but not read; the two ingest paths now count it into their wide event, so
an unsupported notation surfaces as a number rather than as silence.
The German and English lists are the previous alternation split word for word,
and the existing suite covering them passes unchanged.
Document search does whole-token equality over a blind index, so
'Blutdruck' never found the discharge letter that says 'blood pressure'
and 'hypertension' missed every German Arztbrief. A curated German/English
medical synonym list (vitals, common labs, imaging, specialties, document
words — static, no vendor names) now expands the QUERY tokens before they
are hashed: the stored index, its tokenizer version and the schema are
untouched, the expansion happens ahead of the one-way hash, and recall
simply widens. Typed tokens always survive the expansion cap.
- The provider-chain walker now checks the day's recorded spend against
OPERATOR_COST_CAP before invoking an operator-funded admin-* entry. A
request whose primary provider runs on the user's own key reserves
under the generous user-plan ceiling; a fallback hop (or health-ledger
reorder) onto the operator's key could then spend past the operator's
cap unchecked. An exhausted cap skips the candidate like a failed hop.
- summaryState PENDING gets a giver-upper: the detail serialiser
degrades a PENDING older than an hour to UNAVAILABLE (which offers the
manual generate action), and an hourly reaper persists the same heal
following the orphan-ImportJob reconcile precedent — a job dead past
its retry budget no longer leaves the sheet saying "generating"
forever.
The residual scan from the report is a CCITT Group 4 single-image PDF,
the codec class pdfjs rasterization is known to fail on. Chain checks:
a pdfjs open failure or throw already lands as render-failed with the
error text annotated; the zero-pages shape now annotates too instead
of returning silently. An end-to-end test drives the REAL rasterizer
through prepareVisionInput and pins the rasterFailed mapping; the test
header documents why a tiny in-repo CCITT fixture is deliberately not
checked in (probed against this pdfjs version, it renders the page as
blank instead of throwing, so the fixture cannot deterministically
fail and would be flaky across upgrades). The detail view's
raster-failed line now names scanned fax compression formats as a
known cause, in all six locales.
Refs #776
A content-index attempt that failed (raster failure, a scan with no
text layer, a provider error) recorded its reason only in the server's
observability stream. In the vault, hasContentIndex false looked
identical for a document nobody ever tried and one that failed three
times, and the detail sheet had nothing to say about either.
Apply the summaryState pattern to the index. Two additive columns on
InboundDocument (migration 0333) record when the last attempt finished
and why it produced no index, as a closed reason set: local-empty,
local-unsupported, decrypt-error, raster-failed, provider-error,
pdf-needs-anthropic, empty-transcription. The writer sits at the choke
point the auto job and the corpus backfill share, and in the manual
index route; a success clears the reason. The provider path's specific
failure now survives the local fallback, since "no text" is less
actionable than "the PDF would not render". rasterizePdf names its
failure class so a build that cannot rasterize at all is distinguished
from a PDF that would not render. New guard in both index paths: an
empty provider transcription never becomes a successful empty index.
The detail sheet renders the reason in plain words (muted, six
locales). The DTO and OpenAPI carry the two fields; the disaster
recovery backup and restore carry them verbatim like their summary
siblings.
Refs #776
The document-AI actions (read/index, extract, suggest, summarise) shared a
hard-coded 6/hour bucket that lied twice: the 429 said "try again in a few
minutes" while the window is an hour, and several failure paths burned a
slot before any provider was dispatched.
- DOCUMENT_AI_LIMIT_PER_HOUR configures the shared per-user ceiling
(default 6, clamped 1-1000), wired end to end: compose environment
whitelist, .env.production.example, env manifest, operator docs.
- INSIGHTS_RATE_LIMIT_PER_HOUR joins the compose whitelist and manifest;
it was documented but never listed, so an operator value could not reach
the container.
- The 429 mirrors the bucket's reset instant into meta.retryAt (plus the
existing X-RateLimit-Reset header); the client derives "try again in
about N minutes" from it in all six locales, with a plural-aware key.
- A slot is consumed only when a request actually reaches the provider (or
the local index/echo work): malformed bodies, unreadable originals,
refused consent, and exhausted daily budgets refund the charged slot via
a window-scoped refundRateLimit primitive. Consistent across all four
routes sharing the bucket.
Every arm here is the same shape as the one that already exists for
conditions, because the vault's filing model is settled and a second shape
would be the thing to explain rather than the thing to use: encounterIds on
upload and on PATCH with replace-set semantics, linkEncounter / unlinkEncounter
beside the episode pair in the bulk action, and an encounterId list filter.
Both directions ride ONE table. The visit's sheet writes encounter_document_links
from its end and the document reads it from the other, declared as a second
direction in the link service rather than a fourth table. A visit's kind
crosses the wire as the enum constant rather than a rendered name: unlike a
condition label, which is the person's own words, a kind is one of eight closed
values whose human name the reader's own bundle owns.
The suggestion is offered where the document is being looked at, resolved by
the shared rule rather than by this surface: one candidate pre-selects with an
undo, two offer a picker with nothing pre-selected, none offers nothing at all.
It resolves with no AI provider configured, and it renders nothing once the
document already names a visit.
The bulk verb has a caller — the vault's selection bar files a whole selection
against a visit, mirroring the condition verb and hiding itself on an account
with no visits. Nothing here is one-ended.
OpenAPI regenerated in the same commit.
Three narrow link tables landed with the visit record, which is a step
away from the one-shared-service direction the polymorphic-linking work
argues for. This bounds that step to a single module: every caller now
speaks one vocabulary — a source, a target kind, a list of target ids —
so replacing the storage underneath later is a change in one file rather
than a sweep across every route that files something.
The document-condition links move behind the same signature, so there
are two families and one call surface. Behaviour is unchanged and the
existing vault tests are the proof; where a test double had to follow
the new query shape, it says why in the diff.
The rules that used to live at each call site now live in the module and
are proven against real Postgres rather than a fake that ignores a
where clause: both ends re-read scoped to the owner before a row is
written, a repeated link is a success that changes nothing, a call is
capped at a hundred targets, and a tombstoned record is not linkable on
either end. Unlinking one deliberately still works after a delete, or a
filing nobody can clear would be left behind.
Reads return a resolved label and date rather than a bare id, so a
client rendering a list never re-derives what the server already knows.
The restore is the one documented exemption and writes the tables
directly: it rebuilds an account rather than filing something in one.
A lab runs its own method on its own device, so the window a physician
evaluates a value against is the one printed next to it on that report,
not a catalog band averaged over every lab. Reading a report pulled the
value and dropped that range: the extraction asked for it, the commit
handed it to the biomarker resolver, and the resolver only uses it when
it mints a NEW marker. For every analyte the user already had — the
normal case after a first panel — the printed range was read and then
thrown away.
Add `sourceReferenceLow` / `sourceReferenceHigh` / `sourceReferenceText`
to `LabResult` and carry the range from the report to the reading. The
text is stored whether or not numbers can be read out of it, because
"bis 5,0" and "negativ" state a window too; a range that could not be
parsed is kept verbatim, never replaced with invented bounds.
`resolveEffectiveReferenceRange` is the one place the precedence lives:
the source window wins for that single reading, and the catalog stays
the net for readings without one. A printed string with no derivable
bound does not displace the catalog. Every surface that judges a
reading routes through it — the API verdict, the doctor report, the
insights blocks and per-marker assessment, the coach snapshot, the MCP
read and the FHIR `referenceRange` — so a value cannot read in range on
one surface and out on another.
The API keeps `referenceLow` / `referenceHigh` as the window in force,
so a client that reads only those two renders the right range and the
right verdict with no change of its own; the catalog band, the source
window, its origin and a divergence flag ride alongside for clients
that want to show both.
Read side: the chart draws the source window as a second, visually
distinct band with the reading it came from named beneath it, the
reading rows name the printed range and the catalog band where the two
disagree, and the doctor-report PDF prints the report's own string with
a footnote naming the saved range. Entry side: one optional field on
the lab form, folded away until asked for, and the printed range is
editable on the scan review screen.
The range parser covers the notations German and English reports
actually print — two-sided, one-sided, comma or dot decimals, thousands
separators, trailing units — reads units without ever converting them,
and refuses to guess on anything ambiguous.
The chain gains a seventh entry. It builds the OpenAI client against the
base URL the user saved, with the bearer they saved for it and the model
they named, and it reads none of the OpenAI columns — the pinned
api.openai.com arm stays pinned.
Host policy is the Local provider's, unchanged: a public host always, a
private one only where the operator allowlisted it, which is what a LAN
LiteLLM or vLLM needs. Egress keeps riding safeFetch.
The JSON-mode dialect the Local client learns per endpoint moves into a
shared module and now serves both clients, so a gateway that rejects
response_format is detected once and retried without it.
The entry stays out of the default chain: it can only resolve against a
base URL somebody configured.
Replace the internal "[unverified]" token the grounding guard left in a Coach
reply with the editorial elision mark […], and add a quiet muted caption
under the bubble explaining that some figures could not be checked against the
user's data. The mark holds the slot grammar-safely mid-sentence, before a
surviving unit, and inside a range; the notice carries the meaning. The withheld
count rides the existing provenance envelope (unverifiedFigures) through the
stream, persistence, and reload, and the fenced document-chat path carries a
minimal envelope so its notice survives too. New i18n key across all six locales.
Also fix an adjacent reload gap: provenanceFromJson restored only a subset of the
serialised envelope, so a persisted suggestion card, confirm action card, and the
tool trace were dropped on conversation reload despite the render layer expecting
them to survive. Restore them defensively alongside the new field.
Replace the untyped magnitude bag in the Coach numeric verifier with a typed
tokenizer and a normal-form reconciler, and narrow the outbound risk and dose
banks so correct figures and correct refusals stop being blocked while every
fabrication shape still blocks.
Grounding verifier (coach-prose-grounding.ts):
- Classify each prose number before grading — ISO/written dates, clock times,
bare years, ordinals, list markers, hyphenated time spans, ranges, percents,
thousands-separated integers, unit-suffixed measurements. Benign types are
never graded or stripped; ranges decompose into per-endpoint magnitudes.
- Grade against typed authoritative entries (value + kind + aggregation).
Sign-insensitive, canonical rounding, ratio to percent in percent space, and
minutes to hours widen the match only for an aggregate, kinded entry; kind
scoping stops a weight value grounding a blood-pressure claim.
- Thousands/decimal separators follow the reply language.
- Boundary-safe strip so a flagged token never clips a larger grounded number.
Outbound screen (outbound-screen.ts):
- Risk bank requires an asserted figure or a categorical engine result; a bare
engine or horizon mention (education/refusal) passes. Spelled-out percents
count as figures. No refusal-context exemption. EN + DE.
- Dose bank keeps a maintenance-continuation restatement, voided by any change
stem. Taper suffix fixed.
Route + fenced-chat thread the reply locale into the verifier.
Every ingest path in the product can now emit a typed, salience-classified
event into pg-boss, consumed by one worker that claims a durable per-day
reaction marker and fans out to the reaction surfaces.
The recency classifier runs at the seam, before anything is enqueued, so a
mass backfill — a multi-year Apple Health export, a provider re-sync, a
30-day catch-up — emits nothing at all. Non-salient writes are annotated and
dropped.
The worker makes no provider call by construction: it imports no provider
module, writes only markers, and enqueues nothing that is not budget-gated
downstream. Business refusals return a status rather than throwing, so a
refusal is never retried against a ceiling that will not move until the local
day rolls over.
Seams: sleep (one hook, inherited by all seven transports), workouts (the
batch route, the four provider syncs, the export importer), weight and blood
pressure (both measurement routes), labs (manual, OCR commit, document fact
commit).
The provider workout syncs could not previously tell a created row from an
updated one, so a re-sync of a recent session would have re-fired on every
poll; they now recover the distinction from the row the upsert already
returns.
Reaction lines are AES-256-GCM at rest, following the InsightNarrative
precedent, and the column is registered for key rotation. Markers are pruned
after 14 days by a daily maintenance job.
Every staged fact carried a sourceText the model echoed back, stored verbatim
as 'the exact verbatim span of the document the fact came from'. Nothing
checked that the span existed. A paraphrase, a normalised rendering, or an
outright invention was persisted and shown to the reviewer as a transcription
of their own clinical record — an anchor pointing at nothing, or at a span the
document never contained.
Treat the echo as a lookup key instead. anchorSourceText resolves it against
the extracted text (exact, then whitespace-collapsed and case-folded to absorb
OCR reflow) and stores the span the document actually carries, with its offset.
A quote that matches nothing is dropped and the fact is flagged unanchored;
vision mode has no extracted text at all, so its facts are unanchored by
construction. Rows written before this resolve to unanchored on read — their
quote is preserved but never presented as checked.
A READY row whose ciphertext will not open reports UNAVAILABLE, so the view
does not render a heading over nothing. Nothing asserted that, and removing the
degrade left the suite green.
Opening a document said "the summary is being prepared" and kept saying it.
The view had one fallback for a null summary column and used it for every
outcome: a document nobody ever enqueued, a missing vision provider, a spent
budget, a withdrawn consent, an unreadable file, a provider error, and a
summary the outbound screen withheld all rendered as work in progress.
The common case is not the screen. The background job is enqueued at exactly
one place — a fresh upload with the auto-read opt-in already on — and no
backfill reaches anything else. Every document uploaded before the toggle went
on has a null column no job will ever fill, and the view promised all of them a
summary that was never coming.
Record the outcome as `summary_state` (migration 0254, additive, defaults to
NONE which is truthful for every existing row) and branch on it. Only PENDING,
set when a job genuinely exists, may claim generation is running. NONE,
WITHHELD and UNAVAILABLE each say what happened and offer to generate one.
The withhold keeps its rule and loses its dead end. Blocked prose is still
never shown and never stored — only the fact of the refusal is kept, and it is
no longer terminal, so asking again re-runs the gauntlet. That was the original
intent; leaving the column null defeated it, because null read as "generating"
and the guard meant nothing would ever overwrite it.
The on-demand summary now persists what it produces. It is the only repair path
for the documents the job never ran for, and re-deriving the same paragraph on
every open costs a provider call for nothing. Generation is still only ever a
click — nothing runs because a view rendered. Transcription stays session-only.
Also drop the "read by AI" line from the detail footer.
The Coach SNAPSHOT was spliced into the prompt bare. Its JSON leaves are not
all server-computed: lab analyte, panel and unit names are transcribed by a
model out of an uploaded PDF and stored verbatim as the biomarker name, so a
hostile document chooses those strings. The about-me block has been fenced
since v1.16.1; extend the same pattern rather than adding per-field escapes.
- Move the fence markers and the wrapping primitive into a shared data-fence
module. Content is scrubbed against every known marker, not just its own
pair, so text in one block cannot forge another block's boundary.
- Fence the SNAPSHOT with an explicit data/instruction frame.
- Fence the OCR text handed to the document extractor; the previous
'OCR TEXT:' label was a label, not a boundary.
- Sanitise analyte, panel and unit at the labs source, which feeds both the
snapshot prompt and the labs_read tool result, and the medication label in
the adherence storyline.
- Capture a ---REMEMBER--- sentinel as proposed rather than active. It was
the only model-driven write with no confirm step, and the reminder recall
block fed it back into every later turn. Proposed reminders are already
excluded from that block and already carry a confirm control.
The coverage check hung off generateInsight, whose only caller runWithFallback
has no production caller left -- every live surface routes through
runRawCompletionWithFallback instead. The admin quality dashboard was reading a
metric nothing emitted. Widen the input to the structural shape both payloads
share and call it from the comprehensive post-parse step, so the annotation
reflects real generations.
Writing the locale tests found three gaps the English-only banks hid: Polish
verb suffixes fell outside the ASCII word class, a trailing word boundary after
an accented character never closed, and German subordinate clauses put the verb
last so the natural phrasing of a dose instruction never matched. The payload
collector also skipped legacy bare-string recommendations entirely.
The summary is model prose about the document and is contractually
descriptive-only, but it was returned to the browser verbatim with no guard.
Screen it against the insights contracts and let each caller apply its own
policy: the on-demand route replaces the text with a short honest statement
because a user is waiting on it, while the background job persists nothing --
its updateMany is guarded on a null summary, so a stored refusal would stamp
the document permanently with no path to regenerate.
Transcription stays unscreened by construction. It reproduces the user's own
document verbatim; a discharge letter that says to increase a dose is the
prescriber's instruction, and censoring it would delete the user's own record
from their own view. Locale is required on the summary input type and absent
from the transcription input so a caller cannot silently fall back to the
English banks.
The outbound safety screen ran on the Coach only. A dose-change imperative or
a fabricated clinical risk score written onto a per-metric, biomarker or
derived-score status card persisted as that day's cached assessment, while the
identical sentence in the Coach was caught and replaced -- the only transform
between provider and persistence was whitespace normalisation.
Move the pattern banks into a shared screen that takes the reader's locale.
The old signature took no locale, so the dose-refusal and risk-score contracts
had no enforcement for fr/es/it/pl even though the ground rules ship in all six
contract files. Banks now cover all six locales and each screen runs the
reader's bank plus English, since a provider often answers in English
regardless of the language directive.
Status cards withhold on a trip: serve the deterministic stub and persist no
model text. Nobody waits on a background-generated card, and persisting a
refusal would pin it to the card for the day.
Marc-André Bombeck