Commit Graph
31 Commits
Author SHA1 Message Date
Marc-André Bombeck 0219a85e92 feat(documents): filter the archive by procedure or surgery
The vault's filter row gains a procedure facet beside the condition
facet. It lists the record's procedures and surgeries that hold at least
one live document, headed the way the visits page heads them (the
reason, else the practice, else the kind) with the body site and date
beneath, and narrows the list through the existing `encounterId` filter.
Same control, placement and clear behaviour as the condition facet; the
choice rides the URL as `?encounter=`, and an upload dropped into the
filtered view is filed against the procedure.

The usage read publishes the choices as `linkedProcedures` (additive),
decrypting only the label columns, and returns none when a sharing
grant does not cover the visits section.

On a phone an active facet now gives up label width before the search
field does, and the search keeps room for its icon and padding.
2026-10-04 21:59:22 +02:00
Marc-André Bombeck 1564e7c396 Add the sick note document kind
A certificate of incapacity for work (in Germany the
Arbeitsunfähigkeitsbescheinigung) is now its own document kind,
SICK_NOTE, instead of Other. The enum value rides migration 0370.

Uploads, edits, the kind filter, bulk edits, the import script and the
backup format accept it; the vault files it with a thermometer glyph
between prescriptions and referrals; the filing suggestion can propose
it and names the words a sick note carries. The extraction's own kind
field keeps its three values.
2026-10-03 21:40:01 +02:00
Marc-André Bombeck 1eac05b454 fix(documents): key imports by source instance, keep a token on its origin, and harden the import path
- Source keys carry the normalised origin of the Paperless-ngx or Papra
  they came from (source_instance on the document, the alias and the
  purge ledger; unique NULLS NOT DISTINCT). A key stored without one
  (v1.39.2 imports) matches any instance of its system, and a lookup
  without one matches any stored instance. The upload route, the lookup
  route and the import script accept or send it; backups carry it.
- Saving a connection reuses the stored token only for the same origin;
  a new origin needs the token again (422 tokenRequired).
- Document ids are checked against what each system issues (Paperless
  digits, Papra [A-Za-z0-9_-], never . or ..) before any URL is built;
  the Papra organization id likewise.
- Paperless API 9 filters dates on created__gte / created__lte, in the
  picker and the script. A search page the local re-check empties reads
  on, up to five source pages. Tags are read past the first 250.
- The import's key pre-check uses the lookup allowance, a failed fetch
  no longer charges the upload allowance, and one import per key runs at
  a time under an advisory lock, so a double click downloads once.
- Reading and deleting one's own connection works with the documents
  module off and the operator list unset.
2026-09-26 14:20:17 +02:00
Marc-André Bombeck fdc60f69c2 feat(documents): search and import from Paperless-ngx and Papra on request
A person stores one connection per system (base address, Papra
organization id, API token encrypted at rest) and can search it by
document name with tag and date filters, then import picked documents
one per request through the same ingest path as an upload: source-key
answer before any download, content dedup, deleted stays deleted, quota,
provenance, and the person's own AI reading setting. An import can link
the document to the visit, condition or vaccination it was picked for.

The operator lists the instances in DOCUMENT_SOURCE_ORIGINS; unset, the
feature is off. Only listed origins are contacted, through safeFetch with
the origin pinned, redirects refused, timeouts and byte caps. The routes
are cookie-only and owner-only. The token is registered for rotation,
removed by Delete all data, and not carried in backups.
2026-09-26 13:02:36 +02:00
Marc-André Bombeck 10a1c3ca2e fix(documents): keep every import key deleted, answer re-sends cheaply
A keyed upload that matched a stored document by its bytes never recorded
its key, so deleting that document let the next import run store it again.
Such keys are now kept as aliases of the document, the source-key check
reads them, and the purge carries them into the ledger with the row's own.

A source key may ride the query string and is then answered before the
body is read or the hourly allowance charged; an upload answered as a
duplicate hands its slot back. GET /api/documents/inbound/source answers
whether a key is held, under the same documents:write scope, so an
importer can skip a download.

aiRead=defer is stored on the row. The summary catch-up, the summary job
and lab auto-staging skip such documents until an explicit Read with AI
or Generate summary clears the marker; backup and restore carry it.

A documents:write caller no longer sees the quota figures on a full vault.

Refs #1038
2026-09-25 22:35:11 +02:00
Marc-André Bombeck 55a32ff7d1 feat(documents): add a document upload token for imports from other systems
A narrow `documents:write` token, minted cookie-only at
`POST /api/tokens/documents` and accepted by the vault upload route alone.
A scoped caller draws on its own hourly bucket per token
(`DOCUMENT_UPLOAD_LIMIT_PER_HOUR`, default 120) and gets a receipt
`{ id, duplicate }` instead of the stored row.

Uploads take optional `sourceSystem` / `sourceId` fields. The source key is
unique per account across live and deleted documents, and the tombstone
purge carries it to a small ledger, so re-sending a document the owner
deleted answers 200 `deleted: true` and stores nothing. `aiRead=defer`
indexes locally and skips the summary and lab staging.

The detail sheet shows where an imported document came from; backup and
restore carry both fields; Delete All Data clears the ledger.

Refs #1038, #938
2026-09-25 21:48:04 +02:00
Marc-André Bombeck b480ac4543 feat(documents): link a document to visits and vaccinations from its own sheet
A document could show its visits read-only and its vaccinations not at
all, so a childhood record covering a dozen doses had to be attached
from a dozen dose forms. The document's detail sheet now lists the
visits and vaccinations it is filed against and, for someone who may
manage the vault, links and unlinks them in one place, with the records
dated near the document on top. Both ends write the same link tables
through the link service, so the dose's form and the document's sheet
cannot disagree about what is filed where.

The document detail response carries the linked doses, named by
catalogue slug or the person's own wording. It is null rather than
empty for a grant that opens the vault but not the health background,
so a shared page never claims to record no doses. The update route
takes a replace-set of vaccination ids, up to 100 so one record can
cover a whole primary schedule; ownership is re-checked, a foreign id
answers 404 and the change lands in the audit log beside the other link
fields. Linking from the dose suggestion now refreshes the sheet.

Refs #1024, #1005
2026-09-24 09:39:27 +02:00
Marc-André Bombeck bd8ea9d86b feat(ai): gate document, lab and medication extraction on their capabilities
Reading a stored document (suggest, summary, extract, index, chat,
reindex), scanning a lab report and turning medication text into a
schedule now answer under documentAi, labsOcr and medicationExtract. The
operator's master switch and the new reading-documents switch stop the
work and the egress; a module off for the record, a delegate inside
somebody else's record, and a missing receipt refuse with the capability
envelope. Existing no-provider codes keep their status.

The document pick and the labs pick re-check the capability for the
provider they picked, immediately before anything is sent, so a job
enqueued before a switch flipped finds no provider and falls back to the
provider-free text layer. Consent follows the document rule for all
three: any provider that leaves the machine needs an ai_extraction or
ai_full receipt, answered for the provider actually picked.

The fenced Coach chat and attachments answer under coach and documentAi.
Uploads are always stored and indexed; the background summary is only
enqueued while documentAi is open. The capability probes report the
capability beside the provider answer and keep a read offered when only
the receipt is missing. medications/extract keeps its 503 for no provider
and adds ai.provider.none, and its budget refusal gains the
coach.budget.exceeded code.
2026-09-24 08:44:57 +02:00
Marc-André Bombeck 963f61cf0f Drop a reference window whose floor sits above its ceiling
A lab report's reference range reaches the record by four paths. Three of
them validate that the floor does not exceed the ceiling; the document
path did not, and the catalog store writes what it is handed. An
integration run against a real Postgres confirmed the consequence: a
100/30 pair landed on the row and on the freshly minted biomarker, so
every later reading of that analyte inherited a window that cannot be
true and a high value came back reassuring.

Dropping beats swapping. A transposed pair says the transcription is
wrong, not which of the two numbers is, and inventing an order would
state something the report never printed. The verbatim string survives
either way, so nothing the document actually said is lost.

Three points, answering according to what each one is:
- the extraction schema drops the pair, because every field there is
  catch-guarded so one bad value cannot sink a 120-fact document
- the review-screen edit refuses with the wording the lab schemas
  already use, because a person can see both numbers and fix them
- the commit path drops it as the last gate, which is what covers facts
  staged before this rule existed and confirmed afterwards

parseReferenceRange has answered a transposed PRINTED window this way
since it was written, and the commit path already counts a window it
could not read at all. This extends both to the case that was readable
and cannot be true.
2026-08-31 19:38:08 +02:00
Marc-André Bombeck 893db887e9 Record why a document's index attempt failed and say so
A content-index attempt that failed (raster failure, a scan with no
text layer, a provider error) recorded its reason only in the server's
observability stream. In the vault, hasContentIndex false looked
identical for a document nobody ever tried and one that failed three
times, and the detail sheet had nothing to say about either.

Apply the summaryState pattern to the index. Two additive columns on
InboundDocument (migration 0333) record when the last attempt finished
and why it produced no index, as a closed reason set: local-empty,
local-unsupported, decrypt-error, raster-failed, provider-error,
pdf-needs-anthropic, empty-transcription. The writer sits at the choke
point the auto job and the corpus backfill share, and in the manual
index route; a success clears the reason. The provider path's specific
failure now survives the local fallback, since "no text" is less
actionable than "the PDF would not render". rasterizePdf names its
failure class so a build that cannot rasterize at all is distinguished
from a PDF that would not render. New guard in both index paths: an
empty provider transcription never becomes a successful empty index.

The detail sheet renders the reason in plain words (muted, six
locales). The DTO and OpenAPI carry the two fields; the disaster
recovery backup and restore carry them verbatim like their summary
siblings.

Refs #776
2026-08-13 12:40:50 +02:00
Marc-André Bombeck fd104dd0ac feat(documents): review and extract lab values on the document itself
The server chain existed end to end (extract stages PENDING facts, per-fact
edit clears the low-confidence gate, confirm is the only write into the
structured stores) but the document detail offered no control over any of
it: it counted observation facts and linked to Labs. A user whose automatic
staging run was skipped had to re-upload the same PDF through Labs > Scan a
report.

- The detail sheet gains a staged-facts block with honest states: pending
  facts open an inline review (approve/reject per fact; a low-confidence
  fact is checked and saved first via the existing per-fact edit); a
  lab-filed, already-read document with nothing staged offers "Extract lab
  values" over its stored text; otherwise nothing renders.
- The extract route gains { mode: "stored" }: the same text-structuring
  pass the automatic worker runs, over the document's own content index.
  No re-upload, no second read of the original; full gauntlet including
  the shared document-AI bucket with charge-at-dispatch honesty; 422
  notIndexed when no stored text exists.
- A manual "Read with AI" / "Read again" now continues into the same
  lab auto-staging the background worker performs; the response reports
  labFactsStaged and the toast names the values waiting for review.
- Unchecked values are rejected and the copy says so; an unverified
  low-confidence fact receives no decision and stays pending rather than
  being silently discarded. Confirmation remains the only write into
  Labs, conditions, and medications; approved facts carry the
  committedRecordId link back to the source document (existing model, no
  schema change).
- The "lab values from this document" link now counts APPROVED facts
  only — it claims values that actually exist in Labs.
- Copy in all six locales; OpenAPI regenerated for the new extract mode
  and the index response field.
2026-08-13 00:33:14 +02:00
Marc-André Bombeck be9c79974f feat(documents): a document can say which visit it belongs to
Every arm here is the same shape as the one that already exists for
conditions, because the vault's filing model is settled and a second shape
would be the thing to explain rather than the thing to use: encounterIds on
upload and on PATCH with replace-set semantics, linkEncounter / unlinkEncounter
beside the episode pair in the bulk action, and an encounterId list filter.

Both directions ride ONE table. The visit's sheet writes encounter_document_links
from its end and the document reads it from the other, declared as a second
direction in the link service rather than a fourth table. A visit's kind
crosses the wire as the enum constant rather than a rendered name: unlike a
condition label, which is the person's own words, a kind is one of eight closed
values whose human name the reader's own bundle owns.

The suggestion is offered where the document is being looked at, resolved by
the shared rule rather than by this surface: one candidate pre-selects with an
undo, two offer a picker with nothing pre-selected, none offers nothing at all.
It resolves with no AI provider configured, and it renders nothing once the
document already names a visit.

The bulk verb has a caller — the vault's selection bar files a whole selection
against a visit, mirroring the condition verb and hiding itself on an account
with no visits. Nothing here is one-ended.

OpenAPI regenerated in the same commit.
2026-08-08 13:41:42 +02:00
Marc-André Bombeck fd2bd67ff3 feat(labs): keep the reference range a report prints beside a value
A lab runs its own method on its own device, so the window a physician
evaluates a value against is the one printed next to it on that report,
not a catalog band averaged over every lab. Reading a report pulled the
value and dropped that range: the extraction asked for it, the commit
handed it to the biomarker resolver, and the resolver only uses it when
it mints a NEW marker. For every analyte the user already had — the
normal case after a first panel — the printed range was read and then
thrown away.

Add `sourceReferenceLow` / `sourceReferenceHigh` / `sourceReferenceText`
to `LabResult` and carry the range from the report to the reading. The
text is stored whether or not numbers can be read out of it, because
"bis 5,0" and "negativ" state a window too; a range that could not be
parsed is kept verbatim, never replaced with invented bounds.

`resolveEffectiveReferenceRange` is the one place the precedence lives:
the source window wins for that single reading, and the catalog stays
the net for readings without one. A printed string with no derivable
bound does not displace the catalog. Every surface that judges a
reading routes through it — the API verdict, the doctor report, the
insights blocks and per-marker assessment, the coach snapshot, the MCP
read and the FHIR `referenceRange` — so a value cannot read in range on
one surface and out on another.

The API keeps `referenceLow` / `referenceHigh` as the window in force,
so a client that reads only those two renders the right range and the
right verdict with no change of its own; the catalog band, the source
window, its origin and a divergence flag ride alongside for clients
that want to show both.

Read side: the chart draws the source window as a second, visually
distinct band with the reading it came from named beneath it, the
reading rows name the printed range and the catalog band where the two
disagree, and the doctor-report PDF prints the report's own string with
a footnote naming the saved range. Entry side: one optional field on
the lab form, folded away until asked for, and the printed range is
editable on the scan review screen.

The range parser covers the notations German and English reports
actually print — two-sided, one-sided, comma or dot decimals, thousands
separators, trailing units — reads units without ever converting them,
and refuses to guess on anything ambiguous.
2026-08-07 21:40:41 +02:00
Marc-André Bombeck 0627648bea Merge remote-tracking branch 'origin/work/document-and-model-correctness' into release/v1.30.32 2026-07-19 11:52:41 +02:00
Marc-André Bombeck 8e938a4f45 fix(documents): anchor extracted-fact provenance to the real document span
Every staged fact carried a sourceText the model echoed back, stored verbatim
as 'the exact verbatim span of the document the fact came from'. Nothing
checked that the span existed. A paraphrase, a normalised rendering, or an
outright invention was persisted and shown to the reviewer as a transcription
of their own clinical record — an anchor pointing at nothing, or at a span the
document never contained.

Treat the echo as a lookup key instead. anchorSourceText resolves it against
the extracted text (exact, then whitespace-collapsed and case-folded to absorb
OCR reflow) and stores the span the document actually carries, with its offset.
A quote that matches nothing is dropped and the fact is flagged unanchored;
vision mode has no extracted text at all, so its facts are unanchored by
construction. Rows written before this resolve to unanchored on read — their
quote is preserved but never presented as checked.
2026-07-19 11:06:14 +02:00
Marc-André Bombeck d773f460e5 Tell the four document-summary outcomes apart
Opening a document said "the summary is being prepared" and kept saying it.
The view had one fallback for a null summary column and used it for every
outcome: a document nobody ever enqueued, a missing vision provider, a spent
budget, a withdrawn consent, an unreadable file, a provider error, and a
summary the outbound screen withheld all rendered as work in progress.

The common case is not the screen. The background job is enqueued at exactly
one place — a fresh upload with the auto-read opt-in already on — and no
backfill reaches anything else. Every document uploaded before the toggle went
on has a null column no job will ever fill, and the view promised all of them a
summary that was never coming.

Record the outcome as `summary_state` (migration 0254, additive, defaults to
NONE which is truthful for every existing row) and branch on it. Only PENDING,
set when a job genuinely exists, may claim generation is running. NONE,
WITHHELD and UNAVAILABLE each say what happened and offer to generate one.

The withhold keeps its rule and loses its dead end. Blocked prose is still
never shown and never stored — only the fact of the refusal is kept, and it is
no longer terminal, so asking again re-runs the gauntlet. That was the original
intent; leaving the column null defeated it, because null read as "generating"
and the guard meant nothing would ever overwrite it.

The on-demand summary now persists what it produces. It is the only repair path
for the documents the job never ran for, and re-deriving the same paragraph on
every open costs a provider call for nothing. Generation is still only ever a
click — nothing runs because a view rendered. Transcription stays session-only.

Also drop the "read by AI" line from the detail footer.
2026-07-19 10:01:06 +02:00
Marc-André Bombeck 5cff7473c9 feat(documents): persist an auto summary for uploaded documents
When the auto-AI-read opt-in is on, a freshly uploaded document is summarised
once in the background and the encrypted summary is stored and shown on the
document detail view.

- A background step runs after upload alongside the existing content-index and
  thumbnail steps. It generates a short (3-4 sentence) plain-language summary
  only when the user's documentsAutoAiRead opt-in is on (that flag is also the
  standing AI-egress consent), skips a document that already has one, and
  no-ops when no vision provider is configured. It reuses the same provider
  resolution, egress-consent re-assertion, and AI-budget reserve/reconcile
  helpers as the on-demand route; a provider error refunds the reservation and
  never fails the upload.
- The summary is stored encrypted (summaryEncrypted Bytes) with a
  summaryGeneratedAt timestamp on inbound_documents, registered for key
  rotation, and returned decrypted in the document detail response.
- The on-demand transient summary route is unchanged.

Gated by the same consent, budget, and module checks.
2026-07-17 17:46:28 +02:00
Marc-André Bombeck 502783a3ea feat(coach): attach multiple documents to a conversation
Replace the scalar document_id discriminator on a coach conversation with a
CoachConversationDocument join table and a sticky documentScoped flag, so a
conversation can be grounded on several stored documents at once while the
prompt-injection fence stays intact across the data-model change.

The fence is re-established at both enforcement points, fail-closed:

- Client: resolveCoachSendTarget routes any turn on a fenced conversation (or a
  fresh chat with staged attachments) to the new /api/insights/chat/fenced
  endpoint; the fenced body is strict and never carries scope, guided-question,
  or prefill hints.
- Server: the tool route fetches with documentScoped:false AND 404s when the
  fetched row still carries an attachment (flag/join drift), emitting a
  fence_drift audit alarm. Document text now enters an LLM prompt in exactly one
  code path — the fenced pipeline — with no tools and no health snapshot; a
  module-graph test pins that the tool loop is unreachable from it.

The fenced turn fences each document separately with a server-built, scrubbed
header line (title and filename are untrusted text), runs a per-document egress
consent check on the single picked provider, and grounds numerically over the
live attachment union only. Attachments are capped at five documents and 192 KB
of combined text with honest truncation.

Attach and detach ride dedicated endpoints: attaching validates the document is
owned, live, indexed, and within cap, then flips the conversation fenced (the
one legal, privilege-reducing transition, audit-logged); detaching removes the
document but never clears the flag. The document-sheet chat keeps its public
wire contract and shares the same fenced pipeline over the join.

Migration 0245 adds the join table and the flag, backfills every legacy
single-document chat into a one-attachment fenced conversation, and drops the
scalar column in the same transaction — no dual-truth window.
2026-07-16 23:32:28 +02:00
Marc-André Bombeck ee76672798 feat(documents): preview thumbnails in the vault overview
Render a small JPEG preview for each stored document and show it at the
card's leading edge, falling back to the kind icon when none exists.

- Generate previews in the background on upload (new document-thumbnail
  pg-boss queue) plus a boot-time corpus backfill for existing documents;
  upload never blocks on or fails because of it. Images are downscaled via
  @napi-rs/canvas, a PDF's first page is rasterised then downscaled, every
  other type is skipped. Generation never throws.
- Store the preview encrypted at rest in a 1:1 DocumentThumbnail side table
  (migration 0236) so the list/detail queries that omit the original blob
  never drag the preview either. Same AES-256-GCM string codec + fail-closed
  decrypt as the content index; wired into the key-rotation walk.
- Serve it through an owner-scoped GET /api/documents/inbound/[id]/thumbnail
  (auth + module gate + userId-scoped query + fail-closed decrypt), always
  image/jpeg + nosniff + private, no-store; 404 on miss. The card renders an
  authed lazy <img>, onError falls back to the icon.
- Add hasThumbnail to the document DTO so the card only fetches a preview
  that exists.

Also drop the "search looks inside your documents" caption from the overview
(the index-all prompt stays) and the per-card searchable marker; both were
noise. Remove the now-unused i18n keys across all six locales.
2026-07-10 01:15:32 +02:00
Marc-André Bombeck 09bc94a1f0 chore(crypto): register + rotate verbatim index column; format 2026-07-09 03:00:24 +02:00
Marc-André Bombeck 6e84048217 feat(documents): document-chat SSE route + history GET + OpenAPI (no-tools, no-snapshot, fenced, consent/budget/rate) 2026-07-09 02:44:55 +02:00
Marc-André Bombeck 4d6aab2816 Merge branch 'fix/docai-provider-governance' into fix/docai-governance-rebased
# Conflicts:
#	src/components/documents/__tests__/document-ai-section.test.tsx
#	src/components/documents/document-ai-section.tsx
2026-07-09 01:05:35 +02:00
Marc-André BombeckandMarc-André Bombeck c825d4b501 feat(documents): per-egress notice + document-AI capability probe
Add GET /api/documents/inbound/capability (document provider order + vendor-blind
egress class), repoint the vault capability hook to it, and show a per-egress
notice in the detail sheet before a document read leaves the machine. Cover the
document provider order, the egress consent gate, and the notice with tests.
i18n x6 + OpenAPI regenerated.

Co-Authored-By: Marc-André Bombeck
2026-07-08 23:59:56 +02:00
Marc-André Bombeck bb0ebb1dce feat(documents): expose content-index provenance on the document DTO
Surface the content-search index's source (vision vs local extraction) on
the list + detail DTO so the vault UI can tell an AI-read document from a
locally-indexed one. Threads the source through both read paths and the
serialisers, regenerates the OpenAPI contract, and adds a narrowing helper
for the free-String column.
2026-07-08 21:15:52 +02:00
Marc-André Bombeck 85a343a68c feat(documents): assist, summary, content-index endpoints + search union
P2-1-T2/T3/T4/T5/T6.

- assist.ts + [id]/suggest: one provider call proposes a { title, kind,
  documentDate } filing draft over the stored original or browser-OCR text.
  Runs the full extract gauntlet (module → provider → consent → rate-limit →
  budget) but writes nothing — never touches the fact rail or status.
- describe.ts + [id]/summary?mode=summary|text: session-only summary or raw
  transcribed text. Descriptive only, no diagnosis; persists only the budget
  ledger + audit.
- content-index.ts + [id]/index: normalise → tokenise (de-accent, stopword
  drop, whole-word) → HMAC-SHA256 under an HKDF-derived subkey → store the
  token array plus AES-256-GCM ciphertext of the text. Vision path transcribes
  via the provider (consent + budget); text path indexes posted local-OCR text
  (no egress). Gated on the existing AI consent — no new per-user toggle.
- crypto.deriveSubkey: HKDF-SHA256 purpose-bound subkey off the active key.
- list search now unions title/filename ILIKE with a GIN-accelerated
  whole-word content-token match; DTO gains hasContentIndex.
- usage gains assistAvailable + contentIndex { enabled, indexedCount,
  totalCount }; detail gains hasContentIndex.
2026-07-07 18:17:13 +02:00
Marc-André Bombeck d937991af4 fix(documents): source the condition chips from the usage endpoint
The filter bar derived its condition chips from the loaded corpus, so an
episode whose only linked documents sat pages deep in the timeline never
surfaced a chip — the condition route into an older document silently
disappeared at four-digit corpus sizes. The usage endpoint (already read
on page mount) now returns `linkedEpisodes` — every episode carrying at
least one live document link, one indexed grouped query, no blobs — and
the view unions it with the active-filter fallback. Additive DTO change;
OpenAPI regenerated.
2026-07-07 14:15:40 +02:00
Marc-André Bombeck b76873ee1f feat(documents): rebuild upload and list on the vault contract
POST /api/documents/inbound now runs the policy layer end to end: bounded
read aborts at the admin-set cap (413 + meta.reason=fileTooLarge with the
limit), magic-byte classification (415 + unsupportedType), per-user quota
checked inside the insert transaction over every non-purged row —
tombstones keep counting, so deleted bytes are never invisible weight
(413 + quotaExceeded with the figures), sha256 dedupe (a same-bytes
re-upload answers 200 + meta.duplicate with the existing live row; the
partial unique index closes the race), an honoured Idempotency-Key, the
binary2 codec on write, and optional episodeIds[] pre-linking narrowed to
the caller's live episodes.

GET stops fetching the encrypted blob per row (omit: contentEncrypted —
a 50-row page dragged up to 50 × cap ciphertext before), learns kind[] /
episodeId / year facets on top of q/from/to, and returns conditionLinks +
servingClass on every DTO via one grouped link query (no N+1). PATCH gains
replace-set episodeIds with the same 404-shaped refusal for a foreign
episode. The extract path decrypts through the codec dispatcher.

The document-kind labels gain REFERRAL in all six locales.

Tasks: W1-T5, W1-T6, W1-T7.
2026-07-07 11:05:12 +02:00
Marc-André Bombeck eb1779d5c1 fix(documents): close v1.25.1 library QA findings (backend)
Default documentDate at store time. A store-only upload no longer lands
with a NULL filing date: when the client omits one, default it to the
upload day so the column is always set on a stored row. The library list
sorts, groups, and range-filters on the same effective date the UI shows,
so undated uploads no longer mis-sort under nulls-last, break date-group
headers, or fall through the range filter. The date stays user-editable
and may be cleared back to NULL via PATCH.

Block re-extraction once any fact is APPROVED. A partially-confirmed
document stays EXTRACTED, which the CONFIRMED guard does not catch;
re-extracting would delete the approved facts (severing the committed
record's provenance) and re-stage them as PENDING, letting a second
approval duplicate the committed lab / condition / medication. The
extract route now refuses with 409 (documents.inbound.alreadyPartlyConfirmed)
when approved facts exist; the user must finish reviewing or discard first.

RFC 5987-encode the original-download filename. The Content-Disposition
now carries both an ASCII filename= fallback and a filename*=UTF-8'' value
so a non-ASCII name survives intact; the stored set is inline-safe by
construction, so the dead attachment branch is removed.

Add a light per-user rate-limit backstop to the original-read, PATCH, and
DELETE routes (generous; self-DoS only — the routes are owner-scoped).

Lean the list counts: tally facts for the whole page in one grouped count
instead of materialising fact-id rows, and exclude REJECTED facts from
factCount. Extend the documentDate index with deleted_at to cover the
list scan. Document the deliberate plaintext-title PHI tradeoff at the
column and the store site.
2026-06-28 23:48:43 +02:00
Marc-André Bombeck 05465a35a5 feat(documents): store-first library with optional extraction
Invert the inbound-documents surface from an OCR inbox into a document
library: a file can be stored without any AI provider, and extraction
becomes an opt-in action on top.

- Migration 0222: add `title` + `document_date` columns, a
  `(user_id, document_date)` index, the richer kind taxonomy
  (lab/imaging/prescription/insurance/vaccination), and a STORED status
  made the column default. The status enum is recreated so the new value
  can be set as the default within the migration transaction; kind values
  use ADD VALUE. Nullable columns, no backfill.
- POST /api/documents/inbound is now store-only: bounded multipart read,
  magic-byte MIME sniff, AES-256-GCM at rest, persisted as STORED with no
  provider/consent/budget call and no egress.
- POST /api/documents/inbound/[id]/extract carries the relocated
  extraction gauntlet (resolve provider, consent, rate-limit, budget, run,
  reconcile, stage facts) against an already-stored row; absent a provider
  it 422s the enhancement only and leaves the stored document intact.
- PATCH /api/documents/inbound/[id] edits title/kind/documentDate
  field-by-field, owner-scoped by session userId.
- GET /api/documents/inbound gains search, kind filter, documentDate
  range, sort, and keyset pagination via the parameter-bound query builder.
- Extend the validation schemas + DTO, regenerate the OpenAPI contract,
  and add route tests for the store/extract/edit/list paths.
2026-06-28 23:04:19 +02:00
Marc-André Bombeck 0f2b548138 style: apply prettier 3.9.1 formatting
Mechanical reflow from the prettier 3.9.1 bump; no behaviour change.
2026-06-28 19:26:16 +02:00
Marc-André Bombeck 98ca0b9d61 feat(documents): extract structured facts from clinical documents
Ingest an uploaded doctor report / discharge letter through the existing
dedicated OCR/vision provider rails (the v1.22 document-scan provider/key,
reused — no new provider) and transcribe structured facts into staging.

The safety line is extract, never interpret: the prompt frames the document
as untrusted data to reproduce verbatim. Diagnoses map to Condition,
measured values to Observation, medications to MedicationStatement — with
STATED status only. A code survives only when the document writes it and its
system matches the resource (SNOMED/ICD-10 for a diagnosis, LOINC for a
value, RxNorm/ATC for a medication); nothing is inferred, no value is
range-flagged, no condition is linked to a medication. Every fact carries
per-field provenance and a confidence score; a fact below the confidence
floor fails closed — it cannot be approved until the user edits it.

Routes (apiHandler + Zod + session-narrowed userId, opt-in module-gated):
- POST/GET /api/documents/inbound — upload+extract (vision or local-OCR
  text), and list. The raw document is stored encrypted at rest.
- GET/DELETE /api/documents/inbound/[id] — review detail; soft-delete.
- PATCH .../facts/[factId] — correct a staged fact before approval.
- POST .../confirm — the only write path: approved facts flow into the
  labs / conditions / medications stores through their normal create paths;
  rejected facts are discarded; low-confidence facts are refused.
2026-06-28 16:57:06 +02:00