Commit Graph
7 Commits
Author SHA1 Message Date
Marc-André Bombeck 85a343a68c feat(documents): assist, summary, content-index endpoints + search union
P2-1-T2/T3/T4/T5/T6.

- assist.ts + [id]/suggest: one provider call proposes a { title, kind,
  documentDate } filing draft over the stored original or browser-OCR text.
  Runs the full extract gauntlet (module → provider → consent → rate-limit →
  budget) but writes nothing — never touches the fact rail or status.
- describe.ts + [id]/summary?mode=summary|text: session-only summary or raw
  transcribed text. Descriptive only, no diagnosis; persists only the budget
  ledger + audit.
- content-index.ts + [id]/index: normalise → tokenise (de-accent, stopword
  drop, whole-word) → HMAC-SHA256 under an HKDF-derived subkey → store the
  token array plus AES-256-GCM ciphertext of the text. Vision path transcribes
  via the provider (consent + budget); text path indexes posted local-OCR text
  (no egress). Gated on the existing AI consent — no new per-user toggle.
- crypto.deriveSubkey: HKDF-SHA256 purpose-bound subkey off the active key.
- list search now unions title/filename ILIKE with a GIN-accelerated
  whole-word content-token match; DTO gains hasContentIndex.
- usage gains assistAvailable + contentIndex { enabled, indexedCount,
  totalCount }; detail gains hasContentIndex.
2026-07-07 18:17:13 +02:00
Marc-André Bombeck 6bab2ce774 fix(documents): serialise the upload quota gate per user
Concurrent uploads all read the same pre-insert usage SUM, so a burst
could overshoot the per-user quota by rate-limit x cap in one window. A
transaction-scoped advisory lock keyed on the user id serialises the
sum-and-insert; uploads by different users never queue on each other.
Integration probes pin both races: four parallel uploads against a
two-file quota land exactly two, and two racing identical uploads
collapse onto one row via the partial unique index.

Also aligns the round-trip integration test with the serve-route CSP
posture: the proxy carve-out owns the response CSP, the route sets none.
2026-07-07 14:00:38 +02:00
Marc-André Bombeck b76873ee1f feat(documents): rebuild upload and list on the vault contract
POST /api/documents/inbound now runs the policy layer end to end: bounded
read aborts at the admin-set cap (413 + meta.reason=fileTooLarge with the
limit), magic-byte classification (415 + unsupportedType), per-user quota
checked inside the insert transaction over every non-purged row —
tombstones keep counting, so deleted bytes are never invisible weight
(413 + quotaExceeded with the figures), sha256 dedupe (a same-bytes
re-upload answers 200 + meta.duplicate with the existing live row; the
partial unique index closes the race), an honoured Idempotency-Key, the
binary2 codec on write, and optional episodeIds[] pre-linking narrowed to
the caller's live episodes.

GET stops fetching the encrypted blob per row (omit: contentEncrypted —
a 50-row page dragged up to 50 × cap ciphertext before), learns kind[] /
episodeId / year facets on top of q/from/to, and returns conditionLinks +
servingClass on every DTO via one grouped link query (no N+1). PATCH gains
replace-set episodeIds with the same 404-shaped refusal for a foreign
episode. The extract path decrypts through the codec dispatcher.

The document-kind labels gain REFERRAL in all six locales.

Tasks: W1-T5, W1-T6, W1-T7.
2026-07-07 11:05:12 +02:00
Marc-André Bombeck eb1779d5c1 fix(documents): close v1.25.1 library QA findings (backend)
Default documentDate at store time. A store-only upload no longer lands
with a NULL filing date: when the client omits one, default it to the
upload day so the column is always set on a stored row. The library list
sorts, groups, and range-filters on the same effective date the UI shows,
so undated uploads no longer mis-sort under nulls-last, break date-group
headers, or fall through the range filter. The date stays user-editable
and may be cleared back to NULL via PATCH.

Block re-extraction once any fact is APPROVED. A partially-confirmed
document stays EXTRACTED, which the CONFIRMED guard does not catch;
re-extracting would delete the approved facts (severing the committed
record's provenance) and re-stage them as PENDING, letting a second
approval duplicate the committed lab / condition / medication. The
extract route now refuses with 409 (documents.inbound.alreadyPartlyConfirmed)
when approved facts exist; the user must finish reviewing or discard first.

RFC 5987-encode the original-download filename. The Content-Disposition
now carries both an ASCII filename= fallback and a filename*=UTF-8'' value
so a non-ASCII name survives intact; the stored set is inline-safe by
construction, so the dead attachment branch is removed.

Add a light per-user rate-limit backstop to the original-read, PATCH, and
DELETE routes (generous; self-DoS only — the routes are owner-scoped).

Lean the list counts: tally facts for the whole page in one grouped count
instead of materialising fact-id rows, and exclude REJECTED facts from
factCount. Extend the documentDate index with deleted_at to cover the
list scan. Document the deliberate plaintext-title PHI tradeoff at the
column and the store site.
2026-06-28 23:48:43 +02:00
Marc-André Bombeck 05465a35a5 feat(documents): store-first library with optional extraction
Invert the inbound-documents surface from an OCR inbox into a document
library: a file can be stored without any AI provider, and extraction
becomes an opt-in action on top.

- Migration 0222: add `title` + `document_date` columns, a
  `(user_id, document_date)` index, the richer kind taxonomy
  (lab/imaging/prescription/insurance/vaccination), and a STORED status
  made the column default. The status enum is recreated so the new value
  can be set as the default within the migration transaction; kind values
  use ADD VALUE. Nullable columns, no backfill.
- POST /api/documents/inbound is now store-only: bounded multipart read,
  magic-byte MIME sniff, AES-256-GCM at rest, persisted as STORED with no
  provider/consent/budget call and no egress.
- POST /api/documents/inbound/[id]/extract carries the relocated
  extraction gauntlet (resolve provider, consent, rate-limit, budget, run,
  reconcile, stage facts) against an already-stored row; absent a provider
  it 422s the enhancement only and leaves the stored document intact.
- PATCH /api/documents/inbound/[id] edits title/kind/documentDate
  field-by-field, owner-scoped by session userId.
- GET /api/documents/inbound gains search, kind filter, documentDate
  range, sort, and keyset pagination via the parameter-bound query builder.
- Extend the validation schemas + DTO, regenerate the OpenAPI contract,
  and add route tests for the store/extract/edit/list paths.
2026-06-28 23:04:19 +02:00
Marc-André Bombeck e3342dc19e fix(documents): encrypt staged extracted-fact PHI at rest
The inbound-document store encrypts the raw upload
(`inbound_documents.content_encrypted`) because it is sensitive clinical
data, but the per-fact staging columns persisted the same class of data in
the clear: `extracted_facts.data_json` held the transcribed clinical values
(diagnosis text, lab values, medication names, stated codes) and
`provenance_json` held the verbatim source span (up to 2000 chars copied
straight out of the report). The at-rest guarantee was defeated for staged
facts, which persist through PENDING and are retained after the decision.

Convert both columns to the AES-256-GCM `*_encrypted` BYTEA convention shared
with the note columns, going through the same Bytes codec. The upload and
fact-edit paths encrypt on write; the confirm path decrypts at commit time to
write into the normal structured stores, so the review-then-confirm flow is
unchanged. Reads are fail-closed (a bad key id throws).

Register both columns in the encrypted-columns registry and the key-rotation
script so a dropped legacy key cannot strand them (the guard test enforces
both). Migration 0220 swaps the plaintext JSONB columns for the encrypted
BYTEA columns; pre-confirm staged rows are ephemeral and cannot be
re-encrypted in SQL, so any left by a pre-release build are dropped rather
than retained as plaintext (their parent document keeps its encrypted
original).
2026-06-28 18:12:51 +02:00
Marc-André Bombeck 98ca0b9d61 feat(documents): extract structured facts from clinical documents
Ingest an uploaded doctor report / discharge letter through the existing
dedicated OCR/vision provider rails (the v1.22 document-scan provider/key,
reused — no new provider) and transcribe structured facts into staging.

The safety line is extract, never interpret: the prompt frames the document
as untrusted data to reproduce verbatim. Diagnoses map to Condition,
measured values to Observation, medications to MedicationStatement — with
STATED status only. A code survives only when the document writes it and its
system matches the resource (SNOMED/ICD-10 for a diagnosis, LOINC for a
value, RxNorm/ATC for a medication); nothing is inferred, no value is
range-flagged, no condition is linked to a medication. Every fact carries
per-field provenance and a confidence score; a fact below the confidence
floor fails closed — it cannot be approved until the user edits it.

Routes (apiHandler + Zod + session-narrowed userId, opt-in module-gated):
- POST/GET /api/documents/inbound — upload+extract (vision or local-OCR
  text), and list. The raw document is stored encrypted at rest.
- GET/DELETE /api/documents/inbound/[id] — review detail; soft-delete.
- PATCH .../facts/[factId] — correct a staged fact before approval.
- POST .../confirm — the only write path: approved facts flow into the
  labs / conditions / medications stores through their normal create paths;
  rejected facts are discarded; low-confidence facts are refused.
2026-06-28 16:57:06 +02:00