mirror of
https://github.com/MBombeck/HealthLog.git
synced 2026-11-15 06:52:04 +00:00
fix(documents): read a long document through the lab scan's read core
Extracting lab values from a stored document still sent the whole text in one call. A multi-page report produced more JSON than one call's output ceiling, the answer was cut mid-fact, the parse failed twice, and the read came back as 422 "Couldn't read the document. Try a clearer copy." The lab scan had already learned to read the same report a page or a part at a time, but each surface carried its own copy of the read, so the document path never got it. It also ran under the shared 60 s timeout, dropped every fact past 120 (or every fact when one did not match the schema), and charged a flat reservation. The read now lives once, in lib/documents/ocr-read.ts and ocr-plan.ts: the per-read limits, the corrective retry, the salvage of a cut answer's complete rows, the wire re-check and budget reservation before each read, and the planning of a PDF a page at a time and text in parts at line boundaries. The lab scan and the document extraction (the extract route, its background run and the lab auto-staging) both read through it, and the background run plans its reads on the same deadline as a lab scan. The per-read budget moves to a shared metered budget. The document path validates the answer fact by fact, keeps the lab scan's row cap (400), and says why nothing came back: 422 documents.inbound.outputLimit when the reads stopped at the output limit, 429 when the budget refused them, extractFailed only for answers that were not the schema. A run whose reads were cut or not reached reports `incomplete` on its result. A guard holds the class: the output ceiling is set only in the core, both surfaces read through runOcrReads, both runs share the deadline, and a lab read and a document read go out under the same limits. Refs #1257
This commit is contained in:
1 parent
3d47c1e6b4
commit
5a7bd95c7b
31 files changed
+2333
-1034
No files matched your search
+29
-15
@@ -19662,19 +19662,25 @@ paths:
|
||||
capability envelope when it is closed), then rate / budget gated. With no provider configured this returns 422
|
||||
(`documents.inbound.providerUnsupported`, with `meta.capability` and `meta.reason = \"no_provider\"`) — the
|
||||
stored document is untouched, only the enhancement fails. Three modes: VISION (empty body) decrypts and scans
|
||||
the stored original (PDF needs an Anthropic vision provider); TEXT (`application/json`, opt-in local OCR) `{
|
||||
mode: \"text\", text }` structures browser-OCR'd text; STORED (`application/json`) `{ mode: \"stored\" }`
|
||||
structures the document's own stored extracted text (its content index) — the manual recovery for a
|
||||
skipped/failed automatic staging run, 422 `documents.inbound.notIndexed` when no stored text exists. Extraction
|
||||
reproduces what the document states — it never interprets. Nothing reaches the structured stores here; the
|
||||
confirm route is the only write path. With `Prefer: respond-async` (v1.40) the extraction runs in the
|
||||
background: after the same quick refusals the route answers 202 with a run id, a second request while an
|
||||
extraction of this document from the same input is queued or running answers with that run, and `GET
|
||||
/api/ai-runs/{id}` serves `DocumentExtractRunResult` (`{ documentId, factsStaged, status }`, not the document
|
||||
detail) as the run's `result`. Staging re-checks the document under a row lock, so a review finished while the
|
||||
read ran fails the run with `documents.inbound.alreadyPartlyConfirmed` (409) or
|
||||
`documents.inbound.alreadyConfirmed` (422) and leaves the review untouched; the synchronous form answers the
|
||||
same in that race."
|
||||
the stored original (a PDF is rendered and read a page at a time; one that cannot be rendered needs a provider
|
||||
that reads PDFs natively, else 422 `documents.inbound.pdfNeedsAnthropic`); TEXT (`application/json`, opt-in
|
||||
local OCR) `{ mode: \"text\", text }` structures browser-OCR'd text; STORED (`application/json`) `{ mode:
|
||||
\"stored\" }` structures the document's own stored extracted text (its content index) — the manual recovery for
|
||||
a skipped/failed automatic staging run, 422 `documents.inbound.notIndexed` when no stored text exists. A long
|
||||
document is read a page or a part at a time under the same per-read limits as the lab scan (`LABS_OCR_EXTRACT`),
|
||||
each read after the first reserved against the day's budget on its own; an extraction in which no read produced
|
||||
a complete fact fails with 422 `documents.inbound.outputLimit` when the reads stopped at the provider's output
|
||||
limit, 429 `documents.inbound.budgetExceeded` when the budget refused the reads, and 422
|
||||
`documents.inbound.extractFailed` only when the provider's answers were not the schema. Extraction reproduces
|
||||
what the document states — it never interprets. Nothing reaches the structured stores here; the confirm route is
|
||||
the only write path. With `Prefer: respond-async` (v1.40) the extraction runs in the background: after the same
|
||||
quick refusals the route answers 202 with a run id, a second request while an extraction of this document from
|
||||
the same input is queued or running answers with that run, and `GET /api/ai-runs/{id}` serves
|
||||
`DocumentExtractRunResult` (`{ documentId, factsStaged, status }`, plus `incomplete` when part of the document
|
||||
may be missing from the staged facts; not the document detail) as the run's `result`. Staging re-checks the
|
||||
document under a row lock, so a review finished while the read ran fails the run with
|
||||
`documents.inbound.alreadyPartlyConfirmed` (409) or `documents.inbound.alreadyConfirmed` (422) and leaves the
|
||||
review untouched; the synchronous form answers the same in that race."
|
||||
parameters:
|
||||
- name: id
|
||||
in: path
|
||||
@@ -35482,7 +35488,7 @@ components:
|
||||
properties:
|
||||
decisions:
|
||||
minItems: 1
|
||||
maxItems: 120
|
||||
maxItems: 400
|
||||
type: array
|
||||
items:
|
||||
type: object
|
||||
@@ -38381,7 +38387,7 @@ components:
|
||||
documents.inbound.factNotPending, documents.inbound.factTypeMismatch, documents.inbound.fileTooLarge,
|
||||
documents.inbound.fileType, documents.inbound.invalidMetadata, documents.inbound.invalidQuery,
|
||||
documents.inbound.invalidUpdate, documents.inbound.localOcrDisabled, documents.inbound.notFound,
|
||||
documents.inbound.notIndexed, documents.inbound.pdfNeedsAnthropic,
|
||||
documents.inbound.notIndexed, documents.inbound.outputLimit, documents.inbound.pdfNeedsAnthropic,
|
||||
documents.inbound.providerUnsupported, documents.inbound.quotaExceeded, documents.inbound.rateLimited,
|
||||
documents.inbound.restoreDuplicate, documents.inbound.restoreGone, documents.inbound.sourceAliasLimit,
|
||||
documents.inbound.uploadBusy, documents.inbound.uploadTimeout, documents.inbound.vaccinationNotFound,
|
||||
@@ -75913,6 +75919,14 @@ components:
|
||||
- FAILED
|
||||
- CONFIRMED
|
||||
- DISCARDED
|
||||
incomplete:
|
||||
description: "Present when part of the document's facts may be missing from the staged ones: a long document is read a
|
||||
page or a part at a time, and a read stopped at the provider's output limit (its complete facts kept), a
|
||||
part could not be read or was not reached (the day's budget, the run's deadline, AI turned off meanwhile),
|
||||
or the document holds more facts than one review takes. The staged facts are still worth reviewing; the
|
||||
person should check them against the document."
|
||||
type: boolean
|
||||
const: true
|
||||
required:
|
||||
- documentId
|
||||
- factsStaged
|
||||
|
||||
Reference in new issue
Block a user