A chain whose first provider reasoned without answering and whose second
one answered billed only the answer: result.tokensUsed is the answering
call's alone, and the failed hop's spend, already reported on its
FallbackHop, was dropped by every caller that reconciles the day's ledger.
The raw runners now report tokensSpent on success too, the sum the failed
hops before the answer reported. The briefing, status, Coach (no-tools path
and every tool-loop round) and medication-extract reconciles charge it on
top of the answer. On failure, the Coach turn ledger's close keeps what the
failed chain reported spending instead of refunding everything, and the
medication-extract and document-chat routes charge it too, as the status
and briefing paths already did. Document indexing resolves a single provider
and already charged its failed call.
Refs #1279
The route reserved one read's budget and checked the wire once, and a
20-page report then ran up to 40 calls for many minutes on that single
admission. Every read after the first now reserves its own share of the
day's budget and asks the wire again (aiEgressRefusal) before it is sent;
when either refuses, the scan stops there, keeps the pages already read
and names the rest with stoppedBy budget or unavailable. The run settles
against everything it reserved, with the provider's reported usage net of
its cached share, and a scan that fails after several calls is charged
for them instead of refunded in full.
A rendered PDF page now goes out with its own text layer, read by pdfjs
beside the image, so a digitally produced report's digits, decimal
separators and units come from its characters rather than from a 1600 px
picture. Splitting the PDF into single-page documents would need a PDF
writer the project does not ship; the text layer needs none and helps
every provider. A scanned page has no text layer and reads as before.
Codex's cut reply keeps the text it streamed (on a non-enumerable
property, so no log or event serialises it), and its closed rows are
salvaged like every other provider's. The merge folds a reading only into
one an earlier page listed, never two rows of the same page, and the
printed range is part of the key. OCR'd text the deadline cannot split
small enough for one read is refused up front as labs.ocr.tooLarge.
A scan sent the whole report to one call under an 8000-token ceiling and
validated the reply against a 100-row envelope. A long multi-panel report
lost everything either way: the reply stopped mid-row and failed to parse,
which the person saw as "try a clearer photo", or it held more than 100
rows and the schema refused all of them. One malformed row refused the lot
as well. Reasoning models made it worse, because the call sent no
reasoning setting and OpenAI, Codex and Claude's 5 generation reason by
default against the same output allowance.
A PDF of more than one page is now rendered and read a page per call on
every provider (up to 20 pages), as many reads as the run's deadline
allows, and the pages are merged in order with a reading printed twice
listed once. A photo, a one-page PDF on a provider that reads PDFs, and
OCR'd text up to 4000 characters stay one read; longer text is split at
line boundaries. Each read carries an explicit reasoning off, so a model
that can stop reasoning does and one that cannot gets its minimum budget
on top of the answer's. A reply cut at the output limit is detected from
the finish reason (Codex's incomplete stream included), keeps the rows
that closed before the cut and is named in the result instead of retried;
when no read produced a row the run fails as labs.ocr.outputLimit. A
provider failure after the first page keeps the pages already read.
Rows are validated one by one, the row cap is 400 for the scan and the
commit, and a report past it keeps its first 400 and says how many it
had. The run settles what the provider reported it spent rather than the
fixed reservation, so a page-by-page read still counts against the day's
budget in full.
"Read with AI" and the lab scan used to run their model calls inside the
request. At the default response time a read of three calls outlasted the
100 s a proxy in front of the server commonly allows, and the person saw a
cut connection for a read the server was still doing.
Both now run as a background run (`DocumentAiRun`, migration 0366):
- `POST /api/documents/inbound/{id}/index` with `Prefer: respond-async`
answers 202 with a run id once every quick refusal has passed. Without
the header it answers exactly as before, which the iPhone app relies on.
A second request while a read of the same document is queued or running
attaches to that run.
- `POST /api/labs/ocr/extract` always answers 202. Only the web client
calls it.
- `GET /api/ai-runs/{id}` serves the run: the same body the synchronous
route answers with, or the failure code and status it would have given.
Scoped to the caller; another account's run is a 404.
The worker (`document-ai-run`, a long queue with a one-hour expiry, one
job per run under `lockedPass`) claims the run, asks the capability again
for the record and re-checks the wire for the provider it picks, sets the
run's deadline from the person's AI response time and refuses a read that
cannot fit the job's expiry. Sync route and worker run one body per kind,
so the two paths cannot drift. Budget is reserved at enqueue and settled
once by whoever ends the run.
Input and result are sealed with labelled AES-256-GCM, registered for key
rotation; the input is dropped when the run ends. A five-minute reaper
fails a run no worker took within fifteen minutes (and hands its
reservation back), fails a run past its deadline, and deletes a finished
run after an hour. Runs are outside backups and inside the data wipe.
The web client polls through the query-key factory and says, calmly, that
the read continues in the background, and when no worker has picked it up.
Each refusal is checked twice, at the response and at a spy standing in
for every model call, beside an all-open twin in which the spy is called.
Two background cases run the index and summary job handlers directly and
show that a job enqueued before document reading was turned off finds no
provider when it runs.
Marc-André Bombeck