feat(labs): add local-OCR text mode for image-blind AI providers

Add an opt-in in-browser OCR path so a user whose provider cannot read
images (ChatGPT-OAuth/Codex, a text-only model) can still scan a lab
report. The capability probe gains a `mode` field: `vision` when the
provider reads the image directly, `text` when local OCR is enabled and
any provider is configured, or null. The extract route grows a text
branch that accepts the in-browser-OCR'd text as JSON and structures it
through the existing schema + review/commit flow; the raw image never
reaches the server in text mode. New `labsLocalOcrEnabled` user column
(migration 0184, default off) gates it, with a dedicated PATCH endpoint.
This commit is contained in:
Marc-André Bombeck committed 2026-06-20 16:50:06 +02:00
1 parent fa37941add
commit 3d039d5220
13 files changed
+634 -77

No files matched your search

+26 -2
View File
@@ -197,8 +197,32 @@ export interface OcrCommitResponseDto {
/** The capability-probe response (drives whether the UI shows the scan entry). */
export interface OcrCapabilityDto {
available: boolean;
/**
* How the scan runs when available:
* - "vision": the provider reads the image directly (higher accuracy).
* - "text": the image is OCR'd in the browser (tesseract.js) and only the
* extracted text is sent to a text-only provider (less accurate; opt-in
* via `labsLocalOcrEnabled`). The UI loads the WASM only for this mode.
* - null: scanning is unavailable (see `reason`).
*/
mode: "vision" | "text" | null;
/** Why scanning is unavailable, when it is. */
reason: "no-provider" | "text-only-model" | null;
/** Whether PDF uploads are accepted (Anthropic-only in v1). */
reason: "no-provider" | "text-only-model" | "enable-local-ocr" | null;
/** Whether PDF uploads are accepted (Anthropic vision provider only). */
pdfSupported: boolean;
}
/**
* The body of a TEXT-mode extract call. The browser OCR's the image and POSTs
* the extracted text here (no image bytes ever reach the server). The route
* forwards the text to the text-only provider for structuring, reusing the
* exact review + commit flow the vision path uses.
*/
const OCR_TEXT_MAX_CHARS = 200_000;
export const ocrTextExtractSchema = z.object({
mode: z.literal("text"),
text: z.string().trim().min(1).max(OCR_TEXT_MAX_CHARS),
});
export type OcrTextExtractInput = z.infer<typeof ocrTextExtractSchema>;