mirror of
https://github.com/MBombeck/HealthLog.git
synced 2026-11-13 14:32:03 +00:00
feat(labs): add local-OCR text mode for image-blind AI providers
Add an opt-in in-browser OCR path so a user whose provider cannot read images (ChatGPT-OAuth/Codex, a text-only model) can still scan a lab report. The capability probe gains a `mode` field: `vision` when the provider reads the image directly, `text` when local OCR is enabled and any provider is configured, or null. The extract route grows a text branch that accepts the in-browser-OCR'd text as JSON and structures it through the existing schema + review/commit flow; the raw image never reaches the server in text mode. New `labsLocalOcrEnabled` user column (migration 0184, default off) gates it, with a dedicated PATCH endpoint.
This commit is contained in:
1 parent
fa37941add
commit
3d039d5220
13 files changed
+634
-77
No files matched your search
@@ -197,8 +197,32 @@ export interface OcrCommitResponseDto {
|
||||
/** The capability-probe response (drives whether the UI shows the scan entry). */
|
||||
export interface OcrCapabilityDto {
|
||||
available: boolean;
|
||||
/**
|
||||
* How the scan runs when available:
|
||||
* - "vision": the provider reads the image directly (higher accuracy).
|
||||
* - "text": the image is OCR'd in the browser (tesseract.js) and only the
|
||||
* extracted text is sent to a text-only provider (less accurate; opt-in
|
||||
* via `labsLocalOcrEnabled`). The UI loads the WASM only for this mode.
|
||||
* - null: scanning is unavailable (see `reason`).
|
||||
*/
|
||||
mode: "vision" | "text" | null;
|
||||
/** Why scanning is unavailable, when it is. */
|
||||
reason: "no-provider" | "text-only-model" | null;
|
||||
/** Whether PDF uploads are accepted (Anthropic-only in v1). */
|
||||
reason: "no-provider" | "text-only-model" | "enable-local-ocr" | null;
|
||||
/** Whether PDF uploads are accepted (Anthropic vision provider only). */
|
||||
pdfSupported: boolean;
|
||||
}
|
||||
|
||||
/**
|
||||
* The body of a TEXT-mode extract call. The browser OCR's the image and POSTs
|
||||
* the extracted text here (no image bytes ever reach the server). The route
|
||||
* forwards the text to the text-only provider for structuring, reusing the
|
||||
* exact review + commit flow the vision path uses.
|
||||
*/
|
||||
const OCR_TEXT_MAX_CHARS = 200_000;
|
||||
|
||||
export const ocrTextExtractSchema = z.object({
|
||||
mode: z.literal("text"),
|
||||
text: z.string().trim().min(1).max(OCR_TEXT_MAX_CHARS),
|
||||
});
|
||||
|
||||
export type OcrTextExtractInput = z.infer<typeof ocrTextExtractSchema>;
|
||||
Reference in new issue
Block a user