mirror of
https://github.com/MBombeck/HealthLog.git
synced 2026-11-08 12:32:43 +00:00
Merge pull request #456 from MBombeck/release/v1.28.12
release/v1.28.12 — lab scans use your AI provider; tidier dashboard
This commit is contained in:
28 files changed
+203
-778
No files matched your search
@@ -2,6 +2,15 @@
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
## [1.28.12] — 2026-07-09 — Lab scans use your AI provider; tidier dashboard
|
||||
|
||||
Scanning a lab report now uses the AI provider you already configured — including
|
||||
a scanned PDF, which is rendered to images the same way documents are, so a
|
||||
subscription or local vision provider reads it instead of only a Claude one. The
|
||||
separate lab-scan provider setting is gone; there is one place to configure AI.
|
||||
On the dashboard, the "log dose" action for an overdue medication now sits
|
||||
directly under its line instead of far to the right.
|
||||
|
||||
## [1.28.11] — 2026-07-09 — Strava workouts + Garmin guidance
|
||||
|
||||
Connect Strava to bring your activities in as workouts. Like the other wearable
|
||||
|
||||
+11
-45
@@ -1,7 +1,7 @@
|
||||
openapi: 3.1.0
|
||||
info:
|
||||
title: HealthLog API
|
||||
version: 1.28.11
|
||||
version: 1.28.12
|
||||
description: >-
|
||||
Self-hosted personal-health-tracking PWA — public API surface for the iOS native client and external ingest.
|
||||
|
||||
@@ -7612,13 +7612,14 @@ paths:
|
||||
- Labs
|
||||
summary: Extract lab readings from a photo, PDF, or OCR'd text
|
||||
description: "Read-only (NOT idempotent) extraction. Two modes by content-type. VISION (`multipart/form-data`): a `file`
|
||||
(JPEG/PNG/WebP, or PDF on an Anthropic vision provider; ≤ 12 MiB) is run through the user's vision-capable
|
||||
provider; the upload lives in memory only and is never persisted or logged. TEXT (`application/json`, opt-in
|
||||
local OCR): the browser OCR's the image (tesseract.js) and POSTs `{ mode: \"text\", text }` — only the extracted
|
||||
text reaches the server, so a text-only provider (e.g. ChatGPT-OAuth) reaches the same review/commit flow. Both
|
||||
modes are gated by AI consent, a 6/hour rate bucket, and the per-day token budget, and return proposed rows for
|
||||
the mandatory human review screen — nothing is written. Extracted content is treated as untrusted
|
||||
(prompt-injection); the review step is the safety boundary."
|
||||
(JPEG/PNG/WebP, or PDF — read natively on Anthropic, else rasterized to page images for any other vision
|
||||
provider; ≤ 12 MiB) is run through the user's vision-capable provider; the upload lives in memory only and is
|
||||
never persisted or logged. TEXT (`application/json`, opt-in local OCR): the browser OCR's the image
|
||||
(tesseract.js) and POSTs `{ mode: \"text\", text }` — only the extracted text reaches the server, so a text-only
|
||||
provider (e.g. ChatGPT-OAuth) reaches the same review/commit flow. Both modes are gated by AI consent, a 6/hour
|
||||
rate bucket, and the per-day token budget, and return proposed rows for the mandatory human review screen —
|
||||
nothing is written. Extracted content is treated as untrusted (prompt-injection); the review step is the safety
|
||||
boundary."
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
@@ -18492,35 +18493,6 @@ components:
|
||||
- type: "null"
|
||||
description: v1.22 (#89) — per-user response timeout for AI generation, in seconds (10–600). Null = the built-in
|
||||
comprehensive-briefing default (~120 s). Mainly for slow local/self-hosted backends.
|
||||
ocrEnabled:
|
||||
type: boolean
|
||||
description: v1.22 (#90) — whether a dedicated document-scan (Lab-OCR) provider is active. When false the OCR path uses
|
||||
the main provider chain.
|
||||
ocrProvider:
|
||||
anyOf:
|
||||
- type: string
|
||||
enum:
|
||||
- OPENAI
|
||||
- ANTHROPIC
|
||||
- LOCAL
|
||||
- type: "null"
|
||||
description: The dedicated document-scan provider, or null when unset.
|
||||
ocrModel:
|
||||
anyOf:
|
||||
- type: string
|
||||
- type: "null"
|
||||
ocrBaseUrl:
|
||||
anyOf:
|
||||
- type: string
|
||||
- type: "null"
|
||||
description: Custom base URL (LOCAL OCR provider only); null otherwise.
|
||||
hasOcrKey:
|
||||
type: boolean
|
||||
ocrKeyPreview:
|
||||
anyOf:
|
||||
- type: string
|
||||
- type: "null"
|
||||
description: "`...` + last 4 chars of the stored document-scan provider key, or null."
|
||||
required:
|
||||
- provider
|
||||
- model
|
||||
@@ -18533,12 +18505,6 @@ components:
|
||||
- hasOpenaiKey
|
||||
- openaiKeyPreview
|
||||
- responseTimeoutSeconds
|
||||
- ocrEnabled
|
||||
- ocrProvider
|
||||
- ocrModel
|
||||
- ocrBaseUrl
|
||||
- hasOcrKey
|
||||
- ocrKeyPreview
|
||||
additionalProperties: false
|
||||
description: The calling user's AI-provider configuration plus the effective availability (`aiAvailable` + `managedBy`).
|
||||
Reports key presence + a masked preview only — never a plaintext key.
|
||||
@@ -25372,8 +25338,8 @@ components:
|
||||
description: Whether the caller's configured AI provider can ingest a lab report (drives the UI's scan affordance).
|
||||
`mode` is `vision` when the provider reads the image directly, `text` when the image is OCR'd in-browser and
|
||||
only the extracted text is sent (opt-in local OCR for text-only providers), or null when unavailable. `reason`
|
||||
explains an unavailable state; `pdfSupported` is true only for an Anthropic vision provider (native PDF). No
|
||||
provider call is made.
|
||||
explains an unavailable state; `pdfSupported` is true when the provider reads PDFs natively (Anthropic) or the
|
||||
server-side rasterizer can render the pages for any other vision provider. No provider call is made.
|
||||
OcrExtractEnvelope:
|
||||
type: object
|
||||
properties:
|
||||
|
||||
@@ -4775,18 +4775,6 @@
|
||||
"responseTimeoutLabel": "Antwort-Timeout (Sekunden)",
|
||||
"responseTimeoutPlaceholder": "Standard (~120)",
|
||||
"responseTimeoutInvalid": "Gib eine Zahl zwischen 10 und 600 Sekunden ein.",
|
||||
"ocrHeading": "Dokumentenscan (Laborberichte)",
|
||||
"ocrBody": "Verwende einen eigenen Anbieter zum Auslesen hochgeladener Laborberichte. Ist die Option aus, nutzt der Scan deinen Hauptanbieter.",
|
||||
"ocrEnableLabel": "Eigenen Anbieter für den Dokumentenscan aktivieren",
|
||||
"ocrProviderLabel": "Scan-Anbieter",
|
||||
"ocrProviderOption": {
|
||||
"OPENAI": "OpenAI (dein API-Key)",
|
||||
"ANTHROPIC": "Anthropic (Claude)",
|
||||
"LOCAL": "Lokales Modell (OpenAI-kompatibel)"
|
||||
},
|
||||
"ocrModelPlaceholder": "z. B. gpt-4o",
|
||||
"ocrVisionHint": "Dieses Modell kann möglicherweise keine Bilder lesen. Wähle für den Dokumentenscan ein Modell mit Bildverständnis.",
|
||||
"ocrKeyLabel": "API-Key",
|
||||
"autoRead": {
|
||||
"title": "Dokumente automatisch mit KI lesen",
|
||||
"subLabel": "Neu hochgeladene Dokumente werden von deinem konfigurierten KI-Anbieter gelesen und indiziert, ohne dass du jedes Dokument einzeln bestätigen musst.",
|
||||
|
||||
@@ -4775,18 +4775,6 @@
|
||||
"responseTimeoutLabel": "Response timeout (seconds)",
|
||||
"responseTimeoutPlaceholder": "Default (~120)",
|
||||
"responseTimeoutInvalid": "Enter a number between 10 and 600 seconds.",
|
||||
"ocrHeading": "Document scanning (lab reports)",
|
||||
"ocrBody": "Use a dedicated provider for reading uploaded lab reports. When off, scanning uses your main provider.",
|
||||
"ocrEnableLabel": "Enable a dedicated document-scanning provider",
|
||||
"ocrProviderLabel": "Scanning provider",
|
||||
"ocrProviderOption": {
|
||||
"OPENAI": "OpenAI (your API key)",
|
||||
"ANTHROPIC": "Anthropic (Claude)",
|
||||
"LOCAL": "Local model (OpenAI-compatible)"
|
||||
},
|
||||
"ocrModelPlaceholder": "e.g. gpt-4o",
|
||||
"ocrVisionHint": "This model may not be able to read images. Choose a vision-capable model for document scanning.",
|
||||
"ocrKeyLabel": "API key",
|
||||
"autoRead": {
|
||||
"title": "Read documents automatically with AI",
|
||||
"subLabel": "Newly uploaded documents are read and indexed by your configured AI provider without a per-document tap.",
|
||||
|
||||
@@ -4775,18 +4775,6 @@
|
||||
"responseTimeoutLabel": "Tiempo de espera de la respuesta (segundos)",
|
||||
"responseTimeoutPlaceholder": "Predeterminado (~120)",
|
||||
"responseTimeoutInvalid": "Introduce un número entre 10 y 600 segundos.",
|
||||
"ocrHeading": "Escaneo de documentos (informes de laboratorio)",
|
||||
"ocrBody": "Usa un proveedor dedicado para leer los informes de laboratorio subidos. Si está desactivado, el escaneo usa tu proveedor principal.",
|
||||
"ocrEnableLabel": "Activar un proveedor dedicado para el escaneo de documentos",
|
||||
"ocrProviderLabel": "Proveedor de escaneo",
|
||||
"ocrProviderOption": {
|
||||
"OPENAI": "OpenAI (tu clave de API)",
|
||||
"ANTHROPIC": "Anthropic (Claude)",
|
||||
"LOCAL": "Modelo local (compatible con OpenAI)"
|
||||
},
|
||||
"ocrModelPlaceholder": "p. ej. gpt-4o",
|
||||
"ocrVisionHint": "Puede que este modelo no pueda leer imágenes. Elige un modelo con capacidad de visión para el escaneo de documentos.",
|
||||
"ocrKeyLabel": "Clave de API",
|
||||
"autoRead": {
|
||||
"title": "Leer documentos automáticamente con IA",
|
||||
"subLabel": "Los documentos recién subidos son leídos e indexados por tu proveedor de IA configurado, sin confirmar cada documento.",
|
||||
|
||||
@@ -4775,18 +4775,6 @@
|
||||
"responseTimeoutLabel": "Délai de réponse (secondes)",
|
||||
"responseTimeoutPlaceholder": "Par défaut (~120)",
|
||||
"responseTimeoutInvalid": "Saisissez un nombre entre 10 et 600 secondes.",
|
||||
"ocrHeading": "Numérisation de documents (comptes rendus de laboratoire)",
|
||||
"ocrBody": "Utilisez un fournisseur dédié pour lire les comptes rendus de laboratoire importés. Désactivé, la numérisation utilise votre fournisseur principal.",
|
||||
"ocrEnableLabel": "Activer un fournisseur dédié à la numérisation de documents",
|
||||
"ocrProviderLabel": "Fournisseur de numérisation",
|
||||
"ocrProviderOption": {
|
||||
"OPENAI": "OpenAI (votre clé API)",
|
||||
"ANTHROPIC": "Anthropic (Claude)",
|
||||
"LOCAL": "Modèle local (compatible OpenAI)"
|
||||
},
|
||||
"ocrModelPlaceholder": "p. ex. gpt-4o",
|
||||
"ocrVisionHint": "Ce modèle ne peut peut-être pas lire les images. Choisissez un modèle capable de vision pour la numérisation de documents.",
|
||||
"ocrKeyLabel": "Clé API",
|
||||
"autoRead": {
|
||||
"title": "Lire les documents automatiquement avec l'IA",
|
||||
"subLabel": "Les documents nouvellement téléversés sont lus et indexés par votre fournisseur d'IA configuré, sans validation document par document.",
|
||||
|
||||
@@ -4775,18 +4775,6 @@
|
||||
"responseTimeoutLabel": "Timeout della risposta (secondi)",
|
||||
"responseTimeoutPlaceholder": "Predefinito (~120)",
|
||||
"responseTimeoutInvalid": "Inserisci un numero tra 10 e 600 secondi.",
|
||||
"ocrHeading": "Scansione documenti (referti di laboratorio)",
|
||||
"ocrBody": "Usa un provider dedicato per leggere i referti di laboratorio caricati. Se disattivato, la scansione usa il tuo provider principale.",
|
||||
"ocrEnableLabel": "Attiva un provider dedicato per la scansione dei documenti",
|
||||
"ocrProviderLabel": "Provider di scansione",
|
||||
"ocrProviderOption": {
|
||||
"OPENAI": "OpenAI (la tua chiave API)",
|
||||
"ANTHROPIC": "Anthropic (Claude)",
|
||||
"LOCAL": "Modello locale (compatibile con OpenAI)"
|
||||
},
|
||||
"ocrModelPlaceholder": "es. gpt-4o",
|
||||
"ocrVisionHint": "Questo modello potrebbe non essere in grado di leggere le immagini. Scegli un modello con capacità di visione per la scansione dei documenti.",
|
||||
"ocrKeyLabel": "Chiave API",
|
||||
"autoRead": {
|
||||
"title": "Leggi i documenti automaticamente con l'IA",
|
||||
"subLabel": "I documenti appena caricati vengono letti e indicizzati dal provider di IA configurato, senza confermare ogni documento.",
|
||||
|
||||
@@ -4775,18 +4775,6 @@
|
||||
"responseTimeoutLabel": "Limit czasu odpowiedzi (sekundy)",
|
||||
"responseTimeoutPlaceholder": "Domyślnie (~120)",
|
||||
"responseTimeoutInvalid": "Podaj liczbę od 10 do 600 sekund.",
|
||||
"ocrHeading": "Skanowanie dokumentów (wyniki badań)",
|
||||
"ocrBody": "Użyj dedykowanego dostawcy do odczytu przesłanych wyników badań. Gdy wyłączone, skanowanie używa Twojego głównego dostawcy.",
|
||||
"ocrEnableLabel": "Włącz dedykowanego dostawcę do skanowania dokumentów",
|
||||
"ocrProviderLabel": "Dostawca skanowania",
|
||||
"ocrProviderOption": {
|
||||
"OPENAI": "OpenAI (Twój klucz API)",
|
||||
"ANTHROPIC": "Anthropic (Claude)",
|
||||
"LOCAL": "Model lokalny (zgodny z OpenAI)"
|
||||
},
|
||||
"ocrModelPlaceholder": "np. gpt-4o",
|
||||
"ocrVisionHint": "Ten model może nie odczytywać obrazów. Wybierz model obsługujący obraz do skanowania dokumentów.",
|
||||
"ocrKeyLabel": "Klucz API",
|
||||
"autoRead": {
|
||||
"title": "Automatyczne czytanie dokumentów przez AI",
|
||||
"subLabel": "Nowo przesłane dokumenty są czytane i indeksowane przez skonfigurowanego dostawcę AI bez potwierdzania każdego dokumentu.",
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "healthlog",
|
||||
"version": "1.28.11",
|
||||
"version": "1.28.12",
|
||||
"description": "Self-hosted personal-health-tracking PWA with Withings integration, AI insights, and doctor-report PDF export.",
|
||||
"license": "PolyForm-Noncommercial-1.0.0",
|
||||
"homepage": "https://healthlog.dev",
|
||||
|
||||
@@ -0,0 +1,21 @@
|
||||
-- Retire the dedicated document-scan (Lab-OCR) provider.
|
||||
--
|
||||
-- The separate `ai_ocr_*` provider entry is gone: lab-report OCR now always
|
||||
-- uses the user's main configured provider chain (the same providers, incl.
|
||||
-- codex / OAuth, that the Coach and Insights already use), and a PDF is read
|
||||
-- natively on Anthropic or rasterized to page images for any other vision
|
||||
-- provider. The five per-user columns that backed the standalone provider are
|
||||
-- no longer read or written anywhere, so drop them.
|
||||
--
|
||||
-- `ai_ocr_key_encrypted` was an AES-256-GCM column; it is removed from the
|
||||
-- encrypted-column registry + the rotation script in the same change. Dropping
|
||||
-- it here means those rows are gone before any future key rotation runs.
|
||||
--
|
||||
-- Idempotent (IF EXISTS); applies clean on a 0233 database and on a fresh
|
||||
-- database as part of the ordered replay.
|
||||
ALTER TABLE "users"
|
||||
DROP COLUMN IF EXISTS "ai_ocr_enabled",
|
||||
DROP COLUMN IF EXISTS "ai_ocr_provider",
|
||||
DROP COLUMN IF EXISTS "ai_ocr_model",
|
||||
DROP COLUMN IF EXISTS "ai_ocr_base_url",
|
||||
DROP COLUMN IF EXISTS "ai_ocr_key_encrypted";
|
||||
@@ -312,19 +312,6 @@ model User {
|
||||
// default timed the Coach out before the first token landed.
|
||||
aiResponseTimeoutSeconds Int? @map("ai_response_timeout_seconds")
|
||||
|
||||
// v1.22 (#90) — dedicated document-scan (Lab-OCR) provider. OFF by default:
|
||||
// the lab-report OCR path uses the main provider chain unchanged. When
|
||||
// `aiOcrEnabled` is true the OCR ingestion resolves THIS provider/model/key
|
||||
// instead, so a user can keep a cheap text Coach on one provider and route
|
||||
// expensive vision OCR to a vision-capable one. The key is AES-256-GCM at
|
||||
// rest like every other `*Encrypted` column (registered in
|
||||
// `src/lib/crypto/encrypted-columns.ts` + the rotation script).
|
||||
aiOcrEnabled Boolean @default(false) @map("ai_ocr_enabled")
|
||||
aiOcrProvider String? @map("ai_ocr_provider") // "OPENAI" | "ANTHROPIC" | "LOCAL"
|
||||
aiOcrModel String? @map("ai_ocr_model")
|
||||
aiOcrBaseUrl String? @map("ai_ocr_base_url") // LOCAL OCR provider only
|
||||
aiOcrKeyEncrypted String? @map("ai_ocr_key_encrypted")
|
||||
|
||||
// Automatic AI reading of uploaded vault documents. OFF by default: the vault
|
||||
// stays local-first — a newly stored document is only read externally when the
|
||||
// user explicitly taps a per-document AI action (which still needs an active
|
||||
|
||||
+1
-1
@@ -36,7 +36,7 @@ try {
|
||||
// v1.4.38.4 → v1.4.42. Do not hand-edit; bump `package.json` and rebuild.
|
||||
const CACHE_VERSION =
|
||||
(typeof self !== "undefined" && self.__APP_VERSION__) ||
|
||||
/* @sw-version-fallback */ "v1.28.11";
|
||||
/* @sw-version-fallback */ "v1.28.12";
|
||||
const STATIC_CACHE = `healthlog-static-${CACHE_VERSION}`;
|
||||
const PAGE_CACHE = `healthlog-pages-${CACHE_VERSION}`;
|
||||
// v1.18.6 — read-only data cache for a curated allowlist of safe GET `/api/*`
|
||||
|
||||
@@ -204,7 +204,6 @@ async function main() {
|
||||
"aiAnthropicKeyEncrypted",
|
||||
"aiLocalKeyEncrypted",
|
||||
"aiOpenaiKeyEncrypted",
|
||||
"aiOcrKeyEncrypted",
|
||||
"insuranceNumberEncrypted",
|
||||
// v1.23 — TOTP shared secret (second factor).
|
||||
"totpSecretEncrypted",
|
||||
|
||||
@@ -104,7 +104,6 @@ const LEGIT_IDENTICAL = new Set<string>([
|
||||
"insights.healthScore.provenance.sources.appleHealth",
|
||||
"measurements.sourceAppleHealth",
|
||||
"onboarding.source.more.appleHealth",
|
||||
"settings.ai.ocrProviderOption.ANTHROPIC",
|
||||
"settings.ai.providerChain.types.anthropic",
|
||||
"settings.ai.providerChain.types.codex",
|
||||
"settings.ai.providerSelect.anthropic",
|
||||
|
||||
@@ -30,6 +30,9 @@ vi.mock("@/lib/labs/ocr-capability", () => ({
|
||||
resolveTextProvider: vi.fn(),
|
||||
resolveVisionProvider: vi.fn(),
|
||||
}));
|
||||
vi.mock("@/lib/documents/rasterize-pdf", () => ({
|
||||
rasterizePdf: vi.fn(),
|
||||
}));
|
||||
vi.mock("@/lib/ai/consent-guard", () => ({
|
||||
assertConsentForChain: vi.fn().mockResolvedValue(undefined),
|
||||
}));
|
||||
@@ -54,7 +57,11 @@ import { POST } from "../route";
|
||||
import { AI_BUDGETS } from "@/lib/ai/ai-budgets";
|
||||
import { requireAuth } from "@/lib/api-handler";
|
||||
import { prisma } from "@/lib/db";
|
||||
import { resolveTextProvider } from "@/lib/labs/ocr-capability";
|
||||
import {
|
||||
resolveTextProvider,
|
||||
resolveVisionProvider,
|
||||
} from "@/lib/labs/ocr-capability";
|
||||
import { rasterizePdf } from "@/lib/documents/rasterize-pdf";
|
||||
import { reserveBudget, reconcileSpend } from "@/lib/ai/coach/budget";
|
||||
import { OcrExtractError, runOcrExtraction } from "@/lib/labs/ocr-extract";
|
||||
|
||||
@@ -130,3 +137,78 @@ describe("POST /api/labs/ocr/extract — text mode budget", () => {
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe("POST /api/labs/ocr/extract — vision PDF rasterization", () => {
|
||||
function pdfReq(): Request {
|
||||
// A minimal `%PDF-` header is all `detectOcrMimeType` needs to sniff a PDF.
|
||||
const bytes = new Uint8Array([
|
||||
0x25, 0x50, 0x44, 0x46, 0x2d, 0x31, 0x2e, 0x34, 0x0a, 0x25, 0xe2, 0xe3,
|
||||
0xcf, 0xd3, 0x0a,
|
||||
]);
|
||||
const file = new File([bytes], "report.pdf", {
|
||||
type: "application/pdf",
|
||||
});
|
||||
const form = new FormData();
|
||||
form.append("file", file);
|
||||
return new Request("http://localhost/api/labs/ocr/extract", {
|
||||
method: "POST",
|
||||
body: form,
|
||||
});
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
// A non-Anthropic vision provider (codex): no native PDF block, so the
|
||||
// route must rasterize.
|
||||
vi.mocked(resolveVisionProvider).mockResolvedValue({
|
||||
chain: [{ providerType: "codex", instance: {} as never }],
|
||||
localOcrEnabled: false,
|
||||
pick: {
|
||||
entry: { providerType: "codex", instance: {} as never },
|
||||
providerType: "codex",
|
||||
pdfSupported: false,
|
||||
},
|
||||
} as never);
|
||||
vi.mocked(reserveBudget).mockResolvedValue({
|
||||
allowed: true,
|
||||
reserved: AI_BUDGETS.ocrExtract.maxTokens ?? 0,
|
||||
totalAfter: AI_BUDGETS.ocrExtract.maxTokens ?? 0,
|
||||
} as never);
|
||||
});
|
||||
|
||||
it("rasterizes a PDF for a non-Anthropic vision provider and sends the page images", async () => {
|
||||
vi.mocked(rasterizePdf).mockResolvedValue({
|
||||
ok: true,
|
||||
images: [{ mediaType: "image/jpeg", dataBase64: "cGFnZQ==" }],
|
||||
});
|
||||
vi.mocked(runOcrExtraction).mockResolvedValue({ rows: [] } as never);
|
||||
|
||||
const res = await POST(pdfReq());
|
||||
expect(res.status).toBe(200);
|
||||
expect(rasterizePdf).toHaveBeenCalledOnce();
|
||||
// The rendered page images flow through as `input_image`s; no native PDF
|
||||
// document block is sent for a non-Anthropic provider.
|
||||
expect(runOcrExtraction).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
images: [{ mediaType: "image/jpeg", dataBase64: "cGFnZQ==" }],
|
||||
documents: [],
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it("falls back to pdfNeedsAnthropic when rasterization fails", async () => {
|
||||
vi.mocked(rasterizePdf).mockResolvedValue({ ok: false });
|
||||
|
||||
const res = await POST(pdfReq());
|
||||
expect(res.status).toBe(422);
|
||||
const body = (await res.json()) as { error: string | null };
|
||||
expect(body.error).toBeTruthy();
|
||||
// A failed render never reaches the provider — and refunds the reservation.
|
||||
expect(runOcrExtraction).not.toHaveBeenCalled();
|
||||
expect(reconcileSpend).toHaveBeenCalledWith(
|
||||
"user-1",
|
||||
AI_BUDGETS.ocrExtract.maxTokens,
|
||||
0,
|
||||
"2026-06-26",
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -38,6 +38,7 @@ import {
|
||||
resolveDailyCap,
|
||||
} from "@/lib/ai/coach/budget";
|
||||
import { prisma } from "@/lib/db";
|
||||
import { rasterizePdf } from "@/lib/documents/rasterize-pdf";
|
||||
import {
|
||||
resolveTextProvider,
|
||||
resolveVisionProvider,
|
||||
@@ -323,27 +324,52 @@ async function handleVisionExtract(
|
||||
});
|
||||
}
|
||||
|
||||
// 7. PDF gate — only Anthropic reads a native document block in v1.
|
||||
if (mime === "application/pdf" && !pick.pdfSupported) {
|
||||
annotate({
|
||||
action: { name: "labs.ocr.fileRejected" },
|
||||
meta: { reason: "pdf_provider_unsupported" },
|
||||
});
|
||||
await reconcileSpend(userId, reservation.reserved, 0, dateKey);
|
||||
return apiError(
|
||||
"PDF scanning needs a Claude vision provider; upload a photo instead.",
|
||||
422,
|
||||
{ errorCode: "labs.ocr.pdfNeedsAnthropic" },
|
||||
);
|
||||
}
|
||||
// 7. PDF handling — mirror the document vault's `prepareVisionInput`
|
||||
// (`src/lib/documents/ai-route-support.ts`) exactly:
|
||||
// - Anthropic (`pick.pdfSupported`) reads the native `document` block at
|
||||
// highest fidelity;
|
||||
// - every other vision provider (codex / any image-only wire) reads the
|
||||
// PDF via `rasterizePdf`, which renders the pages to JPEG `input_image`
|
||||
// parts the provider already handles. Rendering is pure local compute
|
||||
// (no egress). `rasterizePdf` never throws: a malformed / encrypted /
|
||||
// unrenderable PDF returns `{ ok: false }`, and we keep the shipped
|
||||
// `pdfNeedsAnthropic` reject as the graceful fallback.
|
||||
let images: {
|
||||
mediaType: "image/jpeg" | "image/png" | "image/webp";
|
||||
dataBase64: string;
|
||||
}[];
|
||||
let documents: { mediaType: "application/pdf"; dataBase64: string }[];
|
||||
|
||||
const dataBase64 = buffer.toString("base64");
|
||||
const images =
|
||||
mime === "application/pdf" ? [] : [{ mediaType: mime, dataBase64 }];
|
||||
const documents =
|
||||
mime === "application/pdf"
|
||||
? [{ mediaType: "application/pdf" as const, dataBase64 }]
|
||||
: [];
|
||||
if (mime === "application/pdf") {
|
||||
if (pick.pdfSupported) {
|
||||
images = [];
|
||||
documents = [
|
||||
{
|
||||
mediaType: "application/pdf",
|
||||
dataBase64: buffer.toString("base64"),
|
||||
},
|
||||
];
|
||||
} else {
|
||||
const raster = await rasterizePdf(buffer);
|
||||
if (!raster.ok) {
|
||||
annotate({
|
||||
action: { name: "labs.ocr.fileRejected" },
|
||||
meta: { reason: "pdf_rasterize_failed" },
|
||||
});
|
||||
await reconcileSpend(userId, reservation.reserved, 0, dateKey);
|
||||
return apiError(
|
||||
"Couldn't read this PDF; upload a photo instead.",
|
||||
422,
|
||||
{ errorCode: "labs.ocr.pdfNeedsAnthropic" },
|
||||
);
|
||||
}
|
||||
images = raster.images;
|
||||
documents = [];
|
||||
}
|
||||
} else {
|
||||
images = [{ mediaType: mime, dataBase64: buffer.toString("base64") }];
|
||||
documents = [];
|
||||
}
|
||||
|
||||
// 8. Run the extraction. The actual token spend reconciles the reservation.
|
||||
let actualTokens = 0;
|
||||
|
||||
@@ -139,7 +139,7 @@ describe("GET /api/user/ai-provider availability", () => {
|
||||
expect(body.data.managedBy).toBeNull();
|
||||
});
|
||||
|
||||
it("returns the v1.22 timeout + OCR fields", async () => {
|
||||
it("returns the v1.22 response timeout field", async () => {
|
||||
vi.mocked(prisma.user.findUnique).mockResolvedValue({
|
||||
aiProvider: "OPENAI",
|
||||
aiModel: "gpt-4o",
|
||||
@@ -148,11 +148,6 @@ describe("GET /api/user/ai-provider availability", () => {
|
||||
aiLocalKeyEncrypted: null,
|
||||
aiOpenaiKeyEncrypted: null,
|
||||
aiResponseTimeoutSeconds: 240,
|
||||
aiOcrEnabled: true,
|
||||
aiOcrProvider: "ANTHROPIC",
|
||||
aiOcrModel: "claude-sonnet-4-6",
|
||||
aiOcrBaseUrl: null,
|
||||
aiOcrKeyEncrypted: "enc-ocr-1234",
|
||||
} as never);
|
||||
vi.mocked(resolveProviderAvailability).mockResolvedValue({
|
||||
aiAvailable: true,
|
||||
@@ -161,21 +156,10 @@ describe("GET /api/user/ai-provider availability", () => {
|
||||
|
||||
const res = await (GET as () => Promise<Response>)();
|
||||
const body = (await res.json()) as {
|
||||
data: {
|
||||
responseTimeoutSeconds: number | null;
|
||||
ocrEnabled: boolean;
|
||||
ocrProvider: string | null;
|
||||
hasOcrKey: boolean;
|
||||
ocrKeyPreview: string | null;
|
||||
};
|
||||
data: { responseTimeoutSeconds: number | null };
|
||||
};
|
||||
|
||||
expect(body.data.responseTimeoutSeconds).toBe(240);
|
||||
expect(body.data.ocrEnabled).toBe(true);
|
||||
expect(body.data.ocrProvider).toBe("ANTHROPIC");
|
||||
expect(body.data.hasOcrKey).toBe(true);
|
||||
// Preview is `...` + last 4 of the decrypted value (decrypt → `dec:...`).
|
||||
expect(body.data.ocrKeyPreview).toMatch(/^\.\.\./);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -207,32 +191,4 @@ describe("PATCH /api/user/ai-provider — v1.22 fields", () => {
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it("persists OCR provider config and encrypts the key", async () => {
|
||||
const res = await patch(
|
||||
patchRequest({
|
||||
ocrEnabled: true,
|
||||
ocrProvider: "OPENAI",
|
||||
ocrModel: "gpt-4o",
|
||||
ocrKey: "sk-ocr-secret",
|
||||
}),
|
||||
);
|
||||
expect(res.status).toBe(200);
|
||||
expect(vi.mocked(prisma.user.update)).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
data: expect.objectContaining({
|
||||
aiOcrEnabled: true,
|
||||
aiOcrProvider: "OPENAI",
|
||||
aiOcrModel: "gpt-4o",
|
||||
aiOcrKeyEncrypted: "enc:sk-ocr-secret",
|
||||
}),
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it("rejects an invalid OCR provider", async () => {
|
||||
await expect(
|
||||
patch(patchRequest({ ocrProvider: "CHATGPT_OAUTH" })),
|
||||
).rejects.toThrow(/Invalid OCR provider/);
|
||||
});
|
||||
});
|
||||
@@ -11,9 +11,6 @@ import { annotate } from "@/lib/logging/context";
|
||||
export const dynamic = "force-dynamic";
|
||||
|
||||
const ALLOWED = new Set(["OPENAI", "ANTHROPIC", "LOCAL", "CHATGPT_OAUTH"]);
|
||||
// v1.22 (#90) — the dedicated document-scan provider has no OAuth path, so the
|
||||
// allowlist is the three keyed/self-hosted providers only (no CHATGPT_OAUTH).
|
||||
const OCR_PROVIDERS = new Set(["OPENAI", "ANTHROPIC", "LOCAL"]);
|
||||
|
||||
export const GET = apiHandler(async () => {
|
||||
const { user } = await requireAuth();
|
||||
@@ -28,13 +25,8 @@ export const GET = apiHandler(async () => {
|
||||
aiAnthropicKeyEncrypted: true,
|
||||
aiLocalKeyEncrypted: true,
|
||||
aiOpenaiKeyEncrypted: true,
|
||||
// v1.22 (#89 + #90)
|
||||
// v1.22 (#89)
|
||||
aiResponseTimeoutSeconds: true,
|
||||
aiOcrEnabled: true,
|
||||
aiOcrProvider: true,
|
||||
aiOcrModel: true,
|
||||
aiOcrBaseUrl: true,
|
||||
aiOcrKeyEncrypted: true,
|
||||
},
|
||||
});
|
||||
|
||||
@@ -62,16 +54,6 @@ export const GET = apiHandler(async () => {
|
||||
: null,
|
||||
// v1.22 (#89) — per-user response timeout, in seconds (null = default).
|
||||
responseTimeoutSeconds: u?.aiResponseTimeoutSeconds ?? null,
|
||||
// v1.22 (#90) — dedicated document-scan (Lab-OCR) provider config. Key
|
||||
// material is presence + 4-char preview only, like every other provider.
|
||||
ocrEnabled: Boolean(u?.aiOcrEnabled),
|
||||
ocrProvider: u?.aiOcrProvider ?? null,
|
||||
ocrModel: u?.aiOcrModel ?? null,
|
||||
ocrBaseUrl: u?.aiOcrBaseUrl ?? null,
|
||||
hasOcrKey: Boolean(u?.aiOcrKeyEncrypted),
|
||||
ocrKeyPreview: u?.aiOcrKeyEncrypted
|
||||
? `...${decrypt(u.aiOcrKeyEncrypted).slice(-4)}`
|
||||
: null,
|
||||
});
|
||||
});
|
||||
|
||||
@@ -179,62 +161,6 @@ export const PATCH = apiHandler(async (request: NextRequest) => {
|
||||
}
|
||||
}
|
||||
|
||||
// ── v1.22 (#90) — dedicated document-scan (Lab-OCR) provider ──
|
||||
if (body.ocrEnabled !== undefined) {
|
||||
if (typeof body.ocrEnabled === "boolean") {
|
||||
updates.aiOcrEnabled = body.ocrEnabled;
|
||||
} else {
|
||||
throw new HttpError(422, "Invalid ocrEnabled");
|
||||
}
|
||||
}
|
||||
|
||||
if (body.ocrProvider !== undefined) {
|
||||
if (body.ocrProvider === null || body.ocrProvider === "") {
|
||||
updates.aiOcrProvider = null;
|
||||
} else if (
|
||||
typeof body.ocrProvider === "string" &&
|
||||
OCR_PROVIDERS.has(body.ocrProvider)
|
||||
) {
|
||||
updates.aiOcrProvider = body.ocrProvider;
|
||||
} else {
|
||||
throw new HttpError(422, "Invalid OCR provider");
|
||||
}
|
||||
}
|
||||
|
||||
if (body.ocrModel !== undefined) {
|
||||
if (body.ocrModel === null || body.ocrModel === "") {
|
||||
updates.aiOcrModel = null;
|
||||
} else if (typeof body.ocrModel === "string") {
|
||||
updates.aiOcrModel = body.ocrModel.trim() || null;
|
||||
}
|
||||
}
|
||||
|
||||
if (body.ocrBaseUrl !== undefined) {
|
||||
if (body.ocrBaseUrl === null || body.ocrBaseUrl === "") {
|
||||
updates.aiOcrBaseUrl = null;
|
||||
} else if (typeof body.ocrBaseUrl === "string") {
|
||||
const trimmed = body.ocrBaseUrl.trim();
|
||||
// Same SSRF guard as the main LOCAL base URL: reject private/internal
|
||||
// hosts unless the operator opted in via ALLOW_LOCAL_AI_PRIVATE_HOSTS.
|
||||
const allowPrivate = isLocalAiHostAllowed(trimmed);
|
||||
if (!allowPrivate && !isPublicUrl(trimmed)) {
|
||||
return apiError(
|
||||
"OCR base URL points to an internal/private host. Ops must allow it on this instance via ALLOW_LOCAL_AI_PRIVATE_HOSTS.",
|
||||
422,
|
||||
);
|
||||
}
|
||||
updates.aiOcrBaseUrl = trimmed;
|
||||
}
|
||||
}
|
||||
|
||||
if (body.ocrKey !== undefined) {
|
||||
if (body.ocrKey === null || body.ocrKey === "") {
|
||||
updates.aiOcrKeyEncrypted = null;
|
||||
} else if (typeof body.ocrKey === "string") {
|
||||
updates.aiOcrKeyEncrypted = encrypt(body.ocrKey.trim());
|
||||
}
|
||||
}
|
||||
|
||||
if (Object.keys(updates).length === 0) {
|
||||
throw new HttpError(422, "No valid fields");
|
||||
}
|
||||
|
||||
@@ -399,17 +399,18 @@ export function DashboardHero({
|
||||
>
|
||||
{welcomeText}
|
||||
</p>
|
||||
{/* Text left, action right, one row — the PageHeader/TileHeader
|
||||
inline-action pattern. The sentence takes the available width and
|
||||
wraps internally (`min-w-0 flex-1`); the button never wraps to its
|
||||
own line (`shrink-0`), so the action stays beside the text. */}
|
||||
<div className="flex items-start justify-between gap-3">
|
||||
{/* Sentence then action, stacked — the overdue-dose button sits
|
||||
DIRECTLY beneath the verdict text (tight `gap-1.5`), left-aligned
|
||||
and intrinsic width (`self-start w-auto`), on every viewport. An
|
||||
earlier `justify-between` shoved the button to the far container
|
||||
edge on desktop, stranding it from the sentence it acts on. */}
|
||||
<div className="flex flex-col items-start gap-1.5">
|
||||
{sentenceHref ? (
|
||||
<Link
|
||||
href={sentenceHref}
|
||||
data-slot="dashboard-hero-verdict"
|
||||
data-verdict-variant={verdict.variant}
|
||||
className="text-foreground/90 hover:text-foreground min-w-0 flex-1 text-base font-medium transition-colors"
|
||||
className="text-foreground/90 hover:text-foreground min-w-0 text-base font-medium transition-colors"
|
||||
>
|
||||
{sentence}
|
||||
</Link>
|
||||
@@ -417,7 +418,7 @@ export function DashboardHero({
|
||||
<p
|
||||
data-slot="dashboard-hero-verdict"
|
||||
data-verdict-variant={verdict.variant}
|
||||
className="text-foreground/90 min-w-0 flex-1 text-base font-medium"
|
||||
className="text-foreground/90 min-w-0 text-base font-medium"
|
||||
>
|
||||
{sentence}
|
||||
</p>
|
||||
@@ -428,7 +429,7 @@ export function DashboardHero({
|
||||
asChild
|
||||
size="default"
|
||||
variant="outline"
|
||||
className="min-h-11 shrink-0 sm:min-h-9"
|
||||
className="min-h-11 w-auto self-start sm:min-h-9"
|
||||
data-slot="dashboard-hero-cta"
|
||||
>
|
||||
<Link href={cta.href}>{t(ctaLabelKey)}</Link>
|
||||
@@ -436,7 +437,7 @@ export function DashboardHero({
|
||||
) : (
|
||||
<Button
|
||||
size="default"
|
||||
className="min-h-11 shrink-0 sm:min-h-9"
|
||||
className="min-h-11 w-auto self-start sm:min-h-9"
|
||||
data-slot="dashboard-hero-cta"
|
||||
onClick={() => onQuickEntry(cta.target)}
|
||||
>
|
||||
|
||||
@@ -33,7 +33,6 @@ import { AdminOpenAIProviderForm } from "./admin-openai-provider-form";
|
||||
import { AnthropicProviderForm } from "./anthropic-provider-form";
|
||||
import { AutoReadCard } from "./auto-read-card";
|
||||
import { CodexProviderForm } from "./codex-provider-form";
|
||||
import { DocumentScanCard } from "./document-scan-card";
|
||||
import { FallbackChainCard } from "./fallback-chain-card";
|
||||
import { LocalProviderForm } from "./local-provider-form";
|
||||
import { OpenAIProviderForm } from "./openai-provider-form";
|
||||
@@ -179,9 +178,6 @@ export function AiInsightsCard({
|
||||
local/self-hosted backends). */}
|
||||
<ResponseTimeoutCard userProvider={userProvider} />
|
||||
|
||||
{/* v1.22 (#90) — dedicated document-scan (Lab-OCR) provider. */}
|
||||
<DocumentScanCard userProvider={userProvider} />
|
||||
|
||||
{/* Read uploaded vault documents automatically with AI (opt-in). */}
|
||||
<AutoReadCard />
|
||||
|
||||
|
||||
@@ -1,212 +0,0 @@
|
||||
"use client";
|
||||
|
||||
/* ────────────────────────────────────────────────────────────────
|
||||
* v1.22 (#90) — Document scanning (Lab reports).
|
||||
*
|
||||
* Opt-in dedicated provider for the Lab-OCR ingestion path. OFF by default:
|
||||
* scanning uses the main provider chain. When enabled, document/lab-report
|
||||
* processing routes to THIS provider/model/key instead — so a user can keep a
|
||||
* cheap text Coach on one provider and send expensive vision OCR to a
|
||||
* vision-capable one. Surfaces a hint when the chosen model can't read images.
|
||||
* ──────────────────────────────────────────────────────────────── */
|
||||
|
||||
import { useState } from "react";
|
||||
import { useMutation, useQueryClient } from "@tanstack/react-query";
|
||||
import { Loader2, Save } from "lucide-react";
|
||||
|
||||
import { Button } from "@/components/ui/button";
|
||||
import { Input } from "@/components/ui/input";
|
||||
import { Label } from "@/components/ui/label";
|
||||
import { NativeSelect } from "@/components/ui/native-select";
|
||||
import { PasswordInput } from "@/components/ui/password-input";
|
||||
import { Switch } from "@/components/ui/switch";
|
||||
import { apiPatch } from "@/lib/api/api-fetch";
|
||||
import {
|
||||
supportsVisionForConfig,
|
||||
type VisionProviderType,
|
||||
} from "@/lib/ai/vision-capability";
|
||||
import { useTranslations } from "@/lib/i18n/context";
|
||||
import { queryKeys } from "@/lib/query-keys";
|
||||
|
||||
import { OCR_PROVIDER_TYPES, type UserAIProvider } from "./shared";
|
||||
|
||||
/** Map the OCR provider enum to the vision-capability tag. */
|
||||
function toVisionType(provider: string): VisionProviderType {
|
||||
switch (provider) {
|
||||
case "ANTHROPIC":
|
||||
return "anthropic";
|
||||
case "OPENAI":
|
||||
return "openai";
|
||||
case "LOCAL":
|
||||
return "local";
|
||||
default:
|
||||
return "none";
|
||||
}
|
||||
}
|
||||
|
||||
export function DocumentScanCard({
|
||||
userProvider,
|
||||
}: {
|
||||
userProvider: UserAIProvider | null | undefined;
|
||||
}) {
|
||||
const { t } = useTranslations();
|
||||
const queryClient = useQueryClient();
|
||||
|
||||
const [enabled, setEnabled] = useState(false);
|
||||
const [provider, setProvider] = useState<string>("OPENAI");
|
||||
const [model, setModel] = useState("");
|
||||
const [baseUrl, setBaseUrl] = useState("");
|
||||
const [apiKey, setApiKey] = useState("");
|
||||
const [msg, setMsg] = useState<string | null>(null);
|
||||
const [ok, setOk] = useState(false);
|
||||
|
||||
const seededKey =
|
||||
userProvider != null
|
||||
? `${userProvider.ocrEnabled}|${userProvider.ocrProvider ?? ""}|${userProvider.ocrModel ?? ""}|${userProvider.ocrBaseUrl ?? ""}`
|
||||
: null;
|
||||
const [previousSeed, setPreviousSeed] = useState<string | null>(null);
|
||||
if (seededKey != null && seededKey !== previousSeed) {
|
||||
setPreviousSeed(seededKey);
|
||||
setEnabled(userProvider?.ocrEnabled ?? false);
|
||||
setProvider(userProvider?.ocrProvider ?? "OPENAI");
|
||||
setModel(userProvider?.ocrModel ?? "");
|
||||
setBaseUrl(userProvider?.ocrBaseUrl ?? "");
|
||||
}
|
||||
|
||||
// Vision-capability hint: warn when a model is chosen that can't read images
|
||||
// (local is trust-by-default; the warning surfaces only for cloud providers).
|
||||
const visionOk =
|
||||
!enabled ||
|
||||
model.trim() === "" ||
|
||||
supportsVisionForConfig(toVisionType(provider), model.trim());
|
||||
|
||||
const saveMutation = useMutation({
|
||||
mutationFn: async () => {
|
||||
const body: Record<string, unknown> = {
|
||||
ocrEnabled: enabled,
|
||||
ocrProvider: provider,
|
||||
ocrModel: model.trim() || null,
|
||||
ocrBaseUrl: provider === "LOCAL" ? baseUrl.trim() || null : null,
|
||||
};
|
||||
if (apiKey.trim()) body.ocrKey = apiKey.trim();
|
||||
await apiPatch("/api/user/ai-provider", body);
|
||||
},
|
||||
onSuccess: () => {
|
||||
setOk(true);
|
||||
setMsg(t("settings.ai.saved"));
|
||||
setApiKey("");
|
||||
queryClient.invalidateQueries({ queryKey: queryKeys.userAiProvider() });
|
||||
},
|
||||
onError: (e) => {
|
||||
setOk(false);
|
||||
setMsg(e instanceof Error ? e.message : t("settings.ai.errorGeneric"));
|
||||
},
|
||||
});
|
||||
|
||||
return (
|
||||
<div className="bg-muted/50 space-y-4 rounded-lg p-4">
|
||||
<div className="flex items-start justify-between gap-3">
|
||||
<div>
|
||||
<p className="text-sm font-medium">{t("settings.ai.ocrHeading")}</p>
|
||||
<p className="text-muted-foreground text-xs">
|
||||
{t("settings.ai.ocrBody")}
|
||||
</p>
|
||||
</div>
|
||||
<Switch
|
||||
checked={enabled}
|
||||
onCheckedChange={setEnabled}
|
||||
aria-label={t("settings.ai.ocrEnableLabel")}
|
||||
data-testid="ai-ocr-enable"
|
||||
/>
|
||||
</div>
|
||||
|
||||
{enabled && (
|
||||
<div className="space-y-4">
|
||||
<div>
|
||||
<Label htmlFor="ai-ocr-provider">
|
||||
{t("settings.ai.ocrProviderLabel")}
|
||||
</Label>
|
||||
<NativeSelect
|
||||
id="ai-ocr-provider"
|
||||
value={provider}
|
||||
onChange={(e) => setProvider(e.target.value)}
|
||||
className="mt-1 sm:max-w-md"
|
||||
>
|
||||
{OCR_PROVIDER_TYPES.map((p) => (
|
||||
<option key={p} value={p}>
|
||||
{t(`settings.ai.ocrProviderOption.${p}` as const)}
|
||||
</option>
|
||||
))}
|
||||
</NativeSelect>
|
||||
</div>
|
||||
|
||||
<div>
|
||||
<Label htmlFor="ai-ocr-model">{t("settings.ai.modelLabel")}</Label>
|
||||
<Input
|
||||
id="ai-ocr-model"
|
||||
value={model}
|
||||
onChange={(e) => setModel(e.target.value)}
|
||||
placeholder={t("settings.ai.ocrModelPlaceholder")}
|
||||
className="mt-1"
|
||||
/>
|
||||
{!visionOk && (
|
||||
<p className="text-warning mt-1 text-xs">
|
||||
{t("settings.ai.ocrVisionHint")}
|
||||
</p>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{provider === "LOCAL" && (
|
||||
<div>
|
||||
<Label htmlFor="ai-ocr-base-url">
|
||||
{t("settings.ai.baseUrlLabel")}
|
||||
</Label>
|
||||
<Input
|
||||
id="ai-ocr-base-url"
|
||||
value={baseUrl}
|
||||
onChange={(e) => setBaseUrl(e.target.value)}
|
||||
placeholder="http://localhost:11434/v1"
|
||||
className="mt-1"
|
||||
/>
|
||||
</div>
|
||||
)}
|
||||
|
||||
<div>
|
||||
<Label htmlFor="ai-ocr-key">{t("settings.ai.ocrKeyLabel")}</Label>
|
||||
<PasswordInput
|
||||
id="ai-ocr-key"
|
||||
value={apiKey}
|
||||
onChange={(e) => setApiKey(e.target.value)}
|
||||
placeholder={
|
||||
userProvider?.hasOcrKey ? t("settings.ai.savedShort") : ""
|
||||
}
|
||||
className="mt-1"
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
|
||||
<div>
|
||||
<Button
|
||||
size="sm"
|
||||
className="min-h-11 sm:min-h-9"
|
||||
onClick={() => saveMutation.mutate()}
|
||||
disabled={saveMutation.isPending}
|
||||
>
|
||||
{saveMutation.isPending ? (
|
||||
<Loader2 className="h-4 w-4 animate-spin motion-reduce:animate-none" />
|
||||
) : (
|
||||
<Save className="h-4 w-4" />
|
||||
)}
|
||||
{t("settings.ai.saveCta")}
|
||||
</Button>
|
||||
</div>
|
||||
|
||||
{msg && (
|
||||
<p className={`text-xs ${ok ? "text-success" : "text-destructive"}`}>
|
||||
{msg}
|
||||
</p>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -29,18 +29,8 @@ export interface UserAIProvider {
|
||||
openaiKeyPreview: string | null;
|
||||
// v1.22 (#89) — per-user response timeout, in seconds (null = default).
|
||||
responseTimeoutSeconds: number | null;
|
||||
// v1.22 (#90) — dedicated document-scan (Lab-OCR) provider config.
|
||||
ocrEnabled: boolean;
|
||||
ocrProvider: string | null;
|
||||
ocrModel: string | null;
|
||||
ocrBaseUrl: string | null;
|
||||
hasOcrKey: boolean;
|
||||
ocrKeyPreview: string | null;
|
||||
}
|
||||
|
||||
/** v1.22 (#90) — providers the dedicated document-scan path accepts. */
|
||||
export const OCR_PROVIDER_TYPES = ["OPENAI", "ANTHROPIC", "LOCAL"] as const;
|
||||
|
||||
/**
|
||||
* v1.4.16 phase B2 — provider tags exposed to the UI. Mirrors
|
||||
* `PROVIDER_CHAIN_TYPES` server-side; kept as a const-array here to
|
||||
|
||||
@@ -1,120 +0,0 @@
|
||||
import { describe, it, expect, vi, beforeEach } from "vitest";
|
||||
|
||||
// v1.22 (#90) — dedicated document-scan provider resolution.
|
||||
|
||||
vi.mock("@/lib/db", () => ({
|
||||
prisma: {
|
||||
user: { findUnique: vi.fn() },
|
||||
appSettings: { findUnique: vi.fn() },
|
||||
},
|
||||
}));
|
||||
vi.mock("@/lib/crypto", () => ({
|
||||
decrypt: vi.fn((v: string) => `decrypted:${v}`),
|
||||
encrypt: vi.fn((v: string) => `encrypted:${v}`),
|
||||
}));
|
||||
vi.mock("@/lib/ai/codex-oauth", () => ({
|
||||
refreshDeviceTokens: vi.fn(),
|
||||
encryptCodexCreds: vi.fn(),
|
||||
decryptCodexCreds: vi.fn(),
|
||||
}));
|
||||
|
||||
import { resolveOcrProviderChain } from "../provider";
|
||||
import { prisma } from "@/lib/db";
|
||||
|
||||
describe("resolveOcrProviderChain", () => {
|
||||
beforeEach(() => {
|
||||
vi.resetAllMocks();
|
||||
});
|
||||
|
||||
it("uses the dedicated OCR provider when enabled with a key", async () => {
|
||||
// Single read: the OCR columns. The dedicated provider builds, so the main
|
||||
// chain is never resolved.
|
||||
vi.mocked(prisma.user.findUnique).mockResolvedValueOnce({
|
||||
aiOcrEnabled: true,
|
||||
aiOcrProvider: "OPENAI",
|
||||
aiOcrModel: "gpt-4o",
|
||||
aiOcrBaseUrl: null,
|
||||
aiOcrKeyEncrypted: "enc-key",
|
||||
} as never);
|
||||
|
||||
const res = await resolveOcrProviderChain("u1");
|
||||
|
||||
expect(res.dedicated).toBe(true);
|
||||
expect(res.ocrModelOverride).toBe("gpt-4o");
|
||||
expect(res.chain).toHaveLength(1);
|
||||
expect(res.chain[0].providerType).toBe("openai");
|
||||
// Only the OCR-column read happened — no main-chain resolution.
|
||||
expect(prisma.user.findUnique).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("maps LOCAL OCR provider to the local chain tag via base URL", async () => {
|
||||
vi.mocked(prisma.user.findUnique).mockResolvedValueOnce({
|
||||
aiOcrEnabled: true,
|
||||
aiOcrProvider: "LOCAL",
|
||||
aiOcrModel: "llava",
|
||||
aiOcrBaseUrl: "http://localhost:11434/v1",
|
||||
aiOcrKeyEncrypted: null,
|
||||
} as never);
|
||||
|
||||
const res = await resolveOcrProviderChain("u1");
|
||||
|
||||
expect(res.dedicated).toBe(true);
|
||||
expect(res.chain[0].providerType).toBe("local");
|
||||
});
|
||||
|
||||
it("falls back to the main chain when OCR is disabled", async () => {
|
||||
// First read: OCR columns (disabled). Second read: the main provider chain
|
||||
// resolution (no enabled chain / creds) → empty main chain.
|
||||
vi.mocked(prisma.user.findUnique)
|
||||
.mockResolvedValueOnce({
|
||||
aiOcrEnabled: false,
|
||||
aiOcrProvider: null,
|
||||
aiOcrModel: null,
|
||||
aiOcrBaseUrl: null,
|
||||
aiOcrKeyEncrypted: null,
|
||||
} as never)
|
||||
.mockResolvedValueOnce({
|
||||
aiProvider: null,
|
||||
aiModel: null,
|
||||
aiBaseUrl: null,
|
||||
aiAnthropicKeyEncrypted: null,
|
||||
aiLocalKeyEncrypted: null,
|
||||
aiOpenaiKeyEncrypted: null,
|
||||
aiProviderChain: null,
|
||||
} as never);
|
||||
|
||||
const res = await resolveOcrProviderChain("u1");
|
||||
|
||||
expect(res.dedicated).toBe(false);
|
||||
expect(res.ocrModelOverride).toBeNull();
|
||||
// The OCR-column read plus the main-chain resolution both ran (the
|
||||
// fallback path), so more than the single dedicated-path read happened.
|
||||
expect(
|
||||
vi.mocked(prisma.user.findUnique).mock.calls.length,
|
||||
).toBeGreaterThanOrEqual(2);
|
||||
});
|
||||
|
||||
it("falls back to the main chain when enabled but the key is missing", async () => {
|
||||
vi.mocked(prisma.user.findUnique)
|
||||
.mockResolvedValueOnce({
|
||||
aiOcrEnabled: true,
|
||||
aiOcrProvider: "OPENAI",
|
||||
aiOcrModel: "gpt-4o",
|
||||
aiOcrBaseUrl: null,
|
||||
aiOcrKeyEncrypted: null, // missing credential → cannot build
|
||||
} as never)
|
||||
.mockResolvedValueOnce({
|
||||
aiProvider: null,
|
||||
aiModel: null,
|
||||
aiBaseUrl: null,
|
||||
aiAnthropicKeyEncrypted: null,
|
||||
aiLocalKeyEncrypted: null,
|
||||
aiOpenaiKeyEncrypted: null,
|
||||
aiProviderChain: null,
|
||||
} as never);
|
||||
|
||||
const res = await resolveOcrProviderChain("u1");
|
||||
|
||||
expect(res.dedicated).toBe(false);
|
||||
});
|
||||
});
|
||||
@@ -314,114 +314,6 @@ export async function resolveProviderChain(
|
||||
return resolved;
|
||||
}
|
||||
|
||||
/**
|
||||
* v1.22 (#90) — dedicated document-scan (Lab-OCR) provider columns.
|
||||
*/
|
||||
type UserOcrRow = {
|
||||
aiOcrEnabled: boolean | null;
|
||||
aiOcrProvider: string | null;
|
||||
aiOcrModel: string | null;
|
||||
aiOcrBaseUrl: string | null;
|
||||
aiOcrKeyEncrypted: string | null;
|
||||
};
|
||||
|
||||
/**
|
||||
* Build the dedicated OCR provider instance from the per-user `aiOcr*` columns.
|
||||
* Returns the constructed instance paired with the chain tag the consent gate +
|
||||
* vision-capability resolver understand, or null when the OCR provider is
|
||||
* unconfigured / missing its credential (the caller then falls back to the main
|
||||
* provider chain so document scanning is never silently lost).
|
||||
*/
|
||||
function buildOcrProvider(
|
||||
row: UserOcrRow,
|
||||
): { providerType: ProviderChainType; instance: AIProvider } | null {
|
||||
if (!row.aiOcrEnabled) return null;
|
||||
const choice = row.aiOcrProvider?.toUpperCase();
|
||||
switch (choice) {
|
||||
case "ANTHROPIC": {
|
||||
if (!row.aiOcrKeyEncrypted) return null;
|
||||
return {
|
||||
providerType: "anthropic",
|
||||
instance: new AnthropicClient({
|
||||
apiKey: decrypt(row.aiOcrKeyEncrypted),
|
||||
model: row.aiOcrModel ?? "claude-sonnet-4-6",
|
||||
}),
|
||||
};
|
||||
}
|
||||
case "OPENAI": {
|
||||
if (!row.aiOcrKeyEncrypted) return null;
|
||||
return {
|
||||
providerType: "openai",
|
||||
instance: new OpenAIClient({
|
||||
apiKey: decrypt(row.aiOcrKeyEncrypted),
|
||||
model: row.aiOcrModel ?? "gpt-4o",
|
||||
baseUrl: "https://api.openai.com/v1",
|
||||
}),
|
||||
};
|
||||
}
|
||||
case "LOCAL": {
|
||||
if (!row.aiOcrBaseUrl) return null;
|
||||
return {
|
||||
providerType: "local",
|
||||
instance: new LocalOpenAICompatibleClient({
|
||||
apiKey: row.aiOcrKeyEncrypted ? decrypt(row.aiOcrKeyEncrypted) : null,
|
||||
model: row.aiOcrModel ?? "local-model",
|
||||
baseUrl: row.aiOcrBaseUrl,
|
||||
}),
|
||||
};
|
||||
}
|
||||
default:
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* v1.22 (#90) — resolve the provider chain for the Lab-OCR ingestion path.
|
||||
*
|
||||
* When the user enabled a dedicated document-scan provider (Settings → AI →
|
||||
* Document scanning), this returns a single-entry chain built from the `aiOcr*`
|
||||
* columns plus the OCR model that drives the vision-capability allowlist.
|
||||
* Otherwise (the default) it returns the MAIN provider chain unchanged, so the
|
||||
* OCR path behaves exactly as before the feature existed. A dedicated provider
|
||||
* that fails to build (missing key / base URL) also falls through to the main
|
||||
* chain so scanning is never silently dropped.
|
||||
*/
|
||||
export async function resolveOcrProviderChain(userId: string): Promise<{
|
||||
chain: ProviderChainResolved[];
|
||||
/** The model driving the dedicated provider's vision allowlist, else null. */
|
||||
ocrModelOverride: string | null;
|
||||
/** True when the dedicated document-scan provider served this chain. */
|
||||
dedicated: boolean;
|
||||
}> {
|
||||
const row = await prisma.user.findUnique({
|
||||
where: { id: userId },
|
||||
select: {
|
||||
aiOcrEnabled: true,
|
||||
aiOcrProvider: true,
|
||||
aiOcrModel: true,
|
||||
aiOcrBaseUrl: true,
|
||||
aiOcrKeyEncrypted: true,
|
||||
},
|
||||
});
|
||||
|
||||
if (row?.aiOcrEnabled) {
|
||||
const built = buildOcrProvider(row);
|
||||
if (built) {
|
||||
return {
|
||||
chain: [built],
|
||||
ocrModelOverride: row.aiOcrModel ?? null,
|
||||
dedicated: true,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
chain: await resolveProviderChain(userId),
|
||||
ocrModelOverride: null,
|
||||
dedicated: false,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Credential-presence subset of the `User` row that
|
||||
* `userRowHasProviderCredential` evaluates. Mirrors the columns
|
||||
|
||||
@@ -84,8 +84,6 @@ export const ENCRYPTED_COLUMNS: readonly EncryptedColumn[] = [
|
||||
{ model: "User", field: "aiAnthropicKeyEncrypted", kind: "string" },
|
||||
{ model: "User", field: "aiLocalKeyEncrypted", kind: "string" },
|
||||
{ model: "User", field: "aiOpenaiKeyEncrypted", kind: "string" },
|
||||
// v1.22 (#90) — dedicated document-scan (Lab-OCR) provider key.
|
||||
{ model: "User", field: "aiOcrKeyEncrypted", kind: "string" },
|
||||
{ model: "User", field: "insuranceNumberEncrypted", kind: "string" },
|
||||
// v1.23 — TOTP shared secret (second factor). String ciphertext like the
|
||||
// other User credential columns.
|
||||
|
||||
@@ -10,8 +10,9 @@
|
||||
* vision allowlist.
|
||||
*/
|
||||
import type { ProviderChainResolved } from "@/lib/ai/provider-runner";
|
||||
import { resolveProvider, resolveOcrProviderChain } from "@/lib/ai/provider";
|
||||
import { resolveProvider, resolveProviderChain } from "@/lib/ai/provider";
|
||||
import { resolveCodexVisionSlug } from "@/lib/ai/codex-client";
|
||||
import { RASTERIZATION_AVAILABLE } from "@/lib/documents/rasterize-pdf";
|
||||
import { prisma } from "@/lib/db";
|
||||
import {
|
||||
supportsPdfForProvider,
|
||||
@@ -89,18 +90,14 @@ export async function resolveVisionProvider(
|
||||
|
||||
const localOcrEnabled = userRow?.labsLocalOcrEnabled ?? false;
|
||||
|
||||
// v1.22 (#90) — resolve the OCR chain: the dedicated document-scan provider
|
||||
// when the user enabled one, else the main provider chain unchanged. When a
|
||||
// dedicated provider is active, ITS model drives the vision allowlist.
|
||||
const ocr = await resolveOcrProviderChain(userId);
|
||||
// The OCR path always uses the main configured provider chain — the user's
|
||||
// model drives the vision allowlist.
|
||||
const ctx: ModelContext = {
|
||||
userModel: ocr.dedicated
|
||||
? (ocr.ocrModelOverride ?? userRow?.aiModel ?? null)
|
||||
: (userRow?.aiModel ?? null),
|
||||
userModel: userRow?.aiModel ?? null,
|
||||
adminModel: settings?.adminAiModel ?? null,
|
||||
};
|
||||
|
||||
const resolvedChain = ocr.chain;
|
||||
const resolvedChain = await resolveProviderChain(userId);
|
||||
if (resolvedChain.length === 0) {
|
||||
// Mirror the Coach: fall back to the legacy single-provider resolution and
|
||||
// tag it as the admin-managed entry the consent gate recognises.
|
||||
@@ -148,9 +145,9 @@ export async function resolveTextProvider(
|
||||
chain: ProviderChainResolved[];
|
||||
pick: { entry: ProviderChainResolved; providerType: string } | null;
|
||||
}> {
|
||||
// v1.22 (#90) — use the dedicated document-scan provider when enabled, else
|
||||
// the main chain (the text-mode structuring pass needs no vision).
|
||||
const resolvedChain = (await resolveOcrProviderChain(userId)).chain;
|
||||
// The text-mode structuring pass needs no vision — any configured provider in
|
||||
// the main chain can structure the OCR'd text.
|
||||
const resolvedChain = await resolveProviderChain(userId);
|
||||
if (resolvedChain.length === 0) {
|
||||
const legacy = await resolveProvider(userId);
|
||||
if (legacy.type !== "none") {
|
||||
@@ -183,13 +180,18 @@ export async function resolveOcrCapability(
|
||||
);
|
||||
|
||||
// Prefer the native vision path whenever it is available — it is more
|
||||
// accurate and the client does not download the OCR WASM.
|
||||
// accurate and the client does not download the OCR WASM. PDFs are readable
|
||||
// whenever the picked provider natively supports them (Anthropic) OR the
|
||||
// server-side rasterizer is available (every other vision provider reads a
|
||||
// PDF via rasterized page images), so the scan-a-PDF affordance shows for a
|
||||
// codex / OAuth provider too. The extract route keeps the native-block vs
|
||||
// raster decision on `pick.pdfSupported` itself.
|
||||
if (pick) {
|
||||
return {
|
||||
available: true,
|
||||
mode: "vision",
|
||||
reason: null,
|
||||
pdfSupported: pick.pdfSupported,
|
||||
pdfSupported: pick.pdfSupported || RASTERIZATION_AVAILABLE,
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
@@ -35,7 +35,7 @@ const capabilityResponse = z
|
||||
.meta({
|
||||
id: "OcrCapabilityResponse",
|
||||
description:
|
||||
"Whether the caller's configured AI provider can ingest a lab report (drives the UI's scan affordance). `mode` is `vision` when the provider reads the image directly, `text` when the image is OCR'd in-browser and only the extracted text is sent (opt-in local OCR for text-only providers), or null when unavailable. `reason` explains an unavailable state; `pdfSupported` is true only for an Anthropic vision provider (native PDF). No provider call is made.",
|
||||
"Whether the caller's configured AI provider can ingest a lab report (drives the UI's scan affordance). `mode` is `vision` when the provider reads the image directly, `text` when the image is OCR'd in-browser and only the extracted text is sent (opt-in local OCR for text-only providers), or null when unavailable. `reason` explains an unavailable state; `pdfSupported` is true when the provider reads PDFs natively (Anthropic) or the server-side rasterizer can render the pages for any other vision provider. No provider call is made.",
|
||||
});
|
||||
|
||||
const extractConfidence = z.object({
|
||||
@@ -137,7 +137,7 @@ export const ocrPaths: NonNullable<ZodOpenApiObject["paths"]> = {
|
||||
tags: ["Labs"],
|
||||
summary: "Extract lab readings from a photo, PDF, or OCR'd text",
|
||||
description:
|
||||
"Read-only (NOT idempotent) extraction. Two modes by content-type. VISION (`multipart/form-data`): a `file` (JPEG/PNG/WebP, or PDF on an Anthropic vision provider; ≤ 12 MiB) is run through the user's vision-capable provider; the upload lives in memory only and is never persisted or logged. TEXT (`application/json`, opt-in local OCR): the browser OCR's the image (tesseract.js) and POSTs `{ mode: \"text\", text }` — only the extracted text reaches the server, so a text-only provider (e.g. ChatGPT-OAuth) reaches the same review/commit flow. Both modes are gated by AI consent, a 6/hour rate bucket, and the per-day token budget, and return proposed rows for the mandatory human review screen — nothing is written. Extracted content is treated as untrusted (prompt-injection); the review step is the safety boundary.",
|
||||
"Read-only (NOT idempotent) extraction. Two modes by content-type. VISION (`multipart/form-data`): a `file` (JPEG/PNG/WebP, or PDF — read natively on Anthropic, else rasterized to page images for any other vision provider; ≤ 12 MiB) is run through the user's vision-capable provider; the upload lives in memory only and is never persisted or logged. TEXT (`application/json`, opt-in local OCR): the browser OCR's the image (tesseract.js) and POSTs `{ mode: \"text\", text }` — only the extracted text reaches the server, so a text-only provider (e.g. ChatGPT-OAuth) reaches the same review/commit flow. Both modes are gated by AI consent, a 6/hour rate bucket, and the per-day token budget, and return proposed rows for the mandatory human review screen — nothing is written. Extracted content is treated as untrusted (prompt-injection); the review step is the safety boundary.",
|
||||
requestBody: {
|
||||
required: true,
|
||||
content: {
|
||||
|
||||
@@ -341,27 +341,6 @@ const aiProviderResponse = z
|
||||
.describe(
|
||||
"v1.22 (#89) — per-user response timeout for AI generation, in seconds (10–600). Null = the built-in comprehensive-briefing default (~120 s). Mainly for slow local/self-hosted backends.",
|
||||
),
|
||||
ocrEnabled: z
|
||||
.boolean()
|
||||
.describe(
|
||||
"v1.22 (#90) — whether a dedicated document-scan (Lab-OCR) provider is active. When false the OCR path uses the main provider chain.",
|
||||
),
|
||||
ocrProvider: z
|
||||
.enum(["OPENAI", "ANTHROPIC", "LOCAL"])
|
||||
.nullable()
|
||||
.describe("The dedicated document-scan provider, or null when unset."),
|
||||
ocrModel: z.string().nullable(),
|
||||
ocrBaseUrl: z
|
||||
.string()
|
||||
.nullable()
|
||||
.describe("Custom base URL (LOCAL OCR provider only); null otherwise."),
|
||||
hasOcrKey: z.boolean(),
|
||||
ocrKeyPreview: z
|
||||
.string()
|
||||
.nullable()
|
||||
.describe(
|
||||
"`...` + last 4 chars of the stored document-scan provider key, or null.",
|
||||
),
|
||||
})
|
||||
.meta({
|
||||
id: "AiProviderResponse",
|
||||
|
||||
Reference in new issue
Block a user