feat(labs): make the lab-scan limit operator-tunable

A self-hoster scanning a stack of lab reports ran into a fixed six scans
an hour with no way to raise it. LABS_OCR_LIMIT_PER_HOUR now sets the
per-user ceiling (default 6, clamped to 1-1000, falling back to the
default when unset or not a number), the same shape as
DOCUMENT_AI_LIMIT_PER_HOUR. It is on the compose whitelist, in the
production example, the env manifest and the AI provider guide.

The route also charged the slot before it had looked at the request, so
a malformed body, an unsupported or oversized file, a PDF that could not
be rendered or a scan the daily budget refused still used up the hour.
Those failures hand the slot back now; a scan that reached the provider
keeps it. The 429 carries meta.retryAt. Refs #985
This commit is contained in:
Marc-André Bombeck committed 2026-09-15 23:01:10 +02:00
1 parent 8457e4ffcb
commit ed8a93afc2
10 files changed
+304 -37

No files matched your search

+7
View File
@@ -290,6 +290,13 @@ API_TOKEN_HMAC_KEY="<openssl rand -hex 32>"
# docs/integrations/ai-providers.md.
# DOCUMENT_AI_LIMIT_PER_HOUR=""
# Per-user hourly ceiling for lab-report scans (optional). The photo/PDF scan
# and the browser-OCR text scan share one bucket per user. Default 6, clamped
# to 1-1000. Raise it when users scan a stack of reports at once against a
# generous provider budget; cost stays bounded by the daily AI budget. See
# docs/integrations/ai-providers.md.
# LABS_OCR_LIMIT_PER_HOUR=""
# Extra hosts the ADMIN-configured AI base URL may point at, on top of the
# built-in public providers (comma-separated bare hostnames). Set it when you
# run a gateway this instance should trust. It widens the admin setting only;
+6
View File
@@ -290,6 +290,12 @@ x-healthlog-env: &healthlog_env
# (the `environment:` block is a whitelist). See
# docs/integrations/ai-providers.md.
DOCUMENT_AI_LIMIT_PER_HOUR: "${DOCUMENT_AI_LIMIT_PER_HOUR:-}"
# Per-user hourly ceiling for lab-report scans (photo/PDF and browser-OCR
# text share one bucket; optional; default 6, clamped 1-1000). A scan that
# fails before the provider is called hands its slot back. Listed here so an
# operator value in .env reaches the container. See
# docs/integrations/ai-providers.md.
LABS_OCR_LIMIT_PER_HOUR: "${LABS_OCR_LIMIT_PER_HOUR:-}"
# v1.17.1 — SMTP transport for the email notification channel (optional,
# all-or-none: HOST + PORT + FROM enable it; USER/PASS optional; SECURE
# toggles implicit TLS). Listed here so operator values in .env reach the
+4 -2
View File
@@ -13489,8 +13489,10 @@ paths:
provider; ≤ 12 MiB) is run through the user's vision-capable provider; the upload lives in memory only and is
never persisted or logged. TEXT (`application/json`, opt-in local OCR): the browser OCR's the image
(tesseract.js) and POSTs `{ mode: \"text\", text }` — only the extracted text reaches the server, so a text-only
provider (e.g. ChatGPT-OAuth) reaches the same review/commit flow. Both modes are gated by AI consent, a 6/hour
rate bucket, and the per-day token budget, and return proposed rows for the mandatory human review screen —
provider (e.g. ChatGPT-OAuth) reaches the same review/commit flow. Both modes are gated by AI consent, a
per-user hourly rate bucket (default 6, operator-tunable via `LABS_OCR_LIMIT_PER_HOUR`; a slot is only consumed
when the scan reaches the provider, and the 429 carries `Retry-After`, the `X-RateLimit-*` triple and
`meta.retryAt`), and the per-day token budget, and return proposed rows for the mandatory human review screen —
nothing is written. Extracted content is treated as untrusted (prompt-injection); the review step is the safety
boundary."
requestBody:
+9 -1
View File
@@ -412,7 +412,15 @@ many documents against a generous provider budget; lower it on a
tight one. The 429 names the actual wait (the window is an hour) and
carries the reset instant in `X-RateLimit-Reset`.
All three variables must also be listed in the compose
`LABS_OCR_LIMIT_PER_HOUR` (default `6`, clamped to 1–1000) does the
same for lab-report scans. The photo or PDF scan and the browser-OCR
text scan share one bucket per user, and a scan that fails before the
provider is called (malformed body, file too large or not an image or
PDF, a PDF that cannot be rendered, exhausted daily budget) hands its
slot back. Raise it when users scan a stack of reports at once; the
daily AI budget still bounds the cost.
All four variables must also be listed in the compose
`environment:` whitelist to reach the container; the shipped
`docker-compose.yml` lists them.
+4
View File
@@ -137,6 +137,10 @@
{
"name": "DOCUMENT_AI_LIMIT_PER_HOUR",
"purpose": "Per-user hourly ceiling for the document-AI actions (read/index, extract, suggest, summarise share one bucket). Default 6, clamped to 1-1000."
},
{
"name": "LABS_OCR_LIMIT_PER_HOUR",
"purpose": "Per-user hourly ceiling for lab-report scans (photo/PDF and browser-OCR text share one bucket). Default 6, clamped to 1-1000."
}
]
},
@@ -37,7 +37,8 @@ vi.mock("@/lib/ai/consent-guard", () => ({
assertConsentForChain: vi.fn().mockResolvedValue(undefined),
}));
vi.mock("@/lib/rate-limit", () => ({
checkRateLimit: vi.fn().mockResolvedValue({ allowed: true }),
checkRateLimit: vi.fn(),
refundRateLimit: vi.fn().mockResolvedValue(undefined),
rateLimitHeaders: vi.fn(() => ({})),
}));
vi.mock("@/lib/ai/coach/budget", () => ({
@@ -65,6 +66,7 @@ import {
} from "@/lib/labs/ocr-capability";
import { rasterizePdf } from "@/lib/documents/rasterize-pdf";
import { reserveBudget, reconcileSpend } from "@/lib/ai/coach/budget";
import { checkRateLimit, refundRateLimit } from "@/lib/rate-limit";
import { OcrExtractError, runOcrExtraction } from "@/lib/labs/ocr-extract";
const SESSION_OK = {
@@ -82,6 +84,12 @@ function textReq(text = "Glucose 95 mg/dL"): Request {
beforeEach(() => {
vi.clearAllMocks();
vi.unstubAllEnvs();
vi.mocked(checkRateLimit).mockResolvedValue({
allowed: true,
remaining: 5,
resetAt: Date.now() + 3_600_000,
} as never);
vi.mocked(requireAuth).mockResolvedValue(SESSION_OK as never);
vi.mocked(prisma.user.findUnique).mockResolvedValue({
labsLocalOcrEnabled: true,
@@ -217,6 +225,25 @@ describe("POST /api/labs/ocr/extract — vision PDF rasterization", () => {
);
});
it("hands the slot back when the upload is not an image or PDF", async () => {
const form = new FormData();
form.append(
"file",
new File([new TextEncoder().encode("plain text")], "note.txt", {
type: "text/plain",
}),
);
const res = await POST(
new Request("http://localhost/api/labs/ocr/extract", {
method: "POST",
body: form,
}) as never,
);
expect(res.status).toBe(415);
expect(refundRateLimit).toHaveBeenCalledWith("labs-ocr:user-1");
expect(runOcrExtraction).not.toHaveBeenCalled();
});
it("falls back to pdfNeedsAnthropic when rasterization fails", async () => {
vi.mocked(rasterizePdf).mockResolvedValue({
ok: false,
@@ -239,3 +266,71 @@ describe("POST /api/labs/ocr/extract — vision PDF rasterization", () => {
);
});
});
describe("POST /api/labs/ocr/extract — the hourly scan bucket", () => {
it("charges the ceiling the operator set", async () => {
vi.stubEnv("LABS_OCR_LIMIT_PER_HOUR", "40");
vi.mocked(runOcrExtraction).mockResolvedValue({ rows: [] } as never);
const res = await POST(textReq() as never);
expect(res.status).toBe(200);
expect(checkRateLimit).toHaveBeenCalledWith(
"labs-ocr:user-1",
40,
60 * 60 * 1000,
);
});
it("answers 429 with the reset instant when the bucket is spent", async () => {
vi.mocked(checkRateLimit).mockResolvedValue({
allowed: false,
remaining: 0,
resetAt: Date.parse("2026-09-15T15:00:00Z"),
} as never);
const res = await POST(textReq() as never);
expect(res.status).toBe(429);
const body = (await res.json()) as {
meta: { errorCode: string; retryAt: string };
};
expect(body.meta.errorCode).toBe("labs.ocr.rateLimited");
expect(body.meta.retryAt).toBe("2026-09-15T15:00:00.000Z");
expect(runOcrExtraction).not.toHaveBeenCalled();
});
it("hands the slot back for a malformed text body", async () => {
const res = await POST(
new Request("http://localhost/api/labs/ocr/extract", {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify({ mode: "text" }),
}) as never,
);
expect(res.status).toBe(422);
expect(refundRateLimit).toHaveBeenCalledWith("labs-ocr:user-1");
});
it("hands the slot back when the daily budget turns the scan away", async () => {
vi.mocked(reserveBudget).mockResolvedValue({
allowed: false,
reserved: 0,
totalAfter: 999_999,
owner: "operator",
operatorAfter: 999_999,
} as never);
const res = await POST(textReq() as never);
expect(res.status).toBe(429);
expect(refundRateLimit).toHaveBeenCalledWith("labs-ocr:user-1");
expect(runOcrExtraction).not.toHaveBeenCalled();
});
it("keeps the slot once the provider was actually called", async () => {
vi.mocked(runOcrExtraction).mockRejectedValue(new Error("provider down"));
const failed = await POST(textReq() as never);
expect(failed.status).toBe(502);
expect(refundRateLimit).not.toHaveBeenCalled();
vi.mocked(runOcrExtraction).mockResolvedValue({ rows: [] } as never);
const ok = await POST(textReq() as never);
expect(ok.status).toBe(200);
expect(refundRateLimit).not.toHaveBeenCalled();
});
});
+27 -32
View File
@@ -18,8 +18,10 @@
* `labsLocalOcrEnabled` preference.
*
* Guards mirror the Coach's discipline in both modes:
* requireAuth → resolve provider → assertConsentForChain → rate-limit (6/h)
* → reserveBudget → run extraction → reconcile budget.
* requireAuth → resolve provider → assertConsentForChain → rate-limit
* (`LABS_OCR_LIMIT_PER_HOUR`, default 6/h) → reserveBudget → run extraction
* → reconcile budget. A slot is charged early so a 429 stays cheap, and it is
* handed back when the scan fails before the provider is called.
*
* Extracted text is UNTRUSTED (prompt-injection): the server never acts on an
* instruction inside the document — the human review step is the safety
@@ -58,15 +60,15 @@ import {
readBoundedBody,
} from "@/lib/labs/ocr-upload";
import { annotate } from "@/lib/logging/context";
import { checkRateLimit, rateLimitHeaders } from "@/lib/rate-limit";
import {
checkLabsOcrRateLimit,
labsOcrRateLimited,
refundLabsOcrSlot,
} from "@/lib/labs/ocr-rate-limit";
import { ocrTextExtractSchema } from "@/lib/validations/labs-ocr";
export const dynamic = "force-dynamic";
/** 6 extraction calls per hour — they are expensive (vision) / metered (text). */
const EXTRACT_LIMIT_PER_HOUR = 6;
const EXTRACT_WINDOW_MS = 60 * 60 * 1000;
/** OCR'd text is bounded in the schema; cap the JSON body proportionally. */
const TEXT_BODY_MAX_BYTES = 512 * 1024;
@@ -138,29 +140,23 @@ async function handleTextExtract(
// egress is the user's own act (the toggle is the consent for local OCR).
await assertConsentForChain({ userId, chain, surface: "insights" });
const rl = await checkRateLimit(
`labs-ocr:${userId}`,
EXTRACT_LIMIT_PER_HOUR,
EXTRACT_WINDOW_MS,
);
const rl = await checkLabsOcrRateLimit(userId);
if (!rl.allowed) {
annotate({ action: { name: "labs.ocr.rateLimited" } });
const response = apiError("Too many scans. Try again later.", 429, {
errorCode: "labs.ocr.rateLimited",
});
for (const [k, v] of Object.entries(rateLimitHeaders(rl))) {
response.headers.set(k, v);
}
return response;
return labsOcrRateLimited(rl);
}
const { data: body, error: jsonError } = await safeJson(request, {
maxBytes: TEXT_BODY_MAX_BYTES,
});
if (jsonError) return jsonError;
if (jsonError) {
await refundLabsOcrSlot(userId);
return jsonError;
}
const parsed = ocrTextExtractSchema.safeParse(body);
if (!parsed.success) {
await refundLabsOcrSlot(userId);
return apiValidationError(
"Invalid OCR text payload",
sanitiseZodIssues(parsed.error.issues),
@@ -192,6 +188,7 @@ async function handleTextExtract(
action: { name: "labs.ocr.budget.exceeded" },
meta: { totalAfter: reservation.totalAfter, mode: "text" },
});
await refundLabsOcrSlot(userId);
return apiError("Your AI usage budget for today is reached.", 429, {
errorCode: "labs.ocr.budgetExceeded",
});
@@ -263,20 +260,10 @@ async function handleVisionExtract(
await assertConsentForChain({ userId, chain, surface: "insights" });
// 3. Rate-limit (vision calls are costly).
const rl = await checkRateLimit(
`labs-ocr:${userId}`,
EXTRACT_LIMIT_PER_HOUR,
EXTRACT_WINDOW_MS,
);
const rl = await checkLabsOcrRateLimit(userId);
if (!rl.allowed) {
annotate({ action: { name: "labs.ocr.rateLimited" } });
const response = apiError("Too many scans. Try again later.", 429, {
errorCode: "labs.ocr.rateLimited",
});
for (const [k, v] of Object.entries(rateLimitHeaders(rl))) {
response.headers.set(k, v);
}
return response;
return labsOcrRateLimited(rl);
}
// 4. Reserve the day's budget BEFORE the provider call (atomic, TOCTOU-safe).
@@ -297,6 +284,7 @@ async function handleVisionExtract(
action: { name: "labs.ocr.budget.exceeded" },
meta: { totalAfter: reservation.totalAfter },
});
await refundLabsOcrSlot(userId);
return apiError("Your AI usage budget for today is reached.", 429, {
errorCode: "labs.ocr.budgetExceeded",
});
@@ -317,6 +305,7 @@ async function handleVisionExtract(
servedBy: null,
reservedOwner: reservation.owner,
});
await refundLabsOcrSlot(userId);
return apiError("File is too large (max 12 MB).", 413, {
errorCode: "labs.ocr.fileTooLarge",
});
@@ -333,6 +322,7 @@ async function handleVisionExtract(
servedBy: null,
reservedOwner: reservation.owner,
});
await refundLabsOcrSlot(userId);
if (err instanceof BodyTooLargeError) {
annotate({
action: { name: "labs.ocr.fileRejected" },
@@ -351,6 +341,7 @@ async function handleVisionExtract(
servedBy: null,
reservedOwner: reservation.owner,
});
await refundLabsOcrSlot(userId);
return apiError("Field 'file' must be a file", 422);
}
@@ -362,6 +353,7 @@ async function handleVisionExtract(
servedBy: null,
reservedOwner: reservation.owner,
});
await refundLabsOcrSlot(userId);
return apiError("Failed to read uploaded file", 400);
}
@@ -376,6 +368,7 @@ async function handleVisionExtract(
servedBy: null,
reservedOwner: reservation.owner,
});
await refundLabsOcrSlot(userId);
return apiError("Upload a JPEG, PNG, WebP, or PDF.", 415, {
errorCode: "labs.ocr.fileType",
});
@@ -417,6 +410,7 @@ async function handleVisionExtract(
servedBy: null,
reservedOwner: reservation.owner,
});
await refundLabsOcrSlot(userId);
return apiError(
"Couldn't read this PDF; upload a photo instead.",
422,
@@ -485,6 +479,7 @@ async function handleVisionExtract(
servedBy: null,
reservedOwner: reservation.owner,
}).catch(() => {});
await refundLabsOcrSlot(userId);
throw err;
}
}
@@ -0,0 +1,75 @@
import { afterEach, describe, expect, it, vi } from "vitest";
vi.mock("@/lib/rate-limit", () => ({
checkRateLimit: vi.fn(),
refundRateLimit: vi.fn(),
rateLimitHeaders: vi.fn(() => ({ "Retry-After": "1800" })),
}));
import {
checkLabsOcrRateLimit,
labsOcrRateLimited,
refundLabsOcrSlot,
resolveLabsOcrLimitPerHour,
} from "../ocr-rate-limit";
import { checkRateLimit, refundRateLimit } from "@/lib/rate-limit";
afterEach(() => {
vi.unstubAllEnvs();
vi.clearAllMocks();
});
describe("LABS_OCR_LIMIT_PER_HOUR", () => {
it("defaults to 6 when unset or not a number", () => {
vi.stubEnv("LABS_OCR_LIMIT_PER_HOUR", "");
expect(resolveLabsOcrLimitPerHour()).toBe(6);
vi.stubEnv("LABS_OCR_LIMIT_PER_HOUR", "lots");
expect(resolveLabsOcrLimitPerHour()).toBe(6);
});
it("uses an operator value inside the range", () => {
vi.stubEnv("LABS_OCR_LIMIT_PER_HOUR", "40");
expect(resolveLabsOcrLimitPerHour()).toBe(40);
});
it("clamps to 1-1000 so a typo can neither switch scans off nor remove the cap", () => {
vi.stubEnv("LABS_OCR_LIMIT_PER_HOUR", "0");
expect(resolveLabsOcrLimitPerHour()).toBe(1);
vi.stubEnv("LABS_OCR_LIMIT_PER_HOUR", "-5");
expect(resolveLabsOcrLimitPerHour()).toBe(1);
vi.stubEnv("LABS_OCR_LIMIT_PER_HOUR", "50000");
expect(resolveLabsOcrLimitPerHour()).toBe(1000);
});
it("charges the existing bucket key with the resolved ceiling and an hour window", async () => {
vi.stubEnv("LABS_OCR_LIMIT_PER_HOUR", "25");
await checkLabsOcrRateLimit("user-1");
expect(checkRateLimit).toHaveBeenCalledWith(
"labs-ocr:user-1",
25,
60 * 60 * 1000,
);
});
it("refunds the same bucket and never throws when the refund fails", async () => {
vi.mocked(refundRateLimit).mockRejectedValueOnce(new Error("db down"));
await expect(refundLabsOcrSlot("user-1")).resolves.toBeUndefined();
expect(refundRateLimit).toHaveBeenCalledWith("labs-ocr:user-1");
});
it("names the reset instant on the 429", async () => {
const resetAt = Date.parse("2026-09-15T15:00:00Z");
const res = labsOcrRateLimited({
allowed: false,
remaining: 0,
resetAt,
} as never);
expect(res.status).toBe(429);
expect(res.headers.get("Retry-After")).toBe("1800");
const body = (await res.json()) as {
meta: { errorCode: string; retryAt: string };
};
expect(body.meta.errorCode).toBe("labs.ocr.rateLimited");
expect(body.meta.retryAt).toBe("2026-09-15T15:00:00.000Z");
});
});
+75
View File
@@ -0,0 +1,75 @@
import { apiError } from "@/lib/api-response";
import {
checkRateLimit,
rateLimitHeaders,
refundRateLimit,
type RateLimitResult,
} from "@/lib/rate-limit";
/**
* The lab-report scan (`POST /api/labs/ocr/extract`, both the vision and the
* browser-OCR text mode) charges one per-user hourly bucket. The ceiling is
* operator-tunable via `LABS_OCR_LIMIT_PER_HOUR` (default 6, clamped to
* 1-1000 so a typo can neither switch the scan off nor remove the cap), the
* same posture as `DOCUMENT_AI_LIMIT_PER_HOUR` for the document vault. Unset or
* non-numeric falls back to the default. Cost is bounded separately by the
* daily AI budget; this bucket only bounds the request rate.
*/
const LABS_OCR_LIMIT_DEFAULT = 6;
const LABS_OCR_LIMIT_MIN = 1;
const LABS_OCR_LIMIT_MAX = 1000;
export function resolveLabsOcrLimitPerHour(): number {
const raw = process.env.LABS_OCR_LIMIT_PER_HOUR;
if (!raw) return LABS_OCR_LIMIT_DEFAULT;
const parsed = Number.parseInt(raw, 10);
if (!Number.isFinite(parsed)) return LABS_OCR_LIMIT_DEFAULT;
return Math.min(LABS_OCR_LIMIT_MAX, Math.max(LABS_OCR_LIMIT_MIN, parsed));
}
export const LABS_OCR_WINDOW_MS = 60 * 60 * 1000;
/** Bucket key prefix. Unchanged, so a running window survives the upgrade. */
export const LABS_OCR_BUCKET = "labs-ocr";
/** Charge one slot of the per-user lab-scan bucket. */
export async function checkLabsOcrRateLimit(
userId: string,
): Promise<RateLimitResult> {
return checkRateLimit(
`${LABS_OCR_BUCKET}:${userId}`,
resolveLabsOcrLimitPerHour(),
LABS_OCR_WINDOW_MS,
);
}
/**
* Give a charged slot back when the scan failed BEFORE the provider was
* called: a malformed body, a file that is too large or not an image or PDF,
* a PDF that could not be rendered, or a daily budget that turned it away.
* The bucket is charged early so a 429 stays cheap, but a slot must stand for
* a scan that was actually sent. Best-effort: a failed refund never masks the
* response the caller is about to return.
*/
export async function refundLabsOcrSlot(userId: string): Promise<void> {
try {
await refundRateLimit(`${LABS_OCR_BUCKET}:${userId}`);
} catch {
// The original response matters more than the bookkeeping.
}
}
/**
* The 429 for the lab-scan bucket. Carries the standard `X-RateLimit-*`
* headers and mirrors the reset instant into `meta.retryAt` (ISO 8601) so the
* client can name the actual wait.
*/
export function labsOcrRateLimited(rl: RateLimitResult): Response {
const response = apiError("Too many scans. Try again later.", 429, {
errorCode: "labs.ocr.rateLimited",
retryAt: new Date(rl.resetAt).toISOString(),
});
for (const [k, v] of Object.entries(rateLimitHeaders(rl))) {
response.headers.set(k, v);
}
return response;
}
+1 -1
View File
@@ -165,7 +165,7 @@ export const ocrPaths: NonNullable<ZodOpenApiObject["paths"]> = {
tags: ["Labs"],
summary: "Extract lab readings from a photo, PDF, or OCR'd text",
description:
"Read-only (NOT idempotent) extraction. Two modes by content-type. VISION (`multipart/form-data`): a `file` (JPEG/PNG/WebP, or PDF — read natively on Anthropic, else rasterized to page images for any other vision provider; ≤ 12 MiB) is run through the user's vision-capable provider; the upload lives in memory only and is never persisted or logged. TEXT (`application/json`, opt-in local OCR): the browser OCR's the image (tesseract.js) and POSTs `{ mode: \"text\", text }` — only the extracted text reaches the server, so a text-only provider (e.g. ChatGPT-OAuth) reaches the same review/commit flow. Both modes are gated by AI consent, a 6/hour rate bucket, and the per-day token budget, and return proposed rows for the mandatory human review screen — nothing is written. Extracted content is treated as untrusted (prompt-injection); the review step is the safety boundary.",
"Read-only (NOT idempotent) extraction. Two modes by content-type. VISION (`multipart/form-data`): a `file` (JPEG/PNG/WebP, or PDF — read natively on Anthropic, else rasterized to page images for any other vision provider; ≤ 12 MiB) is run through the user's vision-capable provider; the upload lives in memory only and is never persisted or logged. TEXT (`application/json`, opt-in local OCR): the browser OCR's the image (tesseract.js) and POSTs `{ mode: \"text\", text }` — only the extracted text reaches the server, so a text-only provider (e.g. ChatGPT-OAuth) reaches the same review/commit flow. Both modes are gated by AI consent, a per-user hourly rate bucket (default 6, operator-tunable via `LABS_OCR_LIMIT_PER_HOUR`; a slot is only consumed when the scan reaches the provider, and the 429 carries `Retry-After`, the `X-RateLimit-*` triple and `meta.retryAt`), and the per-day token budget, and return proposed rows for the mandatory human review screen — nothing is written. Extracted content is treated as untrusted (prompt-injection); the review step is the safety boundary.",
requestBody: {
required: true,
content: {