feat(labs): unlock native Codex vision for lab OCR

Image input rides the ChatGPT-OAuth codex/responses endpoint via the
input_image content block (protocol spec 2b) — no vision API key. The
codex client folds params.images into input_image blocks alongside the
input_text; text-only calls keep the prior byte-identical wire. The
vision gate now resolves codex capability against the working slug
(GPT-5.x / gpt-4o-class multimodal) instead of a blanket false, with the
OCR capability resolver feeding the codex slug rather than the user's
aiModel. A non-multimodal slug or an image-block rejection degrades to a
clean extract failure / local-OCR text mode.
This commit is contained in:
Marc-André Bombeck committed 2026-06-20 18:59:37 +02:00
1 parent 4aebaaf399
commit 178d4d7610
5 files changed
+156 -5

No files matched your search

+31 -2
View File
@@ -56,6 +56,31 @@ function openaiSupportsVision(model: string): boolean {
);
}
/**
* Codex (ChatGPT-OAuth) vision models. Image INPUT is reachable over the
* `codex/responses` endpoint (docs/codex-protocol-spec.md §2b) on a multimodal
* slug — no API key needed. The accepted GPT-5.x line is text+vision; the older
* `-codex` specialist slugs and the legacy `gpt-4`-class non-multimodal pins
* are text-only. The slug we test is the codex client's resolved working slug
* (cached or chain head), so a slug rotation onto a non-vision model degrades
* to false rather than 400-ing the user at the image block.
*
* `gpt-4o` / `o*` are folded in for completeness (a future codex slug could
* land there), reusing the same multimodal families the OpenAI gate accepts.
*/
function codexSupportsVision(model: string): boolean {
const m = model.toLowerCase();
return (
m.startsWith("gpt-5") || // GPT-5.x line is multimodal (text + vision)
m.startsWith("gpt-4o") ||
m.startsWith("gpt-4.1") ||
m.startsWith("gpt-4-turbo") ||
m.startsWith("o1") ||
m.startsWith("o3") ||
m.startsWith("o4")
);
}
/**
* Logical provider tags the OCR path reasons about. Mirrors the
* `ProviderChainType` vocabulary plus the legacy `admin-key` single-provider
@@ -79,8 +104,11 @@ export type VisionProviderType =
* configuring a self-hosted model and we cannot sniff
* its capabilities; a non-vision local model surfaces
* a clear extract failure rather than a silent gate.
* - codex / none → false (no vision over the Codex wire; nothing
* configured).
* - codex → vision when the resolved codex slug is a
* multimodal model (GPT-5.x / gpt-4o-class). Image
* input rides the ChatGPT plan — no API key. A
* non-multimodal slug (or unknown) → false.
* - none → false (nothing configured).
*/
export function supportsVisionForConfig(
providerType: VisionProviderType,
@@ -96,6 +124,7 @@ export function supportsVisionForConfig(
case "local":
return true;
case "codex":
return model ? codexSupportsVision(model) : false;
case "none":
default:
return false;