Surface what an OpenAI-compatible gateway actually refused

A gateway can answer a failed generation with HTTP 200 and put the reason
in the body: a top-level `error` object and no `choices`, or
`choices[0].error` beside `finish_reason: "error"` when a generation dies
mid-flight. Both fell through to the empty-content throw, so a dead credit
balance, a data-policy block, a rate limit and a model that is down all
recorded the same hop: status 0, reason "returned empty content", no
excerpt. Nothing on the provider health card told them apart.

Raise the shape a non-2xx reply raises instead, with the gateway's own
message leading and its `error.code` as the status when that code is an
HTTP error status. The chain classifier, the health ledger and the card
then see 402 / 403 / 429 / 503 and treat each of them the way they treat
the same status on the wire. Message and excerpt are bounded and
key-redacted by the same helper the non-2xx path uses, so remote text is
no more trusted here than it is there.

Gated on the gateway tag: api.openai.com signals every failure with a
status, so the pinned tags parse exactly as before, and a genuine empty
reply stays the empty-content throw it was. The local-endpoint provider
had the same hole on its buffered path, its buffered-reply fallback and
its stream, and gets the same treatment. The body classifier learned the
two account-side refusals a gateway words in its own vocabulary rather
than the vendor's.
This commit is contained in:
Marc-André Bombeck committed 2026-09-05 17:56:47 +02:00
1 parent 0913e8f2b7
commit 703ca2744e
8 files changed
+679 -10

No files matched your search

+10
View File
@@ -21,6 +21,16 @@
uploads.** Recording a measurement, editing a medication, restoring a
backup — anything that wrote — failed from the moment the app started
labelling writes this way in v1.37.18. Those all work again.
- **A gateway that refused the call reported nothing but "empty content".**
Some OpenAI-compatible gateways, OpenRouter among them, answer a refused
generation with HTTP 200 and the reason inside the body. The provider health
card therefore recorded status 0 and "returned empty content" whether the
account was out of credits, blocked by its own data policy, rate-limited, or
pointed at a model that was down, and no excerpt survived to tell them
apart. The gateway's own status and message now reach the card and the
fallback chain, which treats each of them exactly as it treats the same
status on the wire. The local-endpoint provider had the same blind spot on
its buffered and its streaming path and got the same treatment.
- **The import guide link under Settings → Import went the long way round.**
Both the CSV and the JSON card now link straight to the guide instead of
through a redirect.
+9
View File
@@ -197,6 +197,15 @@ JSON surfaces send the standard
the field is detected on the first refusal and retried without it,
so strict gateways and older local servers both work unmodified.
**When the gateway refuses.** OpenRouter answers a refused generation
with HTTP 200 and the reason inside the body, so every refusal used to
reach the provider health card as an empty reply with no status. It now
carries the gateway's own status and message — 402 out of credits, 403
blocked, 429 rate-limited, 502 model down. The two account settings
behind most refusals are an empty credit balance and a data-policy or
zero-retention restriction that leaves no eligible provider for the
model you picked; both are changed in the gateway account, not here.
**The Local provider still works for this** and nothing about it
changed — it speaks the same wire and honours the same base URL. Pick
the gateway provider when the endpoint is a gateway (it says so in
@@ -72,6 +72,23 @@ describe("upstream error-body classification", () => {
);
});
it("names a gateway's own wording for the two account-side refusals", () => {
// A gateway meters prepaid credit and enforces the account's data policy
// in its own vocabulary, not OpenAI's. Both used to land in the
// unclassified bucket, which is exactly the pair a person needs told
// apart: top the account up, or widen the routing policy.
expect(
classifyErrorBody(
'{"error":{"code":402,"message":"Insufficient credits"}}',
),
).toBe("classified:quota_or_billing");
expect(
classifyErrorBody(
'{"error":{"code":403,"message":"No endpoints found matching your data policy"}}',
),
).toBe("classified:no_eligible_provider");
});
it("passes through absence honestly", () => {
expect(classifyErrorBody(null)).toBeNull();
expect(classifyErrorBody("")).toBeNull();
@@ -0,0 +1,455 @@
import { beforeEach, describe, expect, it, vi } from "vitest";
/**
* A gateway that refuses inside a 200.
*
* OpenRouter — and every gateway that copies its wire — answers a
* provider-side refusal with HTTP 200 and the failure in the body: a
* top-level `error` object and no `choices`, or `choices[0].error` beside
* `finish_reason: "error"` when a generation dies mid-flight. Both used to
* reach the empty-content throw, so "no credits", "blocked by the account's
* data policy" and "the model is down" all surfaced identically: status 0,
* reason "returned empty content", no excerpt. Nobody reading the provider
* card could act on that.
*
* These cases pin the whole path: the client raises the shape a non-2xx reply
* raises, the chain runner carries the real status into the hop, the health
* ledger records it, and the pinned `api.openai.com` tags parse exactly as
* they did.
*/
// safeFetch's requirePublicHost path runs through undici's own `fetch`.
// Delegate it to the global stub so the interception still applies.
vi.mock("undici", async (importOriginal) => {
const actual = await importOriginal<typeof import("undici")>();
return {
...actual,
fetch: (input: unknown, init?: unknown) =>
(globalThis.fetch as unknown as (i: unknown, n?: unknown) => unknown)(
input,
init,
),
};
});
vi.mock("@/lib/logging/context", () => ({ annotate: vi.fn() }));
// The chain runner reaches for Postgres on both of these at import time; the
// chain semantics under test need neither.
vi.mock("../coach/budget", async () => {
const actual =
await vi.importActual<typeof import("../coach/budget")>("../coach/budget");
return { ...actual, readDailySpend: vi.fn(async () => 0) };
});
vi.mock("../provider-health-ledger", async () => {
const actual = await vi.importActual<
typeof import("../provider-health-ledger")
>("../provider-health-ledger");
return {
...actual,
postgresProviderHealthLedger: {
async getSkipHints() {
return new Map();
},
async recordSuccess() {},
async recordFailure() {},
},
};
});
import { OpenAIClient } from "../openai-client";
import { LocalOpenAICompatibleClient } from "../local-client";
import { resetJsonModeDialectCache } from "../json-dialect";
import {
singleUserTurn,
type AIProvider,
type CompletionResult,
} from "../types";
import {
AllProvidersFailedError,
clearLastWorkingProviderCache,
runRawCompletionWithFallback,
} from "../provider-runner";
import { createInMemoryProviderHealthLedger } from "../provider-health-ledger";
import { annotate } from "@/lib/logging/context";
function reply200(body: unknown) {
return vi.fn().mockResolvedValue({
ok: true,
status: 200,
headers: { get: () => "application/json" },
json: () => Promise.resolve(body),
});
}
const turn = () => singleUserTurn({ system: "s", user: "u" });
const jsonTurn = () =>
singleUserTurn({ system: "s", user: "u", responseFormat: "json" });
function gateway(baseUrl = "https://gateway.example.com/api/v1") {
return new OpenAIClient({
apiKey: "gateway-secret",
model: "vendor/model-5",
baseUrl,
providerType: "openai-compatible",
});
}
beforeEach(() => {
vi.restoreAllMocks();
resetJsonModeDialectCache();
clearLastWorkingProviderCache();
vi.mocked(annotate).mockClear();
});
describe("gateway 200 with a top-level error", () => {
it("surfaces the embedded code as the HTTP status", async () => {
vi.stubGlobal(
"fetch",
reply200({ error: { code: 402, message: "Insufficient credits" } }),
);
await expect(gateway().generateCompletion(turn())).rejects.toMatchObject({
httpStatus: 402,
kind: "embedded_error",
upstream: "openai-compatible",
model: "vendor/model-5",
});
});
it("leads the message with the gateway's own words", async () => {
vi.stubGlobal(
"fetch",
reply200({ error: { code: 402, message: "Insufficient credits" } }),
);
await expect(gateway().generateCompletion(turn())).rejects.toThrow(
/^Insufficient credits \(OpenAI-compatible gateway\)$/,
);
});
it("carries a bounded, key-redacted excerpt of the body", async () => {
vi.stubGlobal(
"fetch",
reply200({
error: {
code: 403,
message: `blocked ${"x".repeat(400)}`,
metadata: { key: "sk-abcdefghijklmnop", reasons: ["data policy"] },
},
}),
);
const err = (await gateway()
.generateCompletion(turn())
.catch((e) => e)) as Error & { bodyExcerpt?: string };
expect(err.bodyExcerpt).toBeTypeOf("string");
expect(err.bodyExcerpt!.length).toBeLessThanOrEqual(500);
expect(err.bodyExcerpt).not.toContain("sk-abcdefghijklmnop");
// The thrown message is bounded independently of the excerpt.
expect(err.message.length).toBeLessThanOrEqual(240);
});
it("keeps the 0 sentinel when the embedded code is not an HTTP status", async () => {
vi.stubGlobal(
"fetch",
reply200({
error: { code: "insufficient_quota", message: "Out of quota" },
}),
);
await expect(gateway().generateCompletion(turn())).rejects.toMatchObject({
httpStatus: 0,
message: "Out of quota (OpenAI-compatible gateway)",
});
});
});
describe("gateway 200 with choices[0].error", () => {
it("surfaces the partial-generation failure with its status", async () => {
vi.stubGlobal(
"fetch",
reply200({
choices: [
{
error: {
code: 403,
message:
"No endpoints found matching your data policy (Free model publication)",
},
finish_reason: "error",
},
],
}),
);
await expect(gateway().generateCompletion(turn())).rejects.toMatchObject({
httpStatus: 403,
kind: "embedded_error",
});
await expect(gateway().generateCompletion(turn())).rejects.toThrow(
/^No endpoints found matching your data policy/,
);
});
});
describe("what the fix must not touch", () => {
it("leaves a genuine empty reply as empty_response with status 0", async () => {
vi.stubGlobal(
"fetch",
reply200({ choices: [{ message: { content: "" } }] }),
);
await expect(gateway().generateCompletion(turn())).rejects.toMatchObject({
message: "OpenAI-compatible gateway returned empty content",
httpStatus: 0,
kind: "empty_response",
});
});
it("leaves a normal reply alone", async () => {
vi.stubGlobal(
"fetch",
reply200({
choices: [{ message: { content: "hello" }, finish_reason: "stop" }],
usage: { total_tokens: 5 },
}),
);
const result = await gateway().generateCompletion(turn());
expect(result.content).toBe("hello");
expect(result.tokensUsed).toBe(5);
});
it("does not read an embedded error on api.openai.com", async () => {
// The shape is gateway-only: the pinned vendor endpoint signals every
// failure with a non-2xx status, so its parsing must be unchanged.
vi.stubGlobal(
"fetch",
reply200({ error: { code: 402, message: "Insufficient credits" } }),
);
await expect(
new OpenAIClient({
apiKey: "sk-user-openai-key",
model: "gpt-4o",
baseUrl: "https://api.openai.com/v1",
}).generateCompletion(jsonTurn()),
).rejects.toMatchObject({
message: "OpenAI returned empty content",
httpStatus: 0,
kind: "empty_response",
});
});
});
describe("the chain runner and the health ledger", () => {
class ThrowingProvider implements AIProvider {
readonly type = "openai-compatible" as const;
constructor(private readonly error: Error) {}
async generateCompletion(): Promise<CompletionResult> {
throw this.error;
}
}
class OkProvider implements AIProvider {
readonly type = "local" as const;
async generateCompletion(): Promise<CompletionResult> {
return {
content: "second provider answered",
tokensUsed: 1,
model: "m",
providerType: "local",
};
}
}
async function embeddedErrorFrom(body: unknown): Promise<Error> {
vi.stubGlobal("fetch", reply200(body));
return (await gateway()
.generateCompletion(turn())
.catch((e) => e)) as Error;
}
it("records a 402 hop as HTTP 402 with the gateway's reason", async () => {
const error = await embeddedErrorFrom({
error: { code: 402, message: "Insufficient credits" },
});
const ledger = createInMemoryProviderHealthLedger();
const recordFailure = vi.spyOn(ledger, "recordFailure");
const failure = (await runRawCompletionWithFallback({
userId: "u1",
providers: [
{
providerType: "openai-compatible",
instance: new ThrowingProvider(error),
},
],
params: turn(),
ledger,
}).catch((e) => e)) as AllProvidersFailedError;
expect(failure).toBeInstanceOf(AllProvidersFailedError);
expect(failure.attempts[0].httpStatus).toBe(402);
expect(failure.attempts[0].failureReason).toBe(
"HTTP 402: Insufficient credits (OpenAI-compatible gateway)",
);
expect(recordFailure).toHaveBeenCalledWith("u1", "openai-compatible", 402);
const hopMeta = vi
.mocked(annotate)
.mock.calls.map(([arg]) => arg?.meta)
.find((m) => m && "ai_chain_hop_1_status" in m);
expect(hopMeta?.ai_chain_hop_1_status).toBe(402);
expect(hopMeta?.ai_chain_hop_1_body).toBe("classified:quota_or_billing");
});
it("benches the provider as a dead credential on an embedded 403", async () => {
const error = await embeddedErrorFrom({
choices: [
{
error: { code: 403, message: "Blocked by your data policy" },
finish_reason: "error",
},
],
});
const ledger = createInMemoryProviderHealthLedger();
const failure = (await runRawCompletionWithFallback({
userId: "u2",
providers: [
{
providerType: "openai-compatible",
instance: new ThrowingProvider(error),
},
],
params: turn(),
ledger,
}).catch((e) => e)) as AllProvidersFailedError;
expect(failure.attempts[0].httpStatus).toBe(403);
expect(failure.primaryCredentialExpired).toBe(true);
expect(ledger.inspect("u2").get("openai-compatible")?.reason).toBe(
"credential_expired",
);
const hopMeta = vi
.mocked(annotate)
.mock.calls.map(([arg]) => arg?.meta)
.find((m) => m && "ai_chain_hop_1_body" in m);
expect(hopMeta?.ai_chain_hop_1_body).toBe(
"classified:no_eligible_provider",
);
});
it("still advances to the next provider", async () => {
const error = await embeddedErrorFrom({
error: { code: 503, message: "No provider is available for this model" },
});
const outcome = await runRawCompletionWithFallback({
userId: "u3",
providers: [
{
providerType: "openai-compatible",
instance: new ThrowingProvider(error),
},
{ providerType: "local", instance: new OkProvider() },
],
params: turn(),
ledger: createInMemoryProviderHealthLedger(),
});
expect(outcome.result.content).toBe("second provider answered");
expect(outcome.fallbackHops[0].httpStatus).toBe(503);
});
});
describe("the local provider has the same hole", () => {
function local() {
return new LocalOpenAICompatibleClient({
apiKey: null,
model: "llama3",
baseUrl: "https://gateway.example.com/api/v1",
});
}
it("surfaces an embedded error on the buffered path", async () => {
vi.stubGlobal(
"fetch",
reply200({ error: { code: 402, message: "Insufficient credits" } }),
);
await expect(local().generateCompletion(turn())).rejects.toMatchObject({
httpStatus: 402,
kind: "embedded_error",
upstream: "local",
message: "Insufficient credits (local AI endpoint)",
});
});
it("surfaces an embedded error the server sent instead of a stream", async () => {
vi.stubGlobal(
"fetch",
reply200({
choices: [
{
error: { code: 429, message: "Rate limited" },
finish_reason: "error",
},
],
}),
);
await expect(
local().generateCompletionStream(turn(), () => {}),
).rejects.toMatchObject({ httpStatus: 429, kind: "embedded_error" });
});
it("surfaces an error frame that arrived before any token", async () => {
const encoder = new TextEncoder();
vi.stubGlobal(
"fetch",
vi.fn().mockResolvedValue({
ok: true,
status: 200,
headers: {
get: (k: string) =>
k === "content-type" ? "text/event-stream" : null,
},
body: new ReadableStream<Uint8Array>({
start(controller) {
controller.enqueue(
encoder.encode(
'data: {"error":{"code":502,"message":"Model is down"}}\n\ndata: [DONE]\n\n',
),
);
controller.close();
},
}),
}),
);
await expect(
local().generateCompletionStream(turn(), () => {}),
).rejects.toMatchObject({
httpStatus: 502,
kind: "embedded_error",
message: "Model is down (local AI endpoint)",
});
});
it("leaves a genuine empty local reply as empty_response", async () => {
vi.stubGlobal(
"fetch",
reply200({ choices: [{ message: { content: "" } }] }),
);
await expect(local().generateCompletion(turn())).rejects.toMatchObject({
message: "Local AI returned empty content",
httpStatus: 0,
kind: "empty_response",
});
});
});
+44 -5
View File
@@ -7,7 +7,13 @@ import type {
CompletionParams,
CompletionResult,
} from "./types";
import { buildOpenAIMessages, mapFinishReason } from "./openai-wire";
import {
buildOpenAIMessages,
mapFinishReason,
raiseEmbeddedError,
readEmbeddedError,
sanitiseBodyExcerpt,
} from "./openai-wire";
import {
hasLearnedJsonModeDialect,
isResponseFormatRejection,
@@ -214,10 +220,7 @@ export class LocalOpenAICompatibleClient implements AIProvider {
// also commonly leak bearer tokens in error envelopes (proxies), so
// strip anything that looks like a key before logging.
const rawBody = await res.text().catch(() => "");
const bodyExcerpt = rawBody
.slice(0, 500)
.replace(/sk-[A-Za-z0-9_-]{8,}/g, "sk-***redacted***")
.replace(/Bearer\s+[A-Za-z0-9_.-]+/gi, "Bearer ***redacted***");
const bodyExcerpt = sanitiseBodyExcerpt(rawBody);
// v1.28.28 (#470) — dialect self-heal. A 4xx whose body names
// `response_format` (or an unknown parameter) means this endpoint
// rejects the standard JSON flag: learn the no-flag dialect for the
@@ -270,6 +273,18 @@ export class LocalOpenAICompatibleClient implements AIProvider {
usage?: { total_tokens?: number };
};
// Same embedded-failure shape the OpenAI-compatible gateway tag handles:
// this client serves hosted gateways too (the class doc above names
// LiteLLM and OpenRouter), and those answer a provider-side refusal with
// HTTP 200 and the failure inside the body. Unconditional here — every
// base URL on this provider was typed by a person, there is no pinned
// vendor endpoint whose behaviour has to stay untouched.
raiseEmbeddedError(json, {
upstream: "local",
label: "local AI endpoint",
model: this.config.model,
});
const content = json.choices?.[0]?.message?.content;
if (!content) {
// v1.20.1 — sentinel httpStatus + kind so the chain classifier can tell
@@ -370,6 +385,13 @@ export class LocalOpenAICompatibleClient implements AIProvider {
}>;
usage?: { total_tokens?: number };
} | null;
// The server ignored `stream` and answered buffered — the embedded
// failure can ride this body exactly as it rides the non-streaming one.
raiseEmbeddedError(json, {
upstream: "local",
label: "local AI endpoint",
model: this.config.model,
});
const buffered = json?.choices?.[0]?.message?.content;
if (!buffered) {
const err = new Error("Local AI returned empty content");
@@ -397,6 +419,11 @@ export class LocalOpenAICompatibleClient implements AIProvider {
let tokensUsed: number | null = null;
let finishReason: string | undefined;
// An SSE frame can carry the same embedded failure the buffered body
// carries. Remember the first one; it only matters if the stream ends
// without a single token, in which case it is the honest reason.
let streamedError: unknown = null;
let idleTimer: ReturnType<typeof setTimeout> | null = null;
const armIdle = () => {
if (idleTimer) clearTimeout(idleTimer);
@@ -421,9 +448,11 @@ export class LocalOpenAICompatibleClient implements AIProvider {
const payload = line.slice("data:".length).trim();
if (!payload || payload === "[DONE]") continue;
let chunk: {
error?: unknown;
choices?: Array<{
delta?: { content?: string };
finish_reason?: string | null;
error?: unknown;
}>;
usage?: { total_tokens?: number };
};
@@ -433,6 +462,9 @@ export class LocalOpenAICompatibleClient implements AIProvider {
// A malformed chunk line is skipped, not fatal — keep reading.
continue;
}
if (streamedError === null && readEmbeddedError(chunk) !== null) {
streamedError = chunk;
}
const delta = chunk.choices?.[0]?.delta?.content;
if (delta) {
content += delta;
@@ -466,6 +498,13 @@ export class LocalOpenAICompatibleClient implements AIProvider {
}
if (!content) {
// A stream that ended with a failure frame and no token reports the
// failure, not "empty content".
raiseEmbeddedError(streamedError, {
upstream: "local",
label: "local AI endpoint",
model: this.config.model,
});
const err = new Error("Local AI returned empty content");
Object.assign(err, {
httpStatus: 0,
+27 -4
View File
@@ -15,6 +15,8 @@ import {
mapFinishReason,
parseCachedTokens,
parseOpenAIToolCalls,
raiseEmbeddedError,
sanitiseBodyExcerpt,
type OpenAIResponseJson,
} from "./openai-wire";
@@ -179,10 +181,7 @@ export class OpenAIClient implements AIProvider {
// surfacing as an opaque "OpenAI request failed (500)". Strip anything
// that looks like an API key from the excerpt before logging.
const rawBody = await res.text().catch(() => "");
const bodyExcerpt = rawBody
.slice(0, 500)
.replace(/sk-[A-Za-z0-9_-]{8,}/g, "sk-***redacted***")
.replace(/Bearer\s+[A-Za-z0-9_.-]+/gi, "Bearer ***redacted***");
const bodyExcerpt = sanitiseBodyExcerpt(rawBody);
// v1.33.1 (#470) — dialect self-heal, gateway only. A 4xx whose body
// names `response_format` (or an unknown parameter) means this endpoint
// rejects the standard JSON flag: learn the no-flag dialect for the base
@@ -232,6 +231,30 @@ export class OpenAIClient implements AIProvider {
}
const json = (await res.json()) as OpenAIResponseJson;
// A 200 that carries the failure inside the body. OpenRouter (and every
// gateway that copies it) answers a provider-side refusal — no credits,
// a data-policy or moderation block, an upstream timeout, a model that is
// down, no provider matching the routing requirements — with HTTP 200 and
// a top-level `error` object instead of `choices`, and a generation that
// died mid-flight with `choices[0].error` beside `finish_reason: "error"`.
// Both used to fall through to the empty-content throw below, so every one
// of those causes collapsed into "returned empty content", status 0, no
// excerpt: neither the person nor the provider-health card could tell them
// apart. Raise the same shaped error a non-2xx reply raises so the chain
// classifier, the health ledger and the card all see the real status.
//
// Gated on the gateway tag because the shape is gateway-only: on
// `api.openai.com` — the `admin-key` and `codex` tags — a failure is
// always a non-2xx status, so their parsing is untouched.
if (this.isGateway) {
raiseEmbeddedError(json, {
upstream: "openai-compatible",
label: "OpenAI-compatible gateway",
model: this.config.model,
});
}
const choice = json.choices?.[0];
let content = choice?.message?.content;
const toolCalls = parseOpenAIToolCalls(choice);
+97
View File
@@ -154,4 +154,101 @@ export function parseCachedTokens(json: OpenAIResponseJson): number | null {
return json.usage?.prompt_tokens_details?.cached_tokens ?? null;
}
/**
* Bound and key-redact an upstream body before it is attached to a thrown
* error. Both clients inlined this next to their non-2xx handling; it lives
* here once so the embedded-error excerpt below is sanitised by the very same
* code rather than by a copy that can drift.
*/
export function sanitiseBodyExcerpt(raw: string): string {
return raw
.slice(0, 500)
.replace(/sk-[A-Za-z0-9_-]{8,}/g, "sk-***redacted***")
.replace(/Bearer\s+[A-Za-z0-9_.-]+/gi, "Bearer ***redacted***");
}
interface EmbeddedError {
message: string;
/** The gateway's own `error.code` when it is an HTTP error status, else 0. */
httpStatus: number;
}
/** Longest slice of a remote error message that reaches a thrown message. */
const EMBEDDED_MESSAGE_MAX = 200;
function coerceEmbeddedError(candidate: unknown): EmbeddedError | null {
if (typeof candidate === "string") {
const trimmed = candidate.trim();
return trimmed.length > 0 ? { message: trimmed, httpStatus: 0 } : null;
}
if (candidate === null || typeof candidate !== "object") return null;
const raw = candidate as { code?: unknown; message?: unknown };
const message =
typeof raw.message === "string" && raw.message.trim().length > 0
? raw.message.trim()
: "gateway reported a failure without a message";
// Only an error-range status is adopted. A gateway that puts a vendor
// string ("insufficient_quota") or a non-status number in `code` keeps the
// 0 sentinel, which the chain classifier already treats as a hard failure.
const code = raw.code;
const httpStatus =
typeof code === "number" &&
Number.isInteger(code) &&
code >= 400 &&
code <= 599
? code
: 0;
return { message, httpStatus };
}
/**
* Read a failure that a 2xx body carries INSIDE itself.
*
* OpenAI-compatible gateways (OpenRouter and friends) answer a provider-side
* refusal with HTTP 200 and either a top-level `error` object and no `choices`
* at all, or — when a generation dies mid-flight — `choices[0].error` beside
* `finish_reason: "error"`. Returns the embedded failure, or null for a normal
* reply. `api.openai.com` never answers this way; every failure there is a
* non-2xx status.
*/
export function readEmbeddedError(json: unknown): EmbeddedError | null {
if (json === null || typeof json !== "object") return null;
const body = json as {
error?: unknown;
choices?: Array<{ error?: unknown } | null> | null;
};
return (
coerceEmbeddedError(body.error) ??
coerceEmbeddedError(body.choices?.[0]?.error)
);
}
/**
* Throw the same shaped error the non-2xx path throws when a 2xx body carries
* an embedded failure; return quietly when it does not. The gateway's own
* message leads so the hop reason says WHY the call failed instead of
* "returned empty content"; it is bounded and key-redacted on the way in,
* because remote text is never trusted to be short or clean.
*/
export function raiseEmbeddedError(
json: unknown,
ctx: { upstream: string; label: string; model: string },
): void {
const embedded = readEmbeddedError(json);
if (!embedded) return;
const message = sanitiseBodyExcerpt(embedded.message).slice(
0,
EMBEDDED_MESSAGE_MAX,
);
const err = new Error(`${message} (${ctx.label})`);
Object.assign(err, {
httpStatus: embedded.httpStatus,
kind: "embedded_error",
upstream: ctx.upstream,
model: ctx.model,
bodyExcerpt: sanitiseBodyExcerpt(JSON.stringify(json) ?? ""),
});
throw err;
}
export type { OpenAIChoice, OpenAIResponseJson };
+20 -1
View File
@@ -356,9 +356,28 @@ export function classifyErrorBody(body: unknown): string | null {
if (lowered.includes("response_format") || lowered.includes("json_schema")) {
return "classified:response_format_rejected";
}
if (lowered.includes("insufficient_quota") || lowered.includes("billing")) {
if (
lowered.includes("insufficient_quota") ||
lowered.includes("billing") ||
// A gateway that meters prepaid credit says so in its own words rather
// than OpenAI's `insufficient_quota`; it is the same class of refusal.
lowered.includes("insufficient credits") ||
lowered.includes("insufficient_credits")
) {
return "classified:quota_or_billing";
}
// A routing refusal: the account's data policy / zero-retention settings, or
// the model's own availability, leave no provider eligible to serve the
// call. Distinct from quota — nothing is spent and nothing is expired; the
// request simply has nowhere to go.
if (
lowered.includes("data policy") ||
lowered.includes("data_policy") ||
lowered.includes("no allowed providers") ||
lowered.includes("no endpoints found")
) {
return "classified:no_eligible_provider";
}
if (lowered.includes("invalid_api_key") || lowered.includes("unauthorized")) {
return "classified:auth_rejected";
}