mirror of
https://github.com/MBombeck/HealthLog.git
synced 2026-11-08 12:32:43 +00:00
Surface what an OpenAI-compatible gateway actually refused
A gateway can answer a failed generation with HTTP 200 and put the reason in the body: a top-level `error` object and no `choices`, or `choices[0].error` beside `finish_reason: "error"` when a generation dies mid-flight. Both fell through to the empty-content throw, so a dead credit balance, a data-policy block, a rate limit and a model that is down all recorded the same hop: status 0, reason "returned empty content", no excerpt. Nothing on the provider health card told them apart. Raise the shape a non-2xx reply raises instead, with the gateway's own message leading and its `error.code` as the status when that code is an HTTP error status. The chain classifier, the health ledger and the card then see 402 / 403 / 429 / 503 and treat each of them the way they treat the same status on the wire. Message and excerpt are bounded and key-redacted by the same helper the non-2xx path uses, so remote text is no more trusted here than it is there. Gated on the gateway tag: api.openai.com signals every failure with a status, so the pinned tags parse exactly as before, and a genuine empty reply stays the empty-content throw it was. The local-endpoint provider had the same hole on its buffered path, its buffered-reply fallback and its stream, and gets the same treatment. The body classifier learned the two account-side refusals a gateway words in its own vocabulary rather than the vendor's.
This commit is contained in:
1 parent
0913e8f2b7
commit
703ca2744e
8 files changed
+679
-10
No files matched your search
@@ -21,6 +21,16 @@
|
||||
uploads.** Recording a measurement, editing a medication, restoring a
|
||||
backup — anything that wrote — failed from the moment the app started
|
||||
labelling writes this way in v1.37.18. Those all work again.
|
||||
- **A gateway that refused the call reported nothing but "empty content".**
|
||||
Some OpenAI-compatible gateways, OpenRouter among them, answer a refused
|
||||
generation with HTTP 200 and the reason inside the body. The provider health
|
||||
card therefore recorded status 0 and "returned empty content" whether the
|
||||
account was out of credits, blocked by its own data policy, rate-limited, or
|
||||
pointed at a model that was down, and no excerpt survived to tell them
|
||||
apart. The gateway's own status and message now reach the card and the
|
||||
fallback chain, which treats each of them exactly as it treats the same
|
||||
status on the wire. The local-endpoint provider had the same blind spot on
|
||||
its buffered and its streaming path and got the same treatment.
|
||||
- **The import guide link under Settings → Import went the long way round.**
|
||||
Both the CSV and the JSON card now link straight to the guide instead of
|
||||
through a redirect.
|
||||
|
||||
@@ -197,6 +197,15 @@ JSON surfaces send the standard
|
||||
the field is detected on the first refusal and retried without it,
|
||||
so strict gateways and older local servers both work unmodified.
|
||||
|
||||
**When the gateway refuses.** OpenRouter answers a refused generation
|
||||
with HTTP 200 and the reason inside the body, so every refusal used to
|
||||
reach the provider health card as an empty reply with no status. It now
|
||||
carries the gateway's own status and message — 402 out of credits, 403
|
||||
blocked, 429 rate-limited, 502 model down. The two account settings
|
||||
behind most refusals are an empty credit balance and a data-policy or
|
||||
zero-retention restriction that leaves no eligible provider for the
|
||||
model you picked; both are changed in the gateway account, not here.
|
||||
|
||||
**The Local provider still works for this** and nothing about it
|
||||
changed — it speaks the same wire and honours the same base URL. Pick
|
||||
the gateway provider when the endpoint is a gateway (it says so in
|
||||
|
||||
@@ -72,6 +72,23 @@ describe("upstream error-body classification", () => {
|
||||
);
|
||||
});
|
||||
|
||||
it("names a gateway's own wording for the two account-side refusals", () => {
|
||||
// A gateway meters prepaid credit and enforces the account's data policy
|
||||
// in its own vocabulary, not OpenAI's. Both used to land in the
|
||||
// unclassified bucket, which is exactly the pair a person needs told
|
||||
// apart: top the account up, or widen the routing policy.
|
||||
expect(
|
||||
classifyErrorBody(
|
||||
'{"error":{"code":402,"message":"Insufficient credits"}}',
|
||||
),
|
||||
).toBe("classified:quota_or_billing");
|
||||
expect(
|
||||
classifyErrorBody(
|
||||
'{"error":{"code":403,"message":"No endpoints found matching your data policy"}}',
|
||||
),
|
||||
).toBe("classified:no_eligible_provider");
|
||||
});
|
||||
|
||||
it("passes through absence honestly", () => {
|
||||
expect(classifyErrorBody(null)).toBeNull();
|
||||
expect(classifyErrorBody("")).toBeNull();
|
||||
|
||||
@@ -0,0 +1,455 @@
|
||||
import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||
|
||||
/**
|
||||
* A gateway that refuses inside a 200.
|
||||
*
|
||||
* OpenRouter — and every gateway that copies its wire — answers a
|
||||
* provider-side refusal with HTTP 200 and the failure in the body: a
|
||||
* top-level `error` object and no `choices`, or `choices[0].error` beside
|
||||
* `finish_reason: "error"` when a generation dies mid-flight. Both used to
|
||||
* reach the empty-content throw, so "no credits", "blocked by the account's
|
||||
* data policy" and "the model is down" all surfaced identically: status 0,
|
||||
* reason "returned empty content", no excerpt. Nobody reading the provider
|
||||
* card could act on that.
|
||||
*
|
||||
* These cases pin the whole path: the client raises the shape a non-2xx reply
|
||||
* raises, the chain runner carries the real status into the hop, the health
|
||||
* ledger records it, and the pinned `api.openai.com` tags parse exactly as
|
||||
* they did.
|
||||
*/
|
||||
|
||||
// safeFetch's requirePublicHost path runs through undici's own `fetch`.
|
||||
// Delegate it to the global stub so the interception still applies.
|
||||
vi.mock("undici", async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import("undici")>();
|
||||
return {
|
||||
...actual,
|
||||
fetch: (input: unknown, init?: unknown) =>
|
||||
(globalThis.fetch as unknown as (i: unknown, n?: unknown) => unknown)(
|
||||
input,
|
||||
init,
|
||||
),
|
||||
};
|
||||
});
|
||||
|
||||
vi.mock("@/lib/logging/context", () => ({ annotate: vi.fn() }));
|
||||
|
||||
// The chain runner reaches for Postgres on both of these at import time; the
|
||||
// chain semantics under test need neither.
|
||||
vi.mock("../coach/budget", async () => {
|
||||
const actual =
|
||||
await vi.importActual<typeof import("../coach/budget")>("../coach/budget");
|
||||
return { ...actual, readDailySpend: vi.fn(async () => 0) };
|
||||
});
|
||||
vi.mock("../provider-health-ledger", async () => {
|
||||
const actual = await vi.importActual<
|
||||
typeof import("../provider-health-ledger")
|
||||
>("../provider-health-ledger");
|
||||
return {
|
||||
...actual,
|
||||
postgresProviderHealthLedger: {
|
||||
async getSkipHints() {
|
||||
return new Map();
|
||||
},
|
||||
async recordSuccess() {},
|
||||
async recordFailure() {},
|
||||
},
|
||||
};
|
||||
});
|
||||
|
||||
import { OpenAIClient } from "../openai-client";
|
||||
import { LocalOpenAICompatibleClient } from "../local-client";
|
||||
import { resetJsonModeDialectCache } from "../json-dialect";
|
||||
import {
|
||||
singleUserTurn,
|
||||
type AIProvider,
|
||||
type CompletionResult,
|
||||
} from "../types";
|
||||
import {
|
||||
AllProvidersFailedError,
|
||||
clearLastWorkingProviderCache,
|
||||
runRawCompletionWithFallback,
|
||||
} from "../provider-runner";
|
||||
import { createInMemoryProviderHealthLedger } from "../provider-health-ledger";
|
||||
import { annotate } from "@/lib/logging/context";
|
||||
|
||||
function reply200(body: unknown) {
|
||||
return vi.fn().mockResolvedValue({
|
||||
ok: true,
|
||||
status: 200,
|
||||
headers: { get: () => "application/json" },
|
||||
json: () => Promise.resolve(body),
|
||||
});
|
||||
}
|
||||
|
||||
const turn = () => singleUserTurn({ system: "s", user: "u" });
|
||||
const jsonTurn = () =>
|
||||
singleUserTurn({ system: "s", user: "u", responseFormat: "json" });
|
||||
|
||||
function gateway(baseUrl = "https://gateway.example.com/api/v1") {
|
||||
return new OpenAIClient({
|
||||
apiKey: "gateway-secret",
|
||||
model: "vendor/model-5",
|
||||
baseUrl,
|
||||
providerType: "openai-compatible",
|
||||
});
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
resetJsonModeDialectCache();
|
||||
clearLastWorkingProviderCache();
|
||||
vi.mocked(annotate).mockClear();
|
||||
});
|
||||
|
||||
describe("gateway 200 with a top-level error", () => {
|
||||
it("surfaces the embedded code as the HTTP status", async () => {
|
||||
vi.stubGlobal(
|
||||
"fetch",
|
||||
reply200({ error: { code: 402, message: "Insufficient credits" } }),
|
||||
);
|
||||
|
||||
await expect(gateway().generateCompletion(turn())).rejects.toMatchObject({
|
||||
httpStatus: 402,
|
||||
kind: "embedded_error",
|
||||
upstream: "openai-compatible",
|
||||
model: "vendor/model-5",
|
||||
});
|
||||
});
|
||||
|
||||
it("leads the message with the gateway's own words", async () => {
|
||||
vi.stubGlobal(
|
||||
"fetch",
|
||||
reply200({ error: { code: 402, message: "Insufficient credits" } }),
|
||||
);
|
||||
|
||||
await expect(gateway().generateCompletion(turn())).rejects.toThrow(
|
||||
/^Insufficient credits \(OpenAI-compatible gateway\)$/,
|
||||
);
|
||||
});
|
||||
|
||||
it("carries a bounded, key-redacted excerpt of the body", async () => {
|
||||
vi.stubGlobal(
|
||||
"fetch",
|
||||
reply200({
|
||||
error: {
|
||||
code: 403,
|
||||
message: `blocked ${"x".repeat(400)}`,
|
||||
metadata: { key: "sk-abcdefghijklmnop", reasons: ["data policy"] },
|
||||
},
|
||||
}),
|
||||
);
|
||||
|
||||
const err = (await gateway()
|
||||
.generateCompletion(turn())
|
||||
.catch((e) => e)) as Error & { bodyExcerpt?: string };
|
||||
|
||||
expect(err.bodyExcerpt).toBeTypeOf("string");
|
||||
expect(err.bodyExcerpt!.length).toBeLessThanOrEqual(500);
|
||||
expect(err.bodyExcerpt).not.toContain("sk-abcdefghijklmnop");
|
||||
// The thrown message is bounded independently of the excerpt.
|
||||
expect(err.message.length).toBeLessThanOrEqual(240);
|
||||
});
|
||||
|
||||
it("keeps the 0 sentinel when the embedded code is not an HTTP status", async () => {
|
||||
vi.stubGlobal(
|
||||
"fetch",
|
||||
reply200({
|
||||
error: { code: "insufficient_quota", message: "Out of quota" },
|
||||
}),
|
||||
);
|
||||
|
||||
await expect(gateway().generateCompletion(turn())).rejects.toMatchObject({
|
||||
httpStatus: 0,
|
||||
message: "Out of quota (OpenAI-compatible gateway)",
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("gateway 200 with choices[0].error", () => {
|
||||
it("surfaces the partial-generation failure with its status", async () => {
|
||||
vi.stubGlobal(
|
||||
"fetch",
|
||||
reply200({
|
||||
choices: [
|
||||
{
|
||||
error: {
|
||||
code: 403,
|
||||
message:
|
||||
"No endpoints found matching your data policy (Free model publication)",
|
||||
},
|
||||
finish_reason: "error",
|
||||
},
|
||||
],
|
||||
}),
|
||||
);
|
||||
|
||||
await expect(gateway().generateCompletion(turn())).rejects.toMatchObject({
|
||||
httpStatus: 403,
|
||||
kind: "embedded_error",
|
||||
});
|
||||
await expect(gateway().generateCompletion(turn())).rejects.toThrow(
|
||||
/^No endpoints found matching your data policy/,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe("what the fix must not touch", () => {
|
||||
it("leaves a genuine empty reply as empty_response with status 0", async () => {
|
||||
vi.stubGlobal(
|
||||
"fetch",
|
||||
reply200({ choices: [{ message: { content: "" } }] }),
|
||||
);
|
||||
|
||||
await expect(gateway().generateCompletion(turn())).rejects.toMatchObject({
|
||||
message: "OpenAI-compatible gateway returned empty content",
|
||||
httpStatus: 0,
|
||||
kind: "empty_response",
|
||||
});
|
||||
});
|
||||
|
||||
it("leaves a normal reply alone", async () => {
|
||||
vi.stubGlobal(
|
||||
"fetch",
|
||||
reply200({
|
||||
choices: [{ message: { content: "hello" }, finish_reason: "stop" }],
|
||||
usage: { total_tokens: 5 },
|
||||
}),
|
||||
);
|
||||
|
||||
const result = await gateway().generateCompletion(turn());
|
||||
expect(result.content).toBe("hello");
|
||||
expect(result.tokensUsed).toBe(5);
|
||||
});
|
||||
|
||||
it("does not read an embedded error on api.openai.com", async () => {
|
||||
// The shape is gateway-only: the pinned vendor endpoint signals every
|
||||
// failure with a non-2xx status, so its parsing must be unchanged.
|
||||
vi.stubGlobal(
|
||||
"fetch",
|
||||
reply200({ error: { code: 402, message: "Insufficient credits" } }),
|
||||
);
|
||||
|
||||
await expect(
|
||||
new OpenAIClient({
|
||||
apiKey: "sk-user-openai-key",
|
||||
model: "gpt-4o",
|
||||
baseUrl: "https://api.openai.com/v1",
|
||||
}).generateCompletion(jsonTurn()),
|
||||
).rejects.toMatchObject({
|
||||
message: "OpenAI returned empty content",
|
||||
httpStatus: 0,
|
||||
kind: "empty_response",
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("the chain runner and the health ledger", () => {
|
||||
class ThrowingProvider implements AIProvider {
|
||||
readonly type = "openai-compatible" as const;
|
||||
constructor(private readonly error: Error) {}
|
||||
async generateCompletion(): Promise<CompletionResult> {
|
||||
throw this.error;
|
||||
}
|
||||
}
|
||||
class OkProvider implements AIProvider {
|
||||
readonly type = "local" as const;
|
||||
async generateCompletion(): Promise<CompletionResult> {
|
||||
return {
|
||||
content: "second provider answered",
|
||||
tokensUsed: 1,
|
||||
model: "m",
|
||||
providerType: "local",
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
async function embeddedErrorFrom(body: unknown): Promise<Error> {
|
||||
vi.stubGlobal("fetch", reply200(body));
|
||||
return (await gateway()
|
||||
.generateCompletion(turn())
|
||||
.catch((e) => e)) as Error;
|
||||
}
|
||||
|
||||
it("records a 402 hop as HTTP 402 with the gateway's reason", async () => {
|
||||
const error = await embeddedErrorFrom({
|
||||
error: { code: 402, message: "Insufficient credits" },
|
||||
});
|
||||
const ledger = createInMemoryProviderHealthLedger();
|
||||
const recordFailure = vi.spyOn(ledger, "recordFailure");
|
||||
|
||||
const failure = (await runRawCompletionWithFallback({
|
||||
userId: "u1",
|
||||
providers: [
|
||||
{
|
||||
providerType: "openai-compatible",
|
||||
instance: new ThrowingProvider(error),
|
||||
},
|
||||
],
|
||||
params: turn(),
|
||||
ledger,
|
||||
}).catch((e) => e)) as AllProvidersFailedError;
|
||||
|
||||
expect(failure).toBeInstanceOf(AllProvidersFailedError);
|
||||
expect(failure.attempts[0].httpStatus).toBe(402);
|
||||
expect(failure.attempts[0].failureReason).toBe(
|
||||
"HTTP 402: Insufficient credits (OpenAI-compatible gateway)",
|
||||
);
|
||||
expect(recordFailure).toHaveBeenCalledWith("u1", "openai-compatible", 402);
|
||||
|
||||
const hopMeta = vi
|
||||
.mocked(annotate)
|
||||
.mock.calls.map(([arg]) => arg?.meta)
|
||||
.find((m) => m && "ai_chain_hop_1_status" in m);
|
||||
expect(hopMeta?.ai_chain_hop_1_status).toBe(402);
|
||||
expect(hopMeta?.ai_chain_hop_1_body).toBe("classified:quota_or_billing");
|
||||
});
|
||||
|
||||
it("benches the provider as a dead credential on an embedded 403", async () => {
|
||||
const error = await embeddedErrorFrom({
|
||||
choices: [
|
||||
{
|
||||
error: { code: 403, message: "Blocked by your data policy" },
|
||||
finish_reason: "error",
|
||||
},
|
||||
],
|
||||
});
|
||||
const ledger = createInMemoryProviderHealthLedger();
|
||||
|
||||
const failure = (await runRawCompletionWithFallback({
|
||||
userId: "u2",
|
||||
providers: [
|
||||
{
|
||||
providerType: "openai-compatible",
|
||||
instance: new ThrowingProvider(error),
|
||||
},
|
||||
],
|
||||
params: turn(),
|
||||
ledger,
|
||||
}).catch((e) => e)) as AllProvidersFailedError;
|
||||
|
||||
expect(failure.attempts[0].httpStatus).toBe(403);
|
||||
expect(failure.primaryCredentialExpired).toBe(true);
|
||||
expect(ledger.inspect("u2").get("openai-compatible")?.reason).toBe(
|
||||
"credential_expired",
|
||||
);
|
||||
|
||||
const hopMeta = vi
|
||||
.mocked(annotate)
|
||||
.mock.calls.map(([arg]) => arg?.meta)
|
||||
.find((m) => m && "ai_chain_hop_1_body" in m);
|
||||
expect(hopMeta?.ai_chain_hop_1_body).toBe(
|
||||
"classified:no_eligible_provider",
|
||||
);
|
||||
});
|
||||
|
||||
it("still advances to the next provider", async () => {
|
||||
const error = await embeddedErrorFrom({
|
||||
error: { code: 503, message: "No provider is available for this model" },
|
||||
});
|
||||
|
||||
const outcome = await runRawCompletionWithFallback({
|
||||
userId: "u3",
|
||||
providers: [
|
||||
{
|
||||
providerType: "openai-compatible",
|
||||
instance: new ThrowingProvider(error),
|
||||
},
|
||||
{ providerType: "local", instance: new OkProvider() },
|
||||
],
|
||||
params: turn(),
|
||||
ledger: createInMemoryProviderHealthLedger(),
|
||||
});
|
||||
|
||||
expect(outcome.result.content).toBe("second provider answered");
|
||||
expect(outcome.fallbackHops[0].httpStatus).toBe(503);
|
||||
});
|
||||
});
|
||||
|
||||
describe("the local provider has the same hole", () => {
|
||||
function local() {
|
||||
return new LocalOpenAICompatibleClient({
|
||||
apiKey: null,
|
||||
model: "llama3",
|
||||
baseUrl: "https://gateway.example.com/api/v1",
|
||||
});
|
||||
}
|
||||
|
||||
it("surfaces an embedded error on the buffered path", async () => {
|
||||
vi.stubGlobal(
|
||||
"fetch",
|
||||
reply200({ error: { code: 402, message: "Insufficient credits" } }),
|
||||
);
|
||||
|
||||
await expect(local().generateCompletion(turn())).rejects.toMatchObject({
|
||||
httpStatus: 402,
|
||||
kind: "embedded_error",
|
||||
upstream: "local",
|
||||
message: "Insufficient credits (local AI endpoint)",
|
||||
});
|
||||
});
|
||||
|
||||
it("surfaces an embedded error the server sent instead of a stream", async () => {
|
||||
vi.stubGlobal(
|
||||
"fetch",
|
||||
reply200({
|
||||
choices: [
|
||||
{
|
||||
error: { code: 429, message: "Rate limited" },
|
||||
finish_reason: "error",
|
||||
},
|
||||
],
|
||||
}),
|
||||
);
|
||||
|
||||
await expect(
|
||||
local().generateCompletionStream(turn(), () => {}),
|
||||
).rejects.toMatchObject({ httpStatus: 429, kind: "embedded_error" });
|
||||
});
|
||||
|
||||
it("surfaces an error frame that arrived before any token", async () => {
|
||||
const encoder = new TextEncoder();
|
||||
vi.stubGlobal(
|
||||
"fetch",
|
||||
vi.fn().mockResolvedValue({
|
||||
ok: true,
|
||||
status: 200,
|
||||
headers: {
|
||||
get: (k: string) =>
|
||||
k === "content-type" ? "text/event-stream" : null,
|
||||
},
|
||||
body: new ReadableStream<Uint8Array>({
|
||||
start(controller) {
|
||||
controller.enqueue(
|
||||
encoder.encode(
|
||||
'data: {"error":{"code":502,"message":"Model is down"}}\n\ndata: [DONE]\n\n',
|
||||
),
|
||||
);
|
||||
controller.close();
|
||||
},
|
||||
}),
|
||||
}),
|
||||
);
|
||||
|
||||
await expect(
|
||||
local().generateCompletionStream(turn(), () => {}),
|
||||
).rejects.toMatchObject({
|
||||
httpStatus: 502,
|
||||
kind: "embedded_error",
|
||||
message: "Model is down (local AI endpoint)",
|
||||
});
|
||||
});
|
||||
|
||||
it("leaves a genuine empty local reply as empty_response", async () => {
|
||||
vi.stubGlobal(
|
||||
"fetch",
|
||||
reply200({ choices: [{ message: { content: "" } }] }),
|
||||
);
|
||||
|
||||
await expect(local().generateCompletion(turn())).rejects.toMatchObject({
|
||||
message: "Local AI returned empty content",
|
||||
httpStatus: 0,
|
||||
kind: "empty_response",
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -7,7 +7,13 @@ import type {
|
||||
CompletionParams,
|
||||
CompletionResult,
|
||||
} from "./types";
|
||||
import { buildOpenAIMessages, mapFinishReason } from "./openai-wire";
|
||||
import {
|
||||
buildOpenAIMessages,
|
||||
mapFinishReason,
|
||||
raiseEmbeddedError,
|
||||
readEmbeddedError,
|
||||
sanitiseBodyExcerpt,
|
||||
} from "./openai-wire";
|
||||
import {
|
||||
hasLearnedJsonModeDialect,
|
||||
isResponseFormatRejection,
|
||||
@@ -214,10 +220,7 @@ export class LocalOpenAICompatibleClient implements AIProvider {
|
||||
// also commonly leak bearer tokens in error envelopes (proxies), so
|
||||
// strip anything that looks like a key before logging.
|
||||
const rawBody = await res.text().catch(() => "");
|
||||
const bodyExcerpt = rawBody
|
||||
.slice(0, 500)
|
||||
.replace(/sk-[A-Za-z0-9_-]{8,}/g, "sk-***redacted***")
|
||||
.replace(/Bearer\s+[A-Za-z0-9_.-]+/gi, "Bearer ***redacted***");
|
||||
const bodyExcerpt = sanitiseBodyExcerpt(rawBody);
|
||||
// v1.28.28 (#470) — dialect self-heal. A 4xx whose body names
|
||||
// `response_format` (or an unknown parameter) means this endpoint
|
||||
// rejects the standard JSON flag: learn the no-flag dialect for the
|
||||
@@ -270,6 +273,18 @@ export class LocalOpenAICompatibleClient implements AIProvider {
|
||||
usage?: { total_tokens?: number };
|
||||
};
|
||||
|
||||
// Same embedded-failure shape the OpenAI-compatible gateway tag handles:
|
||||
// this client serves hosted gateways too (the class doc above names
|
||||
// LiteLLM and OpenRouter), and those answer a provider-side refusal with
|
||||
// HTTP 200 and the failure inside the body. Unconditional here — every
|
||||
// base URL on this provider was typed by a person, there is no pinned
|
||||
// vendor endpoint whose behaviour has to stay untouched.
|
||||
raiseEmbeddedError(json, {
|
||||
upstream: "local",
|
||||
label: "local AI endpoint",
|
||||
model: this.config.model,
|
||||
});
|
||||
|
||||
const content = json.choices?.[0]?.message?.content;
|
||||
if (!content) {
|
||||
// v1.20.1 — sentinel httpStatus + kind so the chain classifier can tell
|
||||
@@ -370,6 +385,13 @@ export class LocalOpenAICompatibleClient implements AIProvider {
|
||||
}>;
|
||||
usage?: { total_tokens?: number };
|
||||
} | null;
|
||||
// The server ignored `stream` and answered buffered — the embedded
|
||||
// failure can ride this body exactly as it rides the non-streaming one.
|
||||
raiseEmbeddedError(json, {
|
||||
upstream: "local",
|
||||
label: "local AI endpoint",
|
||||
model: this.config.model,
|
||||
});
|
||||
const buffered = json?.choices?.[0]?.message?.content;
|
||||
if (!buffered) {
|
||||
const err = new Error("Local AI returned empty content");
|
||||
@@ -397,6 +419,11 @@ export class LocalOpenAICompatibleClient implements AIProvider {
|
||||
let tokensUsed: number | null = null;
|
||||
let finishReason: string | undefined;
|
||||
|
||||
// An SSE frame can carry the same embedded failure the buffered body
|
||||
// carries. Remember the first one; it only matters if the stream ends
|
||||
// without a single token, in which case it is the honest reason.
|
||||
let streamedError: unknown = null;
|
||||
|
||||
let idleTimer: ReturnType<typeof setTimeout> | null = null;
|
||||
const armIdle = () => {
|
||||
if (idleTimer) clearTimeout(idleTimer);
|
||||
@@ -421,9 +448,11 @@ export class LocalOpenAICompatibleClient implements AIProvider {
|
||||
const payload = line.slice("data:".length).trim();
|
||||
if (!payload || payload === "[DONE]") continue;
|
||||
let chunk: {
|
||||
error?: unknown;
|
||||
choices?: Array<{
|
||||
delta?: { content?: string };
|
||||
finish_reason?: string | null;
|
||||
error?: unknown;
|
||||
}>;
|
||||
usage?: { total_tokens?: number };
|
||||
};
|
||||
@@ -433,6 +462,9 @@ export class LocalOpenAICompatibleClient implements AIProvider {
|
||||
// A malformed chunk line is skipped, not fatal — keep reading.
|
||||
continue;
|
||||
}
|
||||
if (streamedError === null && readEmbeddedError(chunk) !== null) {
|
||||
streamedError = chunk;
|
||||
}
|
||||
const delta = chunk.choices?.[0]?.delta?.content;
|
||||
if (delta) {
|
||||
content += delta;
|
||||
@@ -466,6 +498,13 @@ export class LocalOpenAICompatibleClient implements AIProvider {
|
||||
}
|
||||
|
||||
if (!content) {
|
||||
// A stream that ended with a failure frame and no token reports the
|
||||
// failure, not "empty content".
|
||||
raiseEmbeddedError(streamedError, {
|
||||
upstream: "local",
|
||||
label: "local AI endpoint",
|
||||
model: this.config.model,
|
||||
});
|
||||
const err = new Error("Local AI returned empty content");
|
||||
Object.assign(err, {
|
||||
httpStatus: 0,
|
||||
|
||||
@@ -15,6 +15,8 @@ import {
|
||||
mapFinishReason,
|
||||
parseCachedTokens,
|
||||
parseOpenAIToolCalls,
|
||||
raiseEmbeddedError,
|
||||
sanitiseBodyExcerpt,
|
||||
type OpenAIResponseJson,
|
||||
} from "./openai-wire";
|
||||
|
||||
@@ -179,10 +181,7 @@ export class OpenAIClient implements AIProvider {
|
||||
// surfacing as an opaque "OpenAI request failed (500)". Strip anything
|
||||
// that looks like an API key from the excerpt before logging.
|
||||
const rawBody = await res.text().catch(() => "");
|
||||
const bodyExcerpt = rawBody
|
||||
.slice(0, 500)
|
||||
.replace(/sk-[A-Za-z0-9_-]{8,}/g, "sk-***redacted***")
|
||||
.replace(/Bearer\s+[A-Za-z0-9_.-]+/gi, "Bearer ***redacted***");
|
||||
const bodyExcerpt = sanitiseBodyExcerpt(rawBody);
|
||||
// v1.33.1 (#470) — dialect self-heal, gateway only. A 4xx whose body
|
||||
// names `response_format` (or an unknown parameter) means this endpoint
|
||||
// rejects the standard JSON flag: learn the no-flag dialect for the base
|
||||
@@ -232,6 +231,30 @@ export class OpenAIClient implements AIProvider {
|
||||
}
|
||||
|
||||
const json = (await res.json()) as OpenAIResponseJson;
|
||||
|
||||
// A 200 that carries the failure inside the body. OpenRouter (and every
|
||||
// gateway that copies it) answers a provider-side refusal — no credits,
|
||||
// a data-policy or moderation block, an upstream timeout, a model that is
|
||||
// down, no provider matching the routing requirements — with HTTP 200 and
|
||||
// a top-level `error` object instead of `choices`, and a generation that
|
||||
// died mid-flight with `choices[0].error` beside `finish_reason: "error"`.
|
||||
// Both used to fall through to the empty-content throw below, so every one
|
||||
// of those causes collapsed into "returned empty content", status 0, no
|
||||
// excerpt: neither the person nor the provider-health card could tell them
|
||||
// apart. Raise the same shaped error a non-2xx reply raises so the chain
|
||||
// classifier, the health ledger and the card all see the real status.
|
||||
//
|
||||
// Gated on the gateway tag because the shape is gateway-only: on
|
||||
// `api.openai.com` — the `admin-key` and `codex` tags — a failure is
|
||||
// always a non-2xx status, so their parsing is untouched.
|
||||
if (this.isGateway) {
|
||||
raiseEmbeddedError(json, {
|
||||
upstream: "openai-compatible",
|
||||
label: "OpenAI-compatible gateway",
|
||||
model: this.config.model,
|
||||
});
|
||||
}
|
||||
|
||||
const choice = json.choices?.[0];
|
||||
let content = choice?.message?.content;
|
||||
const toolCalls = parseOpenAIToolCalls(choice);
|
||||
|
||||
@@ -154,4 +154,101 @@ export function parseCachedTokens(json: OpenAIResponseJson): number | null {
|
||||
return json.usage?.prompt_tokens_details?.cached_tokens ?? null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Bound and key-redact an upstream body before it is attached to a thrown
|
||||
* error. Both clients inlined this next to their non-2xx handling; it lives
|
||||
* here once so the embedded-error excerpt below is sanitised by the very same
|
||||
* code rather than by a copy that can drift.
|
||||
*/
|
||||
export function sanitiseBodyExcerpt(raw: string): string {
|
||||
return raw
|
||||
.slice(0, 500)
|
||||
.replace(/sk-[A-Za-z0-9_-]{8,}/g, "sk-***redacted***")
|
||||
.replace(/Bearer\s+[A-Za-z0-9_.-]+/gi, "Bearer ***redacted***");
|
||||
}
|
||||
|
||||
interface EmbeddedError {
|
||||
message: string;
|
||||
/** The gateway's own `error.code` when it is an HTTP error status, else 0. */
|
||||
httpStatus: number;
|
||||
}
|
||||
|
||||
/** Longest slice of a remote error message that reaches a thrown message. */
|
||||
const EMBEDDED_MESSAGE_MAX = 200;
|
||||
|
||||
function coerceEmbeddedError(candidate: unknown): EmbeddedError | null {
|
||||
if (typeof candidate === "string") {
|
||||
const trimmed = candidate.trim();
|
||||
return trimmed.length > 0 ? { message: trimmed, httpStatus: 0 } : null;
|
||||
}
|
||||
if (candidate === null || typeof candidate !== "object") return null;
|
||||
const raw = candidate as { code?: unknown; message?: unknown };
|
||||
const message =
|
||||
typeof raw.message === "string" && raw.message.trim().length > 0
|
||||
? raw.message.trim()
|
||||
: "gateway reported a failure without a message";
|
||||
// Only an error-range status is adopted. A gateway that puts a vendor
|
||||
// string ("insufficient_quota") or a non-status number in `code` keeps the
|
||||
// 0 sentinel, which the chain classifier already treats as a hard failure.
|
||||
const code = raw.code;
|
||||
const httpStatus =
|
||||
typeof code === "number" &&
|
||||
Number.isInteger(code) &&
|
||||
code >= 400 &&
|
||||
code <= 599
|
||||
? code
|
||||
: 0;
|
||||
return { message, httpStatus };
|
||||
}
|
||||
|
||||
/**
|
||||
* Read a failure that a 2xx body carries INSIDE itself.
|
||||
*
|
||||
* OpenAI-compatible gateways (OpenRouter and friends) answer a provider-side
|
||||
* refusal with HTTP 200 and either a top-level `error` object and no `choices`
|
||||
* at all, or — when a generation dies mid-flight — `choices[0].error` beside
|
||||
* `finish_reason: "error"`. Returns the embedded failure, or null for a normal
|
||||
* reply. `api.openai.com` never answers this way; every failure there is a
|
||||
* non-2xx status.
|
||||
*/
|
||||
export function readEmbeddedError(json: unknown): EmbeddedError | null {
|
||||
if (json === null || typeof json !== "object") return null;
|
||||
const body = json as {
|
||||
error?: unknown;
|
||||
choices?: Array<{ error?: unknown } | null> | null;
|
||||
};
|
||||
return (
|
||||
coerceEmbeddedError(body.error) ??
|
||||
coerceEmbeddedError(body.choices?.[0]?.error)
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Throw the same shaped error the non-2xx path throws when a 2xx body carries
|
||||
* an embedded failure; return quietly when it does not. The gateway's own
|
||||
* message leads so the hop reason says WHY the call failed instead of
|
||||
* "returned empty content"; it is bounded and key-redacted on the way in,
|
||||
* because remote text is never trusted to be short or clean.
|
||||
*/
|
||||
export function raiseEmbeddedError(
|
||||
json: unknown,
|
||||
ctx: { upstream: string; label: string; model: string },
|
||||
): void {
|
||||
const embedded = readEmbeddedError(json);
|
||||
if (!embedded) return;
|
||||
const message = sanitiseBodyExcerpt(embedded.message).slice(
|
||||
0,
|
||||
EMBEDDED_MESSAGE_MAX,
|
||||
);
|
||||
const err = new Error(`${message} (${ctx.label})`);
|
||||
Object.assign(err, {
|
||||
httpStatus: embedded.httpStatus,
|
||||
kind: "embedded_error",
|
||||
upstream: ctx.upstream,
|
||||
model: ctx.model,
|
||||
bodyExcerpt: sanitiseBodyExcerpt(JSON.stringify(json) ?? ""),
|
||||
});
|
||||
throw err;
|
||||
}
|
||||
|
||||
export type { OpenAIChoice, OpenAIResponseJson };
|
||||
@@ -356,9 +356,28 @@ export function classifyErrorBody(body: unknown): string | null {
|
||||
if (lowered.includes("response_format") || lowered.includes("json_schema")) {
|
||||
return "classified:response_format_rejected";
|
||||
}
|
||||
if (lowered.includes("insufficient_quota") || lowered.includes("billing")) {
|
||||
if (
|
||||
lowered.includes("insufficient_quota") ||
|
||||
lowered.includes("billing") ||
|
||||
// A gateway that meters prepaid credit says so in its own words rather
|
||||
// than OpenAI's `insufficient_quota`; it is the same class of refusal.
|
||||
lowered.includes("insufficient credits") ||
|
||||
lowered.includes("insufficient_credits")
|
||||
) {
|
||||
return "classified:quota_or_billing";
|
||||
}
|
||||
// A routing refusal: the account's data policy / zero-retention settings, or
|
||||
// the model's own availability, leave no provider eligible to serve the
|
||||
// call. Distinct from quota — nothing is spent and nothing is expired; the
|
||||
// request simply has nowhere to go.
|
||||
if (
|
||||
lowered.includes("data policy") ||
|
||||
lowered.includes("data_policy") ||
|
||||
lowered.includes("no allowed providers") ||
|
||||
lowered.includes("no endpoints found")
|
||||
) {
|
||||
return "classified:no_eligible_provider";
|
||||
}
|
||||
if (lowered.includes("invalid_api_key") || lowered.includes("unauthorized")) {
|
||||
return "classified:auth_rejected";
|
||||
}
|
||||
|
||||
Reference in new issue
Block a user