mirror of
https://github.com/MBombeck/HealthLog.git
synced 2026-11-12 22:26:13 +00:00
Marc-André Bombeck
73cee35917
A gateway that refuses reasoning_effort outright (LiteLLM in front of Ollama: "does not support parameters: ['reasoning_effort']") in front of a model that reasons without answering sent requests until something outside stopped it. The answerless rule learned to ask with "none", the refusal of that "none" was learned as "send no field" over it, the next request came back answerless again, and each lesson undid the other. Each call of the Local and the OpenAI-compatible gateway clients now carries one request budget through every repeat (request-hops.ts): at most two requests, after which the call ends with the failure it has. A structural guard holds both clients to sending only through the budget and to handing it to every repeat. The refusal is now learned as what it is: an endpoint that refuses the field, in front of a model already seen reasoning unasked, keeps both facts (FIELD_REFUSED_REASONS_REGARDLESS), so it is never asked with "none" again and keeps its thinking room. An answerless reply from an endpoint that refuses the field is final. When the refused "none" was the repeat of an answerless reply, the call fails as reasoning_only, which is what it ran into. The refusal handling of both clients lives in wire-refusal.ts. A status run that hits its bound now aborts the call it gave up on instead of leaving it running detached. Refs #1279
70 lines
2.0 KiB
TypeScript
70 lines
2.0 KiB
TypeScript
/**
|
|
* What a refused request teaches the Local and the OpenAI-compatible gateway
|
|
* clients, in one place for both.
|
|
*
|
|
* Only an HTTP 400 that names a field the request carried teaches anything:
|
|
* the reasoning field (`isReasoningParameterRejection`, learned per endpoint
|
|
* and model) or the JSON flag (`isResponseFormatRejection`, learned per
|
|
* endpoint). The reasoning field is checked first, so a body that names it
|
|
* never switches JSON mode off. Whether the call then sends the lesser request
|
|
* is the client's, inside its request budget (`request-hops.ts`).
|
|
*
|
|
* Server-only.
|
|
*/
|
|
import { annotate } from "@/lib/logging/context";
|
|
|
|
import {
|
|
isResponseFormatRejection,
|
|
rememberJsonModeDialect,
|
|
} from "./json-dialect";
|
|
import { learnReasoningFieldRefused } from "./reasoning/answerless";
|
|
import {
|
|
isReasoningParameterRejection,
|
|
type ReasoningDialectKey,
|
|
} from "./reasoning/dialect-cache";
|
|
|
|
export type RefusalLesson = "reasoning" | "json" | null;
|
|
|
|
export function learnFromRefusal(args: {
|
|
status: number;
|
|
body: string;
|
|
model: string;
|
|
/** The reasoning field the request carried, or `null` for none. */
|
|
reasoning: {
|
|
sent: string;
|
|
key: ReasoningDialectKey;
|
|
provider: string;
|
|
/** The parameter names this endpoint's refusals use. */
|
|
names: RegExp;
|
|
} | null;
|
|
/** The endpoint, when the request carried the JSON flag. */
|
|
json: { endpoint: string; action: string } | null;
|
|
}): RefusalLesson {
|
|
const { reasoning, json } = args;
|
|
if (
|
|
reasoning &&
|
|
isReasoningParameterRejection(
|
|
args.status,
|
|
args.body,
|
|
reasoning.names,
|
|
args.model,
|
|
)
|
|
) {
|
|
learnReasoningFieldRefused(
|
|
reasoning.key,
|
|
reasoning.provider,
|
|
reasoning.sent,
|
|
);
|
|
return "reasoning";
|
|
}
|
|
if (json && isResponseFormatRejection(args.status, args.body)) {
|
|
rememberJsonModeDialect(json.endpoint, "none");
|
|
annotate({
|
|
action: { name: json.action },
|
|
meta: { dialect: "none", httpStatus: args.status },
|
|
});
|
|
return "json";
|
|
}
|
|
return null;
|
|
}
|