mirror of
https://github.com/MBombeck/HealthLog.git
synced 2026-11-11 13:52:20 +00:00
Marc-André Bombeck
73cee35917
A gateway that refuses reasoning_effort outright (LiteLLM in front of Ollama: "does not support parameters: ['reasoning_effort']") in front of a model that reasons without answering sent requests until something outside stopped it. The answerless rule learned to ask with "none", the refusal of that "none" was learned as "send no field" over it, the next request came back answerless again, and each lesson undid the other. Each call of the Local and the OpenAI-compatible gateway clients now carries one request budget through every repeat (request-hops.ts): at most two requests, after which the call ends with the failure it has. A structural guard holds both clients to sending only through the budget and to handing it to every repeat. The refusal is now learned as what it is: an endpoint that refuses the field, in front of a model already seen reasoning unasked, keeps both facts (FIELD_REFUSED_REASONS_REGARDLESS), so it is never asked with "none" again and keeps its thinking room. An answerless reply from an endpoint that refuses the field is final. When the refused "none" was the repeat of an answerless reply, the call fails as reasoning_only, which is what it ran into. The refusal handling of both clients lives in wire-refusal.ts. A status run that hits its bound now aborts the call it gave up on instead of leaving it running detached. Refs #1279
50 lines
1.8 KiB
TypeScript
50 lines
1.8 KiB
TypeScript
/**
|
|
* #1279 — the HTTP requests one completion call may send.
|
|
*
|
|
* The Local and the OpenAI-compatible gateway clients send a request again on
|
|
* their own: without a reasoning field the endpoint refused, without the JSON
|
|
* flag it refused, with `reasoning_effort: "none"` after a reply that
|
|
* reasoned without answering, and the Local stream falls back to a buffered
|
|
* request. Each repeat was bounded by what it had just learned, and two
|
|
* lessons that undo each other were not bounded at all: a gateway that
|
|
* refuses `reasoning_effort` in front of a model that reasons unasked made
|
|
* one call send requests until something outside stopped it.
|
|
*
|
|
* So each call carries one budget through every repeat, and the budget bounds
|
|
* it, not the lessons: at most `MAX_REQUESTS_PER_CALL` requests, after which
|
|
* the call ends with the failure it has. What was learned on the way still
|
|
* holds for the next call. `ai-request-hops-guard.test.ts` holds both clients
|
|
* to sending only through here and to handing the budget to every repeat.
|
|
*
|
|
* Server-only.
|
|
*/
|
|
import { safeFetch } from "@/lib/safe-fetch";
|
|
|
|
/** The requests one call may send: the first, and one repeat. */
|
|
export const MAX_REQUESTS_PER_CALL = 2;
|
|
|
|
export class RequestHops {
|
|
private sent = 0;
|
|
|
|
/** True while the call may send one more request. */
|
|
get canSend(): boolean {
|
|
return this.sent < MAX_REQUESTS_PER_CALL;
|
|
}
|
|
|
|
/**
|
|
* `safeFetch`, counted. A request past the budget is not sent: every repeat
|
|
* asks `canSend` first, so this throw marks a repeat that did not.
|
|
*/
|
|
fetch(...args: Parameters<typeof safeFetch>): Promise<Response> {
|
|
if (!this.canSend) {
|
|
return Promise.reject(
|
|
Object.assign(new Error("AI call has no request left to send"), {
|
|
httpStatus: 0,
|
|
}),
|
|
);
|
|
}
|
|
this.sent += 1;
|
|
return safeFetch(...args);
|
|
}
|
|
}
|