mirror of
https://github.com/MBombeck/HealthLog.git
synced 2026-11-11 13:52:20 +00:00
Marc-André Bombeck
73cee35917
A gateway that refuses reasoning_effort outright (LiteLLM in front of Ollama: "does not support parameters: ['reasoning_effort']") in front of a model that reasons without answering sent requests until something outside stopped it. The answerless rule learned to ask with "none", the refusal of that "none" was learned as "send no field" over it, the next request came back answerless again, and each lesson undid the other. Each call of the Local and the OpenAI-compatible gateway clients now carries one request budget through every repeat (request-hops.ts): at most two requests, after which the call ends with the failure it has. A structural guard holds both clients to sending only through the budget and to handing it to every repeat. The refusal is now learned as what it is: an endpoint that refuses the field, in front of a model already seen reasoning unasked, keeps both facts (FIELD_REFUSED_REASONS_REGARDLESS), so it is never asked with "none" again and keeps its thinking room. An answerless reply from an endpoint that refuses the field is final. When the refused "none" was the repeat of an answerless reply, the call fails as reasoning_only, which is what it ran into. The refusal handling of both clients lives in wire-refusal.ts. A status run that hits its bound now aborts the call it gave up on instead of leaving it running detached. Refs #1279
59 lines
2.0 KiB
TypeScript
59 lines
2.0 KiB
TypeScript
/**
|
|
* Request and error shapes of the Local client that need nothing of its state.
|
|
* Split out of `local-client.ts`; that client is their only importer.
|
|
*
|
|
* Server-only.
|
|
*/
|
|
import type { AiMessage } from "./types";
|
|
|
|
const STRICT_JSON_PREFIX =
|
|
"Return strict JSON only, no markdown, no commentary.\n\n";
|
|
|
|
/**
|
|
* Prepend the strict-JSON instruction to the FIRST user turn's text. Local
|
|
* model templates respect an in-message instruction reliably, so it is sent
|
|
* in BOTH dialects — it is the only JSON steering left when an endpoint
|
|
* rejects `response_format` (dialect `"none"`). Mirrors the pre-refactor
|
|
* behaviour, which prepended to the single user prompt.
|
|
*/
|
|
export function withStrictJsonPrefix(messages: AiMessage[]): AiMessage[] {
|
|
const idx = messages.findIndex((m) => m.role === "user");
|
|
if (idx === -1) return messages;
|
|
const out = [...messages];
|
|
const target = out[idx];
|
|
if (typeof target.content === "string") {
|
|
out[idx] = { ...target, content: `${STRICT_JSON_PREFIX}${target.content}` };
|
|
} else {
|
|
// Vision parts: prepend a leading text part with the instruction.
|
|
out[idx] = {
|
|
...target,
|
|
content: [{ type: "text", text: STRICT_JSON_PREFIX }, ...target.content],
|
|
};
|
|
}
|
|
return out;
|
|
}
|
|
|
|
/**
|
|
* The empty-reply error. `httpStatus: 0` is the sentinel the chain classifier
|
|
* treats as a hard failure; `kind` says whether the reply held reasoning and
|
|
* no answer, or nothing at all, so the log names it instead of "network".
|
|
*/
|
|
export function emptyReplyError(
|
|
hadReasoning: boolean,
|
|
spent: number | null,
|
|
): Error {
|
|
const err = new Error(
|
|
hadReasoning
|
|
? "Local AI returned reasoning but no answer"
|
|
: "Local AI returned empty content",
|
|
);
|
|
Object.assign(err, {
|
|
httpStatus: 0,
|
|
kind: hadReasoning ? "reasoning_only" : "empty_response",
|
|
upstream: "local",
|
|
// #1279 — what the reply cost, for the day's ledger to charge.
|
|
...(spent !== null ? { tokensUsed: spent } : {}),
|
|
});
|
|
return err;
|
|
}
|