mirror of
https://github.com/MBombeck/HealthLog.git
synced 2026-11-08 12:32:43 +00:00
fix(ai): bound one completion call to two requests
A gateway that refuses reasoning_effort outright (LiteLLM in front of Ollama: "does not support parameters: ['reasoning_effort']") in front of a model that reasons without answering sent requests until something outside stopped it. The answerless rule learned to ask with "none", the refusal of that "none" was learned as "send no field" over it, the next request came back answerless again, and each lesson undid the other. Each call of the Local and the OpenAI-compatible gateway clients now carries one request budget through every repeat (request-hops.ts): at most two requests, after which the call ends with the failure it has. A structural guard holds both clients to sending only through the budget and to handing it to every repeat. The refusal is now learned as what it is: an endpoint that refuses the field, in front of a model already seen reasoning unasked, keeps both facts (FIELD_REFUSED_REASONS_REGARDLESS), so it is never asked with "none" again and keeps its thinking room. An answerless reply from an endpoint that refuses the field is final. When the refused "none" was the repeat of an answerless reply, the call fails as reasoning_only, which is what it ran into. The refusal handling of both clients lives in wire-refusal.ts. A status run that hits its bound now aborts the call it gave up on instead of leaving it running detached. Refs #1279
This commit is contained in:
1 parent
3984ceb01c
commit
73cee35917
24 files changed
+1058
-296
No files matched your search
@@ -0,0 +1,127 @@
|
||||
/**
|
||||
* Structural guard (#1279): a provider client that sends a request again on
|
||||
* its own sends every request through the call's budget, and hands that
|
||||
* budget to every repeat.
|
||||
*
|
||||
* A client that repeats a request (`this.generateCompletion(` or
|
||||
* `this.generateCompletionStream(` inside its own class) decided each repeat
|
||||
* from what it had just learned, and two lessons that undid each other made
|
||||
* one call send requests until something outside stopped it. The budget in
|
||||
* `src/lib/ai/request-hops.ts` refuses the request past two; it can only do
|
||||
* that if no request goes around it and no repeat starts a fresh one. So, in
|
||||
* every such client:
|
||||
*
|
||||
* - no request goes out through `safeFetch(` directly, only `hops.fetch(`;
|
||||
* - every repeat passes `hops` on;
|
||||
* - each public entry takes the budget as its last parameter, defaulting
|
||||
* to a fresh one, so a caller outside starts one per call.
|
||||
*
|
||||
* `request-hops.test.ts` walks the refusal and answerless combinations
|
||||
* against a loopback server and counts the requests.
|
||||
*
|
||||
* Its limit: it reads source text. A client that repeats through a helper
|
||||
* that is not a method named `generateCompletion*`, or sends through another
|
||||
* transport, is not found here.
|
||||
*
|
||||
* Mutation check: drop `hops` from one repeat, or put one `safeFetch(` back
|
||||
* into either client, and this goes red.
|
||||
*/
|
||||
import { join } from "node:path";
|
||||
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import {
|
||||
readSourceText,
|
||||
requireFloor,
|
||||
stripComments,
|
||||
walkSourceFiles,
|
||||
} from "./helpers/source-files";
|
||||
|
||||
const AI = join(process.cwd(), "src/lib/ai");
|
||||
|
||||
/** The text of the call that starts at `open` (the index of its `(`). */
|
||||
function callText(source: string, open: number): string {
|
||||
let depth = 0;
|
||||
for (let i = open; i < source.length; i += 1) {
|
||||
if (source[i] === "(") depth += 1;
|
||||
else if (source[i] === ")") {
|
||||
depth -= 1;
|
||||
if (depth === 0) return source.slice(open, i + 1);
|
||||
}
|
||||
}
|
||||
return source.slice(open);
|
||||
}
|
||||
|
||||
const REPEAT = /\bthis\.generateCompletion(?:Stream)?\s*\(/g;
|
||||
|
||||
function repeatingClients(): Array<{ rel: string; source: string }> {
|
||||
const out: Array<{ rel: string; source: string }> = [];
|
||||
for (const rel of walkSourceFiles(AI, { floor: 40 })) {
|
||||
if (rel.includes("__tests__/")) continue;
|
||||
const source = stripComments(readSourceText(join(AI, rel)));
|
||||
if (REPEAT.test(source)) out.push({ rel, source });
|
||||
REPEAT.lastIndex = 0;
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
describe("a client's own repeats stay inside the call's request budget", () => {
|
||||
const clients = repeatingClients();
|
||||
|
||||
it("finds the clients that repeat a request", () => {
|
||||
// The Local client and the OpenAI-compatible gateway client today.
|
||||
requireFloor("repeating clients", clients, 2);
|
||||
expect(clients.map((c) => c.rel).sort()).toEqual([
|
||||
"local-client.ts",
|
||||
"openai-client.ts",
|
||||
]);
|
||||
});
|
||||
|
||||
it("sends every request through the budget", () => {
|
||||
for (const { rel, source } of clients) {
|
||||
expect({ rel, direct: /\bsafeFetch\s*\(/.test(source) }).toEqual({
|
||||
rel,
|
||||
direct: false,
|
||||
});
|
||||
expect({ rel, counted: /\bhops\.fetch\s*\(/.test(source) }).toEqual({
|
||||
rel,
|
||||
counted: true,
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
it("hands the budget to every repeat", () => {
|
||||
const repeats: Array<{ rel: string; text: string }> = [];
|
||||
for (const { rel, source } of clients) {
|
||||
for (const m of source.matchAll(REPEAT)) {
|
||||
const open = (m.index ?? 0) + m[0].length - 1;
|
||||
repeats.push({ rel, text: callText(source, open) });
|
||||
}
|
||||
}
|
||||
requireFloor("repeats", repeats, 6);
|
||||
expect(
|
||||
repeats.filter(({ text }) => !/,\s*hops\s*,?\s*\)$/.test(text)),
|
||||
).toEqual([]);
|
||||
});
|
||||
|
||||
it("starts one budget per call at each public entry", () => {
|
||||
const entries: Array<{ rel: string; text: string }> = [];
|
||||
for (const { rel, source } of clients) {
|
||||
for (const m of source.matchAll(
|
||||
/\basync\s+generateCompletion(?:Stream)?\s*\(/g,
|
||||
)) {
|
||||
const open = (m.index ?? 0) + m[0].length - 1;
|
||||
entries.push({ rel, text: callText(source, open) });
|
||||
}
|
||||
}
|
||||
requireFloor("public entries", entries, 3);
|
||||
expect(
|
||||
entries.filter(
|
||||
({ text }) =>
|
||||
!/hops:\s*RequestHops\s*=\s*new RequestHops\(\)\s*,?\s*\)$/.test(
|
||||
text,
|
||||
),
|
||||
),
|
||||
).toEqual([]);
|
||||
});
|
||||
});
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"$comment": "Production files over 800 lines and their length. Maintained by src/__tests__/source-size-ratchet.test.ts.",
|
||||
"count": 87,
|
||||
"total": 97455,
|
||||
"total": 97452,
|
||||
"entries": {
|
||||
"app/medications/page-client.tsx": 839,
|
||||
"components/charts/mood-chart.tsx": 1166,
|
||||
@@ -29,7 +29,7 @@
|
||||
"lib/ai/coach/stream-events.ts": 817,
|
||||
"lib/ai/coach/types.ts": 1011,
|
||||
"lib/ai/codex-client.ts": 1045,
|
||||
"lib/ai/local-client.ts": 824,
|
||||
"lib/ai/local-client.ts": 821,
|
||||
"lib/ai/prompts/insight-generator.ts": 1110,
|
||||
"lib/ai/provider.ts": 1632,
|
||||
"lib/analytics/score/reader.ts": 998,
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"$comment": "Production functions over 200 lines, keyed \"file :: name\", and their length. Maintained by src/__tests__/source-size-ratchet.test.ts.",
|
||||
"count": 351,
|
||||
"total": 113672,
|
||||
"total": 113581,
|
||||
"entries": {
|
||||
"app/achievements/page.tsx :: AchievementsPage": 206,
|
||||
"app/api/admin/backups/upload/route.ts :: apiHandler(arg)": 314,
|
||||
@@ -254,8 +254,8 @@
|
||||
"lib/ai/coach/turn/pipeline.ts :: runTurn": 274,
|
||||
"lib/ai/coach/turn/reply-guards.ts :: guardReply": 331,
|
||||
"lib/ai/codex-client.ts :: parseResponse": 302,
|
||||
"lib/ai/local-client.ts :: generateCompletionStream": 294,
|
||||
"lib/ai/openai-client.ts :: generateCompletion": 335,
|
||||
"lib/ai/local-client.ts :: generateCompletionStream": 263,
|
||||
"lib/ai/openai-client.ts :: generateCompletion": 276,
|
||||
"lib/analytics/bp-in-target-fast-path.ts :: computeFromRollups": 203,
|
||||
"lib/analytics/compliance/display.ts :: calculateCompliance": 212,
|
||||
"lib/analytics/correlations-fast-path.ts :: computeCorrelationHypothesesFastPath": 256,
|
||||
@@ -298,7 +298,7 @@
|
||||
"lib/insights/comprehensive-generate.ts :: generateComprehensiveInsight": 876,
|
||||
"lib/insights/features.ts :: extractFeatures": 796,
|
||||
"lib/insights/mood-aggregates.ts :: fetchMoodAggregates": 246,
|
||||
"lib/insights/status-provider.ts :: runStatusCompletion": 226,
|
||||
"lib/insights/status-provider.ts :: runStatusCompletion": 225,
|
||||
"lib/integrations/connection-health-load.ts :: loadConnectionFacts": 227,
|
||||
"lib/integrations/status.ts :: recordSyncFailure": 292,
|
||||
"lib/jobs/apple-health-import-worker.ts :: handleAppleHealthImport": 300,
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"$comment": "Comment lines that open with a release version, per file under src/. Maintained by src/__tests__/version-stamp-ratchet.test.ts.",
|
||||
"count": 2897,
|
||||
"total": 9272,
|
||||
"total": 9267,
|
||||
"entries": {
|
||||
"__tests__/account-payload-consumer-guard.test.ts": 2,
|
||||
"__tests__/acting-account-boundary-guard.test.ts": 7,
|
||||
@@ -1748,10 +1748,10 @@
|
||||
"lib/ai/insight-interpretation.ts": 1,
|
||||
"lib/ai/legacy-payload.ts": 2,
|
||||
"lib/ai/legacy-private-host-report.ts": 1,
|
||||
"lib/ai/local-client.ts": 18,
|
||||
"lib/ai/local-client.ts": 16,
|
||||
"lib/ai/medical-references.ts": 1,
|
||||
"lib/ai/mock-client.ts": 5,
|
||||
"lib/ai/openai-client.ts": 16,
|
||||
"lib/ai/openai-client.ts": 13,
|
||||
"lib/ai/prompts/__tests__/assessment-prompt-contract.test.ts": 5,
|
||||
"lib/ai/prompts/__tests__/briefing-personalisation.test.ts": 1,
|
||||
"lib/ai/prompts/__tests__/coach-prompt-locale.test.ts": 1,
|
||||
|
||||
@@ -0,0 +1,281 @@
|
||||
/**
|
||||
* #1279 — one completion call sends at most two requests, whatever the
|
||||
* endpoint answers.
|
||||
*
|
||||
* A gateway in front of Ollama (LiteLLM: "ollama does not support
|
||||
* parameters: ['reasoning_effort']") refuses the reasoning field outright,
|
||||
* and the model behind it reasons unasked and answers nothing. The answerless
|
||||
* rule asked again with `none`, the refusal rule wrote "send no field" over
|
||||
* that lesson, the next request was answerless again, and so on: one call
|
||||
* sent requests until something outside stopped it.
|
||||
*
|
||||
* Driven through the real clients, the real `safeFetch` and the pinned
|
||||
* dispatcher against a loopback server that answers each request by a
|
||||
* script, so every combination of refusal and answerless reply is walked.
|
||||
*/
|
||||
import http from "node:http";
|
||||
import type { AddressInfo } from "node:net";
|
||||
import {
|
||||
afterAll,
|
||||
afterEach,
|
||||
beforeAll,
|
||||
beforeEach,
|
||||
describe,
|
||||
expect,
|
||||
it,
|
||||
vi,
|
||||
} from "vitest";
|
||||
|
||||
vi.mock("@/lib/logging/signal", () => ({ emitSignal: () => {} }));
|
||||
|
||||
import { LocalOpenAICompatibleClient } from "../local-client";
|
||||
import { OpenAIClient } from "../openai-client";
|
||||
import { resetJsonModeDialectCache } from "../json-dialect";
|
||||
import { resetReasoningDialectCache } from "../reasoning/dialect-cache";
|
||||
import { MAX_REQUESTS_PER_CALL } from "../request-hops";
|
||||
import { singleUserTurn, type CompletionParams } from "../types";
|
||||
import { _resetPinnedDispatcherForTests } from "@/lib/safe-fetch-dispatcher";
|
||||
|
||||
interface Seen {
|
||||
effort: unknown;
|
||||
json: boolean;
|
||||
stream: boolean;
|
||||
}
|
||||
|
||||
/** Decides one reply from what the request carried. */
|
||||
type Script = (req: Seen) => { status: number; body: unknown };
|
||||
|
||||
const REFUSE_REASONING = {
|
||||
status: 400,
|
||||
body: {
|
||||
error: {
|
||||
message:
|
||||
"litellm.UnsupportedParamsError: ollama does not support parameters: ['reasoning_effort']",
|
||||
},
|
||||
},
|
||||
};
|
||||
|
||||
const REFUSE_JSON = {
|
||||
status: 400,
|
||||
body: { error: { message: "Unsupported parameter: response_format" } },
|
||||
};
|
||||
|
||||
function answerless() {
|
||||
return {
|
||||
status: 200,
|
||||
body: {
|
||||
choices: [
|
||||
{
|
||||
index: 0,
|
||||
message: {
|
||||
role: "assistant",
|
||||
content: null,
|
||||
reasoning_content: "thinking ".repeat(50),
|
||||
},
|
||||
finish_reason: "length",
|
||||
},
|
||||
],
|
||||
usage: { total_tokens: 1000 },
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/** Refuses the reasoning field, reasons without answering otherwise. */
|
||||
const gatewayRefusingTheField: Script = (req) =>
|
||||
req.effort !== undefined ? REFUSE_REASONING : answerless();
|
||||
|
||||
/** Refuses the JSON flag and the reasoning field, answerless otherwise. */
|
||||
const refusesEverything: Script = (req) =>
|
||||
req.json
|
||||
? REFUSE_JSON
|
||||
: req.effort !== undefined
|
||||
? REFUSE_REASONING
|
||||
: answerless();
|
||||
|
||||
/** Refuses every streamed request and the reasoning field. */
|
||||
const refusesStreams: Script = (req) =>
|
||||
req.stream
|
||||
? { status: 400, body: { error: { message: "no stream" } } }
|
||||
: gatewayRefusingTheField(req);
|
||||
|
||||
let server: http.Server;
|
||||
let origin = "";
|
||||
let script: Script = gatewayRefusingTheField;
|
||||
let seen: Seen[] = [];
|
||||
|
||||
beforeAll(async () => {
|
||||
server = http.createServer((req, res) => {
|
||||
let raw = "";
|
||||
req.on("data", (c) => (raw += c));
|
||||
req.on("end", () => {
|
||||
if (!raw) {
|
||||
res.writeHead(404);
|
||||
res.end();
|
||||
return;
|
||||
}
|
||||
const body = JSON.parse(raw) as Record<string, unknown>;
|
||||
const request: Seen = {
|
||||
effort: body.reasoning_effort,
|
||||
json: body.response_format !== undefined,
|
||||
stream: body.stream === true,
|
||||
};
|
||||
seen.push(request);
|
||||
// A backstop for the unbounded case, so a failing run ends.
|
||||
const reply =
|
||||
seen.length > 40
|
||||
? { status: 500, body: { error: { message: "stop" } } }
|
||||
: script(request);
|
||||
res.writeHead(reply.status, { "Content-Type": "application/json" });
|
||||
res.end(JSON.stringify(reply.body));
|
||||
});
|
||||
});
|
||||
await new Promise<void>((r) => server.listen(0, "127.0.0.1", r));
|
||||
origin = `http://127.0.0.1:${(server.address() as AddressInfo).port}`;
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
_resetPinnedDispatcherForTests();
|
||||
await new Promise<void>((r) => server.close(() => r()));
|
||||
});
|
||||
|
||||
const savedOrigins = process.env.AI_PRIVATE_ORIGINS;
|
||||
|
||||
beforeEach(() => {
|
||||
process.env.AI_PRIVATE_ORIGINS = origin;
|
||||
seen = [];
|
||||
resetReasoningDialectCache();
|
||||
resetJsonModeDialectCache();
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
if (savedOrigins === undefined) delete process.env.AI_PRIVATE_ORIGINS;
|
||||
else process.env.AI_PRIVATE_ORIGINS = savedOrigins;
|
||||
});
|
||||
|
||||
function gateway() {
|
||||
return new OpenAIClient({
|
||||
apiKey: "",
|
||||
model: "qwen3",
|
||||
baseUrl: `${origin}/v1`,
|
||||
providerType: "openai-compatible",
|
||||
});
|
||||
}
|
||||
|
||||
function local() {
|
||||
return new LocalOpenAICompatibleClient({
|
||||
baseUrl: `${origin}/v1`,
|
||||
model: "qwen3",
|
||||
});
|
||||
}
|
||||
|
||||
function params(extra: Partial<CompletionParams> = {}): CompletionParams {
|
||||
return {
|
||||
...singleUserTurn({
|
||||
system: "s",
|
||||
user: "u",
|
||||
maxTokens: 300,
|
||||
timeoutMs: 30_000,
|
||||
responseFormat: "json",
|
||||
}),
|
||||
...extra,
|
||||
};
|
||||
}
|
||||
|
||||
type Call = (p: CompletionParams) => Promise<unknown>;
|
||||
|
||||
const CALLERS: ReadonlyArray<[string, () => Call]> = [
|
||||
["the gateway", () => (p) => gateway().generateCompletion(p)],
|
||||
["the Local client", () => (p) => local().generateCompletion(p)],
|
||||
[
|
||||
"the Local client's stream",
|
||||
() => (p) => local().generateCompletionStream(p, () => {}),
|
||||
],
|
||||
];
|
||||
|
||||
/** Run `calls` calls in a row; the requests each one sent. */
|
||||
async function requestsPerCall(
|
||||
call: Call,
|
||||
calls: number,
|
||||
extra: Partial<CompletionParams> = {},
|
||||
) {
|
||||
const out: Array<{
|
||||
requests: Seen[];
|
||||
error: Record<string, unknown> | null;
|
||||
}> = [];
|
||||
for (let i = 0; i < calls; i += 1) {
|
||||
const before = seen.length;
|
||||
const error = await call(params(extra)).then(
|
||||
() => null,
|
||||
(e: unknown) => e as Record<string, unknown>,
|
||||
);
|
||||
out.push({ requests: seen.slice(before), error });
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
describe("one call, at most two requests", () => {
|
||||
it("the budget is two", () => {
|
||||
expect(MAX_REQUESTS_PER_CALL).toBe(2);
|
||||
});
|
||||
|
||||
describe.each(CALLERS)("%s", (_, make) => {
|
||||
it.each([
|
||||
["a gateway that refuses the reasoning field", gatewayRefusingTheField],
|
||||
["an endpoint that refuses the JSON flag too", refusesEverything],
|
||||
["a server that refuses streams", refusesStreams],
|
||||
] as const)("%s", async (_name, s) => {
|
||||
script = s;
|
||||
for (const extra of [
|
||||
{},
|
||||
{ reasoning: { effort: "low" as const, summaries: false } },
|
||||
{ reasoning: { effort: "off" as const, summaries: false } },
|
||||
]) {
|
||||
resetReasoningDialectCache();
|
||||
resetJsonModeDialectCache();
|
||||
const calls = await requestsPerCall(make(), 6, extra);
|
||||
for (const { requests } of calls) {
|
||||
expect(requests.length).toBeLessThanOrEqual(MAX_REQUESTS_PER_CALL);
|
||||
expect(requests.length).toBeGreaterThan(0);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
it("stops asking with `none` once the endpoint refused it, and fails as the reasoning it is", async () => {
|
||||
script = gatewayRefusingTheField;
|
||||
const calls = await requestsPerCall(make(), 4);
|
||||
// The first call: no field, answerless; asked again with `none`,
|
||||
// refused. Every later call sends no field and one request.
|
||||
expect(calls[0].requests.map((r) => r.effort)).toEqual([
|
||||
undefined,
|
||||
"none",
|
||||
]);
|
||||
for (const later of calls.slice(1)) {
|
||||
expect(later.requests.map((r) => r.effort)).toEqual([undefined]);
|
||||
}
|
||||
for (const { error } of calls) {
|
||||
expect(error?.kind).toBe("reasoning_only");
|
||||
expect(error?.httpStatus).toBe(0);
|
||||
}
|
||||
// The first call's answerless request is what it cost.
|
||||
expect(calls[0].error?.tokensUsed).toBe(1000);
|
||||
});
|
||||
|
||||
it("never asks a model with `none` behind an endpoint that refused the field", async () => {
|
||||
script = gatewayRefusingTheField;
|
||||
// A call with a level learns that the field is refused first.
|
||||
const first = await requestsPerCall(make(), 1, {
|
||||
reasoning: { effort: "low", summaries: false },
|
||||
});
|
||||
expect(first[0].requests.map((r) => r.effort)).toEqual([
|
||||
"low",
|
||||
undefined,
|
||||
]);
|
||||
// Its answerless repeat taught that the model reasons unasked; no call
|
||||
// after it sends a reasoning field again.
|
||||
const later = await requestsPerCall(make(), 3);
|
||||
for (const { requests } of later) {
|
||||
expect(requests.map((r) => r.effort)).toEqual([undefined]);
|
||||
}
|
||||
});
|
||||
});
|
||||
});
|
||||
+181
-184
@@ -1,13 +1,7 @@
|
||||
import { safeFetch } from "@/lib/safe-fetch";
|
||||
import { callTimeoutMs } from "./effective-timeout";
|
||||
import { annotate } from "@/lib/logging/context";
|
||||
import { aiEgressPolicyFor } from "./local-host-allowlist";
|
||||
import type {
|
||||
AIProvider,
|
||||
AiMessage,
|
||||
CompletionParams,
|
||||
CompletionResult,
|
||||
} from "./types";
|
||||
import type { AIProvider, CompletionParams, CompletionResult } from "./types";
|
||||
import {
|
||||
buildOpenAIMessages,
|
||||
mapFinishReason,
|
||||
@@ -24,19 +18,20 @@ import {
|
||||
resetJsonModeDialectCache,
|
||||
type JsonModeDialect,
|
||||
} from "./json-dialect";
|
||||
import { emptyReplyError, withStrictJsonPrefix } from "./local-wire";
|
||||
import { RequestHops } from "./request-hops";
|
||||
import { learnFromRefusal } from "./wire-refusal";
|
||||
import type { ReasoningEffort } from "./reasoning-effort";
|
||||
import {
|
||||
ANSWERS_ONLY_UNREASONED,
|
||||
annotateReasoningDowngrade,
|
||||
isReasoningParameterRejection,
|
||||
learnedReasoningDialect,
|
||||
rememberReasoningDialect,
|
||||
refusesReasoningField,
|
||||
} from "./reasoning/dialect-cache";
|
||||
import { uncontrolledThinkingRoom } from "./reasoning/levels";
|
||||
import {
|
||||
askAgainUnreasoned,
|
||||
reasonsUnasked,
|
||||
withEarlierSpend,
|
||||
repeatUnreasoned,
|
||||
} from "./reasoning/answerless";
|
||||
import {
|
||||
ReasoningCollector,
|
||||
@@ -57,9 +52,6 @@ interface LocalClientConfig {
|
||||
operatorTrusted?: boolean;
|
||||
}
|
||||
|
||||
const STRICT_JSON_PREFIX =
|
||||
"Return strict JSON only, no markdown, no commentary.\n\n";
|
||||
|
||||
/**
|
||||
* v1.28.28 (#470) — per-endpoint JSON-mode dialect, learned at runtime.
|
||||
* v1.33.1 — the machinery moved to `json-dialect.ts` when the
|
||||
@@ -83,51 +75,6 @@ export { isResponseFormatRejection };
|
||||
*/
|
||||
const STREAM_ABSOLUTE_CEILING_MS = 10 * 60_000;
|
||||
|
||||
/**
|
||||
* Prepend the strict-JSON instruction to the FIRST user turn's text. Local
|
||||
* model templates respect an in-message instruction reliably, so it is sent
|
||||
* in BOTH dialects — it is the only JSON steering left when an endpoint
|
||||
* rejects `response_format` (dialect `"none"`). Mirrors the pre-refactor
|
||||
* behaviour, which prepended to the single user prompt.
|
||||
*/
|
||||
function withStrictJsonPrefix(messages: AiMessage[]): AiMessage[] {
|
||||
const idx = messages.findIndex((m) => m.role === "user");
|
||||
if (idx === -1) return messages;
|
||||
const out = [...messages];
|
||||
const target = out[idx];
|
||||
if (typeof target.content === "string") {
|
||||
out[idx] = { ...target, content: `${STRICT_JSON_PREFIX}${target.content}` };
|
||||
} else {
|
||||
// Vision parts: prepend a leading text part with the instruction.
|
||||
out[idx] = {
|
||||
...target,
|
||||
content: [{ type: "text", text: STRICT_JSON_PREFIX }, ...target.content],
|
||||
};
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/**
|
||||
* The empty-reply error. `httpStatus: 0` is the sentinel the chain classifier
|
||||
* treats as a hard failure; `kind` says whether the reply held reasoning and
|
||||
* no answer, or nothing at all, so the log names it instead of "network".
|
||||
*/
|
||||
function emptyReplyError(hadReasoning: boolean, spent: number | null): Error {
|
||||
const err = new Error(
|
||||
hadReasoning
|
||||
? "Local AI returned reasoning but no answer"
|
||||
: "Local AI returned empty content",
|
||||
);
|
||||
Object.assign(err, {
|
||||
httpStatus: 0,
|
||||
kind: hadReasoning ? "reasoning_only" : "empty_response",
|
||||
upstream: "local",
|
||||
// #1279 — what the reply cost, for the day's ledger to charge.
|
||||
...(spent !== null ? { tokensUsed: spent } : {}),
|
||||
});
|
||||
return err;
|
||||
}
|
||||
|
||||
/**
|
||||
* Talks to OpenAI-compatible servers: local runtimes (Ollama, LocalAI,
|
||||
* LM Studio, vLLM, …) AND hosted gateways (LiteLLM, OpenRouter, any
|
||||
@@ -279,7 +226,7 @@ export class LocalOpenAICompatibleClient implements AIProvider {
|
||||
// expires, a call without a level of its own included.
|
||||
if (learned === ANSWERS_ONLY_UNREASONED) return "none";
|
||||
if (!params.reasoning) return null;
|
||||
if (learned === "none") return null;
|
||||
if (refusesReasoningField(learned)) return null;
|
||||
return params.reasoning.effort === "off" ? "none" : params.reasoning.effort;
|
||||
}
|
||||
|
||||
@@ -308,31 +255,93 @@ export class LocalOpenAICompatibleClient implements AIProvider {
|
||||
*/
|
||||
private learnAnswerlessReasoning(
|
||||
params: CompletionParams,
|
||||
hops: RequestHops,
|
||||
hadReasoning: boolean,
|
||||
finishReason: string | undefined,
|
||||
): boolean {
|
||||
return askAgainUnreasoned({
|
||||
key: this.reasoningDialectKey(),
|
||||
provider: "local",
|
||||
signals: "ai.local",
|
||||
sent: this.sentReasoningEffort(params),
|
||||
hadReasoning,
|
||||
finishReason,
|
||||
return (
|
||||
// The lesson holds even when the budget leaves no repeat for it.
|
||||
askAgainUnreasoned({
|
||||
key: this.reasoningDialectKey(),
|
||||
provider: "local",
|
||||
signals: "ai.local",
|
||||
sent: this.sentReasoningEffort(params),
|
||||
hadReasoning,
|
||||
finishReason,
|
||||
}) && hops.canSend
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* #1279 — the call once more with `none` (streamed when `onDelta` is
|
||||
* given), carrying this reply's spend; see `repeatUnreasoned`.
|
||||
*/
|
||||
private askUnreasoned(
|
||||
params: CompletionParams,
|
||||
hops: RequestHops,
|
||||
reply: { hadReasoning: boolean; spent: number | null },
|
||||
onDelta?: (delta: string) => void,
|
||||
): Promise<CompletionResult> {
|
||||
return repeatUnreasoned(
|
||||
onDelta
|
||||
? this.generateCompletionStream(params, onDelta, hops)
|
||||
: this.generateCompletion(params, hops),
|
||||
reply.spent,
|
||||
() => emptyReplyError(reply.hadReasoning, reply.spent),
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* A non-2xx reply. A refused reasoning field or JSON flag is learned and
|
||||
* the call asked once more without it (`wire-refusal.ts`), and a refused
|
||||
* stream (`streamed`; some servers 400 on `stream` / `stream_options`)
|
||||
* once more buffered, while the call's request budget allows. Otherwise
|
||||
* the structured error; local endpoints and their proxies commonly leak
|
||||
* bearer tokens in error envelopes, so the excerpt is stripped first.
|
||||
*/
|
||||
private async failed(
|
||||
res: Response,
|
||||
params: CompletionParams,
|
||||
hops: RequestHops,
|
||||
sent: { effort: string | null; json: boolean; streamed: boolean },
|
||||
): Promise<CompletionResult> {
|
||||
const bodyExcerpt = sanitiseBodyExcerpt(await res.text().catch(() => ""));
|
||||
const lesson = learnFromRefusal({
|
||||
status: res.status,
|
||||
body: bodyExcerpt,
|
||||
model: this.config.model,
|
||||
reasoning: sent.effort
|
||||
? {
|
||||
sent: sent.effort,
|
||||
key: this.reasoningDialectKey(),
|
||||
provider: "local",
|
||||
names: /reasoning|think/i,
|
||||
}
|
||||
: null,
|
||||
json: sent.json
|
||||
? { endpoint: this.config.baseUrl, action: "ai.local.jsonDialect" }
|
||||
: null,
|
||||
});
|
||||
if ((lesson || sent.streamed) && hops.canSend) {
|
||||
return this.generateCompletion(params, hops);
|
||||
}
|
||||
throw Object.assign(new Error(`Local AI request failed (${res.status})`), {
|
||||
httpStatus: res.status,
|
||||
upstream: "local",
|
||||
model: this.config.model,
|
||||
baseUrl: this.config.baseUrl,
|
||||
bodyExcerpt,
|
||||
...(lesson === "reasoning" ? { reasoningRefused: true } : {}),
|
||||
});
|
||||
}
|
||||
|
||||
/** #1279 — the streamed call once more, carrying this reply's spend. */
|
||||
private streamAgain(
|
||||
params: CompletionParams,
|
||||
onDelta: (delta: string) => void,
|
||||
spent: number | null,
|
||||
): Promise<CompletionResult> {
|
||||
const again = this.generateCompletionStream(params, onDelta);
|
||||
return withEarlierSpend(again, spent);
|
||||
}
|
||||
|
||||
/**
|
||||
* `hops` is the call's request budget (`request-hops.ts`): a caller never
|
||||
* passes it, every repeat inside the call hands it on.
|
||||
*/
|
||||
async generateCompletion(
|
||||
params: CompletionParams,
|
||||
hops: RequestHops = new RequestHops(),
|
||||
): Promise<CompletionResult> {
|
||||
const { url, headers, body, jsonDialect, room } = this.buildRequest(
|
||||
params,
|
||||
@@ -357,7 +366,7 @@ export class LocalOpenAICompatibleClient implements AIProvider {
|
||||
const egress = aiEgressPolicyFor(url, {
|
||||
operatorTrusted: this.config.operatorTrusted,
|
||||
});
|
||||
const res = await safeFetch(
|
||||
const res = await hops.fetch(
|
||||
url,
|
||||
{
|
||||
method: "POST",
|
||||
@@ -375,55 +384,12 @@ export class LocalOpenAICompatibleClient implements AIProvider {
|
||||
);
|
||||
|
||||
if (!res.ok) {
|
||||
// Mirror the openai-client structured-error pattern. Local endpoints
|
||||
// also commonly leak bearer tokens in error envelopes (proxies), so
|
||||
// strip anything that looks like a key before logging.
|
||||
const rawBody = await res.text().catch(() => "");
|
||||
const bodyExcerpt = sanitiseBodyExcerpt(rawBody);
|
||||
// v1.41 — a server that refuses the reasoning field is asked once more
|
||||
// without it and remembers. Before the JSON dialect, so a complaint
|
||||
// about reasoning never switches JSON mode off.
|
||||
const sentEffort = this.callReasoningEffort(params);
|
||||
if (
|
||||
sentEffort &&
|
||||
isReasoningParameterRejection(
|
||||
res.status,
|
||||
bodyExcerpt,
|
||||
/reasoning|think/i,
|
||||
this.config.model,
|
||||
)
|
||||
) {
|
||||
rememberReasoningDialect(this.reasoningDialectKey(), "none");
|
||||
annotateReasoningDowngrade("local", sentEffort, "none");
|
||||
return this.generateCompletion(params);
|
||||
}
|
||||
// v1.28.28 (#470) — dialect self-heal. A 4xx whose body names
|
||||
// `response_format` (or an unknown parameter) means this endpoint
|
||||
// rejects the standard JSON flag: learn the no-flag dialect for the
|
||||
// baseUrl and retry ONCE without it (the recursion terminates because
|
||||
// the cached dialect is now "none"). Unrelated 4xx/5xx errors are not
|
||||
// retried.
|
||||
if (
|
||||
params.responseFormat === "json" &&
|
||||
jsonDialect === "response_format" &&
|
||||
isResponseFormatRejection(res.status, bodyExcerpt)
|
||||
) {
|
||||
rememberJsonModeDialect(this.config.baseUrl, "none");
|
||||
annotate({
|
||||
action: { name: "ai.local.jsonDialect" },
|
||||
meta: { dialect: "none", httpStatus: res.status },
|
||||
});
|
||||
return this.generateCompletion(params);
|
||||
}
|
||||
const err = new Error(`Local AI request failed (${res.status})`);
|
||||
Object.assign(err, {
|
||||
httpStatus: res.status,
|
||||
upstream: "local",
|
||||
model: this.config.model,
|
||||
baseUrl: this.config.baseUrl,
|
||||
bodyExcerpt,
|
||||
return this.failed(res, params, hops, {
|
||||
effort: this.callReasoningEffort(params),
|
||||
json:
|
||||
params.responseFormat === "json" && jsonDialect === "response_format",
|
||||
streamed: false,
|
||||
});
|
||||
throw err;
|
||||
}
|
||||
|
||||
// v1.28.28 (#470) — the standard `response_format` request succeeded:
|
||||
@@ -474,8 +440,8 @@ export class LocalOpenAICompatibleClient implements AIProvider {
|
||||
);
|
||||
if (!content) {
|
||||
const spent = json.usage?.total_tokens ?? null;
|
||||
if (this.learnAnswerlessReasoning(params, hadReasoning, finish)) {
|
||||
return withEarlierSpend(this.generateCompletion(params), spent);
|
||||
if (this.learnAnswerlessReasoning(params, hops, hadReasoning, finish)) {
|
||||
return this.askUnreasoned(params, hops, { hadReasoning, spent });
|
||||
}
|
||||
// The reasoning that did arrive still reaches `onReasoning`, as before.
|
||||
collector.result(
|
||||
@@ -507,6 +473,68 @@ export class LocalOpenAICompatibleClient implements AIProvider {
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* A server that ignored the stream flag and answered buffered (or sent no
|
||||
* readable body): parsed like the non-streaming path rather than paying a
|
||||
* second round-trip, and the answer handed to `onDelta` whole.
|
||||
*/
|
||||
private async bufferedInsteadOfStream(
|
||||
res: Response,
|
||||
params: CompletionParams,
|
||||
onDelta: (delta: string) => void,
|
||||
hops: RequestHops,
|
||||
room: boolean,
|
||||
): Promise<CompletionResult> {
|
||||
const json = (await res.json().catch(() => null)) as {
|
||||
choices?: Array<{
|
||||
message?: { content?: string };
|
||||
finish_reason?: string;
|
||||
}>;
|
||||
usage?: {
|
||||
total_tokens?: number;
|
||||
prompt_tokens_details?: { cached_tokens?: number };
|
||||
};
|
||||
} | null;
|
||||
// The server ignored `stream` and answered buffered — the embedded
|
||||
// failure can ride this body exactly as it rides the non-streaming one.
|
||||
raiseEmbeddedError(json, {
|
||||
upstream: "local",
|
||||
label: "local AI endpoint",
|
||||
model: this.config.model,
|
||||
});
|
||||
const { content, hadReasoning, collector } = separateMessage(
|
||||
params,
|
||||
json?.choices?.[0]?.message,
|
||||
json?.choices?.[0]?.finish_reason,
|
||||
room,
|
||||
);
|
||||
if (!content) {
|
||||
const spent = json?.usage?.total_tokens ?? null;
|
||||
const finish = json?.choices?.[0]?.finish_reason;
|
||||
if (this.learnAnswerlessReasoning(params, hops, hadReasoning, finish)) {
|
||||
return this.askUnreasoned(
|
||||
params,
|
||||
hops,
|
||||
{ hadReasoning, spent },
|
||||
onDelta,
|
||||
);
|
||||
}
|
||||
collector.result(null);
|
||||
throw emptyReplyError(hadReasoning, spent);
|
||||
}
|
||||
const reasoning = collector.result(null);
|
||||
onDelta(content);
|
||||
return {
|
||||
content,
|
||||
tokensUsed: json?.usage?.total_tokens ?? null,
|
||||
cachedInputTokens: json ? parseCachedTokens(json) : null,
|
||||
model: this.config.model,
|
||||
providerType: "local",
|
||||
finishReason: mapFinishReason(json?.choices?.[0]?.finish_reason),
|
||||
...(reasoning ? { reasoning } : {}),
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* v1.22 (#89) — true token streaming over the OpenAI-compatible
|
||||
* `stream: true` wire. POSTs with `stream: true`, reads the SSE response body
|
||||
@@ -530,8 +558,12 @@ export class LocalOpenAICompatibleClient implements AIProvider {
|
||||
async generateCompletionStream(
|
||||
params: CompletionParams,
|
||||
onDelta: (delta: string) => void,
|
||||
hops: RequestHops = new RequestHops(),
|
||||
): Promise<CompletionResult> {
|
||||
const { url, headers, body, room } = this.buildRequest(params, true);
|
||||
const { url, headers, body, jsonDialect, room } = this.buildRequest(
|
||||
params,
|
||||
true,
|
||||
);
|
||||
const egress = aiEgressPolicyFor(url, {
|
||||
operatorTrusted: this.config.operatorTrusted,
|
||||
});
|
||||
@@ -547,7 +579,7 @@ export class LocalOpenAICompatibleClient implements AIProvider {
|
||||
|
||||
let res: Response;
|
||||
try {
|
||||
res = await safeFetch(
|
||||
res = await hops.fetch(
|
||||
url,
|
||||
{ method: "POST", headers, body },
|
||||
{
|
||||
@@ -558,68 +590,26 @@ export class LocalOpenAICompatibleClient implements AIProvider {
|
||||
signal,
|
||||
},
|
||||
);
|
||||
} catch {
|
||||
} catch (err) {
|
||||
// Network / private-host / abort before the stream opened → fall back to
|
||||
// the buffered path, which surfaces the structured error the chain runner
|
||||
// understands (or succeeds on a server that simply doesn't stream).
|
||||
return this.generateCompletion(params);
|
||||
if (!hops.canSend) throw err;
|
||||
return this.generateCompletion(params, hops);
|
||||
}
|
||||
|
||||
if (!res.ok) {
|
||||
// The server rejected the streaming request (some servers 400 on
|
||||
// `stream`/`stream_options`). Retry once on the buffered path so a
|
||||
// non-streaming server still works and surfaces its real status.
|
||||
return this.generateCompletion(params);
|
||||
return this.failed(res, params, hops, {
|
||||
effort: this.callReasoningEffort(params),
|
||||
json:
|
||||
params.responseFormat === "json" && jsonDialect === "response_format",
|
||||
streamed: true,
|
||||
});
|
||||
}
|
||||
|
||||
const contentType = res.headers.get("content-type") ?? "";
|
||||
if (!contentType.includes("text/event-stream") || !res.body) {
|
||||
// The server ignored the stream flag and returned a buffered completion
|
||||
// (or there is no readable body). Parse it like the non-streaming path
|
||||
// rather than paying a second round-trip.
|
||||
const json = (await res.json().catch(() => null)) as {
|
||||
choices?: Array<{
|
||||
message?: { content?: string };
|
||||
finish_reason?: string;
|
||||
}>;
|
||||
usage?: {
|
||||
total_tokens?: number;
|
||||
prompt_tokens_details?: { cached_tokens?: number };
|
||||
};
|
||||
} | null;
|
||||
// The server ignored `stream` and answered buffered — the embedded
|
||||
// failure can ride this body exactly as it rides the non-streaming one.
|
||||
raiseEmbeddedError(json, {
|
||||
upstream: "local",
|
||||
label: "local AI endpoint",
|
||||
model: this.config.model,
|
||||
});
|
||||
const { content, hadReasoning, collector } = separateMessage(
|
||||
params,
|
||||
json?.choices?.[0]?.message,
|
||||
json?.choices?.[0]?.finish_reason,
|
||||
room,
|
||||
);
|
||||
if (!content) {
|
||||
const spent = json?.usage?.total_tokens ?? null;
|
||||
const finish = json?.choices?.[0]?.finish_reason;
|
||||
if (this.learnAnswerlessReasoning(params, hadReasoning, finish)) {
|
||||
return this.streamAgain(params, onDelta, spent);
|
||||
}
|
||||
collector.result(null);
|
||||
throw emptyReplyError(hadReasoning, spent);
|
||||
}
|
||||
const reasoning = collector.result(null);
|
||||
onDelta(content);
|
||||
return {
|
||||
content,
|
||||
tokensUsed: json?.usage?.total_tokens ?? null,
|
||||
cachedInputTokens: json ? parseCachedTokens(json) : null,
|
||||
model: this.config.model,
|
||||
providerType: "local",
|
||||
finishReason: mapFinishReason(json?.choices?.[0]?.finish_reason),
|
||||
...(reasoning ? { reasoning } : {}),
|
||||
};
|
||||
return this.bufferedInsteadOfStream(res, params, onDelta, hops, room);
|
||||
}
|
||||
|
||||
const reader = res.body.getReader();
|
||||
@@ -803,8 +793,15 @@ export class LocalOpenAICompatibleClient implements AIProvider {
|
||||
// An unclosed inline `<think>` block is reasoning too.
|
||||
const hadReasoning =
|
||||
streamedReasoning || endState === "inside" || tail.took;
|
||||
if (this.learnAnswerlessReasoning(params, hadReasoning, finishReason)) {
|
||||
return this.streamAgain(params, onDelta, tokensUsed);
|
||||
if (
|
||||
this.learnAnswerlessReasoning(params, hops, hadReasoning, finishReason)
|
||||
) {
|
||||
return this.askUnreasoned(
|
||||
params,
|
||||
hops,
|
||||
{ hadReasoning, spent: tokensUsed },
|
||||
onDelta,
|
||||
);
|
||||
}
|
||||
collector.result(reasoningTokens);
|
||||
throw emptyReplyError(hadReasoning, tokensUsed);
|
||||
|
||||
@@ -0,0 +1,58 @@
|
||||
/**
|
||||
* Request and error shapes of the Local client that need nothing of its state.
|
||||
* Split out of `local-client.ts`; that client is their only importer.
|
||||
*
|
||||
* Server-only.
|
||||
*/
|
||||
import type { AiMessage } from "./types";
|
||||
|
||||
const STRICT_JSON_PREFIX =
|
||||
"Return strict JSON only, no markdown, no commentary.\n\n";
|
||||
|
||||
/**
|
||||
* Prepend the strict-JSON instruction to the FIRST user turn's text. Local
|
||||
* model templates respect an in-message instruction reliably, so it is sent
|
||||
* in BOTH dialects — it is the only JSON steering left when an endpoint
|
||||
* rejects `response_format` (dialect `"none"`). Mirrors the pre-refactor
|
||||
* behaviour, which prepended to the single user prompt.
|
||||
*/
|
||||
export function withStrictJsonPrefix(messages: AiMessage[]): AiMessage[] {
|
||||
const idx = messages.findIndex((m) => m.role === "user");
|
||||
if (idx === -1) return messages;
|
||||
const out = [...messages];
|
||||
const target = out[idx];
|
||||
if (typeof target.content === "string") {
|
||||
out[idx] = { ...target, content: `${STRICT_JSON_PREFIX}${target.content}` };
|
||||
} else {
|
||||
// Vision parts: prepend a leading text part with the instruction.
|
||||
out[idx] = {
|
||||
...target,
|
||||
content: [{ type: "text", text: STRICT_JSON_PREFIX }, ...target.content],
|
||||
};
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/**
|
||||
* The empty-reply error. `httpStatus: 0` is the sentinel the chain classifier
|
||||
* treats as a hard failure; `kind` says whether the reply held reasoning and
|
||||
* no answer, or nothing at all, so the log names it instead of "network".
|
||||
*/
|
||||
export function emptyReplyError(
|
||||
hadReasoning: boolean,
|
||||
spent: number | null,
|
||||
): Error {
|
||||
const err = new Error(
|
||||
hadReasoning
|
||||
? "Local AI returned reasoning but no answer"
|
||||
: "Local AI returned empty content",
|
||||
);
|
||||
Object.assign(err, {
|
||||
httpStatus: 0,
|
||||
kind: hadReasoning ? "reasoning_only" : "empty_response",
|
||||
upstream: "local",
|
||||
// #1279 — what the reply cost, for the day's ledger to charge.
|
||||
...(spent !== null ? { tokensUsed: spent } : {}),
|
||||
});
|
||||
return err;
|
||||
}
|
||||
+99
-84
@@ -1,4 +1,3 @@
|
||||
import { safeFetch } from "@/lib/safe-fetch";
|
||||
import { callTimeoutMs } from "./effective-timeout";
|
||||
import { annotate } from "@/lib/logging/context";
|
||||
import { emitSignal } from "@/lib/logging/signal";
|
||||
@@ -15,16 +14,16 @@ import {
|
||||
} from "./openai-capabilities";
|
||||
import {
|
||||
ANSWERS_ONLY_UNREASONED,
|
||||
annotateReasoningDowngrade,
|
||||
isReasoningParameterRejection,
|
||||
learnedReasoningDialect,
|
||||
rememberReasoningDialect,
|
||||
refusesReasoningField,
|
||||
} from "./reasoning/dialect-cache";
|
||||
import {
|
||||
askAgainUnreasoned,
|
||||
reasonsUnasked,
|
||||
withEarlierSpend,
|
||||
repeatUnreasoned,
|
||||
} from "./reasoning/answerless";
|
||||
import { RequestHops } from "./request-hops";
|
||||
import { learnFromRefusal } from "./wire-refusal";
|
||||
import {
|
||||
REASONING_THINKING_BUDGET,
|
||||
uncontrolledThinkingRoom,
|
||||
@@ -39,7 +38,6 @@ import {
|
||||
} from "./reasoning/support";
|
||||
import {
|
||||
hasLearnedJsonModeDialect,
|
||||
isResponseFormatRejection,
|
||||
jsonModeDialectFor,
|
||||
rememberJsonModeDialect,
|
||||
} from "./json-dialect";
|
||||
@@ -167,8 +165,13 @@ export class OpenAIClient implements AIProvider {
|
||||
return this.type === "openai-compatible";
|
||||
}
|
||||
|
||||
/**
|
||||
* `hops` is the call's request budget (`request-hops.ts`): a caller never
|
||||
* passes it, every repeat inside the call hands it on.
|
||||
*/
|
||||
async generateCompletion(
|
||||
params: CompletionParams,
|
||||
hops: RequestHops = new RequestHops(),
|
||||
): Promise<CompletionResult> {
|
||||
const url = `${this.config.baseUrl.replace(/\/$/, "")}/chat/completions`;
|
||||
|
||||
@@ -234,7 +237,7 @@ export class OpenAIClient implements AIProvider {
|
||||
const trimmedKey = this.config.apiKey?.trim();
|
||||
if (trimmedKey) headers.Authorization = `Bearer ${trimmedKey}`;
|
||||
|
||||
const res = await safeFetch(
|
||||
const res = await hops.fetch(
|
||||
url,
|
||||
{
|
||||
method: "POST",
|
||||
@@ -309,68 +312,7 @@ export class OpenAIClient implements AIProvider {
|
||||
);
|
||||
|
||||
if (!res.ok) {
|
||||
// Pull as much of the body as we can so upstream incidents (OpenAI 5xx,
|
||||
// model-not-found, quota exceeded) are diagnosable from logs instead of
|
||||
// surfacing as an opaque "OpenAI request failed (500)". Strip anything
|
||||
// that looks like an API key from the excerpt before logging.
|
||||
const rawBody = await res.text().catch(() => "");
|
||||
const bodyExcerpt = sanitiseBodyExcerpt(rawBody);
|
||||
// v1.41 — an endpoint or model that refuses the reasoning field is
|
||||
// asked once more without it and remembers, so the answer never fails
|
||||
// on a reasoning parameter. Checked before the JSON dialect: a body
|
||||
// naming the reasoning field must not switch JSON mode off.
|
||||
if (
|
||||
reasoningWire.kind !== "none" &&
|
||||
isReasoningParameterRejection(
|
||||
res.status,
|
||||
bodyExcerpt,
|
||||
/reasoning|thinking/i,
|
||||
this.config.model,
|
||||
)
|
||||
) {
|
||||
rememberReasoningDialect(this.reasoningDialectKey(), "none");
|
||||
annotateReasoningDowngrade(this.type, reasoningWire.effort, "none");
|
||||
return this.generateCompletion(params);
|
||||
}
|
||||
// v1.33.1 (#470) — dialect self-heal, gateway only. A 4xx whose body
|
||||
// names `response_format` (or an unknown parameter) means this endpoint
|
||||
// rejects the standard JSON flag: learn the no-flag dialect for the base
|
||||
// URL and retry ONCE without it. The recursion terminates because the
|
||||
// cached dialect is now "none". Unrelated 4xx/5xx errors are not retried
|
||||
// and surface as the structured error they always were.
|
||||
if (
|
||||
this.isGateway &&
|
||||
sendJsonFormat &&
|
||||
isResponseFormatRejection(res.status, bodyExcerpt)
|
||||
) {
|
||||
rememberJsonModeDialect(this.config.baseUrl, "none");
|
||||
annotate({
|
||||
action: { name: "ai.compat.jsonDialect" },
|
||||
meta: { dialect: "none", httpStatus: res.status },
|
||||
});
|
||||
return this.generateCompletion(params);
|
||||
}
|
||||
// v1.42 — a proxy that stopped offering the configured model answers
|
||||
// with a bare 500 that names nothing. Ask its model listing (free, no
|
||||
// completion) and say so plainly when the name is gone, so the chain,
|
||||
// the health card and the log name the actual fix.
|
||||
const modelNotServed = await this.modelNoLongerServed(res.status);
|
||||
const err = new Error(
|
||||
modelNotServed
|
||||
? `${this.isGateway ? "OpenAI-compatible gateway" : "OpenAI"} request failed (${res.status}): model "${this.config.model}" is not offered by this endpoint`
|
||||
: `${this.isGateway ? "OpenAI-compatible gateway" : "OpenAI"} request failed (${res.status})`,
|
||||
);
|
||||
// Keep the body in a structured field rather than the message so
|
||||
// Error.message stays short and the excerpt lands in dedicated log
|
||||
// fields (bodyExcerpt) that can be filtered/truncated centrally.
|
||||
Object.assign(err, {
|
||||
httpStatus: res.status,
|
||||
upstream: this.isGateway ? "openai-compatible" : "openai",
|
||||
model: this.config.model,
|
||||
bodyExcerpt,
|
||||
...(modelNotServed ? { modelNotServed: true } : {}),
|
||||
});
|
||||
throw err;
|
||||
return this.failed(res, params, hops, { reasoningWire, sendJsonFormat });
|
||||
}
|
||||
|
||||
// v1.33.1 (#470) — the standard `response_format` request succeeded: pin
|
||||
@@ -439,7 +381,7 @@ export class OpenAIClient implements AIProvider {
|
||||
// A reply with tool calls and no prose is valid (F1 tool loop); only an
|
||||
// empty reply with neither content NOR tool calls is an error.
|
||||
if (!content && !toolCalls) {
|
||||
return this.answerless(params, {
|
||||
return this.answerless(params, hops, {
|
||||
sent:
|
||||
reasoningWire.kind !== "none" ? reasoningWire.effort : entryEffort,
|
||||
hadReasoning: reply.hadReasoning,
|
||||
@@ -464,8 +406,9 @@ export class OpenAIClient implements AIProvider {
|
||||
).map((field) => ({ [field]: message[field] }));
|
||||
const usage = json.usage as
|
||||
{ completion_tokens_details?: { reasoning_tokens?: number } } | undefined;
|
||||
const learnedNone =
|
||||
learnedReasoningDialect(this.reasoningDialectKey()) === "none";
|
||||
const learnedNone = refusesReasoningField(
|
||||
learnedReasoningDialect(this.reasoningDialectKey()),
|
||||
);
|
||||
|
||||
return {
|
||||
content,
|
||||
@@ -519,6 +462,7 @@ export class OpenAIClient implements AIProvider {
|
||||
*/
|
||||
private answerless(
|
||||
params: CompletionParams,
|
||||
hops: RequestHops,
|
||||
reply: {
|
||||
sent: string | null;
|
||||
hadReasoning: boolean;
|
||||
@@ -526,8 +470,21 @@ export class OpenAIClient implements AIProvider {
|
||||
spent: number | null;
|
||||
},
|
||||
): Promise<CompletionResult> {
|
||||
const failure = () =>
|
||||
Object.assign(
|
||||
new Error(
|
||||
`${this.isGateway ? "OpenAI-compatible gateway" : "OpenAI"} returned empty content`,
|
||||
),
|
||||
{
|
||||
httpStatus: 0,
|
||||
kind: reply.hadReasoning ? "reasoning_only" : "empty_response",
|
||||
upstream: this.isGateway ? "openai-compatible" : "openai",
|
||||
...(reply.spent !== null ? { tokensUsed: reply.spent } : {}),
|
||||
},
|
||||
);
|
||||
if (
|
||||
this.isGateway &&
|
||||
// The lesson holds even when the budget leaves no repeat for it.
|
||||
askAgainUnreasoned({
|
||||
key: this.reasoningDialectKey(),
|
||||
provider: this.type,
|
||||
@@ -535,20 +492,78 @@ export class OpenAIClient implements AIProvider {
|
||||
sent: reply.sent,
|
||||
hadReasoning: reply.hadReasoning,
|
||||
finishReason: reply.finishReason,
|
||||
})
|
||||
}) &&
|
||||
hops.canSend
|
||||
) {
|
||||
return withEarlierSpend(this.generateCompletion(params), reply.spent);
|
||||
return repeatUnreasoned(
|
||||
this.generateCompletion(params, hops),
|
||||
reply.spent,
|
||||
failure,
|
||||
);
|
||||
}
|
||||
const err = new Error(
|
||||
`${this.isGateway ? "OpenAI-compatible gateway" : "OpenAI"} returned empty content`,
|
||||
);
|
||||
Object.assign(err, {
|
||||
httpStatus: 0,
|
||||
kind: reply.hadReasoning ? "reasoning_only" : "empty_response",
|
||||
upstream: this.isGateway ? "openai-compatible" : "openai",
|
||||
...(reply.spent !== null ? { tokensUsed: reply.spent } : {}),
|
||||
throw failure();
|
||||
}
|
||||
|
||||
/**
|
||||
* A non-2xx reply. As much of the body as can be read goes into a
|
||||
* structured field (with anything key-shaped stripped), so an upstream
|
||||
* incident is diagnosable from the log instead of an opaque status.
|
||||
*
|
||||
* An endpoint or model that refuses the reasoning field, or (#470, gateway
|
||||
* only) the JSON flag, is asked once more without it and remembers
|
||||
* (`wire-refusal.ts`), inside the call's request budget. A refusal the
|
||||
* budget leaves no repeat for is marked `reasoningRefused`.
|
||||
*
|
||||
* A proxy that stopped offering the configured model answers with a bare
|
||||
* 500 that names nothing; its model listing (free, no completion) says so
|
||||
* plainly, so the chain, the health card and the log name the fix.
|
||||
*/
|
||||
private async failed(
|
||||
res: Response,
|
||||
params: CompletionParams,
|
||||
hops: RequestHops,
|
||||
sent: { reasoningWire: OpenAIReasoningWire; sendJsonFormat: boolean },
|
||||
): Promise<CompletionResult> {
|
||||
const bodyExcerpt = sanitiseBodyExcerpt(await res.text().catch(() => ""));
|
||||
const wire = sent.reasoningWire;
|
||||
const lesson = learnFromRefusal({
|
||||
status: res.status,
|
||||
body: bodyExcerpt,
|
||||
model: this.config.model,
|
||||
reasoning:
|
||||
wire.kind === "none"
|
||||
? null
|
||||
: {
|
||||
sent: wire.effort,
|
||||
key: this.reasoningDialectKey(),
|
||||
provider: this.type,
|
||||
names: /reasoning|thinking/i,
|
||||
},
|
||||
json:
|
||||
this.isGateway && sent.sendJsonFormat
|
||||
? { endpoint: this.config.baseUrl, action: "ai.compat.jsonDialect" }
|
||||
: null,
|
||||
});
|
||||
throw err;
|
||||
if (lesson && hops.canSend) return this.generateCompletion(params, hops);
|
||||
const modelNotServed = await this.modelNoLongerServed(res.status);
|
||||
const label = this.isGateway ? "OpenAI-compatible gateway" : "OpenAI";
|
||||
// The excerpt rides a structured field, not the message, so it lands in
|
||||
// the log fields that are filtered and truncated centrally.
|
||||
throw Object.assign(
|
||||
new Error(
|
||||
modelNotServed
|
||||
? `${label} request failed (${res.status}): model "${this.config.model}" is not offered by this endpoint`
|
||||
: `${label} request failed (${res.status})`,
|
||||
),
|
||||
{
|
||||
httpStatus: res.status,
|
||||
upstream: this.isGateway ? "openai-compatible" : "openai",
|
||||
model: this.config.model,
|
||||
bodyExcerpt,
|
||||
...(modelNotServed ? { modelNotServed: true } : {}),
|
||||
...(lesson === "reasoning" ? { reasoningRefused: true } : {}),
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
// ── v1.41 reasoning ───────────────────────────────────────────────────────
|
||||
@@ -663,7 +678,7 @@ export class OpenAIClient implements AIProvider {
|
||||
: { kind: "reasoning_effort", effort: "none" };
|
||||
}
|
||||
if (!reasoning) return { kind: "none" };
|
||||
if (learned === "none") return { kind: "none" };
|
||||
if (refusesReasoningField(learned)) return { kind: "none" };
|
||||
const level = reasoning.effort;
|
||||
if (this.isGateway) {
|
||||
const effort = level === "off" ? "none" : level;
|
||||
|
||||
@@ -11,8 +11,14 @@
|
||||
* and a GLM chat template reads an effort it does not know as its maximum
|
||||
* (vllm-project/vllm#54744). That is learned too (`REASONS_REGARDLESS`): later
|
||||
* calls go out as the call asks, with the thinking room #1276 keeps, and an
|
||||
* answerless reply is final. So one call costs two requests at most, and
|
||||
* after the first such call, one.
|
||||
* answerless reply is final. After the first such call, one request.
|
||||
*
|
||||
* An endpoint that refuses the reasoning field altogether can be asked
|
||||
* neither way: it learns `"none"` (no field), and once its model is seen
|
||||
* reasoning unasked, `FIELD_REFUSED_REASONS_REGARDLESS`. Its `none` repeat is
|
||||
* the refused field itself, so a refusal of that repeat is learned as both.
|
||||
* Every request of a call, these repeats included, counts against its budget
|
||||
* of two (`request-hops.ts`).
|
||||
*
|
||||
* What bounds the calls themselves is not here: the chain keeps background
|
||||
* work off a provider that keeps failing (`provider-runner.ts`), and each job
|
||||
@@ -21,6 +27,7 @@
|
||||
import { emitSignal } from "@/lib/logging/signal";
|
||||
import {
|
||||
ANSWERS_ONLY_UNREASONED,
|
||||
FIELD_REFUSED_REASONS_REGARDLESS,
|
||||
REASONS_REGARDLESS,
|
||||
annotateReasoningDowngrade,
|
||||
learnedReasoningDialect,
|
||||
@@ -47,9 +54,19 @@ export function nextAnswerlessStep(
|
||||
sent: string | null,
|
||||
hadReasoning: boolean,
|
||||
): AnswerlessStep {
|
||||
if (learnedReasoningDialect(key) === REASONS_REGARDLESS) {
|
||||
const learned = learnedReasoningDialect(key);
|
||||
if (
|
||||
learned === REASONS_REGARDLESS ||
|
||||
learned === FIELD_REFUSED_REASONS_REGARDLESS
|
||||
) {
|
||||
return { kind: "final", learnedRegardless: false };
|
||||
}
|
||||
// The endpoint refuses the field: there is no `none` to ask with.
|
||||
if (learned === "none") {
|
||||
if (!hadReasoning) return { kind: "final", learnedRegardless: false };
|
||||
rememberReasoningDialect(key, FIELD_REFUSED_REASONS_REGARDLESS);
|
||||
return { kind: "final", learnedRegardless: true };
|
||||
}
|
||||
if (sent === "none") {
|
||||
if (!hadReasoning) return { kind: "final", learnedRegardless: false };
|
||||
rememberReasoningDialect(key, REASONS_REGARDLESS);
|
||||
@@ -111,7 +128,28 @@ export function askAgainUnreasoned(args: {
|
||||
*/
|
||||
export function reasonsUnasked(key: ReasoningDialectKey): boolean {
|
||||
const learned = learnedReasoningDialect(key);
|
||||
return learned === ANSWERS_ONLY_UNREASONED || learned === REASONS_REGARDLESS;
|
||||
return (
|
||||
learned === ANSWERS_ONLY_UNREASONED ||
|
||||
learned === REASONS_REGARDLESS ||
|
||||
learned === FIELD_REFUSED_REASONS_REGARDLESS
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* The endpoint refused the reasoning field `sent`: send none from now on. A
|
||||
* model already known to reason unasked keeps that lesson too, so it is never
|
||||
* asked with `none` again.
|
||||
*/
|
||||
export function learnReasoningFieldRefused(
|
||||
key: ReasoningDialectKey,
|
||||
provider: string,
|
||||
sent: string,
|
||||
): void {
|
||||
rememberReasoningDialect(
|
||||
key,
|
||||
reasonsUnasked(key) ? FIELD_REFUSED_REASONS_REGARDLESS : "none",
|
||||
);
|
||||
annotateReasoningDowngrade(provider, sent, "none");
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -136,6 +174,27 @@ export async function withEarlierSpend<T extends { tokensUsed: number | null }>(
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The `none` repeat after an answerless reply, carrying that reply's spend.
|
||||
* When the endpoint refused the `none` itself on the call's last request
|
||||
* (`reasoningRefused` on the error), the call ends with the first reply's
|
||||
* failure (`answerless`): the model reasoned without answering, and that is
|
||||
* what the call ran into, not the field the client then tried.
|
||||
*/
|
||||
export async function repeatUnreasoned<T extends { tokensUsed: number | null }>(
|
||||
repeat: Promise<T>,
|
||||
earlier: number | null,
|
||||
answerless: () => Error,
|
||||
): Promise<T> {
|
||||
try {
|
||||
return await withEarlierSpend(repeat, earlier);
|
||||
} catch (err) {
|
||||
const refused = (err as { reasoningRefused?: unknown } | null)
|
||||
?.reasoningRefused;
|
||||
throw refused === true ? answerless() : err;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Tokens a failed call spent upstream, as its error carries them
|
||||
* (`tokensUsed`), or `null` when it carries none.
|
||||
|
||||
@@ -58,6 +58,21 @@ export const ANSWERS_ONLY_UNREASONED = "off";
|
||||
*/
|
||||
export const REASONS_REGARDLESS = "unswitchable";
|
||||
|
||||
/**
|
||||
* #1279 — the learned value for an endpoint that refuses the reasoning field
|
||||
* outright (LiteLLM in front of Ollama: "does not support parameters:
|
||||
* ['reasoning_effort']") in front of a model that reasons unasked. Both facts
|
||||
* at once: no field is sent, as with `"none"`, and the thinking room stays
|
||||
* and an answerless reply is final, as with `REASONS_REGARDLESS`. Held as one
|
||||
* value each, the two lessons overwrote each other on every request.
|
||||
*/
|
||||
export const FIELD_REFUSED_REASONS_REGARDLESS = "none-unswitchable";
|
||||
|
||||
/** True when `learned` says the endpoint refuses the reasoning field. */
|
||||
export function refusesReasoningField(learned: string | undefined): boolean {
|
||||
return learned === "none" || learned === FIELD_REFUSED_REASONS_REGARDLESS;
|
||||
}
|
||||
|
||||
/** How long a learned downgrade holds before the full request is tried again. */
|
||||
export const DIALECT_TTL_MS = 60 * 60 * 1000;
|
||||
|
||||
|
||||
@@ -0,0 +1,49 @@
|
||||
/**
|
||||
* #1279 — the HTTP requests one completion call may send.
|
||||
*
|
||||
* The Local and the OpenAI-compatible gateway clients send a request again on
|
||||
* their own: without a reasoning field the endpoint refused, without the JSON
|
||||
* flag it refused, with `reasoning_effort: "none"` after a reply that
|
||||
* reasoned without answering, and the Local stream falls back to a buffered
|
||||
* request. Each repeat was bounded by what it had just learned, and two
|
||||
* lessons that undo each other were not bounded at all: a gateway that
|
||||
* refuses `reasoning_effort` in front of a model that reasons unasked made
|
||||
* one call send requests until something outside stopped it.
|
||||
*
|
||||
* So each call carries one budget through every repeat, and the budget bounds
|
||||
* it, not the lessons: at most `MAX_REQUESTS_PER_CALL` requests, after which
|
||||
* the call ends with the failure it has. What was learned on the way still
|
||||
* holds for the next call. `ai-request-hops-guard.test.ts` holds both clients
|
||||
* to sending only through here and to handing the budget to every repeat.
|
||||
*
|
||||
* Server-only.
|
||||
*/
|
||||
import { safeFetch } from "@/lib/safe-fetch";
|
||||
|
||||
/** The requests one call may send: the first, and one repeat. */
|
||||
export const MAX_REQUESTS_PER_CALL = 2;
|
||||
|
||||
export class RequestHops {
|
||||
private sent = 0;
|
||||
|
||||
/** True while the call may send one more request. */
|
||||
get canSend(): boolean {
|
||||
return this.sent < MAX_REQUESTS_PER_CALL;
|
||||
}
|
||||
|
||||
/**
|
||||
* `safeFetch`, counted. A request past the budget is not sent: every repeat
|
||||
* asks `canSend` first, so this throw marks a repeat that did not.
|
||||
*/
|
||||
fetch(...args: Parameters<typeof safeFetch>): Promise<Response> {
|
||||
if (!this.canSend) {
|
||||
return Promise.reject(
|
||||
Object.assign(new Error("AI call has no request left to send"), {
|
||||
httpStatus: 0,
|
||||
}),
|
||||
);
|
||||
}
|
||||
this.sent += 1;
|
||||
return safeFetch(...args);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,69 @@
|
||||
/**
|
||||
* What a refused request teaches the Local and the OpenAI-compatible gateway
|
||||
* clients, in one place for both.
|
||||
*
|
||||
* Only an HTTP 400 that names a field the request carried teaches anything:
|
||||
* the reasoning field (`isReasoningParameterRejection`, learned per endpoint
|
||||
* and model) or the JSON flag (`isResponseFormatRejection`, learned per
|
||||
* endpoint). The reasoning field is checked first, so a body that names it
|
||||
* never switches JSON mode off. Whether the call then sends the lesser request
|
||||
* is the client's, inside its request budget (`request-hops.ts`).
|
||||
*
|
||||
* Server-only.
|
||||
*/
|
||||
import { annotate } from "@/lib/logging/context";
|
||||
|
||||
import {
|
||||
isResponseFormatRejection,
|
||||
rememberJsonModeDialect,
|
||||
} from "./json-dialect";
|
||||
import { learnReasoningFieldRefused } from "./reasoning/answerless";
|
||||
import {
|
||||
isReasoningParameterRejection,
|
||||
type ReasoningDialectKey,
|
||||
} from "./reasoning/dialect-cache";
|
||||
|
||||
export type RefusalLesson = "reasoning" | "json" | null;
|
||||
|
||||
export function learnFromRefusal(args: {
|
||||
status: number;
|
||||
body: string;
|
||||
model: string;
|
||||
/** The reasoning field the request carried, or `null` for none. */
|
||||
reasoning: {
|
||||
sent: string;
|
||||
key: ReasoningDialectKey;
|
||||
provider: string;
|
||||
/** The parameter names this endpoint's refusals use. */
|
||||
names: RegExp;
|
||||
} | null;
|
||||
/** The endpoint, when the request carried the JSON flag. */
|
||||
json: { endpoint: string; action: string } | null;
|
||||
}): RefusalLesson {
|
||||
const { reasoning, json } = args;
|
||||
if (
|
||||
reasoning &&
|
||||
isReasoningParameterRejection(
|
||||
args.status,
|
||||
args.body,
|
||||
reasoning.names,
|
||||
args.model,
|
||||
)
|
||||
) {
|
||||
learnReasoningFieldRefused(
|
||||
reasoning.key,
|
||||
reasoning.provider,
|
||||
reasoning.sent,
|
||||
);
|
||||
return "reasoning";
|
||||
}
|
||||
if (json && isResponseFormatRejection(args.status, args.body)) {
|
||||
rememberJsonModeDialect(json.endpoint, "none");
|
||||
annotate({
|
||||
action: { name: json.action },
|
||||
meta: { dialect: "none", httpStatus: args.status },
|
||||
});
|
||||
return "json";
|
||||
}
|
||||
return null;
|
||||
}
|
||||
@@ -320,6 +320,27 @@ describe("runStatusCompletion — ledger accounting", () => {
|
||||
expect(ledgerTotal).toBe(0);
|
||||
});
|
||||
|
||||
it("aborts the call its bound cut off, so nothing runs on detached (#1279)", async () => {
|
||||
vi.useFakeTimers();
|
||||
try {
|
||||
let signal: AbortSignal | undefined;
|
||||
runRawCompletionWithFallback.mockImplementation(
|
||||
(args: { params: { signal?: AbortSignal } }) => {
|
||||
signal = args.params.signal;
|
||||
return new Promise(() => {});
|
||||
},
|
||||
);
|
||||
const pending = runStatusCompletion(completionArgs());
|
||||
await vi.waitFor(() => expect(signal).toBeDefined());
|
||||
expect(signal?.aborted).toBe(false);
|
||||
await vi.advanceTimersByTimeAsync(10 * 60_000);
|
||||
expect((await pending).kind).toBe("timeout");
|
||||
expect(signal?.aborted).toBe(true);
|
||||
} finally {
|
||||
vi.useRealTimers();
|
||||
}
|
||||
});
|
||||
|
||||
it("asks no provider about a note inside its negative window (#1279)", async () => {
|
||||
noteWindow = { retryAt: new Date(Date.now() + 60_000) };
|
||||
const result = await runStatusCompletion(
|
||||
|
||||
@@ -228,15 +228,11 @@ export async function runStatusCompletion(
|
||||
return { kind: "none", reason: refusal.reason };
|
||||
}
|
||||
|
||||
// Honour the per-user response-timeout setting the operator dials in for a
|
||||
// slow self-hosted / local backend (Settings → AI). This is the single
|
||||
// chokepoint every status / reference surface funnels through — per-metric
|
||||
// cards, the batched assessment, the derived assessments, and the period
|
||||
// narratives — so resolving it here threads the setting onto all of them at
|
||||
// once. A positive stored value wins (seconds → ms); unset falls back to the
|
||||
// status-path budget. Applied to the upstream call's own `timeoutMs` and,
|
||||
// times the calls a run may make, to the outer `withTimeout` cap, so a
|
||||
// raised value is not silently clipped by a wall-clock sized for less.
|
||||
// The per-user response-timeout setting (Settings → AI), threaded here onto
|
||||
// every status / reference surface at once. A positive stored value wins
|
||||
// (seconds → ms); unset falls back to the status-path budget. Applied to the
|
||||
// call's own `timeoutMs` and, times the calls a run may make, to the outer
|
||||
// `withTimeout` cap, so a raised value is not clipped by a shorter bound.
|
||||
const settingsRow = await prisma.user.findUnique({
|
||||
where: { id: userId },
|
||||
select: { aiResponseTimeoutSeconds: true },
|
||||
@@ -260,12 +256,9 @@ export async function runStatusCompletion(
|
||||
// The answer budget. Each client adds the thinking budget on top itself.
|
||||
const maxTokens = args.maxTokens ?? AI_BUDGETS.status.maxTokens;
|
||||
|
||||
// The day's token ledger. Until now this chokepoint — the provider entry for
|
||||
// EVERY status/reference family (the specialised cards, the generic metric
|
||||
// cards, biomarker cards, the batched assessment, the derived scores, the
|
||||
// period narrative, and the off-request Coach memory workers: rolling
|
||||
// summary, fact extraction, plan proposals) — ran with no accounting at all,
|
||||
// so none of that spend appeared in `coach_usage` and no ceiling applied.
|
||||
// The day's token ledger, for EVERY status/reference family (the cards, the
|
||||
// batched assessment, the derived scores, the period narrative, and the
|
||||
// off-request Coach memory workers).
|
||||
//
|
||||
// The reservation is atomic (single upsert-increment), matching the Coach and
|
||||
// document paths: a read-then-write check would let concurrent generations
|
||||
@@ -302,6 +295,10 @@ export async function runStatusCompletion(
|
||||
return { kind: "error" };
|
||||
}
|
||||
|
||||
// #1279 — the run's bound aborts the call it cuts off, so a provider call
|
||||
// the bound gave up on (and any repeat inside it) stops instead of running
|
||||
// on detached, with its spend reported nowhere.
|
||||
const abort = new AbortController();
|
||||
const raced = await withTimeout(
|
||||
() =>
|
||||
runRawCompletionWithFallback({
|
||||
@@ -324,12 +321,14 @@ export async function runStatusCompletion(
|
||||
responseFormat: args.responseFormat === "text" ? undefined : "json",
|
||||
timeoutMs: effectiveTimeoutMs,
|
||||
}),
|
||||
signal: abort.signal,
|
||||
// v1.41 — only a named job ever carries this; see `reasoningJob`.
|
||||
...(reasoning ? { reasoning } : {}),
|
||||
},
|
||||
}),
|
||||
statusRunBoundMs(effectiveTimeoutMs),
|
||||
null,
|
||||
() => abort.abort(),
|
||||
);
|
||||
|
||||
if (raced.timedOut || raced.errored || raced.value === null) {
|
||||
|
||||
@@ -151,6 +151,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
@@ -260,6 +261,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
@@ -611,6 +613,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"Du schreibst eine kurze, persönliche Einschätzung zu EINER Gesundheitsmetrik dieses Nutzers. Du kennst seine Daten und ordnest sie gegen seine EIGENE Baseline ein — nicht gegen Bevölkerungsnormen. Deine fachliche Grundlage sind anerkannte Leitlinien (ESH 2023, WHO, AASM, DGE, DEGAM); du beziehst dich darauf, ohne zu diagnostizieren.",
|
||||
"",
|
||||
@@ -727,6 +730,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"Du schreibst eine kurze, persönliche Einschätzung zu EINER Gesundheitsmetrik dieses Nutzers. Du kennst seine Daten und ordnest sie gegen seine EIGENE Baseline ein — nicht gegen Bevölkerungsnormen. Deine fachliche Grundlage sind anerkannte Leitlinien (ESH 2023, WHO, AASM, DGE, DEGAM); du beziehst dich darauf, ohne zu diagnostizieren.",
|
||||
"",
|
||||
@@ -844,6 +848,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"Du schreibst eine kurze, persönliche Einschätzung zu EINER Gesundheitsmetrik dieses Nutzers. Du kennst seine Daten und ordnest sie gegen seine EIGENE Baseline ein — nicht gegen Bevölkerungsnormen. Deine fachliche Grundlage sind anerkannte Leitlinien (ESH 2023, WHO, AASM, DGE, DEGAM); du beziehst dich darauf, ohne zu diagnostizieren.",
|
||||
"",
|
||||
@@ -955,6 +960,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"Du schreibst eine kurze, persönliche Einschätzung zu EINER Gesundheitsmetrik dieses Nutzers. Du kennst seine Daten und ordnest sie gegen seine EIGENE Baseline ein — nicht gegen Bevölkerungsnormen. Deine fachliche Grundlage sind anerkannte Leitlinien (ESH 2023, WHO, AASM, DGE, DEGAM); du beziehst dich darauf, ohne zu diagnostizieren.",
|
||||
"",
|
||||
@@ -1074,6 +1080,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"Du schreibst eine kurze, persönliche Einschätzung zu EINER Gesundheitsmetrik dieses Nutzers. Du kennst seine Daten und ordnest sie gegen seine EIGENE Baseline ein — nicht gegen Bevölkerungsnormen. Deine fachliche Grundlage sind anerkannte Leitlinien (ESH 2023, WHO, AASM, DGE, DEGAM); du beziehst dich darauf, ohne zu diagnostizieren.",
|
||||
"",
|
||||
@@ -1189,6 +1196,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"Du schreibst eine kurze, persönliche Einschätzung zu EINER Gesundheitsmetrik dieses Nutzers. Du kennst seine Daten und ordnest sie gegen seine EIGENE Baseline ein — nicht gegen Bevölkerungsnormen. Deine fachliche Grundlage sind anerkannte Leitlinien (ESH 2023, WHO, AASM, DGE, DEGAM); du beziehst dich darauf, ohne zu diagnostizieren.",
|
||||
"",
|
||||
@@ -1301,6 +1309,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"Du schreibst eine kurze, persönliche Einschätzung zu EINER Gesundheitsmetrik dieses Nutzers. Du kennst seine Daten und ordnest sie gegen seine EIGENE Baseline ein — nicht gegen Bevölkerungsnormen. Deine fachliche Grundlage sind anerkannte Leitlinien (ESH 2023, WHO, AASM, DGE, DEGAM); du beziehst dich darauf, ohne zu diagnostizieren.",
|
||||
"",
|
||||
@@ -1412,6 +1421,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"Du schreibst eine kurze, persönliche Einschätzung zu EINER Gesundheitsmetrik dieses Nutzers. Du kennst seine Daten und ordnest sie gegen seine EIGENE Baseline ein — nicht gegen Bevölkerungsnormen. Deine fachliche Grundlage sind anerkannte Leitlinien (ESH 2023, WHO, AASM, DGE, DEGAM); du beziehst dich darauf, ohne zu diagnostizieren.",
|
||||
"",
|
||||
|
||||
@@ -147,6 +147,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
@@ -434,6 +435,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
@@ -847,6 +849,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"Du schreibst eine kurze, persönliche Einschätzung zu EINER Gesundheitsmetrik dieses Nutzers. Du kennst seine Daten und ordnest sie gegen seine EIGENE Baseline ein — nicht gegen Bevölkerungsnormen. Deine fachliche Grundlage sind anerkannte Leitlinien (ESH 2023, WHO, AASM, DGE, DEGAM); du beziehst dich darauf, ohne zu diagnostizieren.",
|
||||
"",
|
||||
@@ -1043,6 +1046,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
|
||||
@@ -25,6 +25,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"Du schreibst eine kurze, persönliche Einschätzung zu EINER Gesundheitsmetrik dieses Nutzers. Du kennst seine Daten und ordnest sie gegen seine EIGENE Baseline ein — nicht gegen Bevölkerungsnormen. Deine fachliche Grundlage sind anerkannte Leitlinien (ESH 2023, WHO, AASM, DGE, DEGAM); du beziehst dich darauf, ohne zu diagnostizieren.",
|
||||
"",
|
||||
@@ -231,6 +232,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
@@ -431,6 +433,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
@@ -670,6 +673,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
@@ -1091,6 +1095,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
@@ -1239,6 +1244,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"Du schreibst eine kurze, persönliche Einschätzung zu EINER Gesundheitsmetrik dieses Nutzers. Du kennst seine Daten und ordnest sie gegen seine EIGENE Baseline ein — nicht gegen Bevölkerungsnormen. Deine fachliche Grundlage sind anerkannte Leitlinien (ESH 2023, WHO, AASM, DGE, DEGAM); du beziehst dich darauf, ohne zu diagnostizieren.",
|
||||
"",
|
||||
@@ -1452,6 +1458,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
|
||||
@@ -155,6 +155,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
@@ -385,6 +386,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
@@ -741,6 +743,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"Du schreibst eine kurze, persönliche Einschätzung zu EINER Gesundheitsmetrik dieses Nutzers. Du kennst seine Daten und ordnest sie gegen seine EIGENE Baseline ein — nicht gegen Bevölkerungsnormen. Deine fachliche Grundlage sind anerkannte Leitlinien (ESH 2023, WHO, AASM, DGE, DEGAM); du beziehst dich darauf, ohne zu diagnostizieren.",
|
||||
"",
|
||||
@@ -945,6 +948,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
|
||||
@@ -25,6 +25,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"Du schreibst eine kurze, persönliche Einschätzung zu EINER Gesundheitsmetrik dieses Nutzers. Du kennst seine Daten und ordnest sie gegen seine EIGENE Baseline ein — nicht gegen Bevölkerungsnormen. Deine fachliche Grundlage sind anerkannte Leitlinien (ESH 2023, WHO, AASM, DGE, DEGAM); du beziehst dich darauf, ohne zu diagnostizieren.",
|
||||
"",
|
||||
@@ -228,6 +229,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
@@ -418,6 +420,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
@@ -647,6 +650,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
@@ -1058,6 +1062,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
@@ -1196,6 +1201,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"Du schreibst eine kurze, persönliche Einschätzung zu EINER Gesundheitsmetrik dieses Nutzers. Du kennst seine Daten und ordnest sie gegen seine EIGENE Baseline ein — nicht gegen Bevölkerungsnormen. Deine fachliche Grundlage sind anerkannte Leitlinien (ESH 2023, WHO, AASM, DGE, DEGAM); du beziehst dich darauf, ohne zu diagnostizieren.",
|
||||
"",
|
||||
@@ -1399,6 +1405,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
|
||||
@@ -191,6 +191,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
@@ -486,6 +487,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
@@ -895,6 +897,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
@@ -1052,6 +1055,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"Du schreibst eine kurze, persönliche Einschätzung zu EINER Gesundheitsmetrik dieses Nutzers. Du kennst seine Daten und ordnest sie gegen seine EIGENE Baseline ein — nicht gegen Bevölkerungsnormen. Deine fachliche Grundlage sind anerkannte Leitlinien (ESH 2023, WHO, AASM, DGE, DEGAM); du beziehst dich darauf, ohne zu diagnostizieren.",
|
||||
"",
|
||||
@@ -1294,6 +1298,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
|
||||
@@ -264,6 +264,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
@@ -654,6 +655,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
@@ -1200,6 +1202,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
@@ -1395,6 +1398,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"Du schreibst eine kurze, persönliche Einschätzung zu EINER Gesundheitsmetrik dieses Nutzers. Du kennst seine Daten und ordnest sie gegen seine EIGENE Baseline ein — nicht gegen Bevölkerungsnormen. Deine fachliche Grundlage sind anerkannte Leitlinien (ESH 2023, WHO, AASM, DGE, DEGAM); du beziehst dich darauf, ohne zu diagnostizieren.",
|
||||
"",
|
||||
@@ -1760,6 +1764,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
@@ -2125,6 +2130,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
@@ -2371,6 +2377,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
@@ -2721,6 +2728,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
@@ -2876,6 +2884,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"Du schreibst eine kurze, persönliche Einschätzung zu EINER Gesundheitsmetrik dieses Nutzers. Du kennst seine Daten und ordnest sie gegen seine EIGENE Baseline ein — nicht gegen Bevölkerungsnormen. Deine fachliche Grundlage sind anerkannte Leitlinien (ESH 2023, WHO, AASM, DGE, DEGAM); du beziehst dich darauf, ohne zu diagnostizieren.",
|
||||
"",
|
||||
@@ -3096,6 +3105,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
@@ -3617,6 +3627,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
@@ -4175,6 +4186,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
@@ -4941,6 +4953,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
@@ -5200,6 +5213,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"Du schreibst eine kurze, persönliche Einschätzung zu EINER Gesundheitsmetrik dieses Nutzers. Du kennst seine Daten und ordnest sie gegen seine EIGENE Baseline ein — nicht gegen Bevölkerungsnormen. Deine fachliche Grundlage sind anerkannte Leitlinien (ESH 2023, WHO, AASM, DGE, DEGAM); du beziehst dich darauf, ohne zu diagnostizieren.",
|
||||
"",
|
||||
@@ -5733,6 +5747,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
|
||||
@@ -25,6 +25,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"Du schreibst eine kurze, persönliche Einschätzung zu EINER Gesundheitsmetrik dieses Nutzers. Du kennst seine Daten und ordnest sie gegen seine EIGENE Baseline ein — nicht gegen Bevölkerungsnormen. Deine fachliche Grundlage sind anerkannte Leitlinien (ESH 2023, WHO, AASM, DGE, DEGAM); du beziehst dich darauf, ohne zu diagnostizieren.",
|
||||
"",
|
||||
@@ -228,6 +229,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
@@ -418,6 +420,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
@@ -647,6 +650,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
@@ -1058,6 +1062,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
@@ -1196,6 +1201,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"Du schreibst eine kurze, persönliche Einschätzung zu EINER Gesundheitsmetrik dieses Nutzers. Du kennst seine Daten und ordnest sie gegen seine EIGENE Baseline ein — nicht gegen Bevölkerungsnormen. Deine fachliche Grundlage sind anerkannte Leitlinien (ESH 2023, WHO, AASM, DGE, DEGAM); du beziehst dich darauf, ohne zu diagnostizieren.",
|
||||
"",
|
||||
@@ -1399,6 +1405,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
|
||||
@@ -25,6 +25,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"Du schreibst eine kurze, persönliche Einschätzung zu EINER Gesundheitsmetrik dieses Nutzers. Du kennst seine Daten und ordnest sie gegen seine EIGENE Baseline ein — nicht gegen Bevölkerungsnormen. Deine fachliche Grundlage sind anerkannte Leitlinien (ESH 2023, WHO, AASM, DGE, DEGAM); du beziehst dich darauf, ohne zu diagnostizieren.",
|
||||
"",
|
||||
@@ -227,6 +228,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
@@ -416,6 +418,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
@@ -644,6 +647,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
@@ -1054,6 +1058,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
@@ -1191,6 +1196,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"Du schreibst eine kurze, persönliche Einschätzung zu EINER Gesundheitsmetrik dieses Nutzers. Du kennst seine Daten und ordnest sie gegen seine EIGENE Baseline ein — nicht gegen Bevölkerungsnormen. Deine fachliche Grundlage sind anerkannte Leitlinien (ESH 2023, WHO, AASM, DGE, DEGAM); du beziehst dich darauf, ohne zu diagnostizieren.",
|
||||
"",
|
||||
@@ -1393,6 +1399,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
|
||||
@@ -25,6 +25,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"Du schreibst eine kurze, persönliche Einschätzung zu EINER Gesundheitsmetrik dieses Nutzers. Du kennst seine Daten und ordnest sie gegen seine EIGENE Baseline ein — nicht gegen Bevölkerungsnormen. Deine fachliche Grundlage sind anerkannte Leitlinien (ESH 2023, WHO, AASM, DGE, DEGAM); du beziehst dich darauf, ohne zu diagnostizieren.",
|
||||
"",
|
||||
@@ -231,6 +232,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
@@ -431,6 +433,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
@@ -670,6 +673,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
@@ -1113,6 +1117,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"Du schreibst eine kurze, persönliche Einschätzung zu EINER Gesundheitsmetrik dieses Nutzers. Du kennst seine Daten und ordnest sie gegen seine EIGENE Baseline ein — nicht gegen Bevölkerungsnormen. Deine fachliche Grundlage sind anerkannte Leitlinien (ESH 2023, WHO, AASM, DGE, DEGAM); du beziehst dich darauf, ohne zu diagnostizieren.",
|
||||
"",
|
||||
@@ -1326,6 +1331,7 @@
|
||||
],
|
||||
"responseFormat": "json",
|
||||
"seed": 1618033,
|
||||
"signal": {},
|
||||
"system": [
|
||||
"You are writing a short, personal assessment of ONE of this user's health metrics. You know their data and you read it against their OWN baseline — never against a population norm. Your clinical grounding is established guidelines (ESH 2023, WHO, AASM, DGE, DEGAM); you reference them without diagnosing.",
|
||||
"",
|
||||
|
||||
Reference in new issue
Block a user