fix(ai): gate OpenAI JSON mode on responseFormat instead of forcing it

The OpenAI client pinned `response_format: { type: "json_object" }` on every
call. The Coach tool loop's forced-final round runs with `tool_choice: "none"`
and expects plain prose, but JSON mode coerced the reply into a JSON object —
the user saw a JSON blob or an empty reply instead of the coach's answer.

Send `response_format` only when the caller opts in with `responseFormat: "json"`
and no tools are present, mirroring the Anthropic client's prefill gate. The
JSON-consuming callers that relied on the implicit mode (insight generate +
briefing retry, medication extraction, the connection-test probe) now pass the
flag explicitly; the prose Coach path and every tool round stay out of JSON
mode. Codex already omitted the flag and is unaffected.
This commit is contained in:
Marc-André Bombeck committed 2026-06-22 08:11:16 +02:00
1 parent 745991a922
commit 7b0dcbf560
6 files changed
+91 -9

No files matched your search

+3
View File
@@ -72,6 +72,9 @@ export const POST = apiHandler(async (request: NextRequest) => {
user: 'Reply with the JSON object {"ok": true} and nothing else.',
temperature: 0,
maxTokens: 32,
// The probe asks for a JSON object — keep the OpenAI / Codex strict
// JSON mode it relied on before the response_format gate landed.
responseFormat: "json",
}),
);
+5
View File
@@ -682,6 +682,9 @@ export const POST = apiHandler((request: NextRequest) =>
user: userPrompt,
temperature: 0.3,
maxTokens: 1500,
// The reply is parsed with `JSON.parse` below, so opt the OpenAI /
// Codex chains into their strict JSON mode (gated on this flag).
responseFormat: "json",
}),
});
result = fallback.result;
@@ -815,6 +818,8 @@ export const POST = apiHandler((request: NextRequest) =>
user: `${userPrompt}\n\n${buildBriefingGroundingCorrection(ungrounded)}`,
temperature: 0.3,
maxTokens: 1500,
// Parsed with `JSON.parse` below — opt OpenAI / Codex into JSON mode.
responseFormat: "json",
}),
});
const parsedRetry = JSON.parse(retry.result.content);
+3
View File
@@ -195,6 +195,9 @@ async function handleExtract(request: NextRequest): Promise<Response> {
// The reply body is a single small JSON object; 600 is well
// over the empirical max we have seen on a 2k-char input.
maxTokens: 600,
// Parsed with `parseModelJson` (JSON.parse) below — opt the OpenAI /
// Codex chains into their strict JSON-object mode (gated on this flag).
responseFormat: "json",
}),
});
} catch (err) {