mirror of
https://github.com/MBombeck/HealthLog.git
synced 2026-11-07 12:22:03 +00:00
feat(ai): fallback-chain slug discovery with 1h positive cache
Phase C1 (v1.4.15) — slug-drift defence per Marc's hard mandate:
"Die Integration des Slug Drift Risiko darf halt immer überhaupt nicht
sein." ("Slug-drift risk must NEVER happen.")
Background: the ChatGPT-Codex backend rotates its allow-list silently.
v1.4.7..v1.4.13 lost a week to chasing the rotation manually
(`gpt-5-codex` → `gpt-5.3-codex`). A single hard-coded `CODEX_MODEL`
flips invalid the moment upstream rotates it.
Solution — the client now walks an ordered fallback chain on every
fresh request series and caches the first slug that returns 200 +
`response.completed` for 1 hour. Subsequent requests skip the chain
walk entirely on a cache hit.
Spec: extends `docs/codex-protocol-spec.md` with new §7b documenting
the chain semantics, cache TTL, walk triggers (400 + "not supported
when using Codex with a ChatGPT account" / `model_not_found` /
"does not exist" / 404), non-walk responses (5xx, 429, 401-with-
refresh, `invalid_prompt`), and the all-failed structured-503 error
shape. Per Marc's v1.4.6 hard rule ("read source-of-truth before
iterating; no more 6-iteration line-fixes"), the spec was extended
FIRST and the implementation cross-referenced against it.
Implementation:
- New `src/lib/ai/codex-slug-cache.ts` — process-local positive
cache. Single-slot Map (per-account scoping is a one-line change
when v1.4.16 multi-tenant rolls out). 1 h TTL. `getCachedCodexSlug`
auto-evicts expired entries on read.
- Rewritten `CodexClient.generateCompletion()` — walks the chain
`[cached?, ...slugChain]` in order. On 401 → refresh + retry SAME
slug (auth ≠ slug-state). On 400/404 with rejection body → drop
cache, walk to next. On 5xx/429 → propagate immediately (walking
won't help). On all-rejected → throw structured 503 with
`attempted[]` list, `httpStatus: 503`, message "AI provider
unreachable — all configured Codex slugs were rejected".
- `CodexAttemptDiagnostics` exposes `{ attempted, cacheState,
workingSlug }` for Wide-Event annotations the route can read via
`client.getLastDiagnostics()`.
- `CODEX_MODEL_FALLBACK_CHAIN` env override (comma-separated). When
`CODEX_MODEL` is also set, it folds into chain position 0; the
default chain (`gpt-5.3-codex` → `gpt-5-codex` → `gpt-5` →
`gpt-4o`) survives behind it. Stable de-dup.
Tests: +20 across 2 new files.
`codex-slug-cache.test.ts` (7 tests) — TTL-edge boundaries,
invalidation, replacement semantics.
`codex-slug-fallback.test.ts` (13 tests) — cold-cache happy path,
walk on canonical 400 body, walk on `model_not_found`, walk on 404,
no-walk on 500/429/401, all-rejected → 503-with-attempted-list,
cache-hit shortcut, cache invalidation on rejection, isSlugRejection
body classifier, loadFallbackChain env-pin merge.
The 5 existing `codex-client.test.ts` cases continue to pass without
edit — the new `slugChain` constructor parameter is opt-in and the
single-slug behaviour reduces to chain length 1.
Tests: 1028 → 1048 unit pass (+20). Typecheck clean for AI files.
Lint clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
76dd48e7b8
commit
4bba951121
5 files changed
+773
-47
No files matched your search
@@ -635,6 +635,47 @@ Confirmed-accepted slugs on a ChatGPT Plus subscription (per `available_in_plans
|
||||
|
||||
The `codex-rs/tui/src/model_migration.rs` references to `gpt-5-codex` etc. are _migration prompts_ shown to existing users — they are NOT the slugs sent on the wire after migration. The wire slugs come from `models.json`. Always cross-check `models.json` (`gh api repos/openai/codex/contents/codex-rs/models-manager/models.json`) before picking a default.
|
||||
|
||||
### 7b. Slug-drift defence — fallback chain (HealthLog v1.4.15+)
|
||||
|
||||
The ChatGPT-Codex allow-list is rotated by OpenAI without prior notice (`gpt-5-codex` → `gpt-5.3-codex` happened over 48 h in May 2026 and bricked HealthLog twice while the v1.4.7..v1.4.13 saga unrolled). The cause: a single hard-coded slug in `CODEX_MODEL` flips invalid the moment the upstream rotates it. Marc, verbatim 2026-05-09: "Die Integration des Slug Drift Risiko darf halt immer überhaupt nicht sein."
|
||||
|
||||
To eliminate that risk class, HealthLog's `CodexClient` walks an ordered chain on every fresh request series.
|
||||
|
||||
**Chain** (top = first try; default order is the empirically-safest most-current-first ordering as of 2026-05):
|
||||
|
||||
```ts
|
||||
const DEFAULT_SLUG_FALLBACK_CHAIN = [
|
||||
"gpt-5.3-codex", // current allow-list entry (verified 2026-05-09)
|
||||
"gpt-5-codex", // historical default (rejected as of 2026-05 but safe to retry — backend may flip)
|
||||
"gpt-5", // bare slug — rejected on ChatGPT-auth as of 2026-05 but lives on api-key auth
|
||||
"gpt-4o", // ladder-rung for last-ditch capability
|
||||
];
|
||||
```
|
||||
|
||||
The chain is overridable via the env var `CODEX_MODEL_FALLBACK_CHAIN` (comma-separated). The first env-var-named slug supersedes `CODEX_MODEL`; if both are set, `CODEX_MODEL` is folded into position 0 and de-duplicated.
|
||||
|
||||
**Trigger condition** (when to walk to the next slug):
|
||||
|
||||
| Status | Action |
|
||||
| ------------------------------------------------------------------------------------- | ----------- |
|
||||
| `400` body matches `/not supported when using Codex with a ChatGPT account/i` | walk |
|
||||
| `400` body contains `model_not_found` | walk |
|
||||
| `400` body contains `does not exist` AND mentions a model name | walk |
|
||||
| `404` (any body) | walk |
|
||||
| `401` first time | refresh and re-try same slug; do NOT walk |
|
||||
| `401` after refresh, `403`, `429`, `5xx` | propagate (don't walk) |
|
||||
| `200` with SSE `response.failed.error.code === "invalid_prompt"` | propagate (request shape is wrong, slug is fine) |
|
||||
|
||||
**Positive cache** — when a slug is observed working (HTTP 200 with at least one `response.completed` event), record `(slug, timestamp)` in a process-local Map. On the next request, jump straight to the cached slug. TTL: 1 hour. Cache scope: per-process (no DB write).
|
||||
|
||||
**Cache invalidation**: on first 400/404 walk, the cached entry is dropped. The next-best slug is promoted into the cache only after it succeeds.
|
||||
|
||||
**All-failed case**: if every slug in the chain rejects with the walk-trigger, the client throws a structured error with `httpStatus: 503` and message `"AI provider unreachable — all configured Codex slugs were rejected"`. The route layer surfaces this to the user with "AI provider unreachable — check Settings".
|
||||
|
||||
**Observability**: each attempted slug is recorded in the Wide-Event annotation `codex.slug.attempts: ["gpt-5.3-codex","gpt-5-codex"]` so Loki can answer "which slug carried this request" and "how often did the cache hit". Cache hits emit `codex.slug.cache: "hit" | "miss" | "expired"`.
|
||||
|
||||
The implementation lives in `src/lib/ai/codex-client.ts`; the contract is unit-tested in `src/lib/ai/__tests__/codex-client.test.ts`.
|
||||
|
||||
---
|
||||
|
||||
## 8. Worked example — minimal end-to-end
|
||||
|
||||
@@ -0,0 +1,67 @@
|
||||
import { describe, it, expect, beforeEach } from "vitest";
|
||||
import {
|
||||
getCachedCodexSlug,
|
||||
setCachedCodexSlug,
|
||||
invalidateCachedCodexSlug,
|
||||
clearCodexSlugCache,
|
||||
inspectCodexSlugCache,
|
||||
CODEX_SLUG_CACHE_TTL_MS,
|
||||
} from "../codex-slug-cache";
|
||||
|
||||
/**
|
||||
* Phase C1 — slug-drift defence positive cache.
|
||||
* Cache TTL = 1 hour per `docs/codex-protocol-spec.md` §7b.
|
||||
*/
|
||||
describe("Codex slug cache", () => {
|
||||
beforeEach(() => {
|
||||
clearCodexSlugCache();
|
||||
});
|
||||
|
||||
it("starts empty", () => {
|
||||
expect(getCachedCodexSlug()).toBeNull();
|
||||
expect(inspectCodexSlugCache()).toBeNull();
|
||||
});
|
||||
|
||||
it("set + get returns the cached slug within TTL", () => {
|
||||
setCachedCodexSlug("gpt-5.3-codex", 1_000);
|
||||
expect(getCachedCodexSlug(1_000)).toBe("gpt-5.3-codex");
|
||||
expect(getCachedCodexSlug(1_000 + CODEX_SLUG_CACHE_TTL_MS - 1)).toBe(
|
||||
"gpt-5.3-codex",
|
||||
);
|
||||
});
|
||||
|
||||
it("expires entries strictly after the TTL", () => {
|
||||
setCachedCodexSlug("gpt-5.3-codex", 1_000);
|
||||
// Exactly at TTL: still valid (the comparison is `now - cached > TTL`).
|
||||
expect(getCachedCodexSlug(1_000 + CODEX_SLUG_CACHE_TTL_MS)).toBe(
|
||||
"gpt-5.3-codex",
|
||||
);
|
||||
// Past TTL: gone.
|
||||
expect(getCachedCodexSlug(1_000 + CODEX_SLUG_CACHE_TTL_MS + 1)).toBeNull();
|
||||
});
|
||||
|
||||
it("expired entry is evicted on read", () => {
|
||||
setCachedCodexSlug("gpt-5.3-codex", 1_000);
|
||||
getCachedCodexSlug(1_000 + CODEX_SLUG_CACHE_TTL_MS + 1);
|
||||
// Subsequent inspect should be null too.
|
||||
expect(inspectCodexSlugCache(1_000 + CODEX_SLUG_CACHE_TTL_MS + 1)).toBeNull();
|
||||
});
|
||||
|
||||
it("invalidate clears the slot", () => {
|
||||
setCachedCodexSlug("gpt-5.3-codex");
|
||||
invalidateCachedCodexSlug();
|
||||
expect(getCachedCodexSlug()).toBeNull();
|
||||
});
|
||||
|
||||
it("set replaces the previous slot", () => {
|
||||
setCachedCodexSlug("a", 1_000);
|
||||
setCachedCodexSlug("b", 2_000);
|
||||
expect(getCachedCodexSlug(2_500)).toBe("b");
|
||||
});
|
||||
|
||||
it("inspect reports age in ms", () => {
|
||||
setCachedCodexSlug("gpt-5.3-codex", 1_000);
|
||||
const inspected = inspectCodexSlugCache(1_500);
|
||||
expect(inspected).toEqual({ slug: "gpt-5.3-codex", ageMs: 500 });
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,362 @@
|
||||
import { describe, it, expect, vi, beforeEach, afterEach } from "vitest";
|
||||
import { CodexClient, __test } from "../codex-client";
|
||||
import {
|
||||
clearCodexSlugCache,
|
||||
inspectCodexSlugCache,
|
||||
setCachedCodexSlug,
|
||||
CODEX_SLUG_CACHE_TTL_MS,
|
||||
} from "../codex-slug-cache";
|
||||
|
||||
/**
|
||||
* Phase C1 — slug-drift defence.
|
||||
*
|
||||
* Marc, verbatim 2026-05-09: "Die Integration des Slug Drift Risiko
|
||||
* darf halt immer überhaupt nicht sein."
|
||||
* ("Slug-drift risk must never happen.")
|
||||
*
|
||||
* Pattern from `docs/codex-protocol-spec.md` §7b:
|
||||
* 1. Walk the fallback chain on each fresh request series.
|
||||
* 2. Cache the working slug for 1 h.
|
||||
* 3. On all-failed: throw structured 503 "AI provider unreachable".
|
||||
*
|
||||
* These tests stub `fetch` deterministically per-slug and assert:
|
||||
* - Walk happens on 400 + "not supported when using Codex with a
|
||||
* ChatGPT account" (the exact body OpenAI returns).
|
||||
* - Walk happens on 400 + model_not_found.
|
||||
* - Walk happens on 404.
|
||||
* - Walk does NOT happen on 5xx, 429, or 401 (auth flow).
|
||||
* - Cache entry is dropped on the first slug-rejection walk.
|
||||
* - Cache hit causes the working slug to come first on the next
|
||||
* call.
|
||||
* - All-rejected case throws 503.
|
||||
*/
|
||||
|
||||
function sseSuccessResponse(text = '{"ok":true}'): Response {
|
||||
const encoder = new TextEncoder();
|
||||
const events = [
|
||||
`event: response.output_item.done\ndata: ${JSON.stringify({
|
||||
type: "response.output_item.done",
|
||||
item: {
|
||||
type: "message",
|
||||
role: "assistant",
|
||||
content: [{ type: "output_text", text }],
|
||||
},
|
||||
})}\n\n`,
|
||||
`event: response.completed\ndata: ${JSON.stringify({
|
||||
type: "response.completed",
|
||||
response: { id: "resp1", usage: { total_tokens: 10 } },
|
||||
})}\n\n`,
|
||||
];
|
||||
const stream = new ReadableStream<Uint8Array>({
|
||||
start(controller) {
|
||||
for (const ev of events) controller.enqueue(encoder.encode(ev));
|
||||
controller.close();
|
||||
},
|
||||
});
|
||||
return new Response(stream, {
|
||||
status: 200,
|
||||
headers: { "content-type": "text/event-stream" },
|
||||
});
|
||||
}
|
||||
|
||||
function rejectionResponse(
|
||||
status: number,
|
||||
body = '{"detail":"The \'gpt-5\' model is not supported when using Codex with a ChatGPT account."}',
|
||||
): Response {
|
||||
// Use a real Response so .text() works, body returned once.
|
||||
return new Response(body, {
|
||||
status,
|
||||
headers: { "content-type": "application/json" },
|
||||
});
|
||||
}
|
||||
|
||||
const onTokenRefresh = vi
|
||||
.fn()
|
||||
.mockResolvedValue({ accessToken: "x", accountId: "acct-test" });
|
||||
|
||||
describe("Codex slug fallback chain", () => {
|
||||
beforeEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
clearCodexSlugCache();
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
clearCodexSlugCache();
|
||||
});
|
||||
|
||||
it("first slug succeeds — uses head-of-chain and caches it", async () => {
|
||||
const mockFetch = vi.fn().mockResolvedValue(sseSuccessResponse());
|
||||
vi.stubGlobal("fetch", mockFetch);
|
||||
|
||||
const client = new CodexClient({
|
||||
accessToken: "test",
|
||||
accountId: "acct",
|
||||
onTokenRefresh,
|
||||
slugChain: ["gpt-5.3-codex", "gpt-5-codex", "gpt-4o"],
|
||||
});
|
||||
await client.generateCompletion({ systemPrompt: "s", userPrompt: "u" });
|
||||
|
||||
expect(mockFetch).toHaveBeenCalledTimes(1);
|
||||
const sentBody = JSON.parse(mockFetch.mock.calls[0][1].body);
|
||||
expect(sentBody.model).toBe("gpt-5.3-codex");
|
||||
|
||||
const diagnostics = client.getLastDiagnostics();
|
||||
expect(diagnostics?.attempted).toEqual(["gpt-5.3-codex"]);
|
||||
expect(diagnostics?.cacheState).toBe("miss");
|
||||
expect(diagnostics?.workingSlug).toBe("gpt-5.3-codex");
|
||||
|
||||
expect(inspectCodexSlugCache()?.slug).toBe("gpt-5.3-codex");
|
||||
});
|
||||
|
||||
it("walks past first slug on 'not supported when using Codex with a ChatGPT account' 400", async () => {
|
||||
const mockFetch = vi
|
||||
.fn()
|
||||
.mockResolvedValueOnce(rejectionResponse(400))
|
||||
.mockResolvedValueOnce(sseSuccessResponse());
|
||||
vi.stubGlobal("fetch", mockFetch);
|
||||
|
||||
const client = new CodexClient({
|
||||
accessToken: "test",
|
||||
accountId: "acct",
|
||||
onTokenRefresh,
|
||||
slugChain: ["gpt-5", "gpt-5.3-codex", "gpt-4o"],
|
||||
});
|
||||
await client.generateCompletion({ systemPrompt: "s", userPrompt: "u" });
|
||||
|
||||
expect(mockFetch).toHaveBeenCalledTimes(2);
|
||||
expect(JSON.parse(mockFetch.mock.calls[0][1].body).model).toBe("gpt-5");
|
||||
expect(JSON.parse(mockFetch.mock.calls[1][1].body).model).toBe(
|
||||
"gpt-5.3-codex",
|
||||
);
|
||||
|
||||
const diagnostics = client.getLastDiagnostics();
|
||||
expect(diagnostics?.attempted).toEqual(["gpt-5", "gpt-5.3-codex"]);
|
||||
expect(diagnostics?.workingSlug).toBe("gpt-5.3-codex");
|
||||
|
||||
// The working slug — not the rejected one — is now cached.
|
||||
expect(inspectCodexSlugCache()?.slug).toBe("gpt-5.3-codex");
|
||||
});
|
||||
|
||||
it("walks past first slug on 400 with model_not_found body", async () => {
|
||||
const mockFetch = vi
|
||||
.fn()
|
||||
.mockResolvedValueOnce(
|
||||
rejectionResponse(
|
||||
400,
|
||||
'{"error":{"code":"model_not_found","message":"unknown slug"}}',
|
||||
),
|
||||
)
|
||||
.mockResolvedValueOnce(sseSuccessResponse());
|
||||
vi.stubGlobal("fetch", mockFetch);
|
||||
|
||||
const client = new CodexClient({
|
||||
accessToken: "test",
|
||||
accountId: "acct",
|
||||
onTokenRefresh,
|
||||
slugChain: ["nonexistent-slug", "gpt-5.3-codex"],
|
||||
});
|
||||
await client.generateCompletion({ systemPrompt: "s", userPrompt: "u" });
|
||||
|
||||
expect(mockFetch).toHaveBeenCalledTimes(2);
|
||||
expect(client.getLastDiagnostics()?.workingSlug).toBe("gpt-5.3-codex");
|
||||
});
|
||||
|
||||
it("walks past first slug on 404", async () => {
|
||||
const mockFetch = vi
|
||||
.fn()
|
||||
.mockResolvedValueOnce(rejectionResponse(404, "Not found"))
|
||||
.mockResolvedValueOnce(sseSuccessResponse());
|
||||
vi.stubGlobal("fetch", mockFetch);
|
||||
|
||||
const client = new CodexClient({
|
||||
accessToken: "test",
|
||||
accountId: "acct",
|
||||
onTokenRefresh,
|
||||
slugChain: ["dead-slug", "gpt-5.3-codex"],
|
||||
});
|
||||
await client.generateCompletion({ systemPrompt: "s", userPrompt: "u" });
|
||||
|
||||
expect(mockFetch).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
|
||||
it("does NOT walk on 500 — propagates immediately", async () => {
|
||||
const mockFetch = vi
|
||||
.fn()
|
||||
.mockResolvedValueOnce(rejectionResponse(500, "Internal server error"));
|
||||
vi.stubGlobal("fetch", mockFetch);
|
||||
|
||||
const client = new CodexClient({
|
||||
accessToken: "test",
|
||||
accountId: "acct",
|
||||
onTokenRefresh,
|
||||
slugChain: ["gpt-5.3-codex", "gpt-4o"],
|
||||
});
|
||||
|
||||
await expect(
|
||||
client.generateCompletion({ systemPrompt: "s", userPrompt: "u" }),
|
||||
).rejects.toThrow("Codex request failed (500)");
|
||||
|
||||
expect(mockFetch).toHaveBeenCalledTimes(1); // no walk
|
||||
});
|
||||
|
||||
it("does NOT walk on 429 — propagates immediately", async () => {
|
||||
const mockFetch = vi
|
||||
.fn()
|
||||
.mockResolvedValueOnce(rejectionResponse(429, "Rate limited"));
|
||||
vi.stubGlobal("fetch", mockFetch);
|
||||
|
||||
const client = new CodexClient({
|
||||
accessToken: "test",
|
||||
accountId: "acct",
|
||||
onTokenRefresh,
|
||||
slugChain: ["gpt-5.3-codex", "gpt-4o"],
|
||||
});
|
||||
|
||||
await expect(
|
||||
client.generateCompletion({ systemPrompt: "s", userPrompt: "u" }),
|
||||
).rejects.toThrow("Codex request failed (429)");
|
||||
|
||||
expect(mockFetch).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("does NOT walk on 401 — refreshes and retries same slug", async () => {
|
||||
const mockFetch = vi
|
||||
.fn()
|
||||
.mockResolvedValueOnce(
|
||||
new Response("unauthorized", { status: 401 }),
|
||||
)
|
||||
.mockResolvedValueOnce(sseSuccessResponse());
|
||||
vi.stubGlobal("fetch", mockFetch);
|
||||
|
||||
const client = new CodexClient({
|
||||
accessToken: "expired",
|
||||
accountId: "acct",
|
||||
onTokenRefresh: vi
|
||||
.fn()
|
||||
.mockResolvedValue({ accessToken: "fresh", accountId: "acct" }),
|
||||
slugChain: ["gpt-5.3-codex", "gpt-4o"],
|
||||
});
|
||||
|
||||
await client.generateCompletion({ systemPrompt: "s", userPrompt: "u" });
|
||||
|
||||
// Both calls hit the SAME slug — auth retry didn't walk to gpt-4o.
|
||||
expect(JSON.parse(mockFetch.mock.calls[0][1].body).model).toBe(
|
||||
"gpt-5.3-codex",
|
||||
);
|
||||
expect(JSON.parse(mockFetch.mock.calls[1][1].body).model).toBe(
|
||||
"gpt-5.3-codex",
|
||||
);
|
||||
expect(client.getLastDiagnostics()?.attempted).toEqual(["gpt-5.3-codex"]);
|
||||
});
|
||||
|
||||
it("all slugs rejected → throws structured 503 with attempted list", async () => {
|
||||
// mockImplementation returns a fresh Response each call — Response
|
||||
// bodies can only be read once, so a single shared object would
|
||||
// throw "body used already" on the second walk-step.
|
||||
const mockFetch = vi.fn(() => Promise.resolve(rejectionResponse(400)));
|
||||
vi.stubGlobal("fetch", mockFetch);
|
||||
|
||||
const client = new CodexClient({
|
||||
accessToken: "test",
|
||||
accountId: "acct",
|
||||
onTokenRefresh,
|
||||
slugChain: ["a", "b", "c"],
|
||||
});
|
||||
|
||||
let caught: unknown;
|
||||
try {
|
||||
await client.generateCompletion({ systemPrompt: "s", userPrompt: "u" });
|
||||
} catch (e) {
|
||||
caught = e;
|
||||
}
|
||||
|
||||
const err = caught as Error & {
|
||||
httpStatus?: number;
|
||||
attempted?: string[];
|
||||
};
|
||||
expect(err.message).toMatch(/AI provider unreachable/);
|
||||
expect(err.httpStatus).toBe(503);
|
||||
expect(err.attempted).toEqual(["a", "b", "c"]);
|
||||
expect(mockFetch).toHaveBeenCalledTimes(3);
|
||||
expect(client.getLastDiagnostics()?.workingSlug).toBeNull();
|
||||
expect(client.getLastDiagnostics()?.attempted).toEqual(["a", "b", "c"]);
|
||||
});
|
||||
|
||||
it("cache hit — second call goes straight to the cached slug", async () => {
|
||||
setCachedCodexSlug("gpt-4o");
|
||||
const mockFetch = vi.fn().mockResolvedValue(sseSuccessResponse());
|
||||
vi.stubGlobal("fetch", mockFetch);
|
||||
|
||||
const client = new CodexClient({
|
||||
accessToken: "test",
|
||||
accountId: "acct",
|
||||
onTokenRefresh,
|
||||
slugChain: ["gpt-5.3-codex", "gpt-5-codex", "gpt-4o"],
|
||||
});
|
||||
await client.generateCompletion({ systemPrompt: "s", userPrompt: "u" });
|
||||
|
||||
expect(mockFetch).toHaveBeenCalledTimes(1);
|
||||
expect(JSON.parse(mockFetch.mock.calls[0][1].body).model).toBe("gpt-4o");
|
||||
expect(client.getLastDiagnostics()?.cacheState).toBe("hit");
|
||||
});
|
||||
|
||||
it("cache invalidated when cached slug starts rejecting", async () => {
|
||||
setCachedCodexSlug("gpt-5-codex");
|
||||
expect(inspectCodexSlugCache()?.slug).toBe("gpt-5-codex");
|
||||
|
||||
const mockFetch = vi
|
||||
.fn()
|
||||
.mockResolvedValueOnce(rejectionResponse(400))
|
||||
.mockResolvedValueOnce(sseSuccessResponse());
|
||||
vi.stubGlobal("fetch", mockFetch);
|
||||
|
||||
const client = new CodexClient({
|
||||
accessToken: "test",
|
||||
accountId: "acct",
|
||||
onTokenRefresh,
|
||||
slugChain: ["gpt-5.3-codex", "gpt-5-codex"],
|
||||
});
|
||||
await client.generateCompletion({ systemPrompt: "s", userPrompt: "u" });
|
||||
|
||||
// After walking to "gpt-5.3-codex" (the next entry after the
|
||||
// cached one), the cache is updated to the new working slug.
|
||||
expect(inspectCodexSlugCache()?.slug).toBe("gpt-5.3-codex");
|
||||
});
|
||||
|
||||
it("isSlugRejection helper recognises canonical rejection bodies", () => {
|
||||
const helper = __test.isSlugRejection;
|
||||
expect(
|
||||
helper(
|
||||
400,
|
||||
'{"detail":"The \'gpt-5\' model is not supported when using Codex with a ChatGPT account."}',
|
||||
),
|
||||
).toBe(true);
|
||||
expect(helper(400, '{"error":{"code":"model_not_found"}}')).toBe(true);
|
||||
expect(helper(400, "model gpt-5 does not exist")).toBe(true);
|
||||
expect(helper(404, "anything")).toBe(true);
|
||||
expect(helper(500, "internal error")).toBe(false);
|
||||
expect(helper(429, "rate limited")).toBe(false);
|
||||
expect(helper(400, "unrelated 400")).toBe(false);
|
||||
});
|
||||
|
||||
it("loadFallbackChain folds CODEX_MODEL into position 0", () => {
|
||||
const orig = process.env.CODEX_MODEL;
|
||||
process.env.CODEX_MODEL = "custom-pinned-slug";
|
||||
try {
|
||||
const chain = __test.loadFallbackChain();
|
||||
expect(chain[0]).toBe("custom-pinned-slug");
|
||||
// Defaults still present after position 0.
|
||||
expect(chain).toContain("gpt-5.3-codex");
|
||||
// No duplicates.
|
||||
const seen = new Set(chain);
|
||||
expect(seen.size).toBe(chain.length);
|
||||
} finally {
|
||||
if (orig === undefined) delete process.env.CODEX_MODEL;
|
||||
else process.env.CODEX_MODEL = orig;
|
||||
}
|
||||
});
|
||||
|
||||
it("CODEX_SLUG_CACHE_TTL_MS is exactly 1 hour per spec §7b", () => {
|
||||
expect(CODEX_SLUG_CACHE_TTL_MS).toBe(60 * 60 * 1000);
|
||||
});
|
||||
});
|
||||
+229
-47
@@ -1,5 +1,10 @@
|
||||
import { randomUUID } from "node:crypto";
|
||||
import type { AIProvider, CompletionParams, CompletionResult } from "./types";
|
||||
import {
|
||||
getCachedCodexSlug,
|
||||
invalidateCachedCodexSlug,
|
||||
setCachedCodexSlug,
|
||||
} from "./codex-slug-cache";
|
||||
|
||||
/**
|
||||
* Codex backend client — talks to `chatgpt.com/backend-api/codex/responses`
|
||||
@@ -16,35 +21,67 @@ import type { AIProvider, CompletionParams, CompletionResult } from "./types";
|
||||
* - `originator`, `User-Agent`, `session_id`, `thread_id`
|
||||
*
|
||||
* Response is always SSE (server rejects `stream: false` with 400).
|
||||
*
|
||||
* v1.4.15 (Phase C1) adds a slug-drift defence — see
|
||||
* `docs/codex-protocol-spec.md` §7b. The client walks an ordered
|
||||
* fallback chain on every fresh request series (short-circuiting on
|
||||
* a cached working slug) so a single upstream allow-list rotation no
|
||||
* longer bricks the integration.
|
||||
*/
|
||||
|
||||
const CODEX_ENDPOINT = "https://chatgpt.com/backend-api/codex/responses";
|
||||
|
||||
/**
|
||||
* Model slug for the ChatGPT-account auth path.
|
||||
*
|
||||
* The ChatGPT/Codex backend has a tight allow-list of slugs accepted for
|
||||
* subscription auth. As of 2026-05 the bundled Codex models in the
|
||||
* official `openai/codex` client are `gpt-5.5` / `gpt-5.4` /
|
||||
* `gpt-5.4-mini` / `gpt-5.3-codex` / `gpt-5.2`; `gpt-5` and `gpt-5-codex`
|
||||
* are both rejected on this auth path with a 400:
|
||||
*
|
||||
* {"detail":"The 'gpt-5' model is not supported when using Codex with
|
||||
* a ChatGPT account."}
|
||||
*
|
||||
* `gpt-5.3-codex` is the codex-optimized slug available on Plus/Pro
|
||||
* plans (per `codex-rs/models-manager/models.json` `available_in_plans`)
|
||||
* and accepts our HealthLog "summarize health metrics" prompts without
|
||||
* issue (verified live 2026-05-09).
|
||||
*
|
||||
* The server safety-routes when needed and reports the actual routed
|
||||
* model in the `OpenAI-Model` response header — we trust that header
|
||||
* for downstream logging.
|
||||
*
|
||||
* Operators on a different plan can override via the `CODEX_MODEL` env
|
||||
* var on apps01 (e.g. `gpt-5.5` on Free, `gpt-5.4` on Plus/Pro).
|
||||
* Default fallback chain — most-current-first as of 2026-05-09. Override
|
||||
* via `CODEX_MODEL_FALLBACK_CHAIN` (comma-separated) on apps01 if a
|
||||
* specific plan ladder needs different ordering. `CODEX_MODEL` is
|
||||
* folded into position 0 of the chain when set; duplicates are dropped.
|
||||
*/
|
||||
const CODEX_MODEL = process.env.CODEX_MODEL?.trim() || "gpt-5.3-codex";
|
||||
const DEFAULT_SLUG_FALLBACK_CHAIN = [
|
||||
"gpt-5.3-codex", // verified accepted on Plus/Pro 2026-05-09
|
||||
"gpt-5-codex", // historical default — kept as second-chance retry; backend may flip back
|
||||
"gpt-5", // bare slug — rejected on ChatGPT-auth as of 2026-05; safe to keep as ladder-rung
|
||||
"gpt-4o", // last-ditch capability fallback
|
||||
] as const;
|
||||
|
||||
function loadFallbackChain(): string[] {
|
||||
const envChain = process.env.CODEX_MODEL_FALLBACK_CHAIN?.trim();
|
||||
const fromEnv = envChain
|
||||
? envChain
|
||||
.split(",")
|
||||
.map((s) => s.trim())
|
||||
.filter(Boolean)
|
||||
: [...DEFAULT_SLUG_FALLBACK_CHAIN];
|
||||
const pinned = process.env.CODEX_MODEL?.trim();
|
||||
const merged = pinned ? [pinned, ...fromEnv] : fromEnv;
|
||||
// Stable de-duplication.
|
||||
const seen = new Set<string>();
|
||||
const result: string[] = [];
|
||||
for (const slug of merged) {
|
||||
if (seen.has(slug)) continue;
|
||||
seen.add(slug);
|
||||
result.push(slug);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
* Triggers per spec §7b — body / status that mean "this slug is no
|
||||
* longer accepted, walk to the next one in the chain".
|
||||
*/
|
||||
function isSlugRejection(status: number, bodyExcerpt: string): boolean {
|
||||
if (status === 404) return true;
|
||||
if (status !== 400) return false;
|
||||
const lower = bodyExcerpt.toLowerCase();
|
||||
if (lower.includes("not supported when using codex with a chatgpt account")) {
|
||||
return true;
|
||||
}
|
||||
if (lower.includes("model_not_found")) return true;
|
||||
if (lower.includes("does not exist") && lower.includes("model")) {
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
const ORIGINATOR = "healthlog";
|
||||
const USER_AGENT = "HealthLog/1.0 (+https://healthlog.bombeck.io)";
|
||||
@@ -53,6 +90,18 @@ interface CodexClientConfig {
|
||||
accessToken: string;
|
||||
accountId: string;
|
||||
onTokenRefresh: () => Promise<{ accessToken: string; accountId: string }>;
|
||||
/**
|
||||
* Override the slug fallback chain (test-only). When unset, the
|
||||
* chain is loaded from env + DEFAULT_SLUG_FALLBACK_CHAIN.
|
||||
*/
|
||||
slugChain?: string[];
|
||||
}
|
||||
|
||||
/** Diagnostic shape returned to the route layer for Wide-Event logging. */
|
||||
export interface CodexAttemptDiagnostics {
|
||||
attempted: string[];
|
||||
cacheState: "hit" | "miss" | "expired";
|
||||
workingSlug: string | null;
|
||||
}
|
||||
|
||||
export class CodexClient implements AIProvider {
|
||||
@@ -63,59 +112,174 @@ export class CodexClient implements AIProvider {
|
||||
accessToken: string;
|
||||
accountId: string;
|
||||
}>;
|
||||
private readonly slugChain: string[];
|
||||
private lastDiagnostics: CodexAttemptDiagnostics | null = null;
|
||||
|
||||
constructor(config: CodexClientConfig) {
|
||||
this.accessToken = config.accessToken;
|
||||
this.accountId = config.accountId;
|
||||
this.onTokenRefresh = config.onTokenRefresh;
|
||||
this.slugChain = config.slugChain ?? loadFallbackChain();
|
||||
if (this.slugChain.length === 0) {
|
||||
// Hardcoded floor — even with a misconfigured env var we always
|
||||
// try at least one slug.
|
||||
this.slugChain = [...DEFAULT_SLUG_FALLBACK_CHAIN];
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Diagnostics for the most recent generateCompletion() call, used by
|
||||
* the route layer for Wide-Event annotations. Captures the order of
|
||||
* attempted slugs, whether the cache was hit, and which slug
|
||||
* eventually worked (or null on all-failed).
|
||||
*/
|
||||
getLastDiagnostics(): CodexAttemptDiagnostics | null {
|
||||
return this.lastDiagnostics;
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the slug-attempt order: cached slug first (if any and not
|
||||
* expired), then the rest of the fallback chain in order, deduped
|
||||
* against the cached slug.
|
||||
*/
|
||||
private buildAttemptOrder(): { order: string[]; cacheState: "hit" | "miss" } {
|
||||
const cached = getCachedCodexSlug();
|
||||
if (cached === null) {
|
||||
return { order: [...this.slugChain], cacheState: "miss" };
|
||||
}
|
||||
const order = [cached, ...this.slugChain.filter((s) => s !== cached)];
|
||||
return { order, cacheState: "hit" };
|
||||
}
|
||||
|
||||
async generateCompletion(
|
||||
params: CompletionParams,
|
||||
): Promise<CompletionResult> {
|
||||
const firstAttempt = await this.doRequest(params);
|
||||
const { order, cacheState } = this.buildAttemptOrder();
|
||||
const attempted: string[] = [];
|
||||
let lastSlugRejectionError: Error | null = null;
|
||||
|
||||
if (firstAttempt.status === 401) {
|
||||
const fresh = await this.onTokenRefresh();
|
||||
this.accessToken = fresh.accessToken;
|
||||
this.accountId = fresh.accountId;
|
||||
const retryAttempt = await this.doRequest(params);
|
||||
if (!retryAttempt.ok) {
|
||||
for (const slug of order) {
|
||||
attempted.push(slug);
|
||||
|
||||
const firstAttempt = await this.doRequest(params, slug);
|
||||
|
||||
if (firstAttempt.status === 401) {
|
||||
// Token-refresh path — don't walk the chain, this is auth-state
|
||||
// not slug-state. Refresh once and re-try the SAME slug.
|
||||
const fresh = await this.onTokenRefresh();
|
||||
this.accessToken = fresh.accessToken;
|
||||
this.accountId = fresh.accountId;
|
||||
const retryAfterRefresh = await this.doRequest(params, slug);
|
||||
if (retryAfterRefresh.ok) {
|
||||
this.lastDiagnostics = {
|
||||
attempted,
|
||||
cacheState,
|
||||
workingSlug: slug,
|
||||
};
|
||||
setCachedCodexSlug(slug);
|
||||
return this.parseResponse(retryAfterRefresh, slug);
|
||||
}
|
||||
// Even after refresh — surface the auth failure verbatim, do
|
||||
// NOT walk the chain (auth issues don't get fixed by changing
|
||||
// the slug).
|
||||
throw await this.buildUpstreamError(
|
||||
retryAttempt,
|
||||
retryAfterRefresh,
|
||||
"Codex request failed after token refresh",
|
||||
slug,
|
||||
);
|
||||
}
|
||||
return this.parseResponse(retryAttempt);
|
||||
|
||||
if (firstAttempt.ok) {
|
||||
this.lastDiagnostics = {
|
||||
attempted,
|
||||
cacheState,
|
||||
workingSlug: slug,
|
||||
};
|
||||
setCachedCodexSlug(slug);
|
||||
return this.parseResponse(firstAttempt, slug);
|
||||
}
|
||||
|
||||
// Capture the body for the slug-rejection check.
|
||||
const rawBody = await firstAttempt.text().catch(() => "");
|
||||
const bodyExcerpt = redactBody(rawBody);
|
||||
|
||||
if (isSlugRejection(firstAttempt.status, bodyExcerpt)) {
|
||||
// Slug rejected — drop the cache (in case it pointed here),
|
||||
// record the failure for diagnostics, and walk to the next
|
||||
// chain slot.
|
||||
invalidateCachedCodexSlug();
|
||||
const slugErr = new Error(
|
||||
`Codex slug "${slug}" rejected (${firstAttempt.status})`,
|
||||
);
|
||||
Object.assign(slugErr, {
|
||||
httpStatus: firstAttempt.status,
|
||||
upstream: "codex",
|
||||
model: slug,
|
||||
bodyExcerpt,
|
||||
});
|
||||
lastSlugRejectionError = slugErr;
|
||||
continue;
|
||||
}
|
||||
|
||||
// Non-slug error (5xx, 429, invalid_prompt, etc.) — propagate
|
||||
// immediately. Walking the chain wouldn't help.
|
||||
const err = new Error(`Codex request failed (${firstAttempt.status})`);
|
||||
Object.assign(err, {
|
||||
httpStatus: firstAttempt.status,
|
||||
upstream: "codex",
|
||||
model: slug,
|
||||
bodyExcerpt,
|
||||
});
|
||||
this.lastDiagnostics = {
|
||||
attempted,
|
||||
cacheState,
|
||||
workingSlug: null,
|
||||
};
|
||||
throw err;
|
||||
}
|
||||
|
||||
if (!firstAttempt.ok) {
|
||||
throw await this.buildUpstreamError(firstAttempt, "Codex request failed");
|
||||
}
|
||||
|
||||
return this.parseResponse(firstAttempt);
|
||||
// All slugs exhausted — every one rejected as not-supported.
|
||||
this.lastDiagnostics = {
|
||||
attempted,
|
||||
cacheState,
|
||||
workingSlug: null,
|
||||
};
|
||||
const message =
|
||||
"AI provider unreachable — all configured Codex slugs were rejected";
|
||||
const err = new Error(message);
|
||||
Object.assign(err, {
|
||||
httpStatus: 503,
|
||||
upstream: "codex",
|
||||
model: attempted[attempted.length - 1] ?? null,
|
||||
bodyExcerpt: lastSlugRejectionError
|
||||
? (lastSlugRejectionError as Error & { bodyExcerpt?: string }).bodyExcerpt
|
||||
: null,
|
||||
attempted,
|
||||
});
|
||||
throw err;
|
||||
}
|
||||
|
||||
private async buildUpstreamError(
|
||||
res: Response,
|
||||
message: string,
|
||||
slug: string,
|
||||
): Promise<Error> {
|
||||
const rawBody = await res.text().catch(() => "");
|
||||
const bodyExcerpt = rawBody
|
||||
.slice(0, 500)
|
||||
.replace(/sk-(?:ant-)?[A-Za-z0-9_-]{8,}/g, "sk-***redacted***")
|
||||
.replace(/Bearer\s+[A-Za-z0-9_.-]+/gi, "Bearer ***redacted***");
|
||||
const bodyExcerpt = redactBody(rawBody);
|
||||
const err = new Error(`${message} (${res.status})`);
|
||||
Object.assign(err, {
|
||||
httpStatus: res.status,
|
||||
upstream: "codex",
|
||||
model: CODEX_MODEL,
|
||||
model: slug,
|
||||
bodyExcerpt,
|
||||
});
|
||||
return err;
|
||||
}
|
||||
|
||||
private async doRequest(params: CompletionParams): Promise<Response> {
|
||||
private async doRequest(
|
||||
params: CompletionParams,
|
||||
slug: string,
|
||||
): Promise<Response> {
|
||||
const sessionId = randomUUID();
|
||||
const threadId = randomUUID();
|
||||
return fetch(CODEX_ENDPOINT, {
|
||||
@@ -133,7 +297,7 @@ export class CodexClient implements AIProvider {
|
||||
"thread-id": threadId,
|
||||
},
|
||||
body: JSON.stringify({
|
||||
model: CODEX_MODEL,
|
||||
model: slug,
|
||||
instructions: params.systemPrompt,
|
||||
input: [
|
||||
{
|
||||
@@ -174,7 +338,10 @@ export class CodexClient implements AIProvider {
|
||||
* NOT folded into the visible text — they belong to a separate
|
||||
* channel.
|
||||
*/
|
||||
private async parseResponse(res: Response): Promise<CompletionResult> {
|
||||
private async parseResponse(
|
||||
res: Response,
|
||||
requestedSlug: string,
|
||||
): Promise<CompletionResult> {
|
||||
if (!res.body) {
|
||||
throw new Error("Codex returned no response body");
|
||||
}
|
||||
@@ -259,7 +426,7 @@ export class CodexClient implements AIProvider {
|
||||
Object.assign(e, {
|
||||
upstream: "codex",
|
||||
errorCode: err?.code ?? null,
|
||||
model: serverModel ?? CODEX_MODEL,
|
||||
model: serverModel ?? requestedSlug,
|
||||
});
|
||||
throw e;
|
||||
}
|
||||
@@ -294,8 +461,23 @@ export class CodexClient implements AIProvider {
|
||||
return {
|
||||
content,
|
||||
tokensUsed,
|
||||
model: serverModel ?? CODEX_MODEL,
|
||||
model: serverModel ?? requestedSlug,
|
||||
providerType: "codex",
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
function redactBody(rawBody: string): string {
|
||||
return rawBody
|
||||
.slice(0, 500)
|
||||
.replace(/sk-(?:ant-)?[A-Za-z0-9_-]{8,}/g, "sk-***redacted***")
|
||||
.replace(/Bearer\s+[A-Za-z0-9_.-]+/gi, "Bearer ***redacted***");
|
||||
}
|
||||
|
||||
// Test-only re-exports — keeps the spec-driven defaults visible to unit
|
||||
// tests without exporting a settable mutable.
|
||||
export const __test = {
|
||||
DEFAULT_SLUG_FALLBACK_CHAIN,
|
||||
isSlugRejection,
|
||||
loadFallbackChain,
|
||||
};
|
||||
@@ -0,0 +1,74 @@
|
||||
/**
|
||||
* Process-local positive cache for working Codex model slugs.
|
||||
*
|
||||
* Phase C1 (v1.4.15) — slug-drift defence per
|
||||
* `docs/codex-protocol-spec.md` §7b. The ChatGPT-Codex backend rotates
|
||||
* its allow-list silently; pinning a single `CODEX_MODEL` slug bricks
|
||||
* the integration the moment upstream flips it. Instead of pinning,
|
||||
* `CodexClient` walks an ordered fallback chain on every fresh
|
||||
* request series and caches the first slug that returns 200 +
|
||||
* `response.completed` for `CACHE_TTL_MS` so subsequent requests skip
|
||||
* the chain walk.
|
||||
*
|
||||
* Lives in its own module so:
|
||||
* - Tests can `clearCodexSlugCache()` between cases.
|
||||
* - The shared Map state stays out of `CodexClient` itself (so
|
||||
* instance creation in `resolveProvider` doesn't reset the cache).
|
||||
* - Future telemetry (cache hit-rate emission) hooks here cleanly.
|
||||
*/
|
||||
|
||||
const CACHE_TTL_MS = 60 * 60 * 1000; // 1 hour per spec §7b
|
||||
|
||||
interface CachedEntry {
|
||||
slug: string;
|
||||
cachedAtMs: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Cache key — currently a single global slot ("codex"). Kept in a Map
|
||||
* so future per-account / per-model-family scoping is a one-line
|
||||
* change.
|
||||
*/
|
||||
const CACHE_KEY = "codex";
|
||||
|
||||
const cache = new Map<string, CachedEntry>();
|
||||
|
||||
export function getCachedCodexSlug(now: number = Date.now()): string | null {
|
||||
const entry = cache.get(CACHE_KEY);
|
||||
if (!entry) return null;
|
||||
if (now - entry.cachedAtMs > CACHE_TTL_MS) {
|
||||
cache.delete(CACHE_KEY);
|
||||
return null;
|
||||
}
|
||||
return entry.slug;
|
||||
}
|
||||
|
||||
export function setCachedCodexSlug(
|
||||
slug: string,
|
||||
now: number = Date.now(),
|
||||
): void {
|
||||
cache.set(CACHE_KEY, { slug, cachedAtMs: now });
|
||||
}
|
||||
|
||||
export function invalidateCachedCodexSlug(): void {
|
||||
cache.delete(CACHE_KEY);
|
||||
}
|
||||
|
||||
/** Test-only: reset the cache between cases. */
|
||||
export function clearCodexSlugCache(): void {
|
||||
cache.clear();
|
||||
}
|
||||
|
||||
/**
|
||||
* Diagnostic helper — exposes (slug, ageMs) for Wide-Event annotation.
|
||||
* Returns null when nothing is cached.
|
||||
*/
|
||||
export function inspectCodexSlugCache(
|
||||
now: number = Date.now(),
|
||||
): { slug: string; ageMs: number } | null {
|
||||
const entry = cache.get(CACHE_KEY);
|
||||
if (!entry) return null;
|
||||
return { slug: entry.slug, ageMs: now - entry.cachedAtMs };
|
||||
}
|
||||
|
||||
export const CODEX_SLUG_CACHE_TTL_MS = CACHE_TTL_MS;
|
||||
Reference in new issue
Block a user