fix(ai): complete Codex backend protocol implementation per spec

Stop iterating on individual 400 errors and implement the full Codex
Responses-API protocol once, from the source. Wrote a 713-line
reference at docs/codex-protocol-spec.md from the official
openai/codex Rust client; this commit is the TS implementation
against that spec.

Required pieces that were missing in v1.4.11:

- `ChatGPT-Account-ID` header — extracted from the `chatgpt_account_id`
  claim of the OAuth id_token (also accepts the URI-namespaced
  variant under `https://api.openai.com/auth`). Without it: 401.
- `originator`, `User-Agent`, `Accept: text/event-stream`,
  `session_id`/`session-id` (both spellings), `thread_id`/`thread-id`.
- Body fields `reasoning: null` and `include: []` are required even
  when no reasoning is requested.
- Model slug is `gpt-5-codex`, not `gpt-5.3-codex` (the latter is a
  test placeholder upstream).
- Refresh endpoint takes a JSON body, not form-urlencoded.

Storage codec rewrite: `codexAccessTokenEncrypted` now holds an
encrypted JSON blob `{ accessToken, accountId, expiresAt }` instead
of just the raw access token, so the account id stays available
across requests. v1.4.7-v1.4.11 records that lack the field cannot
be revived — `decryptCodexCreds` returns null for those rows and
`resolveCodexProvider` flips the connection to `expired`, prompting
the user to re-link once.

Removed the dead browser-flow routes (`/api/auth/codex/authorize`
and `/callback`). They never worked on hosted domains because
Hydra's redirect-URI allow-list for the public Codex client ID
covers only localhost; device-code is the only viable path.

CodexClient SSE parsing extended to handle `response.failed` and
`response.incomplete` terminal events, and to read the actual
routed model from the `OpenAI-Model` response header.

Tests rebuilt: codex-oauth covers the full device-code flow with a
synthesised id_token JWT, the JSON-body refresh, and the
storage-codec round-trip. CodexClient tests carry an `accountId`
through the constructor. All 720 tests pass; typecheck clean.

Refs: docs/codex-protocol-spec.md §1-§6 for every claim.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Marc-André BombeckandClaude Opus 4.7 committed 2026-05-09 14:36:12 +02:00
1 parent e0de659fdd
commit e712f5fe2f
12 files changed
+1311 -680

No files matched your search

+30
View File
@@ -1,5 +1,35 @@
# Changelog
## [1.4.12] — 2026-05-09
### Fixed — KI Insights via ChatGPT (komplette Codex-Integration)
- **Codex-Backend-Aufruf jetzt vollständig nach Spec.** Die letzten
Versionen haben den Connect-Schritt richtig hinbekommen, aber der
eigentliche Insight-Call ist iterativ an immer neuen 400ern
gestorben. Statt weitere Trial-and-Error-Iterationen zu fahren
habe ich das vollständige Codex-Backend-Protokoll aus dem
offiziellen `openai/codex`-Quellcode dokumentiert
(`docs/codex-protocol-spec.md`) und einmal sauber dagegen
implementiert. Was alles fehlte: der Header `ChatGPT-Account-ID`
(ohne den 401, kommt aus dem `chatgpt_account_id`-Claim im JWT
id_token); die Header `originator`, `User-Agent`, `Accept`,
`session_id`, `thread_id`; die Body-Felder `reasoning: null` und
`include: []`; das richtige Modell-Slug `gpt-5-codex` (statt des
Test-Placeholders `gpt-5.3-codex`); JSON-Body beim Refresh
(vorher form-urlencoded); Persistenz des `accountId` neben dem
Access-Token. SSE-Streaming wird unverändert konsumiert
(output_text.delta + output_item.done).
- **Aufräumen:** der Browser-Authorization-Code-Pfad
(`/api/auth/codex/authorize` + `/callback`) ist gelöscht — er
funktioniert auf hosted Domains grundsätzlich nicht (Hydra-
Whitelist nur für localhost), war aber als Safety-Net noch da.
Device-Code ist jetzt der einzige Pfad.
- **Re-Connect nötig:** wer in v1.4.7-v1.4.11 schon connected hat,
muss einmal "Trennen" + "Mit ChatGPT verbinden" — die alte
Storage-Form trug die `accountId` nicht und kann nicht
nach-bereichert werden.
## [1.4.11] — 2026-05-09
### Fixed — KI Insights via ChatGPT
+719
View File
@@ -0,0 +1,719 @@
# Codex Backend Protocol Spec — `chatgpt.com/backend-api/codex/responses`
This is a TypeScript-implementer's reference for the Codex CLI's "Responses-over-HTTP" call to the ChatGPT-backed Codex endpoint. Every claim cites a specific Rust source file in `openai/codex` (path is relative to the repo root).
The endpoint is reverse-engineered from the official client, so this document is the authoritative shape — there is no public OpenAPI for this URL.
---
## 0. Endpoint
- **Method**: `POST`
- **URL**: `https://chatgpt.com/backend-api/codex/responses`
- **Content-Type**: `application/json`
- **Accept**: `text/event-stream` (the client always sets this)
- **Stream**: SSE; non-streaming is not supported on this URL (server returns 400 `"Stream must be set to true"`).
The base URL `https://chatgpt.com/backend-api/codex` is the constant `CHATGPT_CODEX_BASE_URL` in `codex-rs/model-provider-info/src/lib.rs:37`. The path `/responses` is joined onto it by `ResponsesClient::path()` in `codex-rs/codex-api/src/endpoint/responses.rs:102-104` (returns `"responses"`) and the provider's `url_for_path` joins them with a `/`.
The ACCEPT header is forced to `text/event-stream` in `codex-rs/codex-api/src/endpoint/responses.rs:137-140`.
---
## 1. Request body — `ResponsesApiRequest`
Source: `codex-rs/codex-api/src/common.rs:169-190`.
```ts
interface ResponsesApiRequest {
// Required
model: string; // e.g. "gpt-5-codex"
input: ResponseItem[]; // MUST be a list, not a string
tools: unknown[]; // [] is valid
tool_choice: string; // "auto" is what core uses
parallel_tool_calls: boolean; // bool, no default — must serialize
reasoning: Reasoning | null; // null is allowed; field is required
store: boolean; // false on chatgpt backend (only true on Azure)
stream: true; // MUST be true on this URL
include: string[]; // ["reasoning.encrypted_content"] when reasoning is on, else []
// Optional (skipped when empty/None)
instructions?: string; // skipped if "" (serde "skip_serializing_if = String::is_empty")
service_tier?: string; // omit unless model supports it
prompt_cache_key?: string; // core sets this to thread_id
text?: TextControls; // verbosity + JSON-schema output
client_metadata?: Record<string, string>; // free-form, see below
}
```
Serde rules from `codex-rs/codex-api/src/common.rs:169-190`:
| Field | Skip-rule |
| ------------------ | ----------------------------------------------------------------- |
| `instructions` | skipped when empty string |
| `reasoning` | always emitted (Option, but no skip attribute → `null` when None) |
| `service_tier` | skipped when `None` |
| `prompt_cache_key` | skipped when `None` |
| `text` | skipped when `None` |
| `client_metadata` | skipped when `None` |
All other fields are always emitted.
There are NO serde renames — wire keys equal Rust field names verbatim.
### 1a. The `Reasoning` block
Source: `codex-rs/codex-api/src/common.rs:113-119`.
```ts
interface Reasoning {
effort?: "none" | "minimal" | "low" | "medium" | "high" | "xhigh";
summary?: "auto" | "concise" | "detailed" | "none";
}
```
- `effort` enum: `codex-rs/protocol/src/openai_models.rs:42-52` (lowercase serde). `Medium` is the default if you ask for one.
- `summary` enum: `codex-rs/protocol/src/config_types.rs:24-36` (lowercase serde). The Codex CLI sets `summary` to `Some(...)` only when the user asked for summaries; `ReasoningSummary::None` is mapped to `None` (omitted) in `codex-rs/core/src/client.rs:697-701`.
- When the model does not support reasoning summaries, the entire `reasoning` field is set to `None` (i.e. emitted as `null`): `codex-rs/core/src/client.rs:689-705`.
- When `reasoning` is non-null, the client also sets `include: ["reasoning.encrypted_content"]`. When null, `include: []`. See `codex-rs/core/src/client.rs:721-725`.
### 1b. `TextControls`
Source: `codex-rs/codex-api/src/common.rs:142-148`.
```ts
interface TextControls {
verbosity?: "low" | "medium" | "high"; // codex-rs/codex-api/src/common.rs:150-157
format?: TextFormat;
}
interface TextFormat {
type: "json_schema"; // only variant; codex-rs/codex-api/src/common.rs:121-126
strict: boolean;
schema: unknown; // raw JSON-Schema
name: string; // "codex_output_schema" in CLI; arbitrary
}
```
`text` is omitted entirely when no verbosity AND no output schema is requested (`codex-rs/codex-api/src/common.rs:279-297`).
### 1c. Minimal valid body
For a plain text turn with no tools and no reasoning, the smallest body the Codex client itself ever sends is:
```json
{
"model": "gpt-5-codex",
"instructions": "You are a helpful coding agent.",
"input": [
{
"type": "message",
"role": "user",
"content": [{ "type": "input_text", "text": "Hello" }]
}
],
"tools": [],
"tool_choice": "auto",
"parallel_tool_calls": false,
"reasoning": null,
"store": false,
"stream": true,
"include": []
}
```
`tools: []` is acceptable — the field is required but may be the empty array. `tool_choice: "auto"` is the literal string the CLI always sends (`codex-rs/core/src/client.rs:750`).
### 1d. `client_metadata`
Free-form `Record<string,string>`. The CLI uses it to forward an installation id and (over WebSocket) trace-context headers — none of these are mandatory for the HTTP path. Safe to omit (`None` → field skipped).
---
## 2. `ResponseItem` (the `input` array)
Source: `codex-rs/protocol/src/models.rs:741-891`.
```rust
#[serde(tag = "type", rename_all = "snake_case")]
pub enum ResponseItem { Message {...}, Reasoning {...}, ... }
```
So every entry in `input` is an object with a `type` discriminator (snake_case) plus variant-specific fields.
The variants you'll need for outbound traffic are `Message`, `Reasoning`, `FunctionCall`, `FunctionCallOutput`. The server-only variants (`LocalShellCall`, `WebSearchCall`, `ImageGenerationCall`, `ToolSearchCall`, `ToolSearchOutput`, `CustomToolCall*`, `Compaction`, `ContextCompaction`) appear in stream output and only need to round-trip for multi-turn continuations.
### 2a. `Message`
`codex-rs/protocol/src/models.rs:744-756`.
```ts
interface MessageItem {
type: "message";
role: "user" | "assistant" | "system" | "developer"; // free string
content: ContentItem[];
// The `id` field exists in Rust but is `skip_serializing` — never emit it for inbound input
// The `phase` field is for SERVER output; do not send it.
}
```
Note: `id` has `#[serde(skip_serializing)]` (line 745-747) — set client-side but never put into the request.
### 2b. `ContentItem`
`codex-rs/protocol/src/models.rs:697-712`.
```ts
type ContentItem =
| { type: "input_text"; text: string }
| {
type: "input_image";
image_url: string;
detail?: "auto" | "low" | "high" | "original";
}
| { type: "output_text"; text: string };
```
- Use `input_text` and `input_image` for user turns.
- Use `output_text` for assistant-message replays in multi-turn continuations.
- `image_url` accepts `data:image/png;base64,...` (see `codex-rs/protocol/src/models.rs:2369`) or an HTTPS URL.
- `detail` enum: `codex-rs/protocol/src/models.rs:715-723` (default `High` when serializing for the CLI).
### 2c. `Reasoning`
`codex-rs/protocol/src/models.rs:757-767`.
```ts
interface ReasoningItem {
type: "reasoning";
// `id` is skip_serializing — do not send
summary: ReasoningSummary[]; // Vec<ReasoningItemReasoningSummary>
content?: ReasoningContent[]; // omitted unless server gave you one
encrypted_content: string | null;
}
type ReasoningSummary = { type: "summary_text"; text: string };
type ReasoningContent = { type: "reasoning_text" | "text"; text: string };
```
Sources: `codex-rs/protocol/src/models.rs:1192-1203`.
For the simplest "fresh chat" use, you don't construct `Reasoning` items yourself — they only re-appear in `input` when continuing a turn that the server previously emitted them on. When you do replay them, keep `encrypted_content` verbatim.
### 2d. `FunctionCall` (outbound continuation)
`codex-rs/protocol/src/models.rs:778-791`.
```ts
interface FunctionCallItem {
type: "function_call";
// `id` skip_serializing
name: string;
namespace?: string | null; // omitted when None
arguments: string; // JSON-as-string, NOT a parsed object
call_id: string; // required
}
```
Important: `arguments` is a string containing JSON, not an object — the Responses API mandates this on the wire. See the comment at lines 786-789.
### 2e. `FunctionCallOutput`
`codex-rs/protocol/src/models.rs:809-814`.
```ts
interface FunctionCallOutputItem {
type: "function_call_output";
call_id: string;
output: string | { content?: string; content_items?: ContentItem[] };
}
```
The output payload uses `FunctionCallOutputPayload`, which is a string OR a structured `{ content | content_items }` object — see lines 804-808 for the wire-format note.
### 2f. Other variants you might see in stream output
| `type` | Notes |
| ---------------------------------------------------------------- | -------------------------------------- |
| `local_shell_call` | local-shell tool action (server emits) |
| `web_search_call` | server emits when web_search ran |
| `image_generation_call` | server emits |
| `tool_search_call` / `tool_search_output` | dynamic-tool registry |
| `custom_tool_call` / `custom_tool_call_output` | user-defined tools |
| `compaction` (alias `compaction_summary`) / `context_compaction` | turn compaction artifacts |
If you don't intend to round-trip these, keep them out of `input`. The unknown-variant catch-all (`Other`) is `#[serde(other)]` so unrecognized types are deserialized but lossy — don't echo them.
---
## 3. HTTP headers
The Codex CLI sends these on every Responses POST. Required ones are starred.
### 3a. From `BearerAuthProvider::add_auth_headers` — `codex-rs/model-provider/src/bearer_auth_provider.rs:31-46`
| Header | Required | Value |
| -------------------- | --------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
| `Authorization` | \* | `Bearer <access_token>` (the OAuth access token, see §6) |
| `ChatGPT-Account-ID` | \* (for ChatGPT auth) | the `chatgpt_account_id` claim from the id_token; obtained via `auth.get_account_id()` (`codex-rs/login/src/auth/manager.rs:339-344`) |
| `X-OpenAI-Fedramp` | optional | `"true"` only for FedRAMP accounts |
Without `ChatGPT-Account-ID`, `chatgpt_get_request` errors out with "ChatGPT account ID not available, please re-run `codex login`" (`codex-rs/chatgpt/src/chatgpt_client.rs:33-36`).
### 3b. From the default reqwest client — `codex-rs/login/src/auth/default_client.rs:232-248`
| Header | Required | Value |
| ----------------------------------- | -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `originator` | \* | default `"codex_cli_rs"` (constant `DEFAULT_ORIGINATOR` at line 36); identifies the client family |
| `User-Agent` | \* | format: `"<originator>/<version> (<os_type> <os_version>; <arch>) <terminal_user_agent>"` — see `get_codex_user_agent()` lines 133-157. Anything sane works (e.g. `"healthlog/1.0 (Node)"` is fine for a third-party client). |
| `x-openai-internal-codex-residency` | optional | `"us"` only when residency requirement is set |
### 3c. From `build_session_headers` — `codex-rs/codex-api/src/requests/headers.rs:5-16`
Sent on every Responses request when the CLI has a session/thread:
| Header | Value |
| ----------------------------- | -------------------------------------------- |
| `session_id` and `session-id` | UUID-shaped session id (both spellings sent) |
| `thread_id` and `thread-id` | UUID-shaped thread id (both spellings sent) |
Optional but cheap; the server tolerates them as routing hints. Generate one UUID and use it for both names — duplicates are intentional, see lines 7-13.
### 3d. From `endpoint/responses.rs:91-97`
| Header | Required | Value |
| --------------------- | -------- | ----------------------------------------------------------------------------------------------------------- |
| `x-client-request-id` | optional | mirror the `thread_id` so it appears as the request-id in OpenAI logs |
| `x-openai-subagent` | optional | only set for sub-agent flows (review/compact/memory_consolidation/collab_spawn). Omit for first-party chat. |
### 3e. From `core/src/client.rs:135-148, 1648-1670`
These are Codex-specific extras the CLI adds; the server tolerates them all.
| Header | Value |
| ------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `OpenAI-Beta` | only set on the **WebSocket** transport: `"responses_websockets=2026-02-06"` (line 145). NOT sent on HTTP — see line 911 (only the WS handshake adds it). |
| `x-codex-beta-features` | optional, comma-separated feature flags |
| `x-codex-installation-id` | UUID-ish, identifies the install — also embedded in `client_metadata` (line 759-762) |
| `x-codex-turn-state` | sticky-routing token captured from a prior response's `x-codex-turn-state` response header — replay it on subsequent requests in the same "turn" |
| `x-codex-turn-metadata` | optional, observability-only |
| `x-codex-window-id` / `x-codex-parent-thread-id` | sub-agent only |
| `x-openai-memgen-request` | `"true"` for memory-consolidation sub-agent only |
| `x-responsesapi-include-timing-metrics` | `"true"` if the client wants timing info in the response |
### 3f. Minimum third-party header set
For HealthLog's purposes (no sub-agent, no session continuity, no Azure, no sticky routing):
```
POST /backend-api/codex/responses HTTP/1.1
Host: chatgpt.com
Authorization: Bearer <oauth_access_token>
ChatGPT-Account-ID: <account_id>
Content-Type: application/json
Accept: text/event-stream
originator: <something_distinctive>
User-Agent: <descriptive_string>
session_id: <uuid>
thread_id: <uuid>
```
That's enough for the call to succeed.
### 3g. What the CLI does NOT send on HTTP
- `OpenAI-Beta` header on plain HTTP — only on WebSocket. Don't send it, it's not needed.
- `OpenAI-Organization` / `OpenAI-Project` — those are for `api.openai.com` only (`codex-rs/model-provider-info/src/lib.rs:333-343`), NOT for the ChatGPT backend.
---
## 4. SSE response
### 4a. Wire format
Standard SSE: `eventsource_stream::Eventsource` is used (`codex-rs/codex-api/src/sse/responses.rs:439`), so the format is:
```
event: <type>\n
data: <single-line JSON>\n
\n
```
Each event is one frame, separated by a blank line. The implementation tolerates `event:`-only lines without `data:` (used for keep-alives in tests, `codex-rs/codex-api/src/sse/responses.rs:638-642`). There is **no** `data: [DONE]` terminator — the stream ends with a `response.completed` event (or `response.failed`/`response.incomplete`); after that the connection is closed.
If the connection closes without a terminal event, the client surfaces `"stream closed before response.completed"` (line 458).
The data payload is a JSON object with at minimum `{ "type": "<type>", ...}`. The Rust deserializer is forgiving — unknown event kinds are logged at trace level and ignored (`codex-rs/codex-api/src/sse/responses.rs:425-427`).
### 4b. Event types — the complete list
Source: `codex-rs/codex-api/src/sse/responses.rs:300-428`.
The `data:` payload follows the `ResponsesStreamEvent` shape (lines 179-192):
```ts
interface ResponsesStreamEvent {
type: string;
headers?: unknown; // map; only present on metadata frames
metadata?: unknown; // present on response.metadata
response?: unknown; // present on created/completed/failed/incomplete
item?: unknown; // present on output_item.* — a ResponseItem
item_id?: string;
call_id?: string;
delta?: string; // text deltas
summary_index?: number;
content_index?: number;
// ... other fields are tolerated and ignored
}
```
| Event `type` | Carries | Use it for |
| --------------------------------------- | ------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------- |
| `response.created` | `response: {}` (server may add `id`, `headers.OpenAI-Model`, etc.) | Signals the turn started. The CLI emits `ResponseEvent::Created`. |
| `response.metadata` | `metadata.openai_verification_recommendation: string[]` | Optional; CLI uses to surface trust prompts. Ignore for chat. |
| `response.output_item.added` | `item: ResponseItem` | A new top-level item is starting (e.g. assistant message stub, function call). |
| `response.output_item.done` | `item: ResponseItem` | The item is complete — this is where the **final** assistant `Message` arrives with its full `content[].text`. |
| `response.output_text.delta` | `delta: string` | Token-by-token text streaming for the assistant message currently in flight. Concatenate to assemble live text. |
| `response.reasoning_summary_text.delta` | `delta: string`, `summary_index: number` | Reasoning summary stream (when `reasoning.summary` requested). |
| `response.reasoning_summary_part.added` | `summary_index: number` | Marker that a new summary section started. |
| `response.reasoning_text.delta` | `delta: string`, `content_index: number` | Reasoning text stream. |
| `response.custom_tool_call_input.delta` | `item_id: string`, `call_id?: string`, `delta: string` | Custom tool input streaming (also used for `response.function_call_arguments.delta` per line 806-813 of the test fixture). |
| `response.completed` | `response: { id, usage?, end_turn? }` | **Terminal event** — emit your final assembled text + usage. |
| `response.failed` | `response: { error: { code, message, ... } }` | **Terminal event** — error path; see §4c. |
| `response.incomplete` | `response: { incomplete_details: { reason: string } }` | **Terminal event** — partial result, `reason` may be `max_output_tokens` etc. |
Anything else (`response.in_progress`, `response.content_part.added`, etc.) is currently ignored by the official client (line 425-427); a TS implementation should follow suit and silently drop unknown `type` values.
### 4c. Error event — `response.failed`
Schema of `response.error` (`codex-rs/codex-api/src/sse/responses.rs:122-130`):
```ts
interface FailedError {
type?: string;
code?: string; // primary discriminator
message?: string;
plan_type?: string;
resets_at?: number; // unix seconds
}
```
Code-to-meaning mapping (`codex-rs/codex-api/src/sse/responses.rs:547-570`):
| `error.code` | Meaning | Treat as |
| ------------------------------------ | -------------------------------- | ------------------------------------------------------------------------------------------------------------- |
| `context_length_exceeded` | input too big | fatal, surface to user |
| `insufficient_quota` | account quota | fatal |
| `usage_not_included` | feature not in plan | fatal |
| `invalid_prompt` | malformed/safety-rejected prompt | fatal, surface message |
| `cyber_policy` | flagged as security risk | fatal, message + fallback "This request has been flagged for possible cybersecurity risk." (line 142-143) |
| `server_is_overloaded` / `slow_down` | backpressure | retry with backoff |
| `rate_limit_exceeded` | TPM/RPM limit | retry; parse delay from message via regex `(?i)try again in\s*(\d+(?:\.\d+)?)\s*(s\|ms\|seconds?)` (line 586) |
| anything else | unknown server error | retry |
### 4d. Token usage — `response.completed`
Source: `codex-rs/codex-api/src/sse/responses.rs:132-167, 392-407`.
```ts
interface ResponseCompletedData {
id: string;
end_turn?: boolean; // some providers don't emit; treat None as "unknown"
usage?: {
input_tokens: number;
input_tokens_details?: { cached_tokens: number };
output_tokens: number;
output_tokens_details?: { reasoning_tokens: number };
total_tokens: number;
};
}
```
Path: `event.data.response.usage.{input_tokens, output_tokens, total_tokens}`. Cached and reasoning-token sub-totals live one level deeper in `*_details`.
### 4e. Assembling the final text
Two valid strategies:
1. **Live streaming**: concatenate every `response.output_text.delta`'s `delta` field. Stop at `response.completed`. (Reasoning deltas come on a separate channel — `response.reasoning_*.delta` — and should NOT be concatenated into the visible answer.)
2. **End-of-turn snapshot**: ignore deltas, take the `response.output_item.done` whose `item.type == "message"` and `item.role == "assistant"`. Its `item.content[]` will contain `output_text` entries with the full text. This is more robust if you only care about the final answer.
The CLI uses both (deltas drive the live UI; `done` gives the canonical `ResponseItem` for the conversation log).
### 4f. Response headers worth reading
These are on the HTTP response (not in the SSE body). Source: `codex-rs/codex-api/src/sse/responses.rs:69-95`.
| Header | Use |
| ---------------------- | ------------------------------------------------------------------------------------------------- |
| `x-request-id` | Pass to the user when reporting errors |
| `OpenAI-Model` | The actual model the server routed to (may differ from your request when safety routing kicks in) |
| `x-reasoning-included` | If present, server already accounted for prior reasoning tokens — don't re-estimate |
| `x-codex-turn-state` | Sticky-routing token — replay in next request's `x-codex-turn-state` header |
| `X-Models-Etag` | Cache key for the `/models` endpoint |
| `cf-ray` | Cloudflare ray, useful for support |
### 4g. Idle timeout
The CLI uses an idle timeout per stream (default in `codex-rs/model-provider-info/src/lib.rs` constants); if no SSE event arrives within the window, it emits `"idle timeout waiting for SSE"` and closes. A reasonable TS default is **120 seconds**.
---
## 5. HTTP error responses (non-SSE)
Source: `codex-rs/codex-api/src/api_bridge.rs:36-130`.
If the server fails before the SSE stream begins, it returns a JSON body with shape:
```json
{ "error": { "type": "...", "code": "...", "message": "..." } }
```
| Status | Special handling |
| -------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------- |
| `400` with `error.code == "cyber_policy"` | treat as `CyberPolicy` (message or fallback) |
| `400` body containing `"The image data you provided does not represent a valid image"` | invalid image |
| `400` other | `InvalidRequest` — e.g. our two seen failures (`"Input must be a list"`, `"Stream must be set to true"`) |
| `429` with `error.type == "usage_limit_reached"` | parse `plan_type`, `resets_at`, plus rate-limit headers (§5a) |
| `429` with `error.type == "usage_not_included"` | feature/plan rejection |
| `429` other | retryable rate-limit |
| `500` | retry |
| `503` with `error.code` in `{"server_is_overloaded","slow_down"}` | retry with backoff |
| anything else | propagate verbatim |
### 5a. Rate-limit headers
Source: `codex-rs/codex-api/src/rate_limits.rs` (lines 22-345). The Codex backend uses bespoke headers (NOT standard `X-RateLimit-*`). The keys you'll see:
```
x-codex-active-limit
x-codex-primary-used-percent
x-codex-primary-window-minutes
x-codex-primary-reset-at
x-codex-secondary-primary-used-percent
x-codex-secondary-primary-window-minutes
x-codex-secondary-primary-reset-at
x-codex-bengalfox-primary-used-percent
x-codex-bengalfox-limit-name
x-codex-credits-has-credits
x-codex-credits-unlimited
x-codex-credits-balance
x-codex-promo-message
```
Surface them to the user when relevant. Never block on them — they're informational.
### 5b. Other error-relevant headers
| Header | Source line | Use |
| ----------------------------------- | ----------------------- | ---------------------------------------------------------------------- |
| `x-request-id` / `x-oai-request-id` | `api_bridge.rs:136-137` | request id for support |
| `cf-ray` | `api_bridge.rs:138` | Cloudflare ray |
| `x-openai-authorization-error` | `api_bridge.rs:139` | identity-error sub-code |
| `x-error-json` | `api_bridge.rs:140` | base64-JSON-encoded error envelope; decode with standard b64 then JSON |
---
## 6. Authentication
### 6a. What goes in `Authorization`
The OAuth **access token** issued by `auth.openai.com`. For ChatGPT-mode auth, `CodexAuth::get_token()` returns `tokens.access_token` directly — no swap, no extra exchange (`codex-rs/login/src/auth/manager.rs:325-336`). That's what `BearerAuthProvider` puts in `Bearer <token>`.
The token format is a JWT. The Rust client extracts the expiry from the JWT itself (`parse_jwt_expiration` in `manager.rs:1797`), so there is no `expires_in` to track separately.
### 6b. Where the token comes from (device-code flow)
Source: `codex-rs/login/src/server.rs:711-782` and `codex-rs/login/src/device_code_auth.rs`.
1. Issuer: `https://auth.openai.com` (constant `DEFAULT_ISSUER`, `server.rs:51`).
2. The device-code flow ends with a PKCE code exchange at `POST {issuer}/oauth/token` with body:
```
grant_type=authorization_code
code=<authorization_code>
redirect_uri=<redirect>
client_id=app_EMoamEEZ73f0CkXaXp7hrann
code_verifier=<pkce_verifier>
```
Content-Type: `application/x-www-form-urlencoded`.
3. The response is JSON `{ id_token, access_token, refresh_token }` (`server.rs:719-723`). All three are required.
The `client_id` value `app_EMoamEEZ73f0CkXaXp7hrann` is in `codex-rs/login/src/auth/manager.rs:921`.
The OAuth scope is `openid profile email offline_access api.connectors.read api.connectors.invoke` (`codex-rs/login/src/server.rs:493-495`). Plus the non-standard query params `id_token_add_organizations=true`, `codex_cli_simplified_flow=true`, and `originator=<originator>` on the authorize URL (lines 502-506). The token endpoint itself only needs the form fields above.
### 6c. ChatGPT-Account-ID
Read from the `id_token` JWT — specifically the `chatgpt_account_id` claim. The CLI reads it at parse time and exposes it via `CodexAuth::get_account_id()` (`manager.rs:339-344`). For a TS client: decode the JWT (no signature verification needed for our purposes; the server validates), then look at `payload.chatgpt_account_id` (typically nested under `https://api.openai.com/auth.chatgpt_account_id` — see `manager.rs:891` for the namespace).
If you can't find the account id in the id_token, the server returns 401 on the responses POST.
### 6d. Refresh procedure
Source: `codex-rs/login/src/auth/manager.rs:806-848, 906-926`.
```
POST https://auth.openai.com/oauth/token
Content-Type: application/json ← note: refresh uses JSON, not form-encoded
{
"client_id": "app_EMoamEEZ73f0CkXaXp7hrann",
"grant_type": "refresh_token",
"refresh_token": "<refresh_token>"
}
```
Override the URL with env `CODEX_REFRESH_TOKEN_URL_OVERRIDE` if you need a non-default issuer.
Response shape (`manager.rs:913-918`):
```json
{
"id_token": "...", // optional
"access_token": "...", // optional but expected
"refresh_token": "..." // optional; rotated when present
}
```
Persist whichever fields the server returned. **The refresh token is rotated** — the next refresh must use the newly returned `refresh_token` if any; reusing the old one yields `refresh_token_reused` and 401 (lines 856-857).
Failure handling (HTTP 401 body, lines 851-878):
| `error.code` in body | Treat as |
| --------------------------- | ---------------------------------------- |
| `refresh_token_expired` | permanent — user must re-login |
| `refresh_token_reused` | permanent — concurrent refresh; re-login |
| `refresh_token_invalidated` | permanent — token revoked; re-login |
| anything else | log and re-login |
### 6e. When to refresh
The CLI considers a token "stale" when (`manager.rs:1796-1805`):
1. The JWT's `exp` is `<= now`, OR
2. `last_refresh < now - 8 days` (constant `TOKEN_REFRESH_INTERVAL = 8` at line 85; `last_refresh` is the wall-clock time of the previous successful refresh).
For a TS client, refresh proactively when the access token's `exp` claim is within ~5 minutes of expiry, and force-refresh on any `401` response from the responses endpoint.
### 6f. Revocation (optional)
`POST https://auth.openai.com/oauth/revoke` revokes a refresh token. Constant: `REVOKE_TOKEN_URL` at `manager.rs:95`. Override env: `CODEX_REVOKE_TOKEN_URL_OVERRIDE` (`manager.rs:97`).
---
## 7. Model strings
The CLI sends the model slug verbatim in the `model` field. There is no enum on the wire — anything the backend recognises works. Slugs the official client knows about (sample, not exhaustive):
| Slug | Source |
| ------------------ | ----------------------------------------------------------------------------------------------------------------------------- |
| `gpt-5-codex` | `codex-rs/tui/src/model_migration.rs`; `codex-rs/tui/src/bottom_pane/status_line_setup.rs` |
| `gpt-5-codex-mini` | `codex-rs/tui/src/model_migration.rs` |
| `gpt-5.3-codex` | used as a "cyber-restricted" placeholder in tests (`codex-rs/codex-api/src/sse/responses.rs:1374`) — confirms the slug exists |
| `gpt-5.1` | shows up in rate-limit error messages in tests (`codex-rs/codex-api/src/sse/responses.rs:875`) |
The ChatGPT backend ignores `model` strings the user's account doesn't have entitlements for — the server may safety-route to a different model and report the actual model used in the `OpenAI-Model` response header (§4f). Trust that header, not the request value, when logging.
---
## 8. Worked example — minimal end-to-end
Outbound:
```http
POST /backend-api/codex/responses HTTP/1.1
Host: chatgpt.com
Authorization: Bearer eyJhbGciOi...<oauth-access-token>
ChatGPT-Account-ID: a1b2c3d4-...
Content-Type: application/json
Accept: text/event-stream
originator: healthlog
User-Agent: healthlog/1.0
session_id: 11111111-1111-1111-1111-111111111111
thread_id: 22222222-2222-2222-2222-222222222222
```
```json
{
"model": "gpt-5-codex",
"instructions": "",
"input": [
{
"type": "message",
"role": "user",
"content": [{ "type": "input_text", "text": "Say hi in one word." }]
}
],
"tools": [],
"tool_choice": "auto",
"parallel_tool_calls": false,
"reasoning": null,
"store": false,
"stream": true,
"include": []
}
```
Inbound (illustrative — based on `codex-rs/codex-api/src/sse/responses.rs:666-730` test fixtures):
```
event: response.created
data: {"type":"response.created","response":{"id":"resp_abc"}}
event: response.output_item.added
data: {"type":"response.output_item.added","item":{"type":"message","role":"assistant","content":[]}}
event: response.output_text.delta
data: {"type":"response.output_text.delta","delta":"Hi"}
event: response.output_text.delta
data: {"type":"response.output_text.delta","delta":"!"}
event: response.output_item.done
data: {"type":"response.output_item.done","item":{"type":"message","role":"assistant","content":[{"type":"output_text","text":"Hi!"}]}}
event: response.completed
data: {"type":"response.completed","response":{"id":"resp_abc","usage":{"input_tokens":12,"output_tokens":2,"total_tokens":14}}}
```
A TS client should:
1. Stream-parse SSE.
2. Concatenate `delta` from each `response.output_text.delta` for live UI.
3. On `response.output_item.done` with `item.type=="message"` and `item.role=="assistant"`, take `item.content` as the canonical reply.
4. On `response.completed`, store `response.id` (for follow-up turns) and `response.usage` (for token accounting).
5. On `response.failed` / `response.incomplete`, surface error per §4c.
---
## 9. Quick reference — the original errors we hit
- **"Input must be a list"** → `input` was a string. Wrap as `[{ type: "message", role: "user", content: [{ type: "input_text", text: "..." }] }]`.
- **"Stream must be set to true"** → must set `stream: true`. The chatgpt.com endpoint has no non-streaming mode.
Both come back as HTTP 400 with `error.code == "invalid_prompt"` (no `cyber_policy` flag) → mapped to `ApiError::InvalidRequest` per `api_bridge.rs:75-77`.
---
## 10. Source-file index (for re-verification)
All paths relative to `openai/codex` repo:
- `codex-rs/codex-api/src/common.rs` — `ResponsesApiRequest`, `Reasoning`, `TextControls`, `TextFormat`
- `codex-rs/codex-api/src/endpoint/responses.rs` — request construction, headers, ACCEPT
- `codex-rs/codex-api/src/requests/headers.rs` — session/thread/sub-agent headers
- `codex-rs/codex-api/src/sse/responses.rs` — SSE parsing, all event types, error mapping
- `codex-rs/codex-api/src/api_bridge.rs` — HTTP error mapping
- `codex-rs/codex-api/src/rate_limits.rs` — rate-limit headers
- `codex-rs/codex-api/src/auth.rs` — `AuthProvider` trait
- `codex-rs/protocol/src/models.rs` — `ResponseItem`, `ContentItem`, `MessagePhase`, `ImageDetail`, etc.
- `codex-rs/protocol/src/openai_models.rs` — `ReasoningEffort` enum
- `codex-rs/protocol/src/config_types.rs` — `ReasoningSummary`, `Verbosity` enums
- `codex-rs/model-provider/src/bearer_auth_provider.rs` — `Authorization` + `ChatGPT-Account-ID` headers
- `codex-rs/model-provider/src/auth.rs` — auth-provider selection
- `codex-rs/model-provider-info/src/lib.rs` — `CHATGPT_CODEX_BASE_URL`, OpenAI provider creation
- `codex-rs/login/src/auth/manager.rs` — token storage, refresh, expiry, `CLIENT_ID`
- `codex-rs/login/src/server.rs` — code exchange (`exchange_code_for_tokens`), authorize URL (scope)
- `codex-rs/login/src/device_code_auth.rs` — device-code flow
- `codex-rs/login/src/auth/default_client.rs` — `originator`, `User-Agent`, residency
- `codex-rs/chatgpt/src/chatgpt_client.rs` — confirms `chatgpt_base_url + path` pattern and `ChatGPT-Account-ID` requirement
- `codex-rs/core/src/client.rs` — how the CLI builds requests (`build_responses_request`, `build_responses_headers`); the source of `tool_choice="auto"`, `store=false` for chatgpt, `stream=true`, `include=["reasoning.encrypted_content"]` when reasoning is on, `prompt_cache_key=thread_id`
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "healthlog",
"version": "1.4.11",
"version": "1.4.12",
"private": true,
"packageManager": "pnpm@10.31.0",
"scripts": {
-69
View File
@@ -1,69 +0,0 @@
import { NextRequest, NextResponse } from "next/server";
import { apiHandler, requireAuth, HttpError } from "@/lib/api-handler";
import { checkRateLimit } from "@/lib/rate-limit";
import {
generatePKCE,
generateState,
buildAuthorizationUrl,
CodexOAuthNotConfiguredError,
isCodexOAuthConfigured,
} from "@/lib/ai/codex-oauth";
import { cookies } from "next/headers";
import { annotate } from "@/lib/logging/context";
export const GET = apiHandler(async (_request: NextRequest) => {
const { user } = await requireAuth();
// Surface configuration errors as 503 instead of silently redirecting
// to a dead chatgpt.com URL. The v1.4.2 build dropped the `client_id`
// out of the authorize URL — the user landed on the chatgpt.com login
// page, no callback ever fired, and the connect button "did nothing".
if (!isCodexOAuthConfigured()) {
throw new HttpError(
503,
"Codex OAuth is not configured on this instance — set CODEX_OAUTH_CLIENT_ID",
);
}
const rl = await checkRateLimit(`codex-authorize:${user.id}`, 5, 60_000);
if (!rl.allowed) throw new HttpError(429, "Too many requests");
const { verifier, challenge } = generatePKCE();
const state = generateState();
const appUrl = process.env.APP_URL || process.env.NEXT_PUBLIC_APP_URL;
if (!appUrl) throw new Error("APP_URL not configured");
const redirectUri = `${appUrl}/api/auth/codex/callback`;
const cookieStore = await cookies();
const cookieOptions = {
httpOnly: true,
secure: process.env.NODE_ENV === "production",
sameSite: "lax" as const,
path: "/",
maxAge: 300,
};
cookieStore.set("codex_verifier", verifier, cookieOptions);
cookieStore.set("codex_state", state, cookieOptions);
let authUrl: string;
try {
authUrl = buildAuthorizationUrl({
codeChallenge: challenge,
state,
redirectUri,
});
} catch (err) {
// Race between the env check above and the URL build — extremely
// unlikely but the typed error path is cheap insurance.
if (err instanceof CodexOAuthNotConfiguredError) {
throw new HttpError(503, err.message);
}
throw err;
}
annotate({ action: { name: "codex.oauth.authorize" } });
return NextResponse.redirect(authUrl);
});
-105
View File
@@ -1,105 +0,0 @@
import { NextRequest, NextResponse } from "next/server";
import { apiHandler, requireAuth } from "@/lib/api-handler";
import { checkRateLimit } from "@/lib/rate-limit";
import { exchangeCodeForTokens, encryptCodexCreds } from "@/lib/ai/codex-oauth";
import { prisma } from "@/lib/db";
import { auditLog } from "@/lib/auth/audit";
import { getClientIp } from "@/lib/api-response";
import { cookies } from "next/headers";
import { annotate } from "@/lib/logging/context";
export const GET = apiHandler(async (request: NextRequest) => {
const { user } = await requireAuth();
const rl = await checkRateLimit(`codex-callback:${user.id}`, 10, 60_000);
if (!rl.allowed) {
const appUrl = process.env.APP_URL || process.env.NEXT_PUBLIC_APP_URL;
return NextResponse.redirect(
`${appUrl}/settings/integrations?codex_error=rate_limited`,
);
}
const url = new URL(request.url);
const code = url.searchParams.get("code");
const state = url.searchParams.get("state");
const error = url.searchParams.get("error");
const appUrl = process.env.APP_URL || process.env.NEXT_PUBLIC_APP_URL;
if (!appUrl) throw new Error("APP_URL not configured");
if (error) {
annotate({ meta: { oauth_error: error } });
return NextResponse.redirect(
`${appUrl}/settings/integrations?codex_error=${encodeURIComponent(error)}`,
);
}
if (!code || !state) {
return NextResponse.redirect(
`${appUrl}/settings/integrations?codex_error=missing_params`,
);
}
const cookieStore = await cookies();
const storedState = cookieStore.get("codex_state")?.value;
const storedVerifier = cookieStore.get("codex_verifier")?.value;
cookieStore.delete("codex_state");
cookieStore.delete("codex_verifier");
if (!storedState || !storedVerifier || state !== storedState) {
return NextResponse.redirect(
`${appUrl}/settings/integrations?codex_error=invalid_state`,
);
}
const redirectUri = `${appUrl}/api/auth/codex/callback`;
try {
const creds = await exchangeCodeForTokens({
code,
codeVerifier: storedVerifier,
redirectUri,
});
const encrypted = encryptCodexCreds({
apiKey: creds.apiKey,
refreshToken: creds.refreshToken,
});
// The DB field is named `codexAccessTokenEncrypted` for historical
// reasons. Since v1.4.7 the value stored is the **OpenAI API key**
// obtained via the token-exchange grant (RFC 8693), not the OAuth
// access token. The schema name stays for migration-free rollout;
// the helpers (`encryptCodexCreds` / `decryptCodexCreds`) are the
// canonical surface from now on.
await prisma.user.update({
where: { id: user.id },
data: {
codexAccessTokenEncrypted: encrypted.apiKeyEncrypted,
codexRefreshTokenEncrypted: encrypted.refreshEncrypted,
codexTokenExpiresAt: creds.expiresAt,
codexConnectedAt: new Date(),
codexConnectionStatus: "connected",
insightsCachedAt: null,
insightsCachedText: null,
},
});
await auditLog("codex.oauth.connected", {
userId: user.id,
ipAddress: getClientIp(request),
});
annotate({ action: { name: "codex.oauth.callback.success" } });
return NextResponse.redirect(
`${appUrl}/settings/integrations?codex_connected=true`,
);
} catch (err) {
const message = err instanceof Error ? err.message : "unknown";
annotate({ meta: { codex_token_error: message } });
return NextResponse.redirect(
`${appUrl}/settings/integrations?codex_error=token_exchange_failed`,
);
}
});
+9 -10
View File
@@ -1,8 +1,7 @@
import { NextRequest } from "next/server";
import { apiHandler, requireAuth, HttpError } from "@/lib/api-handler";
import { checkRateLimit } from "@/lib/rate-limit";
import { pollDeviceCode } from "@/lib/ai/codex-oauth";
import { encrypt } from "@/lib/crypto";
import { pollDeviceCode, encryptCodexCreds } from "@/lib/ai/codex-oauth";
import { cookies } from "next/headers";
import { decrypt } from "@/lib/crypto";
import { prisma } from "@/lib/db";
@@ -68,17 +67,17 @@ export const POST = apiHandler(async (request: NextRequest) => {
return apiSuccess({ status: "pending" });
}
// status === "connected" → persist OAuth tokens directly. Device-
// code flow uses the access_token against
// https://chatgpt.com/backend-api/codex/responses — no api-key
// exchange, because the id_token here lacks the organization claim
// that exchange would require.
// status === "connected" → persist the full credential blob
// (access token + chatgpt_account_id + refresh token), encrypted.
// The account id is mandatory for the `ChatGPT-Account-ID` header
// on every Codex backend request.
const enc = encryptCodexCreds(result.creds);
await prisma.user.update({
where: { id: user.id },
data: {
codexAccessTokenEncrypted: encrypt(result.tokens.accessToken),
codexRefreshTokenEncrypted: encrypt(result.tokens.refreshToken),
codexTokenExpiresAt: result.tokens.expiresAt,
codexAccessTokenEncrypted: enc.accessEncrypted,
codexRefreshTokenEncrypted: enc.refreshEncrypted,
codexTokenExpiresAt: result.creds.expiresAt,
codexConnectedAt: new Date(),
codexConnectionStatus: "connected",
insightsCachedAt: null,
+23 -5
View File
@@ -51,7 +51,10 @@ describe("CodexClient", () => {
const client = new CodexClient({
accessToken: "test-token",
onTokenRefresh: vi.fn(),
accountId: "acct-test",
onTokenRefresh: vi
.fn()
.mockResolvedValue({ accessToken: "x", accountId: "acct-test" }),
});
const result = await client.generateCompletion({
@@ -95,7 +98,10 @@ describe("CodexClient", () => {
const client = new CodexClient({
accessToken: "test-token",
onTokenRefresh: vi.fn(),
accountId: "acct-test",
onTokenRefresh: vi
.fn()
.mockResolvedValue({ accessToken: "x", accountId: "acct-test" }),
});
const result = await client.generateCompletion({
systemPrompt: "test",
@@ -122,7 +128,11 @@ describe("CodexClient", () => {
const client = new CodexClient({
accessToken: "expired-token",
onTokenRefresh: onRefresh,
accountId: "acct-test",
onTokenRefresh: async () => {
const t = await onRefresh();
return { accessToken: t, accountId: "acct-test" };
},
});
const result = await client.generateCompletion({
@@ -149,7 +159,11 @@ describe("CodexClient", () => {
const client = new CodexClient({
accessToken: "bad",
onTokenRefresh: onRefresh,
accountId: "acct-test",
onTokenRefresh: async () => {
const t = await onRefresh();
return { accessToken: t, accountId: "acct-test" };
},
});
await expect(
@@ -170,7 +184,11 @@ describe("CodexClient", () => {
const onRefresh = vi.fn();
const client = new CodexClient({
accessToken: "test",
onTokenRefresh: onRefresh,
accountId: "acct-test",
onTokenRefresh: async () => {
const t = await onRefresh();
return { accessToken: t, accountId: "acct-test" };
},
});
await expect(
+121 -128
View File
@@ -1,4 +1,4 @@
import { afterEach, beforeEach, describe, it, expect, vi } from "vitest";
import { afterEach, describe, it, expect, vi } from "vitest";
vi.mock("@/lib/crypto", () => ({
encrypt: vi.fn((v: string) => `enc:${v}`),
@@ -8,25 +8,34 @@ vi.mock("@/lib/crypto", () => ({
import {
generatePKCE,
generateState,
buildAuthorizationUrl,
exchangeCodeForTokens,
refreshTokens,
obtainApiKey,
requestDeviceCode,
pollDeviceCode,
refreshDeviceTokens,
encryptCodexCreds,
decryptCodexCreds,
getCodexClientId,
} from "../codex-oauth";
/**
* Minimal id_token JWT for tests — only the payload matters; signature
* is unverified by our client. Payload includes the
* `chatgpt_account_id` claim and a far-future `exp`.
*/
function makeIdToken(claims: Record<string, unknown>): string {
const header = Buffer.from(JSON.stringify({ alg: "none" })).toString(
"base64url",
);
const payload = Buffer.from(JSON.stringify(claims)).toString("base64url");
return `${header}.${payload}.`;
}
describe("codex-oauth", () => {
describe("generatePKCE", () => {
it("generates verifier and challenge of correct format", () => {
const { verifier, challenge } = generatePKCE();
expect(verifier.length).toBeGreaterThanOrEqual(43);
expect(challenge.length).toBeGreaterThanOrEqual(43);
expect(verifier).not.toContain("=");
expect(challenge).not.toContain("=");
});
it("generates different values each time", () => {
const a = generatePKCE();
const b = generatePKCE();
@@ -38,7 +47,6 @@ describe("codex-oauth", () => {
it("generates a non-empty base64url string", () => {
const state = generateState();
expect(state.length).toBeGreaterThan(20);
expect(state).not.toContain("=");
});
});
@@ -48,167 +56,152 @@ describe("codex-oauth", () => {
if (ORIGINAL === undefined) delete process.env.CODEX_OAUTH_CLIENT_ID;
else process.env.CODEX_OAUTH_CLIENT_ID = ORIGINAL;
});
it("falls back to the public Codex CLI client ID when env is unset", () => {
delete process.env.CODEX_OAUTH_CLIENT_ID;
expect(getCodexClientId()).toBe("app_EMoamEEZ73f0CkXaXp7hrann");
});
it("respects an operator override", () => {
process.env.CODEX_OAUTH_CLIENT_ID = "app_private_test";
expect(getCodexClientId()).toBe("app_private_test");
});
});
describe("buildAuthorizationUrl", () => {
it("targets auth.openai.com with the codex CLI scope set", () => {
const url = buildAuthorizationUrl({
codeChallenge: "test-challenge",
state: "test-state",
redirectUri: "https://example.com/callback",
});
const parsed = new URL(url);
// The previous v1.4.6 pointed at chatgpt.com, which is not an
// OAuth issuer at all. Anchoring this assertion explicitly so a
// future regression to the wrong host fails loudly.
expect(parsed.origin).toBe("https://auth.openai.com");
expect(parsed.pathname).toBe("/oauth/authorize");
expect(parsed.searchParams.get("response_type")).toBe("code");
expect(parsed.searchParams.get("client_id")).toBe(
"app_EMoamEEZ73f0CkXaXp7hrann",
);
expect(parsed.searchParams.get("code_challenge")).toBe("test-challenge");
expect(parsed.searchParams.get("code_challenge_method")).toBe("S256");
expect(parsed.searchParams.get("state")).toBe("test-state");
expect(parsed.searchParams.get("redirect_uri")).toBe(
"https://example.com/callback",
);
expect(parsed.searchParams.get("scope")).toContain("offline_access");
expect(parsed.searchParams.get("id_token_add_organizations")).toBe(
"true",
);
expect(parsed.searchParams.get("codex_cli_simplified_flow")).toBe("true");
});
});
describe("token / api-key exchange", () => {
describe("device-code flow", () => {
let originalFetch: typeof fetch;
beforeEach(() => {
originalFetch = global.fetch;
});
beforeEachInit();
afterEach(() => {
global.fetch = originalFetch;
});
function beforeEachInit() {
originalFetch = global.fetch;
}
it("exchanges the auth code for tokens and trades the id_token for an api key", async () => {
const calls: Array<{ url: string; body: string }> = [];
it("requestDeviceCode returns user code + verification URL", async () => {
global.fetch = vi.fn(
async (url: RequestInfo | URL, init?: RequestInit) => {
calls.push({ url: String(url), body: String(init?.body ?? "") });
if (calls.length === 1) {
// Token endpoint — returns id_token + access_token + refresh_token.
return new Response(
JSON.stringify({
id_token: "id-token-abc",
access_token: "oauth-access",
refresh_token: "oauth-refresh",
expires_in: 3600,
token_type: "Bearer",
}),
{ status: 200, headers: { "content-type": "application/json" } },
);
}
// Api-key exchange.
return new Response(
JSON.stringify({ access_token: "sk-from-codex" }),
{
status: 200,
headers: { "content-type": "application/json" },
},
);
},
async () =>
new Response(
JSON.stringify({
device_auth_id: "dev-1",
user_code: "ABCD-1234",
interval: "5",
}),
{ status: 200, headers: { "content-type": "application/json" } },
),
) as never;
const result = await exchangeCodeForTokens({
code: "auth-code",
codeVerifier: "verifier",
redirectUri: "https://example.com/cb",
});
expect(result.apiKey).toBe("sk-from-codex");
expect(result.refreshToken).toBe("oauth-refresh");
expect(result.expiresAt).toBeInstanceOf(Date);
// Both calls hit auth.openai.com with form-urlencoded bodies.
expect(calls[0].url).toBe("https://auth.openai.com/oauth/token");
expect(calls[0].body).toContain("grant_type=authorization_code");
expect(calls[0].body).toContain("code_verifier=verifier");
expect(calls[1].url).toBe("https://auth.openai.com/oauth/token");
expect(calls[1].body).toContain(
"grant_type=urn%3Aietf%3Aparams%3Aoauth%3Agrant-type%3Atoken-exchange",
);
expect(calls[1].body).toContain("subject_token=id-token-abc");
expect(calls[1].body).toContain("requested_token=openai-api-key");
const code = await requestDeviceCode();
expect(code.userCode).toBe("ABCD-1234");
expect(code.deviceAuthId).toBe("dev-1");
expect(code.intervalSeconds).toBe(5);
expect(code.verificationUrl).toBe("https://auth.openai.com/codex/device");
});
it("refreshTokens re-uses the refresh token and re-runs api-key exchange", async () => {
it("pollDeviceCode returns 'pending' on 403", async () => {
global.fetch = vi.fn(
async () => new Response("", { status: 403 }),
) as never;
const r = await pollDeviceCode({ deviceAuthId: "x", userCode: "y" });
expect(r.status).toBe("pending");
});
it("pollDeviceCode resolves to creds with account id from id_token", async () => {
const idToken = makeIdToken({
chatgpt_account_id: "acct-test",
exp: Math.floor(Date.now() / 1000) + 3600,
});
let call = 0;
global.fetch = vi.fn(async () => {
call += 1;
if (call === 1) {
// device-poll: returns auth code + PKCE values
return new Response(
JSON.stringify({
id_token: "id-fresh",
access_token: "oauth-access-fresh",
refresh_token: "oauth-refresh-fresh",
expires_in: 3600,
token_type: "Bearer",
authorization_code: "auth-code",
code_challenge: "challenge",
code_verifier: "verifier",
}),
{ status: 200, headers: { "content-type": "application/json" } },
);
}
return new Response(JSON.stringify({ access_token: "sk-rotated" }), {
status: 200,
headers: { "content-type": "application/json" },
});
// /oauth/token exchange
return new Response(
JSON.stringify({
id_token: idToken,
access_token: "oauth-access",
refresh_token: "refresh-1",
expires_in: 3600,
}),
{ status: 200, headers: { "content-type": "application/json" } },
);
}) as never;
const result = await refreshTokens("old-refresh");
expect(result.apiKey).toBe("sk-rotated");
expect(result.refreshToken).toBe("oauth-refresh-fresh");
const r = await pollDeviceCode({ deviceAuthId: "x", userCode: "y" });
expect(r.status).toBe("connected");
if (r.status === "connected") {
expect(r.creds.accessToken).toBe("oauth-access");
expect(r.creds.refreshToken).toBe("refresh-1");
expect(r.creds.accountId).toBe("acct-test");
}
});
it("obtainApiKey throws when the upstream omits access_token", async () => {
global.fetch = vi.fn(
async () =>
new Response(JSON.stringify({}), {
status: 200,
headers: { "content-type": "application/json" },
it("refreshDeviceTokens uses JSON body and returns rotated tokens", async () => {
const idToken = makeIdToken({
chatgpt_account_id: "acct-test",
exp: Math.floor(Date.now() / 1000) + 3600,
});
const calls: string[] = [];
global.fetch = vi.fn(async (_url: string, init?: RequestInit) => {
calls.push(String(init?.body ?? ""));
return new Response(
JSON.stringify({
id_token: idToken,
access_token: "oauth-fresh",
refresh_token: "refresh-2",
expires_in: 3600,
}),
) as never;
await expect(obtainApiKey("id-token-x")).rejects.toThrow(
/no access_token/i,
);
{ status: 200, headers: { "content-type": "application/json" } },
);
}) as never;
const r = await refreshDeviceTokens("refresh-1");
expect(r.accessToken).toBe("oauth-fresh");
expect(r.refreshToken).toBe("refresh-2");
expect(r.accountId).toBe("acct-test");
// Body must be JSON (not form-urlencoded), per the spec.
expect(calls[0]).toMatch(/^\{/);
expect(JSON.parse(calls[0])).toMatchObject({
grant_type: "refresh_token",
refresh_token: "refresh-1",
});
});
});
describe("encryptCodexCreds / decryptCodexCreds", () => {
it("round-trips api key and refresh token", () => {
it("round-trips access token, refresh token, and account id", () => {
const expiresAt = new Date(Date.now() + 3600_000);
const enc = encryptCodexCreds({
apiKey: "sk-test",
accessToken: "oauth-access",
refreshToken: "refresh-test",
accountId: "acct-123",
expiresAt,
});
expect(enc.apiKeyEncrypted).toBe("enc:sk-test");
expect(enc.refreshEncrypted).toBe("enc:refresh-test");
const dec = decryptCodexCreds({
apiKeyEncrypted: enc.apiKeyEncrypted,
accessEncrypted: enc.accessEncrypted,
refreshEncrypted: enc.refreshEncrypted,
});
expect(dec.apiKey).toBe("sk-test");
expect(dec.refreshToken).toBe("refresh-test");
expect(dec).not.toBeNull();
expect(dec!.accessToken).toBe("oauth-access");
expect(dec!.refreshToken).toBe("refresh-test");
expect(dec!.accountId).toBe("acct-123");
expect(dec!.expiresAt.getTime()).toBe(expiresAt.getTime());
});
it("returns null for legacy v1.4.7-v1.4.11 token storage (raw string)", () => {
// Pre-v1.4.12 stored just the access token plaintext. Decoder
// detects this (not JSON) and returns null so the caller can
// mark the connection expired.
const dec = decryptCodexCreds({
accessEncrypted: "enc:legacy-raw-token",
refreshEncrypted: "enc:refresh",
});
expect(dec).toBeNull();
});
});
});
+19 -9
View File
@@ -11,11 +11,13 @@ vi.mock("@/lib/crypto", () => ({
encrypt: vi.fn((v: string) => `encrypted:${v}`),
}));
vi.mock("@/lib/ai/codex-oauth", () => ({
refreshTokens: vi.fn(),
encryptCodexCreds: vi.fn((c: { apiKey: string; refreshToken: string }) => ({
apiKeyEncrypted: `enc:${c.apiKey}`,
refreshEncrypted: `enc:${c.refreshToken}`,
})),
refreshDeviceTokens: vi.fn(),
encryptCodexCreds: vi.fn(
(c: { accessToken: string; refreshToken: string; accountId: string }) => ({
accessEncrypted: `enc:${c.accessToken}|${c.accountId}`,
refreshEncrypted: `enc:${c.refreshToken}`,
}),
),
decryptCodexCreds: vi.fn(),
}));
@@ -39,8 +41,12 @@ describe("resolveProvider", () => {
vi.mocked(decrypt).mockImplementation((v: string) => `decrypted:${v}`);
vi.mocked(encrypt).mockImplementation((v: string) => `encrypted:${v}`);
vi.mocked(encryptCodexCreds).mockImplementation(
(c: { apiKey: string; refreshToken: string }) => ({
apiKeyEncrypted: `enc:${c.apiKey}`,
(c: {
accessToken: string;
refreshToken: string;
accountId: string;
}) => ({
accessEncrypted: `enc:${c.accessToken}|${c.accountId}`,
refreshEncrypted: `enc:${c.refreshToken}`,
}),
);
@@ -48,8 +54,10 @@ describe("resolveProvider", () => {
it("returns codex provider when user has valid tokens (no explicit choice)", async () => {
vi.mocked(decryptCodexCreds).mockReturnValue({
apiKey: "sk-codex-key",
accessToken: "oauth-access",
refreshToken: "decrypted-refresh",
accountId: "acct-123",
expiresAt: new Date(Date.now() + 3600_000),
});
vi.mocked(prisma.user.findUnique)
@@ -265,8 +273,10 @@ describe("resolveProvider", () => {
it("CHATGPT_OAUTH selection routes through Codex when connected", async () => {
vi.mocked(decryptCodexCreds).mockReturnValue({
apiKey: "sk-codex-key",
accessToken: "oauth-access",
refreshToken: "decrypted-refresh",
accountId: "acct-123",
expiresAt: new Date(Date.now() + 3600_000),
});
vi.mocked(prisma.user.findUnique)
+130 -66
View File
@@ -1,20 +1,55 @@
import { randomUUID } from "node:crypto";
import type { AIProvider, CompletionParams, CompletionResult } from "./types";
/**
* Codex backend client — talks to `chatgpt.com/backend-api/codex/responses`
* via the OpenAI Responses API (SSE streaming, ChatGPT-Account-ID
* header, OAuth Bearer auth). Implemented against the spec at
* `docs/codex-protocol-spec.md`, which mirrors the official `openai/codex`
* CLI.
*
* Required headers:
* - `Authorization: Bearer <oauth-access-token>`
* - `ChatGPT-Account-ID: <chatgpt_account_id JWT claim>`
* - `Content-Type: application/json`
* - `Accept: text/event-stream`
* - `originator`, `User-Agent`, `session_id`, `thread_id`
*
* Response is always SSE (server rejects `stream: false` with 400).
*/
const CODEX_ENDPOINT = "https://chatgpt.com/backend-api/codex/responses";
const CODEX_MODEL = "gpt-5.3-codex";
/**
* `gpt-5.3-codex` is a *test placeholder* in the upstream codebase
* (`codex-rs/codex-api/src/sse/responses.rs:1374`), not a real model
* slug. The real CLI default is `gpt-5-codex`. The server safety-routes
* if needed and reports the actual model in the `OpenAI-Model`
* response header.
*/
const CODEX_MODEL = "gpt-5-codex";
const ORIGINATOR = "healthlog";
const USER_AGENT = "HealthLog/1.0 (+https://healthlog.bombeck.io)";
interface CodexClientConfig {
accessToken: string;
onTokenRefresh: () => Promise<string>;
accountId: string;
onTokenRefresh: () => Promise<{ accessToken: string; accountId: string }>;
}
export class CodexClient implements AIProvider {
readonly type = "codex" as const;
private accessToken: string;
private onTokenRefresh: () => Promise<string>;
private accountId: string;
private onTokenRefresh: () => Promise<{
accessToken: string;
accountId: string;
}>;
constructor(config: CodexClientConfig) {
this.accessToken = config.accessToken;
this.accountId = config.accountId;
this.onTokenRefresh = config.onTokenRefresh;
}
@@ -24,7 +59,9 @@ export class CodexClient implements AIProvider {
const firstAttempt = await this.doRequest(params);
if (firstAttempt.status === 401) {
this.accessToken = await this.onTokenRefresh();
const fresh = await this.onTokenRefresh();
this.accessToken = fresh.accessToken;
this.accountId = fresh.accountId;
const retryAttempt = await this.doRequest(params);
if (!retryAttempt.ok) {
throw await this.buildUpstreamError(
@@ -42,13 +79,6 @@ export class CodexClient implements AIProvider {
return this.parseResponse(firstAttempt);
}
/**
* Build a structured error from a non-OK upstream response. Mirrors the
* pattern used in `OpenAIClient.generateCompletion` so logs and Glitchtip
* issues for both providers carry the same fields (httpStatus, upstream,
* model, bodyExcerpt). The body excerpt is truncated to 500 chars and any
* `sk-…` / `Bearer …` token is masked before it reaches log shipping.
*/
private async buildUpstreamError(
res: Response,
message: string,
@@ -69,15 +99,21 @@ export class CodexClient implements AIProvider {
}
private async doRequest(params: CompletionParams): Promise<Response> {
// The chatgpt.com/backend-api/codex/responses endpoint expects the
// OpenAI Responses API shape: `input` is an array of typed message
// items, NOT a raw string. The Codex CLI's `ResponsesApiRequest`
// (`codex-rs/codex-api/src/common.rs`) is the canonical definition.
const sessionId = randomUUID();
const threadId = randomUUID();
return fetch(CODEX_ENDPOINT, {
method: "POST",
headers: {
"Content-Type": "application/json",
Accept: "text/event-stream",
Authorization: `Bearer ${this.accessToken}`,
"ChatGPT-Account-ID": this.accountId,
originator: ORIGINATOR,
"User-Agent": USER_AGENT,
session_id: sessionId,
"session-id": sessionId,
thread_id: threadId,
"thread-id": threadId,
},
body: JSON.stringify({
model: CODEX_MODEL,
@@ -92,37 +128,34 @@ export class CodexClient implements AIProvider {
tools: [],
tool_choice: "auto",
parallel_tool_calls: false,
// Reasoning is required-but-nullable on the wire. We don't ask
// for reasoning summaries — emit `null` so the JSON has the
// field present and the server stops complaining about a
// missing key (it returned 400 in earlier iterations).
reasoning: null,
store: false,
// Codex backend rejects sync responses with
// `Stream must be set to true`. We have to consume SSE.
stream: true,
// Required field; empty array when not asking for reasoning.
include: [],
}),
});
}
/**
* Consume the Codex SSE stream and assemble the final assistant
* message. The Codex backend emits OpenAI-Responses-API events:
* message. Event types per the spec:
*
* event: response.output_text.delta
* data: { "delta": "Hello" }
* - `response.output_item.done` — final assembled item; an
* assistant `Message` carries the canonical reply in
* `item.content[].text`.
* - `response.output_text.delta` — incremental chunks; fallback
* when no done event arrives.
* - `response.completed` — terminal; carries `usage.total_tokens`.
* - `response.failed` / `response.incomplete` — terminal errors.
*
* event: response.output_item.done
* data: { "type": "message", "role": "assistant",
* "content": [{ "type": "output_text", "text": "Hello world" }] }
*
* event: response.completed
* data: { "type": "response.completed",
* "response": { "id": "...", "usage": { "total_tokens": 123 } } }
*
* Strategy:
* - Prefer the assembled text in `response.output_item.done` (any
* `message` item with `assistant` role and `output_text` content).
* This avoids re-stitching `delta` chunks ourselves.
* - Fall back to concatenating all `output_text.delta` chunks if no
* output_item.done arrived before the stream ended.
* - Pick up token usage from `response.completed` when present.
* - Throw on `response.error` events with the upstream message.
* Reasoning deltas (`response.reasoning_*.delta`) are deliberately
* NOT folded into the visible text — they belong to a separate
* channel.
*/
private async parseResponse(res: Response): Promise<CompletionResult> {
if (!res.body) {
@@ -135,6 +168,12 @@ export class CodexClient implements AIProvider {
let assembledText = "";
let deltaText = "";
let tokensUsed: number | null = null;
let serverModel: string | null = null;
// The server reports the actual routed model in the OpenAI-Model
// response header — useful when safety-routing kicks in.
const headerModel = res.headers.get("OpenAI-Model");
if (headerModel) serverModel = headerModel;
try {
while (true) {
@@ -142,7 +181,6 @@ export class CodexClient implements AIProvider {
if (done) break;
buffer += decoder.decode(value, { stream: true });
// SSE messages are separated by a blank line (\n\n).
let separator: number;
while ((separator = buffer.indexOf("\n\n")) !== -1) {
const rawEvent = buffer.slice(0, separator);
@@ -153,7 +191,7 @@ export class CodexClient implements AIProvider {
.find((l) => l.startsWith("data:"));
if (!dataLine) continue;
const payload = dataLine.slice(5).trim();
if (!payload || payload === "[DONE]") continue;
if (!payload) continue;
let parsed: {
type?: string;
@@ -163,8 +201,12 @@ export class CodexClient implements AIProvider {
role?: string;
content?: Array<{ type?: string; text?: string }>;
};
response?: { usage?: { total_tokens?: number } };
error?: { message?: string };
response?: {
usage?: { total_tokens?: number };
error?: { code?: string; message?: string };
incomplete_details?: { reason?: string };
};
error?: { code?: string; message?: string };
};
try {
parsed = JSON.parse(payload);
@@ -172,32 +214,54 @@ export class CodexClient implements AIProvider {
continue;
}
if (parsed.type === "response.output_text.delta" && parsed.delta) {
deltaText += parsed.delta;
continue;
}
if (parsed.type === "response.output_item.done" && parsed.item) {
const item = parsed.item;
if (item.type === "message" && item.role === "assistant") {
const text = item.content
?.filter((c) => c.type === "output_text" && c.text)
.map((c) => c.text!)
.join("");
if (text) assembledText += text;
switch (parsed.type) {
case "response.output_text.delta": {
if (parsed.delta) deltaText += parsed.delta;
break;
}
continue;
}
if (parsed.type === "response.completed") {
tokensUsed = parsed.response?.usage?.total_tokens ?? null;
continue;
}
if (parsed.type === "response.error" || parsed.type === "error") {
const message =
parsed.error?.message ?? "Codex stream returned an error event";
throw new Error(message);
case "response.output_item.done": {
const item = parsed.item;
if (item?.type === "message" && item.role === "assistant") {
const text = item.content
?.filter((c) => c.type === "output_text" && c.text)
.map((c) => c.text!)
.join("");
if (text) assembledText += text;
}
break;
}
case "response.completed": {
tokensUsed = parsed.response?.usage?.total_tokens ?? null;
break;
}
case "response.failed": {
const err = parsed.response?.error;
const message =
err?.message ?? "Codex stream returned response.failed";
const e = new Error(message);
Object.assign(e, {
upstream: "codex",
errorCode: err?.code ?? null,
model: serverModel ?? CODEX_MODEL,
});
throw e;
}
case "response.incomplete": {
const reason =
parsed.response?.incomplete_details?.reason ??
"incomplete response";
throw new Error(`Codex stream incomplete: ${reason}`);
}
case "error":
case "response.error": {
const message =
parsed.error?.message ?? "Codex stream returned an error event";
throw new Error(message);
}
default:
// Unknown event types are tolerated and ignored, same
// as the official client.
break;
}
}
}
@@ -213,7 +277,7 @@ export class CodexClient implements AIProvider {
return {
content,
tokensUsed,
model: CODEX_MODEL,
model: serverModel ?? CODEX_MODEL,
providerType: "codex",
};
}
+196 -250
View File
@@ -2,44 +2,43 @@ import { createHash, randomBytes } from "node:crypto";
import { encrypt, decrypt } from "@/lib/crypto";
/**
* Codex / ChatGPT-OAuth flow — mirrors the official `openai/codex` CLI
* (`codex-rs/login/src/server.rs`).
* Codex / ChatGPT-OAuth flow — implemented against `docs/codex-protocol-spec.md`,
* which mirrors the official `openai/codex` CLI.
*
* The previous v1.4.2-v1.4.6 implementation hit `chatgpt.com/authorize`
* + `chatgpt.com/oauth/token`, which are NOT OAuth endpoints. ChatGPT
* silently rendered its normal web-app and the flow dead-ended. The
* canonical issuer is `auth.openai.com` for both authorize and token
* exchange.
* Production-only path: **device-code flow**. The browser/auth-code flow
* does not work for hosted apps — Hydra's redirect-URI allow-list for the
* public Codex CLI client ID covers only `localhost:1455/1457`. The
* device-code flow side-steps that: the user types a short code on
* `auth.openai.com/codex/device`, we poll, on success we get an
* authorization-code internally and exchange it for OAuth tokens.
*
* Flow:
* 1. PKCE authorize → user consents at auth.openai.com
* 2. Token exchange → returns `{ id_token, access_token, refresh_token }`
* 3. **API-key exchange** (token-exchange grant, RFC 8693) → trades
* the `id_token` for a regular OpenAI API key that bills against
* the user's ChatGPT subscription.
* 4. HealthLog stores the API key (encrypted) and uses it via the
* standard OpenAIClient against `https://api.openai.com/v1`.
* Refresh tokens are stored too so the API key can be re-issued
* when it ages out — the refresh token is the long-lived secret.
* After the exchange we hold three pieces:
* - `accessToken` — the OAuth access token used as `Authorization: Bearer`
* on every Codex request.
* - `refreshToken` — long-lived; rotates on every refresh.
* - `accountId` — the `chatgpt_account_id` claim from the id_token,
* required as the `ChatGPT-Account-ID` header (without it: 401).
*
* All three are persisted (JSON-encoded then encrypted) on the user row
* so we don't need a schema migration.
*/
const ISSUER = "https://auth.openai.com";
/**
* Public PKCE client ID hardcoded in the official `openai/codex` CLI
* (`codex-rs/login/src/auth/manager.rs`). Allows OPS to override via
* env var if a private OAuth app is registered for this deployment.
* (`codex-rs/login/src/auth/manager.rs`). Operators with a private OAuth
* app can override via env var.
*/
const DEFAULT_CLIENT_ID = "app_EMoamEEZ73f0CkXaXp7hrann";
/**
* Scopes the official Codex CLI requests. Without these the
* `id_token` returned has no `chatgpt_subscription_active_until` claim
* and the api-key exchange downstream rejects it.
*/
/** Same scopes the official CLI requests. */
const CODEX_SCOPES =
"openid profile email offline_access api.connectors.read api.connectors.invoke";
const DEVICE_VERIFICATION_URL = `${ISSUER}/codex/device`;
const DEVICE_REDIRECT_URI = `${ISSUER}/deviceauth/callback`;
function base64url(buffer: Buffer): string {
return buffer.toString("base64url");
}
@@ -55,9 +54,9 @@ export function generateState(): string {
}
/**
* Backwards-compatible no-op type. Kept so any existing callers that
* imported the error type don't break — the helpers below now always
* return a value (the public Codex client ID is the safe default).
* Backwards-compatible no-op type. Older callers that imported the error
* type continue to compile. Codex OAuth always considers itself
* configured because we ship the public client ID as the default.
*/
export class CodexOAuthNotConfiguredError extends Error {
constructor() {
@@ -72,181 +71,118 @@ export function getCodexClientId(): string {
return process.env.CODEX_OAUTH_CLIENT_ID?.trim() || DEFAULT_CLIENT_ID;
}
/**
* Codex OAuth is always considered configured because we ship the
* public Codex CLI client ID as a default. Operators can override via
* env var if they have a private OAuth app registered with OpenAI.
*/
export function isCodexOAuthConfigured(): boolean {
return true;
}
export function buildAuthorizationUrl(params: {
codeChallenge: string;
state: string;
redirectUri: string;
}): string {
const url = new URL(`${ISSUER}/oauth/authorize`);
url.searchParams.set("response_type", "code");
url.searchParams.set("client_id", getCodexClientId());
url.searchParams.set("redirect_uri", params.redirectUri);
url.searchParams.set("scope", CODEX_SCOPES);
url.searchParams.set("code_challenge", params.codeChallenge);
url.searchParams.set("code_challenge_method", "S256");
url.searchParams.set("state", params.state);
// Required for the api-key exchange step to work — without these
// claims the id_token cannot be traded for an OpenAI API key.
url.searchParams.set("id_token_add_organizations", "true");
url.searchParams.set("codex_cli_simplified_flow", "true");
return url.toString();
// ─── JWT inspection ──────────────────────────────────────────────────
interface JwtClaims {
exp?: number;
chatgpt_account_id?: string;
/**
* The official Codex CLI also accepts the URI-namespaced variant
* (`https://api.openai.com/auth.chatgpt_account_id`) in the
* `https://api.openai.com/auth` claim object — see
* `codex-rs/login/src/auth/manager.rs:891`.
*/
"https://api.openai.com/auth"?: { chatgpt_account_id?: string };
[key: string]: unknown;
}
function decodeJwtClaims(token: string): JwtClaims | null {
const parts = token.split(".");
if (parts.length < 2) return null;
try {
const padded = parts[1].padEnd(
parts[1].length + ((4 - (parts[1].length % 4)) % 4),
"=",
);
const json = Buffer.from(padded, "base64url").toString("utf8");
return JSON.parse(json) as JwtClaims;
} catch {
return null;
}
}
function extractAccountId(idToken: string): string | null {
const claims = decodeJwtClaims(idToken);
if (!claims) return null;
if (typeof claims.chatgpt_account_id === "string") {
return claims.chatgpt_account_id;
}
const namespaced = claims["https://api.openai.com/auth"];
if (
namespaced &&
typeof namespaced === "object" &&
typeof namespaced.chatgpt_account_id === "string"
) {
return namespaced.chatgpt_account_id;
}
return null;
}
function extractExpiry(idToken: string): Date | null {
const claims = decodeJwtClaims(idToken);
if (!claims || typeof claims.exp !== "number") return null;
return new Date(claims.exp * 1000);
}
// ─── Token shape persisted on the user row ───────────────────────────
export interface CodexCreds {
/** OAuth access token used directly against chatgpt.com/backend-api. */
accessToken: string;
/** OAuth refresh token; rotates on every refresh. */
refreshToken: string;
/**
* `chatgpt_account_id` claim from the id_token. Required as the
* `ChatGPT-Account-ID` header on every Codex backend request.
*/
accountId: string;
/** Wall-clock expiry. Derived from id_token `exp`, fallback expires_in. */
expiresAt: Date;
}
interface RawTokenResponse {
id_token: string;
access_token: string;
refresh_token: string;
expires_in: number;
token_type: string;
refresh_token?: string;
expires_in?: number;
token_type?: string;
}
async function postForm(
endpoint: string,
fields: Record<string, string>,
): Promise<Response> {
const body = new URLSearchParams(fields).toString();
return fetch(endpoint, {
method: "POST",
headers: { "Content-Type": "application/x-www-form-urlencoded" },
body,
});
}
export interface CodexTokens {
/** OpenAI API key obtained via the token-exchange grant. */
apiKey: string;
/** OAuth refresh token; long-lived, re-issues fresh tokens. */
refreshToken: string;
/**
* Wall-clock expiry of the API key. Once this passes we re-run the
* refresh + api-key exchange before the next request.
*/
expiresAt: Date;
}
export async function exchangeCodeForTokens(params: {
code: string;
codeVerifier: string;
redirectUri: string;
}): Promise<CodexTokens> {
const res = await postForm(`${ISSUER}/oauth/token`, {
grant_type: "authorization_code",
client_id: getCodexClientId(),
code: params.code,
redirect_uri: params.redirectUri,
code_verifier: params.codeVerifier,
});
if (!res.ok) {
const body = await res.text().catch(() => "");
throw new Error(`Codex token exchange failed (${res.status}): ${body}`);
function buildCreds(
raw: RawTokenResponse,
prevRefresh: string | null,
): CodexCreds {
const accountId = extractAccountId(raw.id_token);
if (!accountId) {
throw new Error(
"Codex token exchange returned id_token without chatgpt_account_id",
);
}
const tokens = (await res.json()) as RawTokenResponse;
const apiKey = await obtainApiKey(tokens.id_token);
return {
apiKey,
refreshToken: tokens.refresh_token,
expiresAt: new Date(Date.now() + tokens.expires_in * 1000),
};
}
export async function refreshTokens(
refreshToken: string,
): Promise<CodexTokens> {
const res = await postForm(`${ISSUER}/oauth/token`, {
grant_type: "refresh_token",
client_id: getCodexClientId(),
refresh_token: refreshToken,
// The same scopes must be requested on refresh, otherwise the
// re-issued id_token loses the claims the api-key exchange needs.
scope: CODEX_SCOPES,
});
if (!res.ok) {
const body = await res.text().catch(() => "");
throw new Error(`Codex token refresh failed (${res.status}): ${body}`);
const refreshToken = raw.refresh_token ?? prevRefresh ?? "";
if (!refreshToken) {
throw new Error("Codex token exchange returned no refresh_token");
}
const tokens = (await res.json()) as RawTokenResponse;
const apiKey = await obtainApiKey(tokens.id_token);
const jwtExp = extractExpiry(raw.id_token);
const expiresAt =
jwtExp ?? new Date(Date.now() + (raw.expires_in ?? 3600) * 1000);
return {
apiKey,
refreshToken: tokens.refresh_token ?? refreshToken,
expiresAt: new Date(Date.now() + tokens.expires_in * 1000),
};
}
/**
* Trade an OpenID `id_token` for an OpenAI API key via the OAuth
* token-exchange grant (RFC 8693). The resulting key is special: it
* bills against the user's ChatGPT subscription rather than a separate
* API plan, which is the entire point of the Codex flow.
*/
export async function obtainApiKey(idToken: string): Promise<string> {
const res = await postForm(`${ISSUER}/oauth/token`, {
grant_type: "urn:ietf:params:oauth:grant-type:token-exchange",
client_id: getCodexClientId(),
requested_token: "openai-api-key",
subject_token: idToken,
subject_token_type: "urn:ietf:params:oauth:token-type:id_token",
});
if (!res.ok) {
const body = await res.text().catch(() => "");
throw new Error(`Codex api-key exchange failed (${res.status}): ${body}`);
}
const json = (await res.json()) as { access_token?: string };
if (!json.access_token) {
throw new Error("Codex api-key exchange returned no access_token");
}
return json.access_token;
}
export function encryptCodexCreds(creds: {
apiKey: string;
refreshToken: string;
}): { apiKeyEncrypted: string; refreshEncrypted: string } {
return {
apiKeyEncrypted: encrypt(creds.apiKey),
refreshEncrypted: encrypt(creds.refreshToken),
};
}
export function decryptCodexCreds(encrypted: {
apiKeyEncrypted: string;
refreshEncrypted: string;
}): { apiKey: string; refreshToken: string } {
return {
apiKey: decrypt(encrypted.apiKeyEncrypted),
refreshToken: decrypt(encrypted.refreshEncrypted),
accessToken: raw.access_token,
refreshToken,
accountId,
expiresAt,
};
}
// ─── Device-code flow ────────────────────────────────────────────────
//
// For hosted apps (HealthLog), the standard authorization-code redirect
// flow does not work because OpenAI's Hydra only allow-lists localhost
// callbacks for the public Codex CLI client ID. The device-code flow is
// the documented escape hatch: the user goes to chatgpt.com on any
// device, enters a short code, and approves the connection — no
// redirect URI on our domain is involved at all.
const DEVICE_VERIFICATION_URL = `${ISSUER}/codex/device`;
const DEVICE_REDIRECT_URI = `${ISSUER}/deviceauth/callback`;
export interface DeviceCodeStart {
/** Short user-facing code, e.g. "RGRP-N5F7U". */
userCode: string;
/** URL the user opens in their browser to approve. */
verificationUrl: string;
/** Opaque per-attempt id used for polling. Server-side only. */
deviceAuthId: string;
/** Polling interval in seconds. */
intervalSeconds: number;
}
@@ -278,36 +214,15 @@ export async function requestDeviceCode(): Promise<DeviceCodeStart> {
};
}
export interface DeviceTokens {
/**
* OAuth access token. Used directly against
* `https://chatgpt.com/backend-api/codex/responses` — the Codex
* backend that bills against the ChatGPT subscription.
*
* The device-code flow deliberately does NOT call the api-key
* exchange the browser flow uses, because the id_token returned
* here lacks the `organization_id` claim (that claim only gets
* added when the authorize URL was hit with
* `id_token_add_organizations=true`, which we cannot do during
* device-code auth — the user types the code on chatgpt.com
* directly without us controlling the authorize step).
*/
accessToken: string;
/** OAuth refresh token; used to mint a new access token before expiry. */
refreshToken: string;
/** Wall-clock expiry of `accessToken`. */
expiresAt: Date;
}
export type DevicePollResult =
| { status: "pending" }
| { status: "connected"; tokens: DeviceTokens };
| { status: "connected"; creds: CodexCreds };
/**
* Single poll attempt against the device-auth token endpoint. Returns
* "pending" while the user has not yet approved (Hydra answers with
* 403/404), and "connected" with already-exchanged Codex tokens once
* the user finishes the approval flow on chatgpt.com.
* One poll attempt against the device-auth token endpoint. Returns
* `pending` while the user has not yet approved (Hydra answers with
* 403/404), and `connected` with fully resolved Codex credentials
* once the user finishes the approval on chatgpt.com.
*/
export async function pollDeviceCode(params: {
deviceAuthId: string;
@@ -336,16 +251,19 @@ export async function pollDeviceCode(params: {
code_verifier: string;
};
// Standard PKCE exchange — but with the deviceauth callback URI
// that Hydra uses internally for this flow. The user never visits
// this URL; it's just the value Hydra associates with the
// authorization code so the exchange typechecks.
const tokenRes = await postForm(`${ISSUER}/oauth/token`, {
grant_type: "authorization_code",
client_id: getCodexClientId(),
code: json.authorization_code,
redirect_uri: DEVICE_REDIRECT_URI,
code_verifier: json.code_verifier,
// Standard PKCE exchange against the OAuth token endpoint, with the
// device-auth callback URI Hydra associates with this authorization
// code internally.
const tokenRes = await fetch(`${ISSUER}/oauth/token`, {
method: "POST",
headers: { "Content-Type": "application/x-www-form-urlencoded" },
body: new URLSearchParams({
grant_type: "authorization_code",
client_id: getCodexClientId(),
code: json.authorization_code,
redirect_uri: DEVICE_REDIRECT_URI,
code_verifier: json.code_verifier,
}).toString(),
});
if (!tokenRes.ok) {
const body = await tokenRes.text().catch(() => "");
@@ -356,27 +274,31 @@ export async function pollDeviceCode(params: {
const tokens = (await tokenRes.json()) as RawTokenResponse;
return {
status: "connected",
tokens: {
accessToken: tokens.access_token,
refreshToken: tokens.refresh_token,
expiresAt: new Date(Date.now() + tokens.expires_in * 1000),
},
creds: buildCreds(tokens, null),
};
}
/**
* Refresh an OAuth access token without going through the api-key
* exchange. Used by the device-code path where the resulting id_token
* never has the organization claims the api-key exchange requires.
* Refresh an OAuth access token using the long-lived refresh token.
* Per `docs/codex-protocol-spec.md` §6d, the refresh endpoint takes
* a JSON body (NOT form-urlencoded — that detail bit us once before).
*
* The refresh token rotates on every call; the new value (when
* present in the response) MUST be persisted, otherwise the next
* attempt fails with `refresh_token_reused`.
*/
export async function refreshDeviceTokens(
refreshToken: string,
): Promise<DeviceTokens> {
const res = await postForm(`${ISSUER}/oauth/token`, {
grant_type: "refresh_token",
client_id: getCodexClientId(),
refresh_token: refreshToken,
scope: CODEX_SCOPES,
): Promise<CodexCreds> {
const res = await fetch(`${ISSUER}/oauth/token`, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
client_id: getCodexClientId(),
grant_type: "refresh_token",
refresh_token: refreshToken,
scope: CODEX_SCOPES,
}),
});
if (!res.ok) {
const body = await res.text().catch(() => "");
@@ -385,36 +307,60 @@ export async function refreshDeviceTokens(
);
}
const tokens = (await res.json()) as RawTokenResponse;
return {
accessToken: tokens.access_token,
refreshToken: tokens.refresh_token ?? refreshToken,
expiresAt: new Date(Date.now() + tokens.expires_in * 1000),
};
return buildCreds(tokens, refreshToken);
}
// ─── Backwards-compat shims so older callers keep compiling ──────────
// ─── Encrypted-storage codec ────────────────────────────────────────
//
// We re-purpose `codexAccessTokenEncrypted` to store the entire creds
// blob (access_token + accountId + expiresAt) JSON-encoded then
// encrypted. `codexRefreshTokenEncrypted` continues to hold just the
// refresh token (encrypted), so a future migration to a dedicated
// column shape stays simple. Old v1.4.9-v1.4.11 tokens stored without
// the accountId field will fail to parse — the user has to re-connect
// once after the v1.4.12 deploy. Acceptable since those connects
// never produced a working session anyway.
/** @deprecated use `encryptCodexCreds`. */
export function encryptTokens(t: {
interface StoredAccess {
accessToken: string;
refreshToken: string;
}): { accessEncrypted: string; refreshEncrypted: string } {
return {
accessEncrypted: encrypt(t.accessToken),
refreshEncrypted: encrypt(t.refreshToken),
};
accountId: string;
expiresAt: string; // ISO
}
/** @deprecated use `decryptCodexCreds`. */
export function decryptTokens(e: {
export function encryptCodexCreds(creds: CodexCreds): {
accessEncrypted: string;
refreshEncrypted: string;
}): { accessToken: string; refreshToken: string } {
} {
const stored: StoredAccess = {
accessToken: creds.accessToken,
accountId: creds.accountId,
expiresAt: creds.expiresAt.toISOString(),
};
return {
accessToken: decrypt(e.accessEncrypted),
refreshToken: decrypt(e.refreshEncrypted),
accessEncrypted: encrypt(JSON.stringify(stored)),
refreshEncrypted: encrypt(creds.refreshToken),
};
}
/** @deprecated alias for `refreshTokens`. */
export const refreshAccessToken = refreshTokens;
export function decryptCodexCreds(encrypted: {
accessEncrypted: string;
refreshEncrypted: string;
}): CodexCreds | null {
let parsed: StoredAccess;
try {
const raw = decrypt(encrypted.accessEncrypted);
parsed = JSON.parse(raw) as StoredAccess;
} catch {
// Pre-v1.4.12 stored format (raw access token / api key string).
// We cannot recover the accountId from that — caller must treat
// the connection as expired and ask the user to re-link.
return null;
}
if (!parsed.accessToken || !parsed.accountId) return null;
return {
accessToken: parsed.accessToken,
refreshToken: decrypt(encrypted.refreshEncrypted),
accountId: parsed.accountId,
expiresAt: new Date(parsed.expiresAt),
};
}
+63 -37
View File
@@ -1,11 +1,15 @@
import type { AIProvider, CompletionParams, CompletionResult } from "./types";
import { prisma } from "@/lib/db";
import { decrypt, encrypt } from "@/lib/crypto";
import { decrypt } from "@/lib/crypto";
import { CodexClient } from "./codex-client";
import { OpenAIClient } from "./openai-client";
import { AnthropicClient } from "./anthropic-client";
import { LocalOpenAICompatibleClient } from "./local-client";
import { refreshDeviceTokens } from "./codex-oauth";
import {
refreshDeviceTokens,
encryptCodexCreds,
decryptCodexCreds,
} from "./codex-oauth";
import { isPublicUrl } from "@/lib/validations/notifications";
const TOKEN_REFRESH_BUFFER_MS = 5 * 60 * 1000; // 5 minutes
@@ -102,20 +106,18 @@ async function resolveAdminProvider(): Promise<AIProvider> {
}
/**
* Codex flow (v1.4.8+, device-code path): the user authorised against
* chatgpt.com/codex/device, we store the resulting OAuth access_token
* (encrypted) in `codexAccessTokenEncrypted` and the refresh_token in
* `codexRefreshTokenEncrypted`. The active token is forwarded to
* `https://chatgpt.com/backend-api/codex/responses` — the same Codex
* backend the official CLI uses, which bills against the user's
* ChatGPT subscription. When the access token is about to expire we
* refresh it transparently using the long-lived refresh token.
* Codex provider — device-code path.
*
* Note: v1.4.7's authorisation-code-flow path also wrote to these
* columns, but stored an OpenAI API key (post-RFC-8693 exchange)
* rather than an OAuth access token. That path is dead in production
* because Hydra rejects our redirect URI. The device-code path is the
* canonical replacement.
* `codexAccessTokenEncrypted` stores an encrypted JSON blob with the
* OAuth access token AND the `chatgpt_account_id` claim from the
* id_token (the latter is mandatory in the `ChatGPT-Account-ID`
* header). `codexRefreshTokenEncrypted` continues to hold just the
* refresh token. See `codex-oauth.ts` for the storage codec.
*
* Old v1.4.7-v1.4.11 rows that stored a raw token string instead of
* the JSON envelope cannot be revived (the account id was never
* captured), so `decryptCodexCreds` returns null and we treat the
* connection as expired — the user re-runs the connect flow once.
*/
async function resolveCodexProvider(
userId: string,
@@ -125,7 +127,6 @@ async function resolveCodexProvider(
select: {
codexAccessTokenEncrypted: true,
codexRefreshTokenEncrypted: true,
codexTokenExpiresAt: true,
codexConnectionStatus: true,
},
});
@@ -138,54 +139,79 @@ async function resolveCodexProvider(
return null;
}
const storedAccessToken = decrypt(user.codexAccessTokenEncrypted);
const storedRefreshToken = decrypt(user.codexRefreshTokenEncrypted);
const stored = decryptCodexCreds({
accessEncrypted: user.codexAccessTokenEncrypted,
refreshEncrypted: user.codexRefreshTokenEncrypted,
});
if (!stored) {
// Pre-v1.4.12 record without account_id — the token cannot
// satisfy the ChatGPT-Account-ID header. Mark the row as
// disconnected so the UI prompts the user to re-link.
await prisma.user.update({
where: { id: userId },
data: { codexConnectionStatus: "expired" },
});
return null;
}
let activeAccessToken = storedAccessToken;
let active = stored;
// Proactive refresh if the access token expires within 5 minutes.
const expiresAt = user.codexTokenExpiresAt?.getTime() ?? 0;
if (expiresAt < Date.now() + TOKEN_REFRESH_BUFFER_MS) {
// Proactive refresh if the access token is within 5 min of expiry.
if (stored.expiresAt.getTime() < Date.now() + TOKEN_REFRESH_BUFFER_MS) {
try {
const fresh = await refreshDeviceTokens(storedRefreshToken);
activeAccessToken = fresh.accessToken;
const fresh = await refreshDeviceTokens(stored.refreshToken);
active = fresh;
const enc = encryptCodexCreds(fresh);
await prisma.user.update({
where: { id: userId },
data: {
codexAccessTokenEncrypted: encrypt(fresh.accessToken),
codexRefreshTokenEncrypted: encrypt(fresh.refreshToken),
codexAccessTokenEncrypted: enc.accessEncrypted,
codexRefreshTokenEncrypted: enc.refreshEncrypted,
codexTokenExpiresAt: fresh.expiresAt,
},
});
} catch {
// Proactive refresh failures fall through — the CodexClient
// retries via onTokenRefresh on a real 401 and writes the
// updated tokens then.
// Fall through — CodexClient will trigger an on-401 refresh
// and persist there.
}
}
return new CodexClient({
accessToken: activeAccessToken,
accessToken: active.accessToken,
accountId: active.accountId,
onTokenRefresh: async () => {
const freshUser = await prisma.user.findUnique({
where: { id: userId },
select: { codexRefreshTokenEncrypted: true },
select: {
codexAccessTokenEncrypted: true,
codexRefreshTokenEncrypted: true,
},
});
if (!freshUser?.codexRefreshTokenEncrypted) {
if (
!freshUser?.codexAccessTokenEncrypted ||
!freshUser.codexRefreshTokenEncrypted
) {
throw new Error("No refresh token available");
}
const currentRefresh = decrypt(freshUser.codexRefreshTokenEncrypted);
const fresh = await refreshDeviceTokens(currentRefresh);
const decoded = decryptCodexCreds({
accessEncrypted: freshUser.codexAccessTokenEncrypted,
refreshEncrypted: freshUser.codexRefreshTokenEncrypted,
});
if (!decoded) {
throw new Error("Codex token storage corrupt; user must re-link");
}
const fresh = await refreshDeviceTokens(decoded.refreshToken);
const enc = encryptCodexCreds(fresh);
await prisma.user.update({
where: { id: userId },
data: {
codexAccessTokenEncrypted: encrypt(fresh.accessToken),
codexRefreshTokenEncrypted: encrypt(fresh.refreshToken),
codexAccessTokenEncrypted: enc.accessEncrypted,
codexRefreshTokenEncrypted: enc.refreshEncrypted,
codexTokenExpiresAt: fresh.expiresAt,
},
});
return fresh.accessToken;
return { accessToken: fresh.accessToken, accountId: fresh.accountId };
},
});
}