HealthLog/docs/integrations/ai-providers.md

# AI providers

HealthLog's Coach, daily briefing, weekly report, and per-metric
insights all run through a multi-provider AI client. Pick whichever
provider you prefer — there is no lock-in. The provider is resolved
per-user via the fallback chain in `src/lib/ai/provider.ts` and
`src/lib/ai/provider-chain.ts`, so a user with a Claude key and an
Ollama endpoint can fall back from one to the other automatically.

## Provider matrix

| Provider        | Key acquisition                               | Default model       | Endpoint                                  | Privacy stance                        |
| --------------- | --------------------------------------------- | ------------------- | ----------------------------------------- | ------------------------------------- |
| `OPENAI`        | <https://platform.openai.com/api-keys>        | `gpt-4o`            | `https://api.openai.com/v1`               | Measurement context sent to OpenAI    |
| `ANTHROPIC`     | <https://console.anthropic.com/settings/keys> | `claude-sonnet-4-6` | Anthropic SDK default                     | Measurement context sent to Anthropic |
| `LOCAL`         | None (your endpoint)                          | `local-model`       | OpenAI-compatible URL you control         | Stays on your network                 |
| `CHATGPT_OAUTH` | Sign in with your ChatGPT account             | Codex-routed        | `chatgpt.com/backend-api/codex/responses` | Routed via your ChatGPT subscription  |

The four spellings above are the canonical `User.aiProvider` enum
values. Anything else falls through to the admin-shared OpenAI key
(if configured) or the `NoProvider` stub that surfaces "No AI
provider configured" to the UI.

## BYOK vs admin-shared

HealthLog supports two key-provisioning modes that coexist:

- **BYOK (Bring Your Own Key)** — each user pastes their personal
  API key into `/settings/ai`. The key is AES-256-GCM-encrypted at
  rest and only decrypted in-process when the provider client
  initialises. Cost falls to the user; no shared budget.
- **Admin-shared** — the admin panel accepts a single OpenAI key in
  `AppSettings.adminAiKeyEncrypted`. Users who have not configured
  their own key fall back to the admin key. This is the last entry
  in the default provider chain so it acts as a safety net rather
  than the default route. Since v1.28.14 the operator can also share
  a subscription-based account that users opt into — see
  [Admin-shared subscription access](#admin-shared-subscription-access).

Single-user instances usually pick one mode and stop there.
Multi-user instances tend to run BYOK with the admin-shared key as a
fallback for users who skip the setup.

## The fallback chain

`User.aiProviderChain` is a JSON array of providers tried in priority
order. The default is encoded in
`src/lib/ai/provider-chain.ts:60-66`:

1. `codex` — ChatGPT OAuth (covered by your ChatGPT Plus / Pro
   subscription; no per-token cost on top).
2. `openai` — user's own OpenAI API key.
3. `anthropic` — user's own Anthropic API key.
4. `local` — self-hosted Ollama / LM Studio / vLLM endpoint.
5. `admin-openai` — operator's shared key (last-ditch).

Entries the user has no credential for are silently skipped. The
runner walks the surviving entries in order; the first to return a
non-error completion wins. A 422 fires only when every entry in the
resolved chain returned an error.

Edit the order in `/settings/ai`. Disable an entry without removing
it by toggling its switch — a malformed chain on disk falls back to
the default rather than 500-ing.

## OpenAI setup

1. Sign in at <https://platform.openai.com/api-keys>.
2. **Create new secret key.** Scope: "Restricted → Model capabilities
   → Write" is sufficient for HealthLog's use of the Responses /
   Chat Completions APIs.
3. Copy the `sk-…` key.
4. In HealthLog, open `/settings/ai`. Set provider to **OpenAI**,
   paste the key, optionally override the model.

The default model is `gpt-4o` (`src/lib/ai/provider.ts:80`) — a
full-size model that keeps the headline Insights assessments grounded
and well-reasoned out of the box. Override to a lighter model such as
`gpt-4o-mini` if you want to trade some reasoning headroom for a lower
per-token cost.

**Cost expectation.** A daily briefing for an active account runs at
roughly 4–8k input tokens + ~1k output. At gpt-4o's published rates
that is a fraction of a US cent per briefing; a heavy user generating
multiple Coach turns plus a weekly report typically spends a few
dollars per month. A lighter override drops this further.

## Anthropic setup

1. Sign in at <https://console.anthropic.com/settings/keys>.
2. **Create Key.** A workspace-scoped key is recommended over an
   organisation-scoped key.
3. Copy the `sk-ant-…` key.
4. In HealthLog, open `/settings/ai`. Set provider to **Anthropic**,
   paste the key, optionally override the model.

The default model is `claude-sonnet-4-6` (`src/lib/ai/provider.ts`).
The Anthropic client refuses to forward an Anthropic key to a stale
`aiBaseUrl` even if one is parked in the shared column — Anthropic
has no per-tenant base URL the UI exposes; the SDK default is the
only correct endpoint.

**Cost expectation.** A current Sonnet runs at a higher per-token
cost than a lightweight OpenAI model but produces noticeably stronger
reasoning on the more structured Coach prompts. Most BYOK users on
Claude spend in the low single digits per month.

## Local endpoints (Ollama, LM Studio, vLLM)

Any OpenAI-compatible local endpoint plugs in as the **Local**
provider. Data stays on your network end-to-end; nothing leaves the
host running the model.

**Ollama** — `ollama pull qwen2.5:14b-instruct && ollama serve`.
In `/settings/ai`: provider **Local**, base URL
`http://localhost:11434/v1` (Ollama exposes the OpenAI-compatible
shim there), model `qwen2.5:14b-instruct`, leave the API key blank.
If HealthLog runs in Docker and Ollama runs on the host, swap
`localhost` for `host.docker.internal` (Mac/Windows) or the host's
LAN IP (Linux).

**LM Studio** — start the built-in server from the **Local Server**
tab (usually `http://localhost:1234/v1`) and paste the URL into
HealthLog the same way.

**vLLM** — production-grade shared inference:

```bash
vllm serve Qwen/Qwen2.5-14B-Instruct \
  --host 0.0.0.0 --port 8000 --api-key your-shared-secret
```

Base URL `http://<vllm-host>:8000/v1`, API key `your-shared-secret`,
model `Qwen/Qwen2.5-14B-Instruct`. The local-client encrypts the
API key at rest like every other provider credential.

**SSRF guard.** Local endpoints route through the same validation
as every other outbound URL HealthLog hits. The guard rejects
private-IP-range targets by default. `ALLOW_LOCAL_AI_PRIVATE_HOSTS`
opens the escape hatch and accepts two forms: a comma-separated host
allowlist (e.g. `ALLOW_LOCAL_AI_PRIVATE_HOSTS=ollama.lan,10.0.0.5`),
which permits only those exact hostnames, or the legacy
`ALLOW_LOCAL_AI_PRIVATE_HOSTS=true`, which permits any private host —
including the cloud-metadata endpoint, so prefer the host list. Leave
it unset on public-facing instances unless you specifically run an
internal endpoint.

**Model sizing.** Roughly: 7-8B for 8 GB GPU / Apple Silicon base
(briefings fine, deeper Coach reasoning gets terse), 14-24B for
16-24 GB GPU / M-series Pro/Max (sweet spot for BYOK-style usage),
70B+ for 48 GB+ / multi-GPU (comparable to mid-tier hosted models).

### OpenAI-compatible gateways (LiteLLM, OpenRouter, …)

The **Local** provider is also the gateway path: any service that
speaks the OpenAI `/v1/chat/completions` wire plugs in the same way,
whether it runs on your host or not. The OPENAI provider deliberately
pins `api.openai.com` (an OpenAI key is never forwarded to a custom
host), so a gateway always goes through **Local**:

1. In `/settings/ai`, set provider to **Local (OpenAI-compatible)**.
2. Base URL is the gateway's OpenAI-compatible root and **ends with
   `/v1`** — e.g. `https://litellm.example.com/v1` or
   `https://openrouter.ai/api/v1`.
3. API key is **optional** — set it when the gateway requires a
   Bearer token (OpenRouter key, LiteLLM master key), leave it blank
   otherwise. It is encrypted at rest like every other credential.
4. Model is whatever the gateway routes — e.g.
   `anthropic/claude-sonnet-4-6` on OpenRouter or a LiteLLM alias.

JSON surfaces send the standard
`response_format: { type: "json_object" }`; an endpoint that rejects
the field is detected on the first refusal and retried without it,
so strict gateways and older local servers both work unmodified.

The **admin** server-key path has a separate guard: an admin-set
custom base URL must additionally be allowed via
`ADMIN_AI_BASE_URL_ALLOWLIST` in the container environment. The
user-level Local provider does not read that variable — it is
governed by the SSRF guard above.

## ChatGPT OAuth (Codex)

The `CHATGPT_OAUTH` provider routes generations via the
`chatgpt.com/backend-api/codex/responses` endpoint that the official
Codex CLI uses. Users authenticate with their existing ChatGPT Plus
or Pro subscription via device-code OAuth — no API key, no
per-token billing on top of the subscription.

UI setup:

1. In `/settings/ai`, set provider to **ChatGPT OAuth**.
2. Click **Connect ChatGPT.** HealthLog calls
   `POST /api/auth/codex/device-start` to receive a user code and a
   verification URL.
3. The UI displays the verification URL and the short user code.
   Open the URL in any browser signed in to your ChatGPT account
   and enter the user code.
4. HealthLog polls `POST /api/auth/codex/device-poll` until ChatGPT
   confirms the grant. Access and refresh tokens land in
   `User.codexAccessTokenEncrypted` / `codexRefreshTokenEncrypted`,
   AES-256-GCM-encrypted at rest. The connection status flips to
   `connected`.
5. Token refresh runs automatically; the client triggers a refresh
   on 401 and persists the new tokens transparently
   (`src/lib/ai/provider.ts:122-217`).

Disconnect cleanly from `/settings/ai` — `POST /api/auth/codex/disconnect`
clears the encrypted token columns. The wire-level protocol is
documented in `docs/codex-protocol-spec.md` for anyone who wants to
audit the request shape; users who just want the integration to
work do not need to read it.

## Admin-shared subscription access

Besides the shared API key, the operator can connect a single
subscription-based AI account for the whole instance — an
operator-shared subscription provider that users without any
personal AI setup can opt into.

- **Admin side.** Under **Admin → AI** the operator runs the same
  device-code OAuth flow as the per-user connection
  (`POST /api/admin/central-codex/device-start` +
  `device-poll`). The tokens land AES-256-GCM-encrypted in the
  `AppSettings` singleton and are never returned by any endpoint —
  status and disconnect go through `GET` / `DELETE
/api/admin/central-codex`, which sit behind the cookie-only
  `requireAdmin()` boundary.
- **User side.** Each user opts in individually in `/settings/ai`
  (`PATCH /api/auth/me/use-central-codex`, body
  `{ "useCentralCodex": boolean }`). The opt-in is off by default.
- **Chain semantics.** The shared connection is the internal
  `admin-codex` chain entry. It is never part of the persisted chain
  and never offered in the chain editor — it is appended **last** to
  the resolved chain, and only when the user opted in AND the
  operator has connected the account
  (`src/lib/ai/provider-chain.ts`). A hand-crafted persisted entry
  resolves to nothing.
- **Cost.** Generations through the shared connection run under the
  operator's account, so usage and rate limits land on the operator,
  not the user.

The settings UI states this plainly: the shared access is a single
signed-in AI account the operator connected for everyone on the
server — not the user's own key. Users who want their data to reach
only a provider under their own contract should configure a personal
provider instead; their own entries always resolve first.

## Privacy stance

- **OpenAI** and **Anthropic** see the prompt + structured snapshot
  HealthLog assembles per generation (recent metrics, target ranges,
  Coach conversation history). No raw row exports leave the app, but
  values themselves are visible to the provider for the request's
  duration. Provider retention policy applies.
- **LOCAL** keeps everything on your network. Snapshots reach only
  the endpoint you configured. With Ollama on the same host as
  HealthLog, nothing leaves the host at all.
- **CHATGPT_OAUTH** routes via OpenAI's Codex endpoint. Same caveat
  as OpenAI; the difference is the billing axis (ChatGPT
  subscription instead of API tokens).

The Coach drawer surfaces a "Local" badge when generation resolves
to the local provider, so users who pinned a local endpoint can see
at a glance that their context stayed on the host.

The document vault is local-first by default: an uploaded document is
only read by an external AI provider when you tap a per-document AI
action, and every text-layer PDF is indexed for search locally with no
egress at all — **unless you opt into automatic AI reading in AI
settings** (next section).

## Automatic document reading (opt-in)

**Off by default.** With the switch off, nothing about the vault
changes: a stored document is read externally only when you explicitly
tap "Read with AI" on it, and that tap still needs an active AI-consent
receipt for any provider that leaves the machine. Text-layer PDFs stay
searchable through local extraction with no egress.

Turn **"Read documents automatically with AI"** on (AI settings) and
each document you upload is read, described, and indexed by your
configured AI provider with no per-document confirmation — the
"upload and it just gets read" flow. What that means, plainly:

- **The document's contents leave this machine** to your configured
  provider. The read happens on the server, right after upload.
- **A subscription connection** — a signed-in AI account rather than an
  API key — applies its own consumer data settings. **Subscription
  providers may use the content to improve their models, and no
  data-processing agreement covers it.** Turning the toggle on is the
  standing consent for that trade; it is recorded in the consent audit
  trail.
- **A self-hosted local model never egresses.** Documents read by a
  local provider stay on your server whether the toggle is on or off.
- **Scanned / image-only PDFs** are rendered to page images server-side
  so a provider that only accepts images (a subscription connection) can
  read them too; a native-PDF provider reads the PDF directly. Only the
  first pages of a long document are sent, to bound cost.

Regulated or privacy-sensitive deployments should either leave this off
and read documents per-document on demand, or configure a **BYOK
no-training API key** or a **local model** as the provider — both avoid
the subscription-training trade. The setting is per-user: in a
multi-user deployment each account opts in for its own documents.

## What the Coach is — and isn't

The Coach is an informational assistant that works from your own
logged data. It is not a doctor, what it says is not medical advice,
and it does not diagnose, prescribe, or change treatment — those
boundaries are written into its system prompt as safety contracts, not
left to chance. It describes patterns observationally ("your resting
heart rate has been rising over 14 days"), never as a verdict about a
named condition, and it defers diagnosis, dose, and drug-interaction
questions to a healthcare professional. This is HealthLog's
project-wide self-description standard: a personal health
record-keeping and wellness tool, not a medical device. The same
non-diagnostic framing governs the daily briefing, the insights, and
every surface that turns your data into words.

The Coach can also reach out proactively — a calm, once-a-day morning
check-in when there is something genuine to surface (a recovery trend
easing off, blood pressure running a touch high). The nudge is warm
and localized, references the focus by category rather than quoting
your own words back, and stays silent when there is nothing real to
say or when you have engaged the Coach recently. It is an invitation,
never a streak-guilt prompt, and it inherits the same non-diagnostic
boundaries as the chat.

## Caching contract

Two caches sit in front of every provider call so generation budgets
stay predictable:

1. **24-hour insight cache.** A successful generation is keyed by
   the upstream measurement snapshot's hash and the prompt version.
   The next read within 24 hours of the same snapshot returns the
   cached envelope instead of re-billing the provider. Force-regen
   on the UI bypasses the cache.
2. **Snapshot LRU.** The Coach snapshot builder caches per
   `(userId, scope)` for 60 seconds so a rapid-fire conversation
   reuses the same upstream context without re-querying Postgres.
   See `src/lib/ai/coach/snapshot.ts`.

`INSIGHTS_RATE_LIMIT_PER_HOUR` (default `10`) bounds the number of
`POST /api/insights/generate` calls a single user can fire per hour.
Lower it when running on a tight budget — the 24h cache already
short-circuits read traffic; only force-regens and cache-misses cost
tokens.

`INSIGHTS_MAX_TOKENS` (default `2500`, clamped to 500–8000) sets the
output-token ceiling for the daily-briefing generation. Raise it if a
verbose model gets its briefing cut off mid-JSON — the API reports
that case as "AI response was cut off" rather than the generic
invalid-JSON error.

## Connection test

`POST /api/ai/test` runs a one-shot probe against the resolved
provider and returns the latency in milliseconds plus the model the
provider routed to. The Settings UI surfaces a **Test connection**
button next to each provider section that fires this endpoint. A
failed probe surfaces a localisable `errorCode` so the UI renders
the failure in the user's language rather than the raw upstream
error.

## Troubleshooting

- **"No AI provider configured."** Either set up at least one BYOK
  provider in `/settings/ai`, or configure an admin-shared OpenAI
  key in the admin panel.
- **Local endpoint rejected as "internal/private host".** Set
  `ALLOW_LOCAL_AI_PRIVATE_HOSTS` to the endpoint's exact hostname
  (e.g. `ollama.lan`; comma-separate several) in the container
  environment, or to `true` to allow any private host. Prefer the host
  list; leave it unset on internet-facing instances unless you
  specifically want to allow private-range targets.
- **Codex disconnects after a few weeks of inactivity.** ChatGPT
  refresh tokens lapse after extended idle periods. Reconnect via
  the **Connect ChatGPT** button — HealthLog re-runs the device-code
  flow and the new tokens land in the same encrypted columns.
- **Chain falls through every entry.** Run the per-provider connection
  test from `/settings/ai` to identify which step is failing. The
  most common cause is a stale or revoked BYOK key earlier in the
  chain; remove or disable that entry to skip ahead.