HealthLog/docs/integrations/ai-providers.md

AI providers

HealthLog's Coach, daily briefing, weekly report, and per-metric insights all run through a multi-provider AI client. Pick whichever provider you prefer — there is no lock-in. The provider is resolved per-user via the fallback chain in src/lib/ai/provider.ts and src/lib/ai/provider-chain.ts, so a user with a Claude key and an Ollama endpoint can fall back from one to the other automatically.

Provider matrix

Provider	Key acquisition	Default model	Endpoint	Privacy stance
OPENAI	https://platform.openai.com/api-keys	gpt-4o	https://api.openai.com/v1	Measurement context sent to OpenAI
ANTHROPIC	https://console.anthropic.com/settings/keys	claude-sonnet-4-6	Anthropic SDK default	Measurement context sent to Anthropic
LOCAL	None (your endpoint)	local-model	OpenAI-compatible URL you control	Stays on your network
CHATGPT_OAUTH	Sign in with your ChatGPT account	Codex-routed	chatgpt.com/backend-api/codex/responses	Routed via your ChatGPT subscription

The four spellings above are the canonical User.aiProvider enum values. Anything else falls through to the admin-shared OpenAI key (if configured) or the NoProvider stub that surfaces "No AI provider configured" to the UI.

BYOK vs admin-shared

HealthLog supports two key-provisioning modes that coexist:

• BYOK (Bring Your Own Key) — each user pastes their personal API key into /settings/ai. The key is AES-256-GCM-encrypted at rest and only decrypted in-process when the provider client initialises. Cost falls to the user; no shared budget.
• Admin-shared — the admin panel accepts a single OpenAI key in AppSettings.adminAiKeyEncrypted. Users who have not configured their own key fall back to the admin key. This is the last entry in the default provider chain so it acts as a safety net rather than the default route.

Single-user instances usually pick one mode and stop there. Multi-user instances tend to run BYOK with the admin-shared key as a fallback for users who skip the setup.

The fallback chain

User.aiProviderChain is a JSON array of providers tried in priority order. The default is encoded in src/lib/ai/provider-chain.ts:60-66:

1. codex — ChatGPT OAuth (covered by your ChatGPT Plus / Pro subscription; no per-token cost on top).
2. openai — user's own OpenAI API key.
3. anthropic — user's own Anthropic API key.
4. local — self-hosted Ollama / LM Studio / vLLM endpoint.
5. admin-openai — operator's shared key (last-ditch).

Entries the user has no credential for are silently skipped. The runner walks the surviving entries in order; the first to return a non-error completion wins. A 422 fires only when every entry in the resolved chain returned an error.

Edit the order in /settings/ai. Disable an entry without removing it by toggling its switch — a malformed chain on disk falls back to the default rather than 500-ing.

OpenAI setup

1. Sign in at https://platform.openai.com/api-keys.
2. Create new secret key. Scope: "Restricted → Model capabilities → Write" is sufficient for HealthLog's use of the Responses / Chat Completions APIs.
3. Copy the sk-… key.
4. In HealthLog, open /settings/ai. Set provider to OpenAI, paste the key, optionally override the model.

The default model is gpt-4o (src/lib/ai/provider.ts:80) — a full-size model that keeps the headline Insights assessments grounded and well-reasoned out of the box. Override to a lighter model such as gpt-4o-mini if you want to trade some reasoning headroom for a lower per-token cost.

Cost expectation. A daily briefing for an active account runs at roughly 4–8k input tokens + ~1k output. At gpt-4o's published rates that is a fraction of a US cent per briefing; a heavy user generating multiple Coach turns plus a weekly report typically spends a few dollars per month. A lighter override drops this further.

Anthropic setup

1. Sign in at https://console.anthropic.com/settings/keys.
2. Create Key. A workspace-scoped key is recommended over an organisation-scoped key.
3. Copy the sk-ant-… key.
4. In HealthLog, open /settings/ai. Set provider to Anthropic, paste the key, optionally override the model.

The default model is claude-sonnet-4-6 (src/lib/ai/provider.ts). The Anthropic client refuses to forward an Anthropic key to a stale aiBaseUrl even if one is parked in the shared column — Anthropic has no per-tenant base URL the UI exposes; the SDK default is the only correct endpoint.

Cost expectation. A current Sonnet runs at a higher per-token cost than a lightweight OpenAI model but produces noticeably stronger reasoning on the more structured Coach prompts. Most BYOK users on Claude spend in the low single digits per month.

Local endpoints (Ollama, LM Studio, vLLM)

Any OpenAI-compatible local endpoint plugs in as the Local provider. Data stays on your network end-to-end; nothing leaves the host running the model.

Ollama — ollama pull qwen2.5:14b-instruct && ollama serve. In /settings/ai: provider Local, base URL http://localhost:11434/v1 (Ollama exposes the OpenAI-compatible shim there), model qwen2.5:14b-instruct, leave the API key blank. If HealthLog runs in Docker and Ollama runs on the host, swap localhost for host.docker.internal (Mac/Windows) or the host's LAN IP (Linux).

LM Studio — start the built-in server from the Local Server tab (usually http://localhost:1234/v1) and paste the URL into HealthLog the same way.

vLLM — production-grade shared inference:

vllm serve Qwen/Qwen2.5-14B-Instruct \
  --host 0.0.0.0 --port 8000 --api-key your-shared-secret

Base URL http://<vllm-host>:8000/v1, API key your-shared-secret, model Qwen/Qwen2.5-14B-Instruct. The local-client encrypts the API key at rest like every other provider credential.

SSRF guard. Local endpoints route through the same validation as every other outbound URL HealthLog hits. The guard rejects private-IP-range targets by default. ALLOW_LOCAL_AI_PRIVATE_HOSTS opens the escape hatch and accepts two forms: a comma-separated host allowlist (e.g. ALLOW_LOCAL_AI_PRIVATE_HOSTS=ollama.lan,10.0.0.5), which permits only those exact hostnames, or the legacy ALLOW_LOCAL_AI_PRIVATE_HOSTS=true, which permits any private host — including the cloud-metadata endpoint, so prefer the host list. Leave it unset on public-facing instances unless you specifically run an internal endpoint.

Model sizing. Roughly: 7-8B for 8 GB GPU / Apple Silicon base (briefings fine, deeper Coach reasoning gets terse), 14-24B for 16-24 GB GPU / M-series Pro/Max (sweet spot for BYOK-style usage), 70B+ for 48 GB+ / multi-GPU (comparable to mid-tier hosted models).

ChatGPT OAuth (Codex)

The CHATGPT_OAUTH provider routes generations via the chatgpt.com/backend-api/codex/responses endpoint that the official Codex CLI uses. Users authenticate with their existing ChatGPT Plus or Pro subscription via device-code OAuth — no API key, no per-token billing on top of the subscription.

UI setup:

1. In /settings/ai, set provider to ChatGPT OAuth.
2. Click Connect ChatGPT. HealthLog calls POST /api/auth/codex/device-start to receive a user code and a verification URL.
3. The UI displays the verification URL and the short user code. Open the URL in any browser signed in to your ChatGPT account and enter the user code.
4. HealthLog polls POST /api/auth/codex/device-poll until ChatGPT confirms the grant. Access and refresh tokens land in User.codexAccessTokenEncrypted / codexRefreshTokenEncrypted, AES-256-GCM-encrypted at rest. The connection status flips to connected.
5. Token refresh runs automatically; the client triggers a refresh on 401 and persists the new tokens transparently (src/lib/ai/provider.ts:122-217).

Disconnect cleanly from /settings/ai — POST /api/auth/codex/disconnect clears the encrypted token columns. The wire-level protocol is documented in docs/codex-protocol-spec.md for anyone who wants to audit the request shape; users who just want the integration to work do not need to read it.

Privacy stance

• OpenAI and Anthropic see the prompt + structured snapshot HealthLog assembles per generation (recent metrics, target ranges, Coach conversation history). No raw row exports leave the app, but values themselves are visible to the provider for the request's duration. Provider retention policy applies.
• LOCAL keeps everything on your network. Snapshots reach only the endpoint you configured. With Ollama on the same host as HealthLog, nothing leaves the host at all.
• CHATGPT_OAUTH routes via OpenAI's Codex endpoint. Same caveat as OpenAI; the difference is the billing axis (ChatGPT subscription instead of API tokens).

The Coach drawer surfaces a "Local" badge when generation resolves to the local provider, so users who pinned a local endpoint can see at a glance that their context stayed on the host.

Caching contract

Two caches sit in front of every provider call so generation budgets stay predictable:

1. 24-hour insight cache. A successful generation is keyed by the upstream measurement snapshot's hash and the prompt version. The next read within 24 hours of the same snapshot returns the cached envelope instead of re-billing the provider. Force-regen on the UI bypasses the cache.
2. Snapshot LRU. The Coach snapshot builder caches per (userId, scope) for 60 seconds so a rapid-fire conversation reuses the same upstream context without re-querying Postgres. See src/lib/ai/coach/snapshot.ts.

INSIGHTS_RATE_LIMIT_PER_HOUR (default 10) bounds the number of POST /api/insights/generate calls a single user can fire per hour. Lower it when running on a tight budget — the 24h cache already short-circuits read traffic; only force-regens and cache-misses cost tokens.

Connection test

POST /api/ai/test runs a one-shot probe against the resolved provider and returns the latency in milliseconds plus the model the provider routed to. The Settings UI surfaces a Test connection button next to each provider section that fires this endpoint. A failed probe surfaces a localisable errorCode so the UI renders the failure in the user's language rather than the raw upstream error.

Troubleshooting

• "No AI provider configured." Either set up at least one BYOK provider in /settings/ai, or configure an admin-shared OpenAI key in the admin panel.
• Local endpoint rejected as "internal/private host". Set ALLOW_LOCAL_AI_PRIVATE_HOSTS to the endpoint's exact hostname (e.g. ollama.lan; comma-separate several) in the container environment, or to true to allow any private host. Prefer the host list; leave it unset on internet-facing instances unless you specifically want to allow private-range targets.
• Codex disconnects after a few weeks of inactivity. ChatGPT refresh tokens lapse after extended idle periods. Reconnect via the Connect ChatGPT button — HealthLog re-runs the device-code flow and the new tokens land in the same encrypted columns.
• Chain falls through every entry. Run the per-provider connection test from /settings/ai to identify which step is failing. The most common cause is a stale or revoked BYOK key earlier in the chain; remove or disable that entry to skip ahead.