mirror of
https://github.com/MBombeck/HealthLog.git
synced 2026-11-08 12:32:43 +00:00
docs(integrations): Apple Health, Withings, AI providers guides
Three end-to-end integration walkthroughs closing the docs gaps the
v1.4.34.3 docs-depth audit flagged as critical user-facing surfaces.
Apple Health (docs/integrations/apple-health.md, 200 lines):
- Generating the iOS `export.zip` from the Health app.
- 1.5 GB upload cap, Zip64-aware streaming parser, content-hash
idempotency.
- Per-record / per-workout / per-correlation / per-clinical-record
coverage matrix grounded in the parser's source.
- Cumulative HK-type daily aggregation contract
(`stats:<HKType>:<YYYY-MM-DD>` external IDs).
- Live status-polling endpoint and the admin-import variant.
- Source-priority interaction with the existing Withings ladder.
Withings (docs/integrations/withings.md, 195 lines):
- Developer-portal app registration with the right OAuth scopes
(`user.metrics user.activity`) and redirect URI.
- Env-var layout (`WITHINGS_CLIENT_ID`, `WITHINGS_CLIENT_SECRET`,
`WITHINGS_WEBHOOK_SECRET`).
- Why the webhook URL carries the secret as a path segment
(v1.4.25 W17a — reverse proxies log `query_string` by default;
path segments aren't logged, so the secret stays off the access
log).
- Sync coverage (weight, body composition, BP, sleep, activity,
SpO₂), reauth handling, troubleshooting.
AI providers (docs/integrations/ai-providers.md, 244 lines):
- All four `User.aiProvider` values (OPENAI, ANTHROPIC, LOCAL,
CHATGPT_OAUTH).
- Default models, BYOK vs admin-shared, the provider fallback chain.
- Local-endpoint setup with Ollama, LM Studio, and vLLM examples;
SSRF-guard rationale; ChatGPT device-code OAuth pointer.
- Privacy stance — local endpoints keep measurement context inside
your network; hosted endpoints send the snapshot to the provider.
This commit is contained in:
1 parent
c81ddbdbc8
commit
70788895fa
3 files changed
+639
No files matched your search
@@ -0,0 +1,244 @@
|
||||
# AI providers
|
||||
|
||||
HealthLog's Coach, daily briefing, weekly report, and per-metric
|
||||
insights all run through a multi-provider AI client. Pick whichever
|
||||
provider you prefer — there is no lock-in. The provider is resolved
|
||||
per-user via the fallback chain in `src/lib/ai/provider.ts` and
|
||||
`src/lib/ai/provider-chain.ts`, so a user with a Claude key and an
|
||||
Ollama endpoint can fall back from one to the other automatically.
|
||||
|
||||
## Provider matrix
|
||||
|
||||
| Provider | Key acquisition | Default model | Endpoint | Privacy stance |
|
||||
| -------- | ---------------- | ------------- | -------- | -------------- |
|
||||
| `OPENAI` | <https://platform.openai.com/api-keys> | `gpt-4o-mini` | `https://api.openai.com/v1` | Measurement context sent to OpenAI |
|
||||
| `ANTHROPIC` | <https://console.anthropic.com/settings/keys> | `claude-3-5-sonnet-latest` | Anthropic SDK default | Measurement context sent to Anthropic |
|
||||
| `LOCAL` | None (your endpoint) | `local-model` | OpenAI-compatible URL you control | Stays on your network |
|
||||
| `CHATGPT_OAUTH` | Sign in with your ChatGPT account | Codex-routed | `chatgpt.com/backend-api/codex/responses` | Routed via your ChatGPT subscription |
|
||||
|
||||
The four spellings above are the canonical `User.aiProvider` enum
|
||||
values. Anything else falls through to the admin-shared OpenAI key
|
||||
(if configured) or the `NoProvider` stub that surfaces "No AI
|
||||
provider configured" to the UI.
|
||||
|
||||
## BYOK vs admin-shared
|
||||
|
||||
HealthLog supports two key-provisioning modes that coexist:
|
||||
|
||||
- **BYOK (Bring Your Own Key)** — each user pastes their personal
|
||||
API key into `/settings/ai`. The key is AES-256-GCM-encrypted at
|
||||
rest and only decrypted in-process when the provider client
|
||||
initialises. Cost falls to the user; no shared budget.
|
||||
- **Admin-shared** — the admin panel accepts a single OpenAI key in
|
||||
`AppSettings.adminAiKeyEncrypted`. Users who have not configured
|
||||
their own key fall back to the admin key. This is the last entry
|
||||
in the default provider chain so it acts as a safety net rather
|
||||
than the default route.
|
||||
|
||||
Single-user instances usually pick one mode and stop there.
|
||||
Multi-user instances tend to run BYOK with the admin-shared key as a
|
||||
fallback for users who skip the setup.
|
||||
|
||||
## The fallback chain
|
||||
|
||||
`User.aiProviderChain` is a JSON array of providers tried in priority
|
||||
order. The default is encoded in
|
||||
`src/lib/ai/provider-chain.ts:60-66`:
|
||||
|
||||
1. `codex` — ChatGPT OAuth (covered by your ChatGPT Plus / Pro
|
||||
subscription; no per-token cost on top).
|
||||
2. `openai` — user's own OpenAI API key.
|
||||
3. `anthropic` — user's own Anthropic API key.
|
||||
4. `local` — self-hosted Ollama / LM Studio / vLLM endpoint.
|
||||
5. `admin-openai` — operator's shared key (last-ditch).
|
||||
|
||||
Entries the user has no credential for are silently skipped. The
|
||||
runner walks the surviving entries in order; the first to return a
|
||||
non-error completion wins. A 422 fires only when every entry in the
|
||||
resolved chain returned an error.
|
||||
|
||||
Edit the order in `/settings/ai`. Disable an entry without removing
|
||||
it by toggling its switch — a malformed chain on disk falls back to
|
||||
the default rather than 500-ing.
|
||||
|
||||
## OpenAI setup
|
||||
|
||||
1. Sign in at <https://platform.openai.com/api-keys>.
|
||||
2. **Create new secret key.** Scope: "Restricted → Model capabilities
|
||||
→ Write" is sufficient for HealthLog's use of the Responses /
|
||||
Chat Completions APIs.
|
||||
3. Copy the `sk-…` key.
|
||||
4. In HealthLog, open `/settings/ai`. Set provider to **OpenAI**,
|
||||
paste the key, optionally override the model.
|
||||
|
||||
The default model is `gpt-4o-mini` (`src/lib/ai/provider.ts:80`) —
|
||||
cheap, fast, and grounded enough for the structured-output Coach
|
||||
prompts. Override to `gpt-4o` or `gpt-4.1` if you want more reasoning
|
||||
headroom and accept the per-token cost.
|
||||
|
||||
**Cost expectation.** A daily briefing for an active account runs at
|
||||
roughly 4–8k input tokens + ~1k output. At gpt-4o-mini's published
|
||||
rates that is well under a US cent per briefing; a heavy user
|
||||
generating multiple Coach turns plus a weekly report typically spends
|
||||
a few dollars per month.
|
||||
|
||||
## Anthropic setup
|
||||
|
||||
1. Sign in at <https://console.anthropic.com/settings/keys>.
|
||||
2. **Create Key.** A workspace-scoped key is recommended over an
|
||||
organisation-scoped key.
|
||||
3. Copy the `sk-ant-…` key.
|
||||
4. In HealthLog, open `/settings/ai`. Set provider to **Anthropic**,
|
||||
paste the key, optionally override the model.
|
||||
|
||||
The default model is `claude-3-5-sonnet-latest` (`src/lib/ai/provider.ts:55`).
|
||||
The Anthropic client refuses to forward an Anthropic key to a stale
|
||||
`aiBaseUrl` even if one is parked in the shared column — Anthropic
|
||||
has no per-tenant base URL the UI exposes; the SDK default is the
|
||||
only correct endpoint.
|
||||
|
||||
**Cost expectation.** Claude 3.5 Sonnet runs roughly 5x the per-token
|
||||
cost of `gpt-4o-mini` but produces noticeably stronger reasoning on
|
||||
the more structured Coach prompts. Most BYOK users on Claude spend
|
||||
in the low single digits per month.
|
||||
|
||||
## Local endpoints (Ollama, LM Studio, vLLM)
|
||||
|
||||
Any OpenAI-compatible local endpoint plugs in as the **Local**
|
||||
provider. Data stays on your network end-to-end; nothing leaves the
|
||||
host running the model.
|
||||
|
||||
**Ollama** — `ollama pull qwen2.5:14b-instruct && ollama serve`.
|
||||
In `/settings/ai`: provider **Local**, base URL
|
||||
`http://localhost:11434/v1` (Ollama exposes the OpenAI-compatible
|
||||
shim there), model `qwen2.5:14b-instruct`, leave the API key blank.
|
||||
If HealthLog runs in Docker and Ollama runs on the host, swap
|
||||
`localhost` for `host.docker.internal` (Mac/Windows) or the host's
|
||||
LAN IP (Linux).
|
||||
|
||||
**LM Studio** — start the built-in server from the **Local Server**
|
||||
tab (usually `http://localhost:1234/v1`) and paste the URL into
|
||||
HealthLog the same way.
|
||||
|
||||
**vLLM** — production-grade shared inference:
|
||||
|
||||
```bash
|
||||
vllm serve Qwen/Qwen2.5-14B-Instruct \
|
||||
--host 0.0.0.0 --port 8000 --api-key your-shared-secret
|
||||
```
|
||||
|
||||
Base URL `http://<vllm-host>:8000/v1`, API key `your-shared-secret`,
|
||||
model `Qwen/Qwen2.5-14B-Instruct`. The local-client encrypts the
|
||||
API key at rest like every other provider credential.
|
||||
|
||||
**SSRF guard.** Local endpoints route through the same validation
|
||||
as every other outbound URL HealthLog hits. The guard rejects
|
||||
private-IP-range targets by default — set
|
||||
`ALLOW_LOCAL_AI_PRIVATE_HOSTS=true` in the container environment to
|
||||
allow `localhost`, RFC1918, and link-local destinations
|
||||
(`src/lib/ai/provider.ts:456-462`). Leave it unset on public-facing
|
||||
instances unless you specifically run an internal endpoint.
|
||||
|
||||
**Model sizing.** Roughly: 7-8B for 8 GB GPU / Apple Silicon base
|
||||
(briefings fine, deeper Coach reasoning gets terse), 14-24B for
|
||||
16-24 GB GPU / M-series Pro/Max (sweet spot for BYOK-style usage),
|
||||
70B+ for 48 GB+ / multi-GPU (comparable to mid-tier hosted models).
|
||||
|
||||
## ChatGPT OAuth (Codex)
|
||||
|
||||
The `CHATGPT_OAUTH` provider routes generations via the
|
||||
`chatgpt.com/backend-api/codex/responses` endpoint that the official
|
||||
Codex CLI uses. Users authenticate with their existing ChatGPT Plus
|
||||
or Pro subscription via device-code OAuth — no API key, no
|
||||
per-token billing on top of the subscription.
|
||||
|
||||
UI setup:
|
||||
|
||||
1. In `/settings/ai`, set provider to **ChatGPT OAuth**.
|
||||
2. Click **Connect ChatGPT.** HealthLog calls
|
||||
`POST /api/auth/codex/device-start` to receive a user code and a
|
||||
verification URL.
|
||||
3. The UI displays the verification URL and the short user code.
|
||||
Open the URL in any browser signed in to your ChatGPT account
|
||||
and enter the user code.
|
||||
4. HealthLog polls `POST /api/auth/codex/device-poll` until ChatGPT
|
||||
confirms the grant. Access and refresh tokens land in
|
||||
`User.codexAccessTokenEncrypted` / `codexRefreshTokenEncrypted`,
|
||||
AES-256-GCM-encrypted at rest. The connection status flips to
|
||||
`connected`.
|
||||
5. Token refresh runs automatically; the client triggers a refresh
|
||||
on 401 and persists the new tokens transparently
|
||||
(`src/lib/ai/provider.ts:122-217`).
|
||||
|
||||
Disconnect cleanly from `/settings/ai` — `POST /api/auth/codex/disconnect`
|
||||
clears the encrypted token columns. The wire-level protocol is
|
||||
documented in `docs/codex-protocol-spec.md` for anyone who wants to
|
||||
audit the request shape; users who just want the integration to
|
||||
work do not need to read it.
|
||||
|
||||
## Privacy stance
|
||||
|
||||
- **OpenAI** and **Anthropic** see the prompt + structured snapshot
|
||||
HealthLog assembles per generation (recent metrics, target ranges,
|
||||
Coach conversation history). No raw row exports leave the app, but
|
||||
values themselves are visible to the provider for the request's
|
||||
duration. Provider retention policy applies.
|
||||
- **LOCAL** keeps everything on your network. Snapshots reach only
|
||||
the endpoint you configured. With Ollama on the same host as
|
||||
HealthLog, nothing leaves the host at all.
|
||||
- **CHATGPT_OAUTH** routes via OpenAI's Codex endpoint. Same caveat
|
||||
as OpenAI; the difference is the billing axis (ChatGPT
|
||||
subscription instead of API tokens).
|
||||
|
||||
The Coach drawer surfaces a "Local" badge when generation resolves
|
||||
to the local provider, so users who pinned a local endpoint can see
|
||||
at a glance that their context stayed on the host.
|
||||
|
||||
## Caching contract
|
||||
|
||||
Two caches sit in front of every provider call so generation budgets
|
||||
stay predictable:
|
||||
|
||||
1. **24-hour insight cache.** A successful generation is keyed by
|
||||
the upstream measurement snapshot's hash and the prompt version.
|
||||
The next read within 24 hours of the same snapshot returns the
|
||||
cached envelope instead of re-billing the provider. Force-regen
|
||||
on the UI bypasses the cache.
|
||||
2. **Snapshot LRU.** The Coach snapshot builder caches per
|
||||
`(userId, scope)` for 60 seconds so a rapid-fire conversation
|
||||
reuses the same upstream context without re-querying Postgres.
|
||||
See `src/lib/ai/coach/snapshot.ts`.
|
||||
|
||||
`INSIGHTS_RATE_LIMIT_PER_HOUR` (default `10`) bounds the number of
|
||||
`POST /api/insights/generate` calls a single user can fire per hour.
|
||||
Lower it when running on a tight budget — the 24h cache already
|
||||
short-circuits read traffic; only force-regens and cache-misses cost
|
||||
tokens.
|
||||
|
||||
## Connection test
|
||||
|
||||
`POST /api/ai/test` runs a one-shot probe against the resolved
|
||||
provider and returns the latency in milliseconds plus the model the
|
||||
provider routed to. The Settings UI surfaces a **Test connection**
|
||||
button next to each provider section that fires this endpoint. A
|
||||
failed probe surfaces a localisable `errorCode` so the UI renders
|
||||
the failure in the user's language rather than the raw upstream
|
||||
error.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
- **"No AI provider configured."** Either set up at least one BYOK
|
||||
provider in `/settings/ai`, or configure an admin-shared OpenAI
|
||||
key in the admin panel.
|
||||
- **Local endpoint rejected as "internal/private host".** Set
|
||||
`ALLOW_LOCAL_AI_PRIVATE_HOSTS=true` in the container environment.
|
||||
Leave it unset on internet-facing instances unless you specifically
|
||||
want to allow private-range targets.
|
||||
- **Codex disconnects after a few weeks of inactivity.** ChatGPT
|
||||
refresh tokens lapse after extended idle periods. Reconnect via
|
||||
the **Connect ChatGPT** button — HealthLog re-runs the device-code
|
||||
flow and the new tokens land in the same encrypted columns.
|
||||
- **Chain falls through every entry.** Run the per-provider connection
|
||||
test from `/settings/ai` to identify which step is failing. The
|
||||
most common cause is a stale or revoked BYOK key earlier in the
|
||||
chain; remove or disable that entry to skip ahead.
|
||||
@@ -0,0 +1,200 @@
|
||||
# Apple Health import
|
||||
|
||||
HealthLog accepts a full Apple Health `export.zip` upload and folds
|
||||
the contents into the same timeline as every other ingest source.
|
||||
The importer is streaming end-to-end — multi-gigabyte exports never
|
||||
land in V8 heap — and idempotent on the upload's SHA-256 so
|
||||
re-uploading the same archive merges instead of duplicating.
|
||||
|
||||
## What you need
|
||||
|
||||
- An iPhone with the Apple Health app and at least a few months of
|
||||
history (otherwise there is little to import).
|
||||
- A HealthLog account on an instance you can reach from the device
|
||||
or from a workstation. The upload goes over your authenticated
|
||||
browser session — no separate API token to provision.
|
||||
- Up to 1.5 GB of upload bandwidth. Apple's largest observed exports
|
||||
sit around 800 MB for a 10-year iCloud-synced account; the parser
|
||||
caps inbound bodies at 1.5 GB
|
||||
(`src/app/api/import/apple-health-export/route.ts:41`).
|
||||
|
||||
## 1. Generate `export.zip` on the iPhone
|
||||
|
||||
1. Open the **Health** app.
|
||||
2. Tap the profile picture in the top-right.
|
||||
3. Scroll to the bottom and tap **Export All Health Data**.
|
||||
4. iOS spends a few minutes packaging the archive. When it finishes,
|
||||
share-sheet the resulting `export.zip` somewhere you can reach
|
||||
from your HealthLog instance — AirDrop to a workstation, save to
|
||||
iCloud Drive, send via email or any chat app you control.
|
||||
|
||||
The archive contains `apple_health_export/export.xml` (the canonical
|
||||
record stream) plus a handful of auxiliary files (workout routes,
|
||||
clinical records, electrocardiograms). The importer reads
|
||||
`export.xml` and ignores the rest.
|
||||
|
||||
## 2. Upload to HealthLog
|
||||
|
||||
Open the import page from the in-app settings tree. Drop the
|
||||
`export.zip` onto the upload control. The browser streams the body
|
||||
directly to the server; nothing is buffered into memory on either
|
||||
side.
|
||||
|
||||
The endpoint is `POST /api/import/apple-health-export`. The handler:
|
||||
|
||||
1. Rate-limits to **three uploads per minute per user** to protect
|
||||
against a flood of multi-gigabyte uploads
|
||||
(`src/app/api/import/apple-health-export/route.ts:43-46`).
|
||||
2. Enforces the **1.5 GB cap** twice — once cheaply against the
|
||||
declared `Content-Length`, then a second time at the streaming
|
||||
sink so a missing/wrong header cannot bypass it.
|
||||
3. Streams the body to a temp file at `/tmp/healthlog-apple-health-import-*`
|
||||
while computing a SHA-256 of the bytes inline.
|
||||
4. Looks up any prior `ImportJob` row with the same `userId` +
|
||||
`uploadSha256`. **A re-upload of the same archive short-circuits to
|
||||
the previous job** instead of re-queueing — the response carries
|
||||
`idempotent: true` and the original `jobId`.
|
||||
5. Otherwise creates a fresh `ImportJob` row in `queued`, enqueues
|
||||
the `apple-health-import` pg-boss job, and returns the new `jobId`.
|
||||
|
||||
The response body is `{ "jobId": "...", "status": "queued" }` with a
|
||||
`202 Accepted`.
|
||||
|
||||
## 3. Poll the status endpoint
|
||||
|
||||
`GET /api/import/apple-health-export/{jobId}/status` returns the
|
||||
canonical envelope:
|
||||
|
||||
```json
|
||||
{
|
||||
"data": {
|
||||
"jobId": "clx123…",
|
||||
"status": "parsing",
|
||||
"startedAt": "2026-05-16T12:34:56.000Z",
|
||||
"completedAt": null,
|
||||
"uploadBytes": 412345678,
|
||||
"exportedAt": "2026-04-30T18:12:00.000Z",
|
||||
"progress": { "records_seen": 124000, "workouts_seen": 230 },
|
||||
"result": null,
|
||||
"failureReason": null
|
||||
},
|
||||
"error": null
|
||||
}
|
||||
```
|
||||
|
||||
The lifecycle is `queued → unpacking → parsing → upserting → done`
|
||||
with `failed` as a terminal-error branch
|
||||
(`src/app/api/import/apple-health-export/[jobId]/status/route.ts:8-9`).
|
||||
The iOS native client polls every two seconds while active and every
|
||||
thirty seconds once the job is terminal; the web UI uses the same
|
||||
cadence. Authorisation: only the job owner can read it. The admin
|
||||
who kicked off an import via the admin variant can also read the row
|
||||
they triggered.
|
||||
|
||||
## What gets imported
|
||||
|
||||
The parser folds four XML element types into the existing HealthLog
|
||||
schema (CHANGELOG v1.4.34):
|
||||
|
||||
| XML element | Maps to | Notes |
|
||||
| ----------- | ------- | ----- |
|
||||
| `<Record>` | `Measurement` rows | One row per spot sample. Keyed by `HKMetadataKeyExternalUUID` when present, otherwise a deterministic `sample:<sha256-of-attributes>` fallback so re-imports stay idempotent. |
|
||||
| `<Workout>` | `Workout` rows | Activity type + duration + energy + distance. |
|
||||
| `<Correlation>` | `Measurement` rows | Compound samples like blood pressure (systolic + diastolic) explode into their per-metric children. |
|
||||
| `<ClinicalRecord>` | `Measurement` rows where the type maps | Health-record FHIR snapshots; only the metric-typed fields are ingested. |
|
||||
|
||||
The HK quantity types that map to HealthLog metric types live in
|
||||
`src/lib/measurements/apple-health-mapping.ts`. Anything outside the
|
||||
mapped set (workout routes, ECG voltage traces) is counted in
|
||||
`progress.records_skipped` but not stored.
|
||||
|
||||
### Cumulative HK type → daily aggregate
|
||||
|
||||
Cumulative quantity types — steps, active energy, walking/running
|
||||
distance, flights climbed — get **collapsed into one row per
|
||||
user-local day** to match the iOS daily-aggregation convention.
|
||||
|
||||
- External ID format: `stats:<HKType>:<YYYY-MM-DD>` (e.g.
|
||||
`stats:HKQuantityTypeIdentifierStepCount:2026-05-15`).
|
||||
- The day boundary respects the user's timezone preference. The
|
||||
worker defaults to `Europe/Berlin` when no preference is set
|
||||
(`src/lib/jobs/apple-health-import-worker.ts:132-133`).
|
||||
- A second import of the same archive re-folds the same days onto
|
||||
the same external IDs — the values update in place rather than
|
||||
duplicating.
|
||||
|
||||
### Re-import idempotency
|
||||
|
||||
The two-axis idempotency story:
|
||||
|
||||
1. **File-level** — re-uploading the same `export.zip` (same SHA-256
|
||||
of bytes) short-circuits to the previous job before the parser
|
||||
even starts.
|
||||
2. **Record-level** — when an older `export.zip` is re-exported from
|
||||
iOS (same history, slightly newer device timestamps), every
|
||||
record's external ID stays stable. The upsert path matches on
|
||||
`(userId, externalId)` and updates instead of inserting.
|
||||
|
||||
Practically: a user who exports monthly and re-imports each archive
|
||||
sees fresh data merged in without duplicates. The job result line
|
||||
in the status response reports `recordsInserted` and
|
||||
`recordsUpdated` separately.
|
||||
|
||||
## Admin variant — import on behalf of another user
|
||||
|
||||
`POST /api/admin/import-apple-health-export` is the cookie-only admin
|
||||
variant (Bearer tokens never elevate to admin). The multipart body
|
||||
adds a `userId` text field naming the target user; everything else
|
||||
is identical to the user-facing flow.
|
||||
|
||||
The `ImportJob` row carries `triggeredByAdminId = admin.id` so the
|
||||
status endpoint admits both the target user and the triggering
|
||||
admin. Idempotency is scoped by target user — an admin re-uploading
|
||||
the same archive for the same user resolves to the previous job.
|
||||
|
||||
Typical use: migrating a friend's history into their HealthLog
|
||||
account when they cannot run the import themselves, or backfilling
|
||||
a household member's data after the admin received the `export.zip`
|
||||
out of band.
|
||||
|
||||
## Source-priority interaction
|
||||
|
||||
HealthLog's source ladder is **APPLE_HEALTH ≻ WITHINGS ≻ MANUAL ≻
|
||||
IMPORT** for cumulative metrics (steps, active energy, distance,
|
||||
flights, sleep, HRV, resting HR) and **WITHINGS ≻ APPLE_HEALTH ≻
|
||||
MANUAL** for point measurements where the Withings device is the
|
||||
primary sensor (weight, BP, body fat, body temperature, SpO₂, VO₂
|
||||
max). The defaults live in `src/lib/validations/source-priority.ts:205-220`.
|
||||
|
||||
Concrete consequences:
|
||||
|
||||
- If you already sync steps from a Withings ScanWatch and also import
|
||||
Apple Health, the Apple-Health-aggregated stream wins for the day —
|
||||
iOS HealthKit folds the watch sensor + iPhone motion data into a
|
||||
single canonical stream, so it is the most complete cumulative
|
||||
source. The Withings rows stay in the database as an audit trail
|
||||
but drop out of the per-day aggregation.
|
||||
- If you weigh yourself on a Withings scale and also import Apple
|
||||
Health (which received the same reading via the Health Mate iOS
|
||||
app), the Withings row wins for display. Apple Health is
|
||||
second-hand for that metric.
|
||||
- Manual entries always rank below either device source. The IMPORT
|
||||
source tag — the lowest rank — is reserved for legacy CSV/JSON
|
||||
imports that pre-date the Apple Health passthrough.
|
||||
|
||||
Override per-user via the Sources section of `/settings/thresholds`.
|
||||
|
||||
## Failure modes
|
||||
|
||||
| Failure | What you see | Recovery |
|
||||
| ------- | ------------ | -------- |
|
||||
| Archive is missing `apple_health_export/export.xml` | `failureReason: "Archive is missing the apple_health_export/export.xml member"` | Re-export from the iPhone; a partial AirDrop or a renamed archive can lose the canonical entry. |
|
||||
| Archive is encrypted | `failureReason: "Encrypted ZIP entries are not supported"` | Apple does not encrypt the export — this means the file was repacked by a third-party tool. Re-export from iOS directly. |
|
||||
| Unsupported compression method | `failureReason: "Unsupported ZIP compression method <n> for export.xml"` | Same recovery — re-export from iOS, which always emits method 0 (stored) or method 8 (deflate). |
|
||||
| Worker is not running | `503 Background worker is not running` from the kick-off endpoint | The worker process is down. Check `docker compose ps` for the `app-worker` container (or `HEALTHLOG_PROCESS_TYPE=all` mode on the main `app` container) and restart. |
|
||||
| Rate-limited | `429 Too many import uploads, try again later` | Wait sixty seconds; the limit is three uploads per minute per user. |
|
||||
|
||||
The wide-event log line `import.apple-health.kickoff` (or
|
||||
`import.apple-health.kickoff.denied` on failure) captures every
|
||||
attempt with the upload size, SHA-256, and the resolved `ImportJob`
|
||||
id for debugging.
|
||||
@@ -0,0 +1,195 @@
|
||||
# Withings integration
|
||||
|
||||
HealthLog connects to Withings via OAuth2 plus a push webhook for
|
||||
near-real-time sync. Once linked, every Body+ scale weighing, BPM
|
||||
blood-pressure reading, ScanWatch sleep stage, and activity sample
|
||||
lands in HealthLog within seconds — no polling, no third-party
|
||||
relay.
|
||||
|
||||
## What gets synced
|
||||
|
||||
The Withings client (`src/lib/withings/client.ts`) maps the following
|
||||
metric types into `Measurement` rows tagged `source = WITHINGS`:
|
||||
|
||||
| Withings meastype | HealthLog `MeasurementType` |
|
||||
| ----------------- | --------------------------- |
|
||||
| 1 | `WEIGHT` |
|
||||
| 5 | `FAT_FREE_MASS` |
|
||||
| 9 | `BLOOD_PRESSURE_DIA` |
|
||||
| 10 | `BLOOD_PRESSURE_SYS` |
|
||||
| 11 | `PULSE` |
|
||||
| 54, 35 | `OXYGEN_SATURATION` (ScanWatch / pulse-ox products) |
|
||||
| Plus the body-composition family (fat ratio, muscle mass, bone mass, hydration) and sleep / heart-rate variability via the heart-list endpoint |
|
||||
|
||||
Activity data (steps, active energy, distance, floors climbed) lands
|
||||
via the `user.activity` OAuth scope shipped in v1.4.25. Sleep stages
|
||||
arrive via the sleep series endpoint.
|
||||
|
||||
## 1. Register a Withings developer app
|
||||
|
||||
1. Sign in at <https://developer.withings.com/> and **create a new
|
||||
public-API application**. The "Public" tier is free and supports
|
||||
the OAuth2 flow HealthLog uses; you do not need the partner tier.
|
||||
2. **Callback URL:** `https://your-instance.example.com/api/withings/callback`.
|
||||
Withings allows multiple callbacks per app — add one entry per
|
||||
environment (production, staging, local development against a
|
||||
tunnel).
|
||||
3. **Scopes:** request `user.metrics` AND `user.activity`. Both are
|
||||
needed — `user.metrics` covers weight / BP / heart-list / sleep /
|
||||
SpO₂ / body comp / temperature / VO₂ max; `user.activity` is the
|
||||
workout-equivalent stream for steps / energy / distance / floors.
|
||||
See `src/lib/withings/client.ts:42` (`WITHINGS_OAUTH_SCOPE`).
|
||||
4. Save the application. Withings will issue a **Client ID** and a
|
||||
**Client Secret** — keep the secret out of any chat log; it grants
|
||||
read access to every linked account's full health history.
|
||||
|
||||
## 2. Wire the credentials into HealthLog
|
||||
|
||||
Paste the client ID and secret into `.env`:
|
||||
|
||||
```env
|
||||
WITHINGS_CLIENT_ID="your-client-id"
|
||||
WITHINGS_CLIENT_SECRET="your-client-secret"
|
||||
WITHINGS_REDIRECT_URI="https://your-instance.example.com/api/withings/callback"
|
||||
WITHINGS_WEBHOOK_SECRET="$(openssl rand -hex 32)"
|
||||
```
|
||||
|
||||
`WITHINGS_WEBHOOK_SECRET` is the path-segment token the webhook
|
||||
handler validates against. Generate a fresh random value — anything
|
||||
that fits in a URL path segment works, but a 64-character hex string
|
||||
keeps it indistinguishable from random. Restart the `app` container
|
||||
after saving.
|
||||
|
||||
Per-user override is also possible: the Settings UI accepts a
|
||||
client-ID/secret pair scoped to one user, useful if you want a
|
||||
shared instance where every user brings their own Withings app
|
||||
credentials. Server-level env vars cover the default case.
|
||||
|
||||
## 3. Set the webhook URL
|
||||
|
||||
When HealthLog calls Withings `Notify.subscribe()` for a newly-linked
|
||||
account, it hands Withings the callback URL constructed by
|
||||
`getWithingsWebhookCallbackUrl()` (`src/lib/withings/sync.ts:38-48`).
|
||||
The shape is:
|
||||
|
||||
```
|
||||
https://your-instance.example.com/api/withings/webhook/<WITHINGS_WEBHOOK_SECRET>
|
||||
```
|
||||
|
||||
Withings has no facility for adding HTTP headers or signing webhook
|
||||
bodies, so the callback URL itself is the only authenticity surface
|
||||
a subscriber controls. The v1.4.25 W17a migration moved the secret
|
||||
from a query parameter (`?secret=…`, captured by every reverse-proxy
|
||||
access log) to a path segment, which keeps it out of the
|
||||
`query_string` column most proxies log by default
|
||||
(`src/app/api/withings/webhook/[token]/route.ts:13-23`).
|
||||
|
||||
You do not need to register the webhook URL anywhere in the
|
||||
developer portal — HealthLog subscribes per-user during the OAuth
|
||||
callback. The portal only owns the OAuth callback URL.
|
||||
|
||||
## 4. Link from the Settings page
|
||||
|
||||
With env vars in place and the app restarted, sign in as the target
|
||||
user and open `/settings/integrations/withings`. Click **Connect
|
||||
Withings**. The flow:
|
||||
|
||||
1. The Connect button redirects to Withings' `oauth2/authorize`.
|
||||
2. The user signs in on Withings and grants the requested scopes.
|
||||
3. Withings redirects back to `/api/withings/callback` with an auth
|
||||
code.
|
||||
4. HealthLog exchanges the code for an access + refresh token, stores
|
||||
them AES-256-GCM-encrypted at rest, subscribes the webhook URL,
|
||||
and triggers an initial sync of the past 30 days.
|
||||
5. The settings page now shows **Connected** with the timestamp of
|
||||
the last successful sync.
|
||||
|
||||
Subsequent syncs run automatically on every webhook delivery plus a
|
||||
safety-net cron pull every few hours.
|
||||
|
||||
## Source-priority interaction
|
||||
|
||||
HealthLog's source ladder ranks **WITHINGS ≻ APPLE_HEALTH ≻ MANUAL**
|
||||
for point measurements (weight, BP, pulse, body fat, body
|
||||
temperature, SpO₂, VO₂ max) and **APPLE_HEALTH ≻ WITHINGS ≻ MANUAL**
|
||||
for cumulative metrics (steps, active energy, distance, flights,
|
||||
sleep, HRV, resting HR). The defaults live in
|
||||
`src/lib/validations/source-priority.ts:205-220`.
|
||||
|
||||
Concrete consequences if you run Withings alongside an Apple Health
|
||||
import:
|
||||
|
||||
- Weight readings from a Withings scale win for display. Apple
|
||||
Health rows for the same metric (received second-hand from Health
|
||||
Mate) stay in the DB as audit trail but drop out of the active
|
||||
display.
|
||||
- Steps from a ScanWatch go through HealthKit's daily aggregation
|
||||
when Apple Health is also linked, so the Apple Health stream wins
|
||||
for the cumulative axis. The Withings rows stay in the DB.
|
||||
|
||||
Override per-user via the Sources section of `/settings/thresholds`.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### The Connect button errors out
|
||||
|
||||
- **`Withings credentials not configured`** — `WITHINGS_CLIENT_ID`
|
||||
or `WITHINGS_CLIENT_SECRET` is missing from `.env`. Confirm both
|
||||
are set in the container's runtime environment (`docker compose
|
||||
exec app env | grep WITHINGS`).
|
||||
- **`Callback URI mismatch`** — the URL HealthLog sends to Withings
|
||||
must match an entry in the developer-portal app's callback list.
|
||||
Add the public URL exactly, including the `https://` scheme and
|
||||
the `/api/withings/callback` path.
|
||||
|
||||
### The webhook stops firing
|
||||
|
||||
Webhook subscriptions can lapse after a token revoke or a portal-
|
||||
side cleanup. Re-subscribe by opening `/settings/integrations/withings`
|
||||
and clicking **Disconnect** then **Connect Withings** again — the
|
||||
callback exchange re-runs `Notify.subscribe()` with a fresh callback
|
||||
URL.
|
||||
|
||||
Test the inbound path with a HEAD request from outside the host:
|
||||
|
||||
```bash
|
||||
curl -I https://your-instance.example.com/api/withings/webhook/<WITHINGS_WEBHOOK_SECRET>
|
||||
```
|
||||
|
||||
A `200 OK` means the path-segment secret matches. A `401` means the
|
||||
env var on the running container does not match the secret embedded
|
||||
in the URL Withings was handed at subscribe time. Re-subscribing
|
||||
fixes the mismatch by updating the URL Withings holds.
|
||||
|
||||
### Sync is stuck on "Reauthorisation required"
|
||||
|
||||
Withings revokes refresh tokens on password change, app un-authorise,
|
||||
or after extended inactivity. The `WithingsConnection` row flips to
|
||||
`error_reauth` status and the scheduled sync no-ops until the user
|
||||
re-runs the OAuth flow (`src/lib/withings/sync.ts:155-161`).
|
||||
Click **Connect Withings** again from the settings page; the
|
||||
callback exchanges fresh tokens and the next sync runs through the
|
||||
last 30 days of history to catch up.
|
||||
|
||||
### Disconnect cleanly
|
||||
|
||||
The Settings page **Disconnect** button:
|
||||
|
||||
1. Calls `POST /api/withings/disconnect`.
|
||||
2. Server-side: revokes the access token via Withings'
|
||||
`Notify.revoke`, deletes the `WithingsConnection` row, and audits
|
||||
the action.
|
||||
3. Existing `Measurement` rows tagged `source = WITHINGS` stay in
|
||||
the database — disconnect does not delete data. Re-linking the
|
||||
same Withings account merges back onto the same metric
|
||||
timeline; the upsert path matches on `(userId, type, measuredAt,
|
||||
source)` so previously-synced rows update in place.
|
||||
|
||||
## Admin probe
|
||||
|
||||
`POST /api/integrations/withings/test` (admin-only) runs a
|
||||
read-only round-trip against the user's stored credentials and
|
||||
returns the latency in milliseconds. Use it from the admin panel
|
||||
when a user reports stale data — a green test plus a stale
|
||||
`lastSyncedAt` usually means the webhook URL Withings holds is out
|
||||
of sync (re-subscribe per the steps above).
|
||||
Reference in new issue
Block a user