diff --git a/docs/README.md b/docs/README.md index 5384a5694..b3e5c1186 100644 --- a/docs/README.md +++ b/docs/README.md @@ -13,9 +13,10 @@ notes, and machine-readable specs that ship alongside the source: [`api/mcp-capabilities.md`](./api/mcp-capabilities.md) (tools, resources, prompts, write model) and [`api/mcp-skills.md`](./api/mcp-skills.md) (building a connector / skill). -- [`ops/`](./ops/) — backup / restore + encryption-key-rotation - playbooks. These are the runbooks an operator reaches for during an - incident. +- [`ops/`](./ops/) — operator runbooks: deploy, backup / restore, + encryption-key rotation, password reset and account recovery, env + checks, TLS pinning, data repair. These are what an operator reaches + for during an incident. - [`self-hosting/`](./self-hosting/) — horizontal-scaling notes (`HEALTHLOG_PROCESS_TYPE=web|worker|all`) and the deploy-pipeline recipe. Routine install steps stay on `docs.healthlog.dev`. Includes @@ -33,14 +34,19 @@ notes, and machine-readable specs that ship alongside the source: managed profiles for somebody with no login and the guardians who look after them, what revocation does and does not undo, and which trust properties of the instance are unchanged by any of it. -- [`migration/`](./migration/) — release-by-release migration notes. - Read the entry for the version you're upgrading from. -- [`audit/`](./audit/) — per-release audit summaries archived for - forensic traceability. +- [`migration/`](./migration/) — migration notes. Currently only + v1.3 to v1.4; later releases are covered by `CHANGELOG.md` and + . +- [`integrations/`](./integrations/) — per-integration reference + (Withings, Fitbit, Google Health, WHOOP, Apple Health, AI providers, + data import). +- [`adr/`](./adr/) — architecture decision records. +- [`security/`](./security/) — threat-model notes. +- [`diagrams/`](./diagrams/) — architecture diagrams rendered through + docs.healthlog.dev. +- [`releases/`](./releases/) — release imagery. - [`apple-store-connect-checklist.md`](./apple-store-connect-checklist.md) — the iOS submission gate. -- [`doctor-report.md`](./doctor-report.md) — PDF generator design - notes. - [`codex-protocol-spec.md`](./codex-protocol-spec.md) — Codex / ChatGPT OAuth protocol reverse-engineering notes. - [`ui-guidelines.md`](./ui-guidelines.md) — UI / a11y house style. diff --git a/docs/api/README.md b/docs/api/README.md index a41bf6d84..52e49b52b 100644 --- a/docs/api/README.md +++ b/docs/api/README.md @@ -1,8 +1,12 @@ # HealthLog API Spec -`openapi.yaml` is the canonical OpenAPI 3.1 description for the HealthLog -Next.js API. It is the source of truth for the native iOS client's -DTO codegen. +`openapi.yaml` is the OpenAPI 3.1 description for the HealthLog +Next.js API and the source of truth for the native iOS client's DTO +codegen. The file is **generated**: `pnpm openapi:generate` emits it +from the Zod registry under `src/lib/openapi/` (`registry.ts` plus the +route modules in `src/lib/openapi/routes/`). Do not edit the YAML by +hand; the next generation overwrites it, and CI (`openapi:check`) +fails the build when the committed YAML drifts from the registry. The MCP server is documented separately — it is a distinct contract over JSON-RPC, not part of the REST OpenAPI spec: @@ -34,36 +38,38 @@ npx @redocly/cli build-docs docs/api/openapi.yaml --output docs/api/index.html npx @redocly/cli lint docs/api/openapi.yaml ``` -The spec is expected to be lint-clean (errors = 0). Some warnings are -intentional: - -- `no-server-example.com` — `http://localhost:3000` is the dev server. -- `operation-2xx-response` / `operation-4xx-response` — webhook + redirect - endpoints intentionally omit one or the other. -- `no-unused-components` — `ReminderPhase` is exposed for client codegen - even though no path response references it directly. +The spec is expected to be lint-clean (errors = 0). Warnings such as +`no-server-example.com` for the localhost dev server are tolerated; +re-run the lint to see what it produces against the current file +before treating a warning as a regression. ## Layout -- **`info`** — title, version (mirrored from `package.json`), description. -- **`servers`** — production `https://healthlog.bombeck.io` plus - `http://localhost:3000` for local development. -- **`tags`** — domain groupings (`Auth`, `Measurements`, `Medications`, - `Mood`, `Dashboard`, `Insights`, `Achievements`, `DoctorReport`, - `Integrations`, `User`, `Tokens`, `Notifications`, `Onboarding`, - `Health`, `Settings`, `Analytics`, `Export`, `AuditLog`, - `Monitoring`, `Webhooks`, `Ingest`, `Admin`). +- **`info`** — title, description, and `version` mirrored automatically + from `package.json` at generation time. Never bump it by hand. +- **`servers`** — `https://healthlog.example.com` as the self-hosted + placeholder plus `http://localhost:3000` for local development. +- **`tags`** — domain groupings; the current set lives at the top of + the generated file (`Auth`, `Measurements`, `Medications`, `Mood`, + `Analytics`, `Insights`, `Consent`, `Dashboard`, `Notifications`, + `Devices`, `Export`, `Sync`, `Cycle`, `MeasurementReminders`, + `Labs`, `Illness`, `Documents`, `Records`, `Admin`, `Meta`, + `Retired`). - **`securitySchemes`**: - `bearerAuth` — long-lived `hlk_*` API tokens issued via `POST /api/tokens`. Required on `/api/ingest/medication`. - `sessionCookie` — HttpOnly `healthlog_session` cookie set by the password and passkey login flows. Used by the web app and any iOS client running inside a web view. -- **`components.schemas`** — DTOs for all routes plus the underlying - Prisma models (e.g. `Measurement`, `Medication`, `MedicationIntakeEvent`, - `MoodEntry`, `User`, `DashboardLayout`, `ApiEnvelope`, `ApiError`). -- **`paths`** — all routes from `src/app/api/**/route.ts`, including - the `/api/admin/*` family which is tagged `Admin` and marked internal. +- **`components.schemas`** — DTOs for the documented routes plus the + underlying Prisma models (e.g. `Measurement`, `Medication`, + `MedicationIntakeEvent`, `MoodEntry`, `User`, `DashboardLayout`, + `ApiEnvelope`, `ApiError`). +- **`paths`** — the client-facing REST surface. Not every + `src/app/api/**/route.ts` is in the contract: most `/api/admin/*` + routes, the OAuth connect/callback legs, and the webhook surfaces + are not documented here. The `Retired` tag lists removed paths that + answer `410 Gone`. ## Conventions @@ -79,8 +85,12 @@ intentional: When adding or changing routes: 1. Update the relevant route handler under `src/app/api/**/route.ts`. -2. Mirror the change in `openapi.yaml`. Keep `operationId` camelCase and - stable — the iOS codegen keys off it. -3. Run `npx @redocly/cli lint docs/api/openapi.yaml` and resolve all - errors. Warnings are tolerated when documented above. -4. Bump `info.version` if the change is breaking for clients. +2. Change the Zod schemas and their `.meta()` registrations under + `src/lib/openapi/` (registry entry plus the matching module in + `src/lib/openapi/routes/`). Keep `operationId` camelCase and + stable; the iOS codegen keys off it. +3. Run `pnpm openapi:generate` and commit the regenerated + `openapi.yaml` alongside the schema change. CI's `openapi:check` + fails on drift. +4. Never edit the YAML directly and never bump `info.version` by + hand; the version follows `package.json` at generation time. diff --git a/docs/api/mcp-capabilities.md b/docs/api/mcp-capabilities.md index 92e0a0e23..51d98ae79 100644 --- a/docs/api/mcp-capabilities.md +++ b/docs/api/mcp-capabilities.md @@ -87,6 +87,9 @@ server-authoritative path; no new analytics is computed at the wire. | `get_integration_status` | Sync health of connected devices/services — connected, last sync, reauth-required/failing — to answer "why is my data stale?". No secrets or tokens. | | `get_preventive_care` | What preventive care is coming up: the user's own configured Vorsorge reminders (upcoming/overdue checkups with next-due dates) plus the appointments booked as future visits. Surfaces configured items, invents nothing. | | `get_visits` | The user's own past doctor visits over a bounded window (default 12 months, optional practitioner-name filter): date, status, kind, practitioner + specialty, the visit's own reason/outcome, and any linked condition labels. `{ present: false }` when none was ever recorded. | +| `get_intraday_pulse` | The 10-minute heart-rate shape for one local day (hourly grain outside the dense-retention window), with at most one cautious descriptive elevated-at-rest window. Gated on the `insights` module. | +| `get_ecg_recordings` | Metadata for the user's ECG recordings (time, duration, sampling rate, lead, the device's own rhythm classification). Never the waveform, never a re-classification. Gated on the `insights` module. | +| `get_nutrients` | Without an argument, the presence overview per logged nutrient code; with one, that code's per-day summed series plus its EFSA reference. Gated on the opt-in `nutrients` module. | | `search` | Free-text search over the user's record (metric domains, medications, lab analytes). Returns `{ results: [{ id, title, url }], nextCursor? }`. | | `fetch` | Hydrate one record by the id `search` returned (`metric:weight`, `med:`, `lab:LDL`). Returns `{ id, title, text, url, metadata }` with a citation deep-link. | @@ -189,13 +192,14 @@ completion and listing are user-scoped, so a host only ever sees the metrics / analytes / medications the user actually has — completion doubles as honest discovery. -| Template | Resolves to | -| ------------------------------------------ | --------------------------------------------------------------------------------------- | -| `healthlog://metric/{type}` | One metric's series over the default window. | -| `healthlog://metric/{type}/{window}` | One metric's series over a chosen trailing window. | -| `healthlog://lab/{analyte}` | One lab analyte's recent readings (value, unit, reference range, status). | -| `healthlog://medication/{id}` | One medication by id, with its schedules. User-scoped (`{present:false}` if not yours). | -| `healthlog://report/doctor-visit/{window}` | The doctor-visit summary over a chosen window. | +| Template | Resolves to | +| ------------------------------------------ | ----------------------------------------------------------------------------------------- | +| `healthlog://metric/{type}` | One metric's series over the default window. | +| `healthlog://metric/{type}/{window}` | One metric's series over a chosen trailing window. | +| `healthlog://lab/{analyte}` | One lab analyte's recent readings (value, unit, reference range, status). | +| `healthlog://medication/{id}` | One medication by id, with its schedules. User-scoped (`{present:false}` if not yours). | +| `healthlog://report/doctor-visit/{window}` | The doctor-visit summary over a chosen window. | +| `healthlog://nutrient/{code}` | One nutrient's per-day intake series plus its EFSA reference (opt-in `nutrients` module). | ## Prompts (skills) diff --git a/docs/apple-store-connect-checklist.md b/docs/apple-store-connect-checklist.md index c16fe0e45..fb638afbe 100644 --- a/docs/apple-store-connect-checklist.md +++ b/docs/apple-store-connect-checklist.md @@ -34,7 +34,7 @@ Plus pre-submit final checks. - [ ] Add the description above to the medical-device-justification field if Apple shows one - [ ] Cross-reference the in-app `/privacy` page (https:///privacy) §7 MDR-boundary -**Why this matters**: Apple's first-submission rejection rate for health apps is ~40%. The most common reason is unclear medical-device classification. The above wording mirrors the in-app disclaimer + the per-feature gating; consistency between the form and the app is what Apple looks for. +**Why this matters**: health apps get rejected on first submission most commonly for unclear medical-device classification. The above wording mirrors the in-app disclaimer + the per-feature gating; consistency between the form and the app is what Apple looks for. ## Section 2 — Privacy Nutrition Labels diff --git a/docs/codex-protocol-spec.md b/docs/codex-protocol-spec.md index 2794c4281..1e49cab8c 100644 --- a/docs/codex-protocol-spec.md +++ b/docs/codex-protocol-spec.md @@ -630,8 +630,9 @@ Confirmed-rejected slugs (despite the source-tree references in the table above) Confirmed-accepted slugs on a ChatGPT Plus subscription (per `available_in_plans` in `models.json`): -- `gpt-5.3-codex` — coding-optimized, accepted; HealthLog uses this as default -- `gpt-5.5` / `gpt-5.4` / `gpt-5.4-mini` / `gpt-5.2` — also expected to work per plan +- `gpt-5.5` — HealthLog's default (first chain entry) as of 2026-06 +- `gpt-5.4` / `gpt-5.4-mini` / `gpt-5.2` — accepted per plan +- `gpt-5.3-codex` — was the default (verified 2026-05-09); the backend rotated off the `-codex` line on 2026-06-02 and this slug is now only accepted intermittently, so it sits on a late chain rung The `codex-rs/tui/src/model_migration.rs` references to `gpt-5-codex` etc. are _migration prompts_ shown to existing users — they are NOT the slugs sent on the wire after migration. The wire slugs come from `models.json`. Always cross-check `models.json` (`gh api repos/openai/codex/contents/codex-rs/models-manager/models.json`) before picking a default. @@ -641,17 +642,20 @@ The ChatGPT-Codex allow-list is rotated by OpenAI without prior notice (`gpt-5-c To eliminate that risk class, HealthLog's `CodexClient` walks an ordered chain on every fresh request series. -**Chain** (top = first try; default order is the empirically-safest most-current-first ordering as of 2026-05): +**Chain** (top = first try; default order is the empirically-safest most-current-first ordering as of 2026-06, after the backend rotated off the `-codex` line on 2026-06-02): ```ts const DEFAULT_SLUG_FALLBACK_CHAIN = [ - "gpt-5.3-codex", // current allow-list entry (verified 2026-05-09) - "gpt-5-codex", // historical default (rejected as of 2026-05 but safe to retry — backend may flip) - "gpt-5", // bare slug — rejected on ChatGPT-auth as of 2026-05 but lives on api-key auth - "gpt-4o", // ladder-rung for last-ditch capability + "gpt-5.5", // current default across paid tiers as of 2026-06-02 + "gpt-5.4", // documented fallback + "gpt-5.4-mini", // documented lighter-weight fallback + "gpt-5.3-codex", // late rung — still accepted intermittently + "gpt-5.2", // legacy floor ]; ``` +The shipped constant in `src/lib/ai/codex-client.ts` is authoritative; when the upstream rotates again, that file moves first and this listing follows. + The chain is overridable via the env var `CODEX_MODEL_FALLBACK_CHAIN` (comma-separated). The first env-var-named slug supersedes `CODEX_MODEL`; if both are set, `CODEX_MODEL` is folded into position 0 and de-duplicated. **Trigger condition** (when to walk to the next slug): diff --git a/docs/integrations/ai-providers.md b/docs/integrations/ai-providers.md index 80cc2abc3..af14d614d 100644 --- a/docs/integrations/ai-providers.md +++ b/docs/integrations/ai-providers.md @@ -45,8 +45,8 @@ fallback for users who skip the setup. ## The fallback chain `User.aiProviderChain` is a JSON array of providers tried in priority -order. The default is encoded in -`src/lib/ai/provider-chain.ts:60-66`: +order. The default is encoded as `PROVIDER_CHAIN_DEFAULT` in +`src/lib/ai/provider-chain.ts`: 1. `codex` — ChatGPT OAuth (covered by your ChatGPT Plus / Pro subscription; no per-token cost on top). @@ -79,7 +79,7 @@ the default rather than 500-ing. 4. In HealthLog, open `/settings/ai`. Set provider to **OpenAI**, paste the key, optionally override the model. -The default model is `gpt-4o` (`src/lib/ai/provider.ts:80`) — a +The default model is `gpt-4o` (see `src/lib/ai/provider.ts`) — a full-size model that keeps the headline Insights assessments grounded and well-reasoned out of the box. Override to a lighter model such as `gpt-4o-mini` if you want to trade some reasoning headroom for a lower @@ -234,7 +234,7 @@ UI setup: `connected`. 5. Token refresh runs automatically; the client triggers a refresh on 401 and persists the new tokens transparently - (`src/lib/ai/provider.ts:122-217`). + (`src/lib/ai/provider.ts`). Disconnect cleanly from `/settings/ai` — `POST /api/auth/codex/disconnect` clears the encrypted token columns. The wire-level protocol is @@ -410,12 +410,12 @@ All three variables must also be listed in the compose ## Connection test `POST /api/ai/test` runs a one-shot probe against the resolved -provider and returns the latency in milliseconds plus the model the -provider routed to. The Settings UI surfaces a **Test connection** -button next to each provider section that fires this endpoint. A -failed probe surfaces a localisable `errorCode` so the UI renders -the failure in the user's language rather than the raw upstream -error. +provider and returns the resolved provider type, the model, the token +usage, and a short response sample. The Settings UI surfaces a +**Test connection** button next to each provider section that fires +this endpoint. A failed probe carries a categorised `reasonCode` so +the UI renders the failure in the user's language rather than the raw +upstream error. ## Troubleshooting diff --git a/docs/integrations/apple-health.md b/docs/integrations/apple-health.md index bc21e02ae..456dd96c3 100644 --- a/docs/integrations/apple-health.md +++ b/docs/integrations/apple-health.md @@ -265,7 +265,8 @@ IMPORT** for cumulative metrics (steps, active energy, distance, flights, sleep, HRV, resting HR) and **WITHINGS ≻ APPLE_HEALTH ≻ MANUAL** for point measurements where the Withings device is the primary sensor (weight, BP, body fat, body temperature, SpO₂, VO₂ -max). The defaults live in `src/lib/validations/source-priority.ts:205-220`. +max). The defaults live in `DEFAULT_SOURCE_PRIORITY` in +`src/lib/validations/source-priority.ts`. Concrete consequences: @@ -283,7 +284,7 @@ Concrete consequences: source tag — the lowest rank — is reserved for legacy CSV/JSON imports that pre-date the Apple Health passthrough. -Override per-user via the Sources section of `/settings/thresholds`. +Override per-user via Settings → Sources (`/settings/sources`). ## Failure modes diff --git a/docs/integrations/data-import.md b/docs/integrations/data-import.md index bd4d13097..775093405 100644 --- a/docs/integrations/data-import.md +++ b/docs/integrations/data-import.md @@ -1,6 +1,6 @@ # Data import -HealthLog has three ways to bring data in, all reachable from +HealthLog has four ways to bring data in, all reachable from **Settings → Export & Import**: 1. An **Apple Health `export.zip`** upload — the full archive from the @@ -12,8 +12,11 @@ HealthLog has three ways to bring data in, all reachable from from another tracker. This page documents that format. 3. A **CSV import** — one measurement per row, for a spreadsheet or a meter / sensor export. Documented in section 3 below. +4. A **medication dose-history import** — its own card in the same + panel, posting to `POST /api/medications/intake/dose-history-import` + with its own `?dryRun=1` preview. -All three paths skip rows that already exist, so re-running an import is +All paths skip rows that already exist, so re-running an import is safe — it merges rather than duplicating. Every import reports what it actually wrote. A file where nothing was @@ -33,9 +36,10 @@ the browser — and a background job parses it while a progress indicator polls for the imported / skipped counts. Re-uploading the same archive resolves to the same job by content hash, -so it merges instead of creating duplicates. Clinical records -(electrocardiograms, lab documents) are intentionally skipped. The full -type mapping lives in [`apple-health.md`](./apple-health.md). +so it merges instead of creating duplicates. ECG CSVs and metric-typed +clinical records are imported too; see +[`apple-health.md`](./apple-health.md) for the details and the full +type mapping. ## 2. Generic JSON import @@ -56,15 +60,16 @@ skipped and counted under `skipped` in the response. ### Measurement entry -| Field | Required | Type | Notes | -| ---------------- | -------- | ------ | --------------------------------------------------------------------------------- | -| `type` | yes | enum | One of the `MeasurementType` values in the table below. | -| `value` | yes | number | Plausibility-checked per type; out-of-range values are rejected. | -| `unit` | yes | string | The canonical unit for the type (see the table). | -| `measuredAt` | yes | string | ISO-8601 datetime, e.g. `2026-05-01T08:00:00.000Z`. | -| `source` | no | string | Free-text origin label. Imported rows are tagged `IMPORT` server-side regardless. | -| `notes` | no | string | Optional free text. | -| `glucoseContext` | no | enum | Only for `BLOOD_GLUCOSE`. See [Glucose context](#glucose-context). | +| Field | Required | Type | Notes | +| ---------------- | -------- | ------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `type` | yes | enum | One of the `MeasurementType` values in the table below. | +| `value` | yes | number | Plausibility-checked per type; out-of-range values are rejected. | +| `unit` | yes | string | The canonical unit for the type (see the table). | +| `measuredAt` | yes | string | ISO-8601 datetime, e.g. `2026-05-01T08:00:00.000Z`. | +| `source` | no | string | Free-text origin label. Imported rows are tagged `IMPORT` server-side regardless. | +| `notes` | no | string | Optional free text. | +| `glucoseContext` | no | enum | Only for `BLOOD_GLUCOSE`. See [Glucose context](#glucose-context). | +| `externalId` | no | string | A source-stable id (1–120 chars). When present, a re-import upserts on `(user, type, source, externalId)` so an edit upstream is reflected rather than skipped. | ### Glucose context @@ -95,7 +100,7 @@ moment you know it. | `mood` | yes | enum | One of `SUPER_GUT`, `GUT`, `OKAY`, `SCHLECHT`, `LAUSIG`. | | `score` | yes | integer | `1`–`5`. | | `tags` | no | string | Comma-separated tags. | -| `loggedAt` | no | string | ISO-8601 datetime; defaults to now. | +| `loggedAt` | no | string | ISO-8601 datetime; defaults to the start of the given `date` in the default timezone. | | `externalId` | no | string | A source-stable id (1–120 chars). When present, a re-import upserts on `(user, source, externalId)` so an edit upstream is reflected rather than skipped. | ### `MeasurementType` values and canonical units @@ -172,9 +177,10 @@ hand-authored. | `WALKING_STEADINESS_EVENT` | event | | `BREATHING_DISTURBANCE_EVENT` | event | -The live, machine-readable list of writable types and their units is -also served by `GET /api/meta/capabilities` under `ingest.quantityTypes` -— the authoritative source if this table ever lags a release. +The table is a curated subset and lags the enum by a few +device-derived types. The live, machine-readable list of writable +types and their units is served by `GET /api/meta/capabilities` under +`ingest.quantityTypes` and is the authoritative source. ### Worked example @@ -234,15 +240,15 @@ mints a file with the right header. A header row is required; column order does not matter and header matching is case-insensitive. -| Column | Required | Notes | -| ---------------- | -------- | --------------------------------------------------------------------------------------------------------- | -| `type` | yes | A `MeasurementType` from the table above. | -| `value` | yes | A number, in the row's `unit`. | -| `unit` | yes | The canonical unit, or a recognised alias: `mmol/L` → `mg/dL`, `lb` → `kg`, `in` → `cm`, `°F` → `°C`. | -| `measuredAt` | yes | ISO-8601 **with an explicit offset** (`Z` or `±HH:MM`, and `±HHMM` is accepted). No offset, no import. | -| `glucoseContext` | no | Only for `BLOOD_GLUCOSE`; blank means no context. See [Glucose context](#glucose-context). | -| `notes` | no | Free text, up to 200 characters. | -| `externalId` | no | A source-stable id. With it, a re-upload updates the same row; without it, a re-upload creates a new one. | +| Column | Required | Notes | +| ---------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `type` | yes | A `MeasurementType` from the table above. | +| `value` | yes | A number, in the row's `unit`. | +| `unit` | yes | The canonical unit, or a recognised alias: `mmol/L` → `mg/dL`, `lb` → `kg`, `in` → `cm`, `°F` → `°C`. | +| `measuredAt` | yes | ISO-8601 **with an explicit offset** (`Z` or `±HH:MM`, and `±HHMM` is accepted). No offset, no import. | +| `glucoseContext` | no | Only for `BLOOD_GLUCOSE`; blank means no context. See [Glucose context](#glucose-context). | +| `notes` | no | Free text, up to 200 characters. | +| `externalId` | no | A source-stable id. With it, a re-upload updates the same row even when the value changed; without it, an identical re-upload is skipped via the natural key `(user, type, measuredAt, source)`. | A sensor export therefore needs no glucose column at all: diff --git a/docs/integrations/fitbit.md b/docs/integrations/fitbit.md index 66f0e0c51..7e4310c39 100644 --- a/docs/integrations/fitbit.md +++ b/docs/integrations/fitbit.md @@ -93,8 +93,9 @@ The Fitbit client id/secret are per-user (Settings), not env vars. If Fitbit and another source (Withings, WHOOP, Apple Health) both supply the same vital — resting heart rate, blood oxygen, body temperature or respiratory rate — -you may see both values until a future update settles on a single preferred -source. +the source priority ladder in **Settings → Sources** decides which one wins +per metric. FITBIT is ranked in every default metric ladder; override +per metric on that page. ## Disconnect diff --git a/docs/integrations/google-health.md b/docs/integrations/google-health.md index 410034c24..ca623a9d0 100644 --- a/docs/integrations/google-health.md +++ b/docs/integrations/google-health.md @@ -134,7 +134,7 @@ triggers the verification + CASA requirements above.) If Google Health and another source (Withings, WHOOP, Apple Health) both supply the same vital — resting heart rate, blood oxygen, respiratory rate — the source -priority ladder in **Settings → Integrations → Sources** decides which one wins +priority ladder in **Settings → Sources** decides which one wins per metric. ## Disconnect diff --git a/docs/integrations/withings.md b/docs/integrations/withings.md index 3d08b370a..58253d76d 100644 --- a/docs/integrations/withings.md +++ b/docs/integrations/withings.md @@ -73,7 +73,7 @@ only when the OAuth callback must differ from that default. When HealthLog calls Withings `Notify.subscribe()` for a newly-linked account, it hands Withings the callback URL constructed by -`getWithingsWebhookCallbackUrl()` (`src/lib/withings/sync.ts:38-48`). +`getWithingsWebhookCallbackUrl()` (`src/lib/withings/sync.ts`). The shape is: ``` @@ -107,8 +107,9 @@ With the credentials saved in Settings, open 5. The settings page now shows **Connected** with the timestamp of the last successful sync. -Subsequent syncs run automatically on every webhook delivery plus a -safety-net cron pull every few hours. +Subsequent syncs run automatically on every webhook delivery plus an +hourly safety-net cron pull (measurements at :00, sleep at :15, ECG +at :41). ## Source-priority interaction @@ -117,7 +118,7 @@ for point measurements (weight, BP, pulse, body fat, body temperature, SpO₂, VO₂ max) and **APPLE_HEALTH ≻ WITHINGS ≻ MANUAL** for cumulative metrics (steps, active energy, distance, flights, sleep, HRV, resting HR). The defaults live in -`src/lib/validations/source-priority.ts:205-220`. +`DEFAULT_SOURCE_PRIORITY` in `src/lib/validations/source-priority.ts`. Concrete consequences if you run Withings alongside an Apple Health import: @@ -130,7 +131,7 @@ import: when Apple Health is also linked, so the Apple Health stream wins for the cumulative axis. The Withings rows stay in the DB. -Override per-user via the Sources section of `/settings/thresholds`. +Override per-user via Settings → Sources (`/settings/sources`). ## Troubleshooting @@ -169,7 +170,7 @@ fixes the mismatch by updating the URL Withings holds. Withings revokes refresh tokens on password change, app un-authorise, or after extended inactivity. The `WithingsConnection` row flips to `error_reauth` status and the scheduled sync no-ops until the user -re-runs the OAuth flow (`src/lib/withings/sync.ts:155-161`). +re-runs the OAuth flow (`src/lib/withings/sync.ts`). Click **Connect Withings** again from the settings page; the callback exchanges fresh tokens and the next sync runs through the last 30 days of history to catch up. @@ -179,20 +180,23 @@ last 30 days of history to catch up. The Settings page **Disconnect** button: 1. Calls `POST /api/withings/disconnect`. -2. Server-side: revokes the access token via Withings' - `Notify.revoke`, deletes the `WithingsConnection` row, and audits - the action. +2. Server-side: unsubscribes the webhook categories (best-effort + Withings `Notify` revoke per category), deletes the + `WithingsConnection` row with its stored encrypted tokens, and + audits the action. The Withings-side grant itself persists until + the user un-authorises HealthLog in their Withings account. 3. Existing `Measurement` rows tagged `source = WITHINGS` stay in the database — disconnect does not delete data. Re-linking the same Withings account merges back onto the same metric timeline; the upsert path matches on `(userId, type, measuredAt, source)` so previously-synced rows update in place. -## Admin probe +## Connection test -`POST /api/integrations/withings/test` (admin-only) runs a -read-only round-trip against the user's stored credentials and -returns the latency in milliseconds. Use it from the admin panel -when a user reports stale data — a green test plus a stale -`lastSyncedAt` usually means the webhook URL Withings holds is out -of sync (re-subscribe per the steps above). +`POST /api/integrations/withings/test` runs a read-only round-trip +against the calling user's own stored credentials and returns the +latency in milliseconds (rate-limited to 5/min per user). The +Withings settings card surfaces it as the connection test. Use it +when data looks stale — a green test plus a stale `lastSyncedAt` +usually means the webhook URL Withings holds is out of sync +(re-subscribe per the steps above). diff --git a/docs/migration/v1.3-to-v1.4.md b/docs/migration/v1.3-to-v1.4.md index fd56560fe..5b29c4d24 100644 --- a/docs/migration/v1.3-to-v1.4.md +++ b/docs/migration/v1.3-to-v1.4.md @@ -7,6 +7,12 @@ new env var in §Environment variables below has a safe default. This guide documents what changes for operators and integrators, in case you're extending the API or scaling the deployment. +> Historical note: route paths on this page reflect v1.4. Several have +> since moved or gone: `/api/admin/status-overview` is now +> `/api/admin/status`, the Umami probe is +> `/api/admin/monitoring/umami-test`, and the moodLog integration was +> removed entirely. + ## TL;DR ```bash @@ -64,9 +70,10 @@ worker`. Default `all` keeps the single-container behaviour. - **`ENCRYPTION_KEY`** _(unchanged)_ — still honoured as a fallback when `ENCRYPTION_KEYS` is empty. No action needed for existing installs. -- Rotation: `pnpm tsx scripts/rotate-encryption-key.ts ` +- Rotation: `pnpm dlx tsx scripts/rotate-encryption-key.ts ` re-encrypts every ciphertext under the new key without downtime. - Walk-through in `docs/ops/encryption-key-rotation.md`. + Walk-through in `docs/ops/encryption-key-rotation.md` (the current + playbook; the production image strips `tsx`, hence `pnpm dlx`). ### Off-host backup target _(optional)_ diff --git a/docs/ops/attempt-number-semantics.md b/docs/ops/attempt-number-semantics.md index 59f878944..b7b9ff5f8 100644 --- a/docs/ops/attempt-number-semantics.md +++ b/docs/ops/attempt-number-semantics.md @@ -17,7 +17,8 @@ that captures how many failures the bucket ledger has accumulated since the last success. The persistent-failure alert dispatcher in -`src/lib/integrations/status.ts:453` uses `Math.max(...buckets)` — the +`src/lib/integrations/status.ts` (the v1.4.43 W14 block) uses +`Math.max(...Object.values(buckets))` — the **max** across the three buckets. A 3-deep persistent streak pages even when the transient bucket is empty, and a flapping integration that ping-pongs between transient and reauth_required never trips a @@ -44,7 +45,8 @@ we want to wake someone up for. If you see an audit row with `attemptNumber >= threshold` but no alert was dispatched, check the per-bucket counts on -`IntegrationStatus.failureBuckets` for that user/integration pair. The +`IntegrationStatus.consecutiveFailuresByKind` (DB column +`consecutive_failures_by_kind`) for that user/integration pair. The sum may be high while every individual bucket sits below the threshold. This is working as designed; manually investigate only if a single bucket is at or above the threshold and `alertedAt` is unset (broken diff --git a/docs/ops/backup-restore.md b/docs/ops/backup-restore.md index 33fddad56..b465f2b5d 100644 --- a/docs/ops/backup-restore.md +++ b/docs/ops/backup-restore.md @@ -54,12 +54,15 @@ performs a 1-byte PUT + GET round-trip and returns: ```json { - "endpoint": "https://...r2.cloudflarestorage.com", - "bucket": "healthlog-backups", - "region": "auto", - "putLatencyMs": 142, - "getLatencyMs": 38, - "ok": true + "data": { + "endpoint": "https://...r2.cloudflarestorage.com", + "bucket": "healthlog-backups", + "region": "auto", + "putLatencyMs": 142, + "getLatencyMs": 38, + "ok": true + }, + "error": null } ``` diff --git a/docs/ops/deploy.md b/docs/ops/deploy.md index 57c3fd31e..289efa4dc 100644 --- a/docs/ops/deploy.md +++ b/docs/ops/deploy.md @@ -16,10 +16,12 @@ canonical command sequence on a single-instance host: # 1. Pull the new image. docker pull ghcr.io/mbombeck/healthlog:vX.Y.Z -# 2. Run migrations against the live DB using the new image. +# 2. Run migrations against the live DB using the new image. The +# production image strips pnpm/npm; use the entrypoint's node +# invocation of the bundled Prisma CLI instead. docker run --rm --env-file .env.production \ ghcr.io/mbombeck/healthlog:vX.Y.Z \ - pnpm prisma migrate deploy + sh -c 'NODE_PATH=/opt/prisma-cli/node_modules node /opt/prisma-cli/node_modules/prisma/build/index.js migrate deploy' # 3. Recreate the app container so it picks up the new image. docker compose up -d app @@ -40,9 +42,10 @@ them in order on boot like any other. WHOOP introduces two optional instance env vars, `WHOOP_REDIRECT_URI` and `WHOOP_WEBHOOK_SECRET` (HMAC secret for WHOOP webhook signature verification). Per-user WHOOP client id/secret live in Settings, not -in the environment. See `docs/integrations/whoop.md`. Add the two vars -to the compose `environment:` whitelist if you set them — vars not on -the whitelist never reach the container. +in the environment. See `docs/integrations/whoop.md`. The shipped +`docker-compose.yml` already whitelists both under `environment:`; only +a forked or older compose file needs them added by hand, since vars not +on the whitelist never reach the container. ### What goes wrong if you reverse it diff --git a/docs/ops/encryption-key-rotation.md b/docs/ops/encryption-key-rotation.md index bb236e8d3..9c28dad2a 100644 --- a/docs/ops/encryption-key-rotation.md +++ b/docs/ops/encryption-key-rotation.md @@ -47,8 +47,13 @@ under `v1` (the synthetic id assigned to the existing `ENCRYPTION_KEY`). pnpm dlx tsx scripts/rotate-encryption-key.ts v2 ``` The script is idempotent — running it again is a no-op for rows already - prefixed with `v2.`. It rotates User, WithingsConnection, AppSettings, - NotificationChannel, and PushSubscription rows. The summary line at the + prefixed with `v2.`. It rotates every registered encrypted column across + all models, plus the two non-`*Encrypted` members: + `IntegrationStatus.lastError` and `CoachMessage.encryptedContent` + (Bytes). The canonical registry lives in + `src/lib/crypto/encrypted-columns.ts`; a guard test fails CI if any + `*Encrypted` column in the schema is missing from the registry or not + referenced by the script. The summary line at the end shows `scanned`, `rotated`, and `errors` per table+field; treat `errors > 0` as a hard failure and re-run after fixing the cause. 4. **Drop the old key (optional, recommended).** Once the script has run diff --git a/docs/ops/env-check.md b/docs/ops/env-check.md index 2e186ca46..af9b3e81a 100644 --- a/docs/ops/env-check.md +++ b/docs/ops/env-check.md @@ -46,8 +46,10 @@ states: | `[missing-optional]` | Variable is optional. The script reports it but does not fail. | The grep-friendly format lets a CI pipeline do `grep -q -'\[MISSING-REQUIRED\]'` to gate a deploy. The CI integration itself -is deferred to v1.4.43 — this release ships the CLI tool only. +'\[MISSING-REQUIRED\]'` to gate a deploy. CI already does: the +`env-check.yml` workflow runs `pnpm check-env --file +.env.production.example` on every PR and fails on missing required +vars. ## All-or-none groups @@ -65,11 +67,15 @@ Example output for a partial APNs config: [OK] APNS_TEAM_ID [OK] APNS_BUNDLE_ID [missing-optional] APNS_KEY — Satisfied by any of: APNS_KEY, APNS_KEY_FILE + [MISSING-REQUIRED] — Group is all-or-none but only 3/4 variables are set. Either set all or unset all. ``` -A reviewer immediately sees that `APNS_KEY` (or its `APNS_KEY_FILE` -alternative) is missing while the other three APNs vars are set — the -exact configuration that silently disables iOS push. +The APNs group is `allOrNone: true`, so the partial config also emits +the synthetic `` row at REQUIRED severity and the run +exits 1. A reviewer immediately sees that `APNS_KEY` (or its +`APNS_KEY_FILE` alternative) is missing while the other three APNs +vars are set — the exact configuration that silently disables iOS +push. ## Editing the manifest @@ -88,12 +94,13 @@ so changes go through a PR. To add a new variable: if the manifest change introduces a new classification branch (anyOf, allOrNone, …). -## Future: CI gate +## CI gate -Deferred to v1.4.43: +The GitHub Actions workflow `.github/workflows/env-check.yml` runs +`pnpm check-env --file .env.production.example` on every PR, blocking +changes that introduce new required vars without updating the example +file. -- GitHub Actions workflow that runs `pnpm check-env --file -.env.production.example` on every PR, blocking changes that - introduce new required vars without updating the example file. -- Coolify pre-deploy hook that runs `pnpm check-env` inside the - target container and aborts the deploy on exit code 1. +Still an idea only: a Coolify pre-deploy hook that runs +`pnpm check-env` inside the target container and aborts the deploy on +exit code 1. diff --git a/docs/ops/password-reset.md b/docs/ops/password-reset.md index 8f6f29f72..4ad1eace2 100644 --- a/docs/ops/password-reset.md +++ b/docs/ops/password-reset.md @@ -2,9 +2,13 @@ A self-hosted operator can reset a single user's password from inside the running container. This is an **operator-only** maintenance path — there is no -user-facing or API route for it, and it bypasses the normal account-recovery -flow. Use it only for an account you administer (for example, after a forgotten -password on a single-user instance). +self-service reset route for users, and it bypasses the normal +account-recovery flow. Admins do have an in-app alternative: +`POST /api/admin/users/{id}/reset-password` (cookie-admin only) sets a new +password and destroys the user's sessions, tokens, and trusted devices. The +CLI remains the path when no admin can log in. Use it only for an account you +administer (for example, after a forgotten password on a single-user +instance). > **Operator-only.** The script writes directly to the `users` table and is > intended to be run by the host operator with database access. Treat the new diff --git a/docs/ops/tls-cert-pin.md b/docs/ops/tls-cert-pin.md index fdb9cf293..f521c8cac 100644 --- a/docs/ops/tls-cert-pin.md +++ b/docs/ops/tls-cert-pin.md @@ -1,11 +1,10 @@ # TLS SPKI pinning — coupling, alarm, and re-pin runbook -The native client SPKI-pins the server's TLS chain at the **CA level**. It -ships a small pin set — currently the issuing intermediate plus the root — -and accepts a connection only when the served chain passes baseline X.509 -evaluation AND at least one certificate anywhere in the chain matches a pin -(any-match). The leaf is deliberately **not** pinned, so routine leaf -renewals are a client no-op. The outage risk is a **chain change**: the +The native client SPKI-pins the server's TLS chain at the **CA level**: it +ships a small pin set and rejects a connection whose validated chain +contains no pinned key. The leaf is deliberately **not** pinned, so routine +leaf renewals are a client no-op. The pin set itself and its enforcement +rules live in the client repository, not here. The outage risk is a **chain change**: the served chain stops terminating in the pinned CA keys — the server moves to a different CA, or the CA rotates its intermediate / root keys. This page explains the coupling, the server-side alarm that watches the served leaf, @@ -15,11 +14,11 @@ and the operator / release-owner steps when it fires. A SPKI pin is `base64(sha256(DER subjectPublicKeyInfo))` — a hash of a certificate's public key, not of the whole certificate. The native client -carries a set of these pins (build-time config, ≥ 2 well-formed pins -enforced for release builds) and rejects any TLS connection whose validated -chain contains no pinned key. Hosts outside the client's pinned-host list -fall back to plain system-trust validation, which is the documented model -for self-hosted instances that ship their own chain. +carries a set of these pins in its build-time config and rejects any TLS +connection whose validated chain contains no pinned key; the exact pin-set +rules are the client repository's contract. Self-hosted instances that ship +their own chain sit outside the pinned-host list and validate against +system trust. The app host's certificate auto-renews roughly every 90 days and each renewal mints a new leaf keypair. Because the client pins the CA keys rather diff --git a/docs/ops/v141-followup-issues.md b/docs/ops/v141-followup-issues.md deleted file mode 100644 index ec5d22eb6..000000000 --- a/docs/ops/v141-followup-issues.md +++ /dev/null @@ -1,260 +0,0 @@ -# v1.4.x follow-up issues - -Items identified by the v1.4.1 audit pass that warrant deeper work -than a hardening release should land. Each one carries enough context -that a future contributor can pick it up without re-running the audit. - -## Security — deferred - -### S-FOLLOW-1 — Idempotency-Key concurrent duplicate side-effects (HIGH) - -Audit reference: `~/infra/reports/v14-security-audit.md` H1. - -The `withIdempotency()` wrapper currently does -`findCached → handler → persistCached`. Two requests with the same -`Idempotency-Key` arriving in parallel both miss the SELECT, both -execute the handler (duplicate measurements / intake events), and -one INSERT loses on the unique constraint silently. The repo-doc -contract ("retries with the same Idempotency-Key … replay the -original response — no second side-effect") is therefore violated -whenever a flaky mobile client double-sends. - -**Fix shape**: acquire the slot before the handler runs. -`INSERT … ON CONFLICT DO NOTHING RETURNING id` reserving the -`(userId, key, method, path)` row with a sentinel -`responseStatus = 0`, then UPDATE on completion; concurrent losers -SELECT-loop with backoff for the row to be filled. Or wrap the -whole request in a Postgres advisory lock keyed by hash(userId, key) -via `pg_advisory_xact_lock`. - -Why deferred: needs a careful integration test against the new -testcontainers suite (two concurrent inserts → exactly one handler -invocation) and a public-API contract check that the -sentinel-status row never bleeds into a 2xx replay. Fits a v1.4.2 -PR scope on its own. - ---- - -### S-FOLLOW-2 — Encryption key fallback NODE_ENV gate (MEDIUM) - -Audit reference: `~/infra/reports/v14-security-audit.md` M1. - -`src/lib/crypto.ts:61-74` only requires the 64-hex production key -when `process.env.NODE_ENV === "production"`. A fresh Docker run -that forgets to set `NODE_ENV=production` (raw `node …` invocations, -some operator setups) silently accepts a 32-character key and pads -it deterministically with a SHA-256 of itself, halving the entropy -and making the resulting key recoverable from any leaked partial. - -**Fix shape**: invert the gate to `NODE_ENV !== "development" && -NODE_ENV !== "test"` — fail-closed by default — or require an -explicit `ALLOW_WEAK_DEV_KEYS=1` opt-in. - -Why deferred: needs a migration note for self-hosters whose `.env` -files lack `NODE_ENV` (default Coolify deploys do set it; bespoke -setups may not). Worth a release note + .env.example call-out. - ---- - -### S-FOLLOW-3 — Refresh-token reuse-detection serialisation (MEDIUM) - -Audit reference: `~/infra/reports/v14-security-audit.md` M2. - -`src/lib/auth/refresh-token.ts` reuse-detection is a `findUnique` -followed by a non-transactional `findMany + updateMany`. Concurrent -first-time refreshes can briefly have both clients holding -unrevoked access tokens before the loser's are revoked, and a -parallel reuse attempt doubles the audit-log noise. - -**Fix shape**: wrap the rotation in a single `prisma.$transaction` -with `Serializable` isolation, OR reorder to "claim first, mint -second": `prisma.refreshToken.update({ where: { id, usedAt: null } })` -fails atomically if the row was already consumed; only after the -update succeeds, call `issueAccessAndRefresh`. - -Why deferred: needs an integration test that fires N parallel -refreshes for the same token and asserts exactly one new pair is -issued. The testcontainers suite is the right home; v1.4.2 PR. - ---- - -### S-FOLLOW-4 — moodLog webhook HMAC lookup column (MEDIUM) - -Audit reference: `~/infra/reports/v14-security-audit.md` M3. - -`src/app/api/integrations/moodlog/webhook/route.ts` performs an -O(n) AES-GCM decrypt sweep across all enabled users for every -webhook hit. Side-channels: timing oracle for ordering, DoS -amplification (30/min IP rate-limit × n decrypts). - -**Fix shape**: add a `moodLogWebhookSecretLookupHash` column on -`User` (HMAC-SHA-256 keyed by `API_TOKEN_HMAC_KEY`, populated -alongside the encrypted secret on every write). Webhook does a -single indexed `findUnique` on the lookup hash, bypassing the -candidate iteration. The encryption stays as defence-in-depth. - -Why deferred: schema migration required. Backfill needs a one-off -script that decrypts every existing row and writes the lookup -hash. Owners need to approve the migration window. v1.4.2 or -v1.5. - ---- - -### S-FOLLOW-5 — moodLog `readMoodLogSecret` legacy fallback (MEDIUM) - -Audit reference: `~/infra/reports/v14-security-audit.md` M4. - -If `decrypt()` throws inside `src/lib/moodlog-secret.ts:27-37`, the -function returns the stored value as-is. Intent: legacy plaintext -rows keep working. Side effect: an attacker with read access to -the DB obtains the ciphertext blob and can submit it as the literal -webhook secret. - -**Fix shape**: detect "looks like a v1.4 envelope" -(`/^[A-Za-z0-9_-]{1,32}\..+$/`) and refuse the legacy fallback for -envelope-shaped values. Log a metric so the rotation-on-write -contract can be verified to drain. - -Why deferred: needs a small data-audit run to confirm no production -row has a non-envelope value before the strict check goes live. - ---- - -### S-FOLLOW-6 — Restore script writes decrypted JSON with default permissions (MEDIUM) - -Audit reference: `~/infra/reports/v14-security-audit.md` M5. - -`scripts/restore-backup.ts:73-122` writes the decrypted JSON via -`writeFileSync(out, plaintext, "utf8")` with default mode (0644 -typical). The file contains decrypted PHI for the restored user. - -**Fix shape**: -`writeFileSync(out, plaintext, { encoding: "utf8", mode: 0o600 })`. -Optional: warn if `out` is in a world-readable directory. - -Why deferred: tiny one-line fix; bundling with the next ops-script -update keeps the PR coherent. - ---- - -## Performance — deferred - -### P-FOLLOW-1 — Recharts top-level import on `/insights` (P0) - -Audit reference: `~/infra/reports/v14-performance-audit.md` P0. - -`src/app/insights/page.tsx:42-50` static-imports -`{ ScatterChart, Scatter, XAxis, … } from "recharts"` even though -`HealthChart` and `MoodChart` 8 lines above are already wrapped in -`dynamic(…, { ssr: false })`. The static import pulls the whole -recharts bundle into the route's first JS payload, defeating the -dynamic split. - -**Fix shape**: extract the BP-mood scatter into -`src/components/charts/bp-mood-scatter.tsx` and import it via -`dynamic(…, { ssr: false })` like the others. - -Why deferred: trivial mechanical change but needs a Lighthouse -before/after to quantify the bundle-size win for the v1.4.2 -release notes. - ---- - -### P-FOLLOW-2 — `/api/insights/targets` unbounded glucose history scan (P0) - -Audit reference: `~/infra/reports/v14-performance-audit.md` P0. - -`src/app/api/insights/targets/route.ts:600-604` pulls every glucose -measurement the user ever logged (no `gte: thirtyDaysAgo`) and -filters in JS. Diabetic users with multi-year history will -materialise tens of thousands of rows on every targets fetch. - -**Fix shape**: split into "latest per context" -(`distinct: ["glucoseContext"], orderBy: { measuredAt: "desc" }`) -and a `measuredAt: { gte: thirtyDaysAgo }` window query, mirroring -the pattern at lines 102-121 for `latestEverByType`. - -Why deferred: needs a per-context test asserting the new query -returns the same payload shape. Pairs nicely with the MoodEntry -index migration (P-FOLLOW-3). - ---- - -### P-FOLLOW-3 — `MoodEntry @@index` is on `(userId, date)` not `(userId, moodLoggedAt)` (P0) - -Audit reference: `~/infra/reports/v14-performance-audit.md` P0. - -`prisma/schema.prisma:438` declares `@@index([userId, date])`, but -active queries sort/filter by `moodLoggedAt`: - -- `src/app/api/insights/targets/route.ts:495` -- `src/app/api/insights/comprehensive/route.ts:52` -- `src/app/api/export/route.ts:87` - -The unique constraint `@@unique([userId, date, moodLoggedAt])` is -not selectable by Postgres for windowed `moodLoggedAt`-range reads. - -**Fix shape**: add `@@index([userId, moodLoggedAt])` (do NOT drop -the existing index — the unique still serves the moodlog webhook -upsert path). New migration `0026_moodentry_moodloggedat_index`. - -Why deferred: schema migration. Owner approval for the migration -window. - ---- - -### P-FOLLOW-4 to -7 — P1 / P2 perf items - -- N+1 count() per medication on admin reminder-check - (`src/app/api/admin/notifications/reminder-check/route.ts:84`). - Replace with `groupBy` on `medicationIntakeEvent`. -- `/api/analytics` 8-parallel full-history scan - (`src/app/api/analytics/route.ts:20-40`). Single `findMany` with - `type: { in: types }` and JS-side group. -- `/api/import` sequential `prisma.measurement.create` per row - (`src/app/api/import/route.ts:77-96`). Replace with - `createMany({ skipDuplicates: true })`. -- `pairByTimestamp` not used in `/api/analytics` BP correlation - (`src/app/api/analytics/route.ts:80-86`). Replace the quadratic - loop with the helper from `@/lib/analytics/correlations`. - ---- - -## Test-quality polish (5 minor items) - -Audit reference: `~/infra/reports/v14-test-theatre-audit.md`. - -- `src/app/api/gamification/__tests__/ios-format.test.ts` L111-112: - upgrade `toBeDefined()` to shape predicates. -- `src/lib/analytics/__tests__/classifications.test.ts` L169-170: - add `toMatch(/60/)` body-content assertion to compliance-alert tests. -- `src/lib/__tests__/idempotency.test.ts` L145: upgrade bare - `toHaveBeenCalled()` to `toHaveBeenCalledWith({ where: …key… })`. -- `src/app/api/telegram/webhook/__tests__/route.test.ts` L457: same - shape — upgrade or drop. - ---- - -## i18n hardcoded strings - -Audit reference: `~/infra/reports/v14-i18n-drift-audit.md`. - -35+ hardcoded strings in `src/components/settings/ai-section.tsx` -need to be moved into `messages/{en,de}.json` keys. The settings -section was extracted in #143 with strings inline as a known -follow-up; this is the cleanup PR. - -Why deferred: low blast-radius (only the AI provider settings page -shows English to a German user; the dashboard, dialogs, and -critical UX paths all flow through `t(…)`). v1.4.x or v1.5. - ---- - -## Notes - -These are the items the v1.4.1 audit identified that the hardening -release deliberately did NOT take. The audit also found 0 hallucinated -medical claims, 0 i18n key-parity gaps, and the test theatre rate -across the entire codebase was low (5 minor patterns). The codebase -is in healthy shape; this list is the punch-list for the next -proactive iteration, not a backlog of bugs we know about and ignore. diff --git a/docs/self-hosting/geolite2.md b/docs/self-hosting/geolite2.md index 7910de171..98a1d9b73 100644 --- a/docs/self-hosting/geolite2.md +++ b/docs/self-hosting/geolite2.md @@ -95,10 +95,15 @@ fails soft. ```yaml volumes: - - /srv/healthlog/geolite2:/var/lib/geolite2:ro + - ./geolite2:/opt/geolite2:ro ``` -3. Point the runtime at the container path: + Swap the host side (`./geolite2`) for your directory, e.g. + `/srv/healthlog/geolite2`. + +3. `/opt/geolite2` is the default value of `GEOLITE2_DIR`, so mounting + at that container path needs no env var at all. Only if you mount at + a different container path do you set it: ```env GEOLITE2_DIR="/var/lib/geolite2" @@ -163,7 +168,8 @@ and it is served without a session for exactly that reason. **System status still says the online provider.** The readiness check requires `GeoLite2-City.mmdb` to exist in `GEOLITE2_DIR` and no `.empty` marker file beside it. Check the path from inside the container: -`docker compose exec app ls -l /var/lib/geolite2`. An empty listing means +`docker compose exec app ls -l /opt/geolite2` (or your `GEOLITE2_DIR` +value). An empty listing means the bind mount did not land; a `.empty` file means the image was built without a licence key and you are still reading the baked-in directory rather than your own. diff --git a/docs/self-hosting/getting-started.md b/docs/self-hosting/getting-started.md index 644d3ed8b..9053dae7c 100644 --- a/docs/self-hosting/getting-started.md +++ b/docs/self-hosting/getting-started.md @@ -17,7 +17,8 @@ already installed, longer if a TLS-fronted public hostname is in scope - **Outbound HTTPS** to `ghcr.io` (image pulls) plus whichever integration endpoints you plan to enable (Withings, OpenAI, etc.). The opt-in environmental-context module additionally reaches - `api.open-meteo.com` for weather and daylight; it stays off until you + `archive-api.open-meteo.com` and `geocoding-api.open-meteo.com` for + weather and daylight; it stays off until you enable it and set a home location, and every outbound call goes through the SSRF-guarded egress wrapper. - **Optional but recommended:** a reverse proxy that terminates TLS @@ -191,8 +192,9 @@ NEXT_PUBLIC_DASHBOARD_SNAPSHOT=false pnpm build `app_settings.singleton.registrationEnabled`. The admin panel can flip it back if a previous owner closed it. - **Service unreachable on `localhost:3000`.** Check `docker compose -ps` — the `app` healthcheck polls `/api/version` every 30 seconds - and will mark the container unhealthy if Postgres is still +ps` — the `app` healthcheck polls `/api/health` every 30 seconds + (it returns 503 while the database or worker is down) and will mark + the container unhealthy if Postgres is still starting. Tail `docker compose logs db` to confirm Postgres came up cleanly. - **OAuth callbacks loop back to localhost.** Confirm `APP_URL` and diff --git a/docs/self-hosting/mcp.md b/docs/self-hosting/mcp.md index 9afc189f3..1753438a1 100644 --- a/docs/self-hosting/mcp.md +++ b/docs/self-hosting/mcp.md @@ -30,7 +30,7 @@ Three things must be true before `/mcp` will answer: | -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **`APP_URL` is set** | The endpoint fails closed without a pinned canonical origin. `APP_URL` (or `NEXT_PUBLIC_APP_URL`) is the origin every OAuth document and audience check is anchored to — `Host` is never trusted. Without it the surface stays invisible (`404`). | | **The global API is on** | The operator's instance-wide API switch must be enabled (it is by default). With it off, every token path — including `/mcp` — returns `404`. | -| **The `mcp` module is on** | A per-account opt-in. It is the only module that ships dark; you turn it on in Settings. Until then `/mcp` answers `404` for that account. | +| **The `mcp` module is on** | A per-account opt-in that ships off by default; you turn it on in Settings. Until then `/mcp` answers `404` for that account. | `APP_URL` is the same origin you already use for passkeys and invite links — for example `https://health.example`. It must be reachable by the diff --git a/docs/self-hosting/reverse-proxy.md b/docs/self-hosting/reverse-proxy.md index b75df95b1..d1df00629 100644 --- a/docs/self-hosting/reverse-proxy.md +++ b/docs/self-hosting/reverse-proxy.md @@ -3,8 +3,8 @@ The bundled `docker-compose.yml` exposes HealthLog on plain HTTP at `localhost:3000`. Production deployments front the container with a reverse proxy that terminates TLS, sets the forwarded headers the -rate-limiter trusts, and preserves the WebSocket upgrade path the -streaming Coach surface depends on. +rate-limiter trusts, and passes the Coach's Server-Sent Events stream +through unbuffered with a generous read timeout. This guide gives copy-paste blocks for five common fronts. Pick one; the env-var pairing is identical across them. @@ -21,6 +21,12 @@ Set the URL pair to the public origin (`https://your-instance.example.com`) and restart the `app` container. The image reads both at startup; no rebuild needed. +Note for bundled-compose users: `TRUST_PROXY_HOPS` is not on the +shipped `docker-compose.yml` `environment:` whitelist, and a var not on +the whitelist never reaches the container. If you need a non-default +value, add `TRUST_PROXY_HOPS: "${TRUST_PROXY_HOPS:-1}"` to the `app` +service's `environment:` block yourself. + ### `TRUST_PROXY_HOPS` — what to set it to `src/lib/api-response.ts` (`getClientIp`) reads the `X-Forwarded-For` @@ -56,8 +62,8 @@ your-instance.example.com { } ``` -Caddy handles the WebSocket upgrade automatically. `flush_interval -1` -disables response buffering so the streaming insight events surface +`flush_interval -1` +disables response buffering so the SSE events surface without batching delays. With Caddy as the only hop, leave `TRUST_PROXY_HOPS=1`. @@ -78,8 +84,9 @@ services: - "traefik.http.routers.healthlog.middlewares=healthlog-headers" ``` -Traefik forwards `X-Forwarded-For` by default and respects the WebSocket -upgrade. Pair with `TRUST_PROXY_HOPS=1`. If Traefik sits behind +Traefik forwards `X-Forwarded-For` by default and does not buffer +responses, so the SSE stream passes through as-is. Pair with +`TRUST_PROXY_HOPS=1`. If Traefik sits behind Cloudflare, set it to `2` so the rate-limiter reads the real client IP that Cloudflare placed in the chain. @@ -91,8 +98,9 @@ In the NPM UI: 2. **Domain Names:** `your-instance.example.com`. 3. **Scheme:** `http`; **Forward Hostname / IP:** the Docker host or internal hostname; **Forward Port:** `3000`. -4. Enable **Websockets Support** (required for streaming Coach - responses). +4. **Websockets Support** may stay on or off; the Coach stream is + SSE over plain HTTP, not WebSockets. What matters is the buffering + and timeout config in step 6. 5. **SSL tab:** request a Let's Encrypt certificate, force SSL. 6. **Advanced tab — Custom Nginx Configuration:** diff --git a/docs/self-hosting/scaling.md b/docs/self-hosting/scaling.md index 7d9f00408..940abdc4d 100644 --- a/docs/self-hosting/scaling.md +++ b/docs/self-hosting/scaling.md @@ -88,10 +88,14 @@ endless spinner. ## Healthchecks -- `app` (web) healthcheck: `wget /api/version` every 30s. -- `app-worker` does not expose a port; its liveness is reported into - the `worker_status` table on every reminder pass and surfaced through - `/api/admin/status`. +- `app` (web) healthcheck: `wget /api/health` every 30s. +- `app-worker` does not expose a port; its liveness is tracked + in-memory inside the process (`src/lib/jobs/worker-status.ts`) and + read back by `/api/admin/status`. In a web/worker split this means + the web container cannot see the separate worker container's + liveness: `/api/admin/status` only reports the worker embedded in + the container that serves the request. Watch the worker container's + own logs and restart policy instead. The web container does NOT depend on the worker, and vice versa, so neither container's startup can deadlock the other. The shared @@ -112,13 +116,17 @@ dependency is Postgres; both wait on `db: condition: service_healthy`. ## Postgres connection-pool sizing -Every container that boots the HealthLog image opens its own -`pg.Pool` against the configured `DATABASE_URL`. The pool ceiling is -**20 connections per container by default** (raised from the library -default of 10 in v1.4.40 to keep the dashboard's analytics fan-out -from starving every other query during a cold mount). +Every container that boots the HealthLog image owns a connection +budget against the configured `DATABASE_URL`. The budget is +**20 connections per container by default** and is split between the +Prisma pool and pg-boss (2 connections). The bundled compose exposes +the knob as `DB_CONNECTION_LIMIT`, baked into `DATABASE_URL` as +`connection_limit`; `DATABASE_POOL_MAX` remains a working explicit +override. Resolution order: `DB_CONNECTION_LIMIT`, then +`DATABASE_POOL_MAX`, then the `connection_limit` URL parameter, then +the default of 20. -Plan total Postgres slots as **container_count × `DATABASE_POOL_MAX`**. +Plan total Postgres slots as **container_count × budget**. A stock Postgres 16 container ships with `max_connections = 100`, so the safe envelope is: @@ -139,9 +147,10 @@ transaction-pooling mode. ### Overriding the per-container pool ceiling -Set `DATABASE_POOL_MAX` to a positive integer on every container that -needs a non-default ceiling — both web and worker pick up the same -env var: +With the bundled compose, set `DB_CONNECTION_LIMIT` in `.env`; it +flows into `DATABASE_URL`'s `connection_limit`. On a hand-rolled +setup, `DATABASE_POOL_MAX` as a plain env var works the same way on +every container, web and worker alike: ```yaml # docker-compose.yml — example for a 6-container deployment @@ -176,6 +185,7 @@ tenant, starving every other dashboard query for the duration. The fan-out — keeps 16 slots free for the rest of the dashboard while still sitting well under Postgres's 100-slot stock ceiling. -The implementation lives in `src/lib/db.ts → getPoolMax()`; the +The implementation lives in `src/lib/db.ts → getConnectionBudget()` +(split into `getPrismaPoolMax()` and `getPgBossPoolMax()`); the 20-slot default was chosen after measuring on a 4-CPU production -container — see the inline rationale in `getPoolMax()`. +container — see the inline rationale in `src/lib/db.ts`. diff --git a/docs/self-hosting/sso.md b/docs/self-hosting/sso.md index 853446fe6..deb826f02 100644 --- a/docs/self-hosting/sso.md +++ b/docs/self-hosting/sso.md @@ -115,9 +115,11 @@ Client registration essentials only — the IdP's own docs cover the rest. (the routes return 403; hiding the buttons is not the boundary). Consequences to accept before turning it on: -- **The native iOS app cannot sign in.** It authenticates via - password/passkey only; there is no native SSO flow yet. Do not enable - `OIDC_ONLY` on a deployment with active iOS users. +- **The native iOS app needs a build that ships the SSO flow.** The app + signs in via the native OIDC handoff (`GET /api/auth/oidc/login?client=native` + with PKCE and the fixed `healthlog://` callback) or via password/passkey. + `OIDC_ONLY` works with app builds that carry the handoff; older + password/passkey-only builds are locked out. - **Existing sessions survive.** The flag gates new sign-ins, not live sessions — nobody is thrown out at flip time, including password-authenticated sessions. diff --git a/docs/self-hosting/unraid.md b/docs/self-hosting/unraid.md index 062562321..91e5c89d6 100644 --- a/docs/self-hosting/unraid.md +++ b/docs/self-hosting/unraid.md @@ -7,7 +7,8 @@ Unraid UI, but because CA templates describe a single container you install PostgreSQL separately and wire the two together. HealthLog stores all persistent state in PostgreSQL — the app container itself is -stateless (the offline geo-IP databases are baked into the image). A backup of +stateless (the offline geo-IP databases are either baked in at build time or +mounted read-only; replaceable either way, not state). A backup of the database is a backup of everything. ## What you need diff --git a/docs/ui-guidelines.md b/docs/ui-guidelines.md index 0f2874ec7..f908d4c2a 100644 --- a/docs/ui-guidelines.md +++ b/docs/ui-guidelines.md @@ -320,24 +320,18 @@ tile itself, to avoid accidental hide. ### 3.3 Settings — `/settings/[section]` -The 1.4 release splits the legacy 3150-LOC settings monolith into one -route per top-level section: +The 1.4 release split the legacy 3150-LOC settings monolith into one +route per top-level section. The pattern holds: every section is a +route under `/settings/[section]` with its component under +`src/components/settings/`. The section list has grown far past the +original eight (account, integrations, notifications, ai, api, +advanced, about now sit alongside security, access, sources, coach, +environment, export, mcp, and more); the registry in +`src/components/settings/settings-shell.tsx` is authoritative. +Cross-section search lives in the section sidebar header. -- `/settings/account` — profile, password, passkeys, account deletion. -- `/settings/integrations` — Withings, Nightscout, Telegram, ntfy, Web Push. -- `/settings/notifications` — channel matrix + quiet hours + reminders. -- `/settings/dashboard` — tile layout editor, default ranges, threshold - overrides. -- `/settings/ai` — AI provider selection + connection + test. -- `/settings/api` — API tokens for headless ingest + Bearer issuance. -- `/settings/advanced` — exports, imports, danger zone. -- `/settings/about` — version, build, license, links to docs/changelog. - -Each section's component lives at `src/components/settings/
.tsx` -and stays under 400 LOC. Cross-section search lives in the section -sidebar header. - -The legacy `/settings` route 301-redirects to `/settings/account`. +The legacy `/settings` route is a permanent (308) redirect to +`/settings/account`. ### 3.4 Admin — `/admin/[section]` @@ -505,8 +499,11 @@ is the point. ## 7. Internationalization -All user-facing strings come from `messages/en.json` (default) and -`messages/de.json`. Use `useTranslations()` on the client and +All user-facing strings come from `messages/en.json` (default) plus +the `de`, `es`, `fr`, `it`, and `pl` bundles beside it. Guard tests +(`i18n-call-site-coverage`, `i18n-locale-integrity`) enforce that every +`t()` key resolves in EN and propagates across all six locales. Use +`useTranslations()` on the client and `getServerTranslator()` (`src/lib/i18n/server-translator.ts`) on the server. Numbers, dates, currencies, and units always go through `useFormatters()` — never hand-roll `Intl.NumberFormat(...)` with a