WIP toward v1.4.35. Lands the persistent rollup foundation from the
v1.4.34 cache-aggregation research §9. Reads still served by the
v1.4.34.1 live-SQL aggregator — the parity-safe read-swap
(count/min/max/mean + dailyByType from buckets, slope/r2/sd stay on
live aggregation) ships in a follow-up commit so v1.4.35 is the
single release that flips both foundation + read-swap.
Schema:
- new MeasurementRollup model with composite PK
(userId, type, granularity, bucketStart) and a descending
bucketStart index. RollupGranularity enum (DAY/WEEK/MONTH/YEAR).
- Prisma migration 0067_v1434_measurement_rollups is additive:
CREATE TYPE / CREATE TABLE IF NOT EXISTS / CREATE INDEX IF NOT
EXISTS / ADD CONSTRAINT FK. No ALTER on existing tables.
- db-compat schema-bootstrap path mirrored.
Populator (src/lib/measurements/rollups.ts):
- recomputeBucketsForMeasurement: sync DAY; WEEK/MONTH/YEAR via
pg-boss (singleton-keyed).
- recomputeUserRollups: bounded full-range, used by Apple Health
import + backfill.
- ensureUserRollupsFresh: cheap stale-detection + bounded DAY-90d
recompute. Error-swallowing.
- SQL via $queryRawUnsafe + strict enum whitelist for date_trunc
unit (Postgres 42803 forbids binding the unit literal).
Write hooks on every measurement-mutating endpoint; each wrapped
in try/catch so a populator hiccup never fails the write.
Apple Health import worker recomputes the full range once at
completion (per-row hooks would multiply cost).
Backfill (scripts/backfill-rollups.ts): idempotent, single-user
serial, single-digit minutes for a 100k-row account.
Tests: +14 unit, +3 integration. setup.ts.truncateAllTables
extends to the new table.
Read-swap follow-up:
- comprehensive-aggregator + analytics/route + summaries-slice
each call ensureUserRollupsFresh as a read side-effect (warms
the table). The response stays on the live-SQL aggregator so
byte-shape parity with v1.4.34.5 is preserved.
- Next commit lands the partial read-swap: count/min/max/mean
and dailyByType from buckets (math-identical compose), while
slope/r2/sd stay on indexed live aggregation. Together they
ship as v1.4.35.
Pre-shipping state — do NOT tag v1.4.35 from this commit alone.
Adds the integration suite for the detection worker against a real
Postgres testcontainer. Covers measurement-driven PRs (steps,
RHR, VO2 max), idempotency on re-run, silent-flag propagation,
and the drift guard that every PR-trackable metric type lands in
the scan loop.
Also patches a pre-existing schema gap: the
(userId, metricType, metricSlot, achievedAt) unique index on
personal_records is NULLS-DISTINCT in Postgres, so the index alone
does not dedup the measurement-driven rows (metricSlot = NULL).
The worker now does an application-level pre-flight findFirst on
the same composite key before each insert, keeping both the
workout path (index covers it) and the measurement path
(application-level guard) idempotent without a Migration.
Concurrent-write integration coverage that validates the W10 fix-C
reconciliation pattern carries over correctly to the
/api/workouts/batch endpoint:
- Two batches with overlapping (userId, source, externalId) tuples
posted in parallel resolve to exactly one DB row per tuple (the
composite unique index enforces it; createMany.skipDuplicates
absorbs duplicate-key conflicts).
- Each response body's per-entry envelope sums to its own
aggregate counts — the invariant the iOS sync cursor relies on
to checkpoint correctly.
- Aggregate inserted counts across both responses equal the row
count actually present, cross-validating that createMany.count
is the truth source rather than the older
'stored vs not-stored' diff (which was a no-op under standard
Postgres semantics because every attempted row IS in the DB
after the call, written by whichever batch won the race).
- A two-batch shared-key fixture asserts the shared tuple lands
exactly once across both responses.
Also adds workouts and workout_routes to the integration suite's
truncate list so beforeEach() actually clears them between tests
rather than relying on CASCADE from users.
Persist Coach chat threads with AES-256-GCM-encrypted message bodies
and a per-user, per-day token-spend ledger. Encryption reuses the
active ENCRYPTION_KEYS entry so rotation continues to work without a
backfill. metric_source_json carries provenance labels only (window +
metric names, never raw values), keeping it queryable for analytics.
Cascade-on-user-delete is wired through the FK chain so GDPR delete
removes Coach data with the rest of the account. The integration-suite
truncation list picks up the new tables.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the per-minute snapshot table that backs the new admin host-load
chart above /admin/system-status. One row per minute, captured by the
pg-boss sampler in the next commit, with 7-day retention enforced
inside the worker so the table stays under a few thousand rows.
Schema mirrors what the chart needs:
- load_avg_1/5/15 (DOUBLE PRECISION) — os.loadavg() output
- mem_used_bytes / mem_total_bytes — derived from os.totalmem() and
os.freemem(), kept as raw bytes so the API can compute either a
percent or a MiB delta downstream
- disk_read_bytes / disk_write_bytes — nullable cumulative byte
counters from /proc/diskstats. NULL on macOS/Windows dev hosts;
Linux production picks them up automatically. The API endpoint
converts deltas to bytes-per-second between samples
Indexed by captured_at so the "last 2 hours" query stays index-only.
Migration 0032_host_metric. integration setup.ts adds host_metrics to
the truncate list so testcontainers stays clean across runs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
v1.4.15 Phase B2, criterion 1. Until this release the Settings →
Integrations card showed "Connected / Last sync 23 min ago" and
nothing else. A user whose Withings refresh token had revoked saw a
green badge; a user whose moodLog instance was down saw the same
green badge with a slightly older timestamp. The signal was useless.
This commit adds:
- New `IntegrationStatus` table (`integration_statuses` migration
`0029`). One row per (user, integration) capturing `state`
(`connected | error_transient | error_reauth | disconnected`),
`lastSuccessAt`, `lastAttemptAt`, encrypted `lastError`,
`consecutiveFailures` counter, `alertedAt` window-guard.
- `src/lib/integrations/status.ts` — the only writers
(`recordSyncSuccess`, `recordSyncFailure`, `markReauthRequired`,
`markReconnected`, `markDisconnected`) plus a read-only
`getIntegrationStatus()` for the API. The error column is
AES-256-GCM encrypted at rest because Withings/moodLog 401s can
echo URL/apiKey fragments — we can't ship that into a future
backup tarball.
- `GET /api/integrations/status` — single roundtrip returns both
Withings and moodLog snapshots plus the global threshold (so the
UI's "{n}/{threshold}" string is server-sourced and rotates with
the env var).
- Settings → Integrations cards both gain an
`IntegrationStatusBanner` row above the credentials form: state
badge ("Connected" / "Reconnect required" / "Error" /
"Disconnected"), `{n}/{threshold} consecutive failures` chip,
last-success / last-attempt times, and a destructive-toned
last-error line. The banner is suppressed entirely when the
integration is fresh-connected with no history (no false alarms).
Tests: +20 unit (status helper threshold + alert-window state machine,
banner rendering across all four states) + 8 integration
(`tests/integration/integration-status.test.ts`) confirming
encrypted-at-rest, audit-log row shape, alert idempotency window,
and CASCADE-delete on User. 116 files / 883 unit pass, 11 integration
files / 39 pass.
Encryption uses the existing `src/lib/crypto.ts` AES-256-GCM helper —
no new key material. The `consecutive_failures` counter and
`alerted_at` window are wired to the dispatcher in the next commit
(`feat(integrations): admin Telegram alert on persistent sync
failure`).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Revert "revert: roll source tree back to v1.4.0 to restore production"
This reverts commit e444a90b10.
* fix(v1.4.2): summaries.WEIGHT + medications array guards + healthcheck IPv4
Three production bugs surfaced after the v1.4.0 deploy stabilized:
1. `TypeError: undefined is not an object (evaluating 'E?.summaries.WEIGHT')`
on /dashboard and /insights for users without weight readings —
the optional chain stopped at `data` when it needed to also cover
`data.summaries`. Fixed: `data?.summaries?.WEIGHT` (one level deeper).
2. `TypeError: y?.filter is not a function` on /medications — the
medications query value was non-array (likely a stale
service-worker response from an older API shape). Replaced with
`Array.isArray(medications) ? medications : []` so the page is
immune to any future shape drift.
3. Healthcheck-induced 503 outage (already in 1.4.1 — included here
so 1.4.2 is the single coherent release). The docker-compose
healthcheck used `localhost` which busybox-wget resolves to IPv6
::1; Next.js standalone listens on IPv4 0.0.0.0 only. Pinned
to 127.0.0.1.
This release also includes everything from v1.4.1 (which never
published a clean GHCR tag): the per-section admin extraction, the
testcontainers integration suite, the Playwright + axe-core E2E
foundation, the moodLog SSRF fix, the central error-redaction in
WideEventBuilder + Glitchtip, the medical-citation drift fixes
(ESH 2023, Saint-Maurice attribution, steps source), and the docs
sync.
Co-Authored-By: Marc-André Bombeck <mbombeck@gmail.com>
Production at healthlog.bombeck.io has been 503-ing since the v1.4.1
deploys started landing on apps-01 (Coolify). The container boots —
Next.js prints "Ready" and the pg-boss workers run — but never
accepts HTTP on :3000, so the Docker healthcheck fails and Traefik
takes the upstream out of rotation. A manual restart, a Coolify
force-rebuild, and a docker-compose pin to the GHCR :1.4.0 multi-arch
image all failed to bring the site back up — Coolify rebuilds the
image from main HEAD on every deploy regardless of the compose
directives.
This commit resets the working tree to commit 21bd46d (v1.4.0
release). Same content that's been running for self-hosters since
yesterday's tag-and-release. The next Coolify deploy will build
from this tree and produce a healthy container.
The v1.4.1 work is NOT lost:
- PRs #144, #145, #137, #146, #147, #148, #149, #150 remain in
git history.
- Their commits are still tagged (`v1.4.1`), still on the GHCR
multi-arch image (`ghcr.io/mbombeck/healthlog:1.4.1`), still in
the GitHub Release notes.
- Self-hosters who have already pulled the v1.4.1 image keep it.
- Local development continues from main HEAD with the v1.4.1
code — the regression only surfaced under the Coolify build
flow.
Re-applying v1.4.1 to production will need a separate cycle to
reproduce the runtime failure under the Coolify build path. That
work is tracked in docs/ops/v141-followup-issues.md (added back
when the tree is reapplied) and the deploy gating in
.github/workflows/e2e.yml will catch this class of bug going
forward.
No DB migration. No env-var change. No API contract change.
Co-Authored-By: Marc-André Bombeck <mbombeck@gmail.com>
* test(integration): Postgres testcontainers + 4 critical-path tests
Adds an integration test suite that runs against a real Postgres booted
in-process via testcontainers. Lives under tests/integration/, runs via
`pnpm test:integration`, and is excluded from the default `pnpm test`
loop so the 477-test unit suite stays fast (~1.2s).
Covers four regression-prone behaviours that pure unit tests cannot
validate against a mocked client:
- rate-limit.test.ts — atomic UPSERT under concurrency (5/6 allowed)
and window-reset semantics for src/lib/rate-limit.ts
- idempotency-replay.test.ts — cached replay with X-Idempotent-Replay,
expired-row purge, and the do-not-cache contract for 401/403/408/429/5xx
on top of src/lib/idempotency.ts
- cascade-delete.test.ts — GDPR Art. 17 erasure: user.delete wipes
every onDelete:Cascade table and SetNulls AuditLog/Feedback
- auth-flow.test.ts — createSession/getSession round-trip plus
expired-session purge for src/lib/auth/session.ts
Adds .github/workflows/integration.yml triggering on pull_request and
push to main. Local Docker is not running in this worktree environment;
the suite was verified by collecting all 10 tests via `vitest list` and
will execute in CI.
Co-Authored-By: Marc-André Bombeck <mbombeck@gmail.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(integration): rebase onto v1.4 + exclude live agent worktrees from default test scope
Rebase brings PR #137's testcontainers infrastructure on top of the
user-locale migration drift fix landed in #140 — the integration suite
should now bootstrap a fresh Postgres container with all 25 migrations
applied cleanly.
vitest.config.mts also gains a `.claude/worktrees/**` exclude so live
agent worktrees (which mirror `src/` while their parent agents are
running) don't double-execute the project's tests against possibly-stale
copies.
* test(integration): boot one Postgres container per run via vitest globalSetup
The first iteration of the testcontainers suite started/stopped a
fresh container in each test file's beforeAll/afterAll. That works
in isolation but breaks when files share a worker (`pool: forks`,
`fileParallelism: false`, `isolate: false`):
1. Test file A boots container 1, sets DATABASE_URL.
2. The application's Prisma singleton in src/lib/db.ts is built
when test code dynamic-imports @/lib/auth/session — it captures
container 1's URL.
3. afterAll stops container 1.
4. Test file B boots container 2, sets DATABASE_URL to a new port.
5. Test file B imports application code that resolves to the same
singleton — still pointing at the dead container 1. Queries fail
with a generic PrismaClientKnownRequestError.
Fix: move the container lifecycle to vitest's globalSetup. One
container, booted once, torn down at the end of the run. Tests still
get isolation because beforeEach truncates every personal-data table
in dependency-safe CASCADE order.
- tests/integration/global-setup.ts boots postgres:16-alpine,
runs `pnpm db:migrate:deploy`, exports a teardown for end-of-run.
- tests/integration/setup.ts now exports getPrismaClient() (which
delegates to the application singleton) and truncateAllTables().
No more startTestDb / stopTestDb.
- All four test files lose their beforeAll/afterAll boilerplate.
- vitest.integration.config.mts wires the new globalSetup.
- truncate list adds refresh_tokens (added in 0025_refresh_tokens).
Result: pnpm test:integration → 10/10 pass in 3.9s (was 6/10 with
4 cross-file failures). pnpm test → 658/658 unchanged.
Co-Authored-By: Marc-André Bombeck <mbombeck@gmail.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Marc-André Bombeck