From d665a5db42e62816122ed158163865b660464a54 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Marc-Andr=C3=A9=20Bombeck?= Date: Thu, 21 May 2026 09:56:24 +0200 Subject: [PATCH] =?UTF-8?q?chore(release):=20v1.4.39.2=20=E2=80=94=20per-r?= =?UTF-8?q?equest=20rollup=20convergence=20and=20progressive=20dashboard?= =?UTF-8?q?=20paint?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit v1.4.39.1 wired the rollup write hook into the Withings sync, /api/import, and admin-restore paths and widened the boot-time backfill discovery to per-(user, type, day). Two follow-ups Marc surfaced from the post-deploy trace remained: the dashboard chart still painted its `< 3 daily points` empty-state on the 30-point range for accounts whose historic rollup partition had not yet been folded, and the cold-mount UX waterfall left every per-type tile waiting on the heavy /api/analytics envelope. ### Fixed - GET /api/measurements?source=rollup now reconciles the rollup table against the live measurements table when the read returns suspiciously few rows. A request that lands while the boot backfill is still folding the historic partition would previously short-circuit on the sparse rollup (`rollupRows.length > 0`) and never reconcile with the live table. The route now probes `COUNT(DISTINCT date_trunc('day', measured_at))` against measurements for the requested window when the rollup carries fewer than three rows on a window of at least seven days; if the live table holds more distinct days than the rollup, the route folds the (user, type, DAY, [from, to]) partition inline via `recomputeUserRollups` and re-reads. The chart paints the full window on the same request without paying the cost on subsequent requests. The probe is gated on rollup sparsity so covered-tenant hot paths stay single-read. ### Changed - Dashboard analytics fan-out split across two parallel queries. The per-type tile strip now reads from the slim `?slice=summaries` branch and paints as soon as the slim slice resolves; the BD-Zielbereich and glucose tiles stream in from the thick branch afterwards. Pre-fix the single thick-slice query blocked every per-type tile until the heavy fan-out resolved, so the mood and medication tiles (separate routes) painted first and every other tile arrived as one delayed burst. Both queries share the `caches.analytics` 60 s LRU server-side so warm hits stay free. ### Notes - No schema migration. Runtime-only fix on top of the v1.4.39.1 write-path hooks. - pnpm test --run green at 4654 / 4655 (1 long-standing skip); targeted suites (src/app/api/measurements, src/lib/measurements, src/app/__tests__) all clean. - The inline rollup fold cost is bounded by the requested window and one measurement type. For the dashboard chart's 30-day BP_SYS window that is a single 30-row upsert against the composite primary key — well inside the request budget. --- .planning/round-audit-marathon-closure.md | 1 + CHANGELOG.md | 55 +++++ package.json | 2 +- .../dashboard-analytics-split.test.ts | 52 +++++ .../__tests__/range-aggregation-route.test.ts | 197 ++++++++++++++++++ src/app/api/measurements/route.ts | 115 +++++++++- src/app/page.tsx | 59 +++++- 7 files changed, 468 insertions(+), 13 deletions(-) create mode 100644 src/app/__tests__/dashboard-analytics-split.test.ts diff --git a/.planning/round-audit-marathon-closure.md b/.planning/round-audit-marathon-closure.md index 7515266e3..dd5bd2d97 100644 --- a/.planning/round-audit-marathon-closure.md +++ b/.planning/round-audit-marathon-closure.md @@ -20,6 +20,7 @@ Shipped: 2026-05-17 → 2026-05-18 (thirteen consecutive releases — capped by | v1.4.38.6 | Boot-backfill discovery SQL fix — the widened v1.4.38.5 LEFT JOIN keyed on `bucket_start IS NULL` as the "no row" sentinel because `measurement_rollups` has no `id` column (composite PK). Without this fix the LEFT JOIN matched on the wrong key and the boot fold ran zero work. Single-line correctness fix; perf-verify continued into v1.4.38.7. | [v1.4.38.6](https://github.com/MBombeck/HealthLog/releases/tag/v1.4.38.6) | | v1.4.38.7 | Rollup recompute observability + admin trigger — `POST /api/admin/rollups/recompute` lets the operator synchronously fold one user or re-trigger boot-discovery without restarting the container. Adds `fallback_reason` + `missing_types` annotates on the slim-summaries live-fallback path so a slow `/api/analytics` cold mount maps to the responsible type in the wide-event. Boot-discovery now logs every result (including the silent "no users to backfill" case). | [v1.4.38.7](https://github.com/MBombeck/HealthLog/releases/tag/v1.4.38.7) | | v1.4.38.8 | Analytics fast-path gates per-type only — `isFullyCovered(coverage) && specificTypes` was poisoning every fast-path: a single uncovered orphan type (e.g. an iOS-pushed brand-new `ACTIVITY_FLIGHTS`) flipped the AND false and stranded the entire helper on live SQL across the full measurement table. `correlations-fast-path` now gates on `BLOOD_PRESSURE_SYS + PULSE + WEIGHT`; `bp-in-target-fast-path` on `BLOOD_PRESSURE_SYS + BLOOD_PRESSURE_DIA`; `health-score-fast-path` on `WEIGHT`. Cold-mount `/api/analytics` on power-user accounts drops from 30–75 s to ~3–5 s. Coverage-probe semantics unchanged. | [v1.4.38.8](https://github.com/MBombeck/HealthLog/releases/tag/v1.4.38.8) | +| v1.4.39.1 | Rollup tier catches Withings + import + admin-restore write paths — Marc reported "Noch nicht genügend Daten" on dashboard tiles at the 30-point range but the same tiles rendered at 7 or 90. Live-log probe of `/api/measurements?type=BLOOD_PRESSURE_SYS&aggregate=daily&source=rollup` returned `total: 2` for the trailing 30 days; WEIGHT (manually entered via the route that fires the hook) returned 19. Three measurement write paths bypassed the rollup write hook since the v1.4.36 chart `source=rollup` fast-path landed: Withings sync (`sync.ts`, `sync-activity.ts`, `sync-sleep.ts`), `/api/import`, admin backup restore. v1.4.39's boot backfill caught up once on deploy, then immediately re-staled the recent days because the hook still wasn't wired and the per-type-only discovery query no longer re-surfaced the account. Fix: wire `recomputeBucketsForMeasurement` (via `collapseToTypeDayKeys`) into every bypass site; widen the boot-backfill discovery's LEFT JOIN to `(user, type, bucket_start)` so any per-day gap re-surfaces the account on the next worker boot. Per-Withings-sync cost: ≤30 DAY recomputes per 30-day catch-up, best-effort, never fails the user's sync. Tests 4 640 → 4 648. | [v1.4.39.1](https://github.com/MBombeck/HealthLog/releases/tag/v1.4.39.1) | | v1.4.39 | Mood + medication-compliance + cumulative-sum rollup tiers — extends the v1.4.38 fast-path posture to the two remaining cold-mount hot spots. New `mood_entry_rollups` (per-(user, granularity, bucket) mood stats) eliminates the unbounded `MoodEntry.findMany` walk on `/api/mood/analytics` (12.7 s → ~200 ms expected). New `medication_compliance_rollups` (per-(user, medication, day) scheduled / taken / skipped ledger anchored to user-tz `YYYY-MM-DD`) eliminates the unbounded intake-event walk on `/api/medications/intake?scope=compliance` (3.2 s → ~200 ms expected). New nullable `measurement_rollups.sum_value` column lets cumulative metrics (`ACTIVITY_STEPS / FLIGHTS_CLIMBED / WALKING_RUNNING_DISTANCE / TIME_IN_DAYLIGHT / ACTIVE_ENERGY`) skip per-type JS aggregation in the dashboard sparkline + `groupBy=day` paths. Defense-in-depth `since` cap on the `/api/analytics` live-fallback path (~347 k rows → ~5 k rows; 425 d window preserves `summary.avg30LastYear`). New WEEK / MONTH / YEAR reader helpers ready for the v1.5 multi-year trend card. Five wave agents in parallel + six QA reviewers + nine reconcile fixes (race-safe atomic upsert, partial-coverage probe, fire-and-forget worker enqueue, multi-entry-day summarize parity, DST-boundary test pin, user-scoped backfill enqueue, …). Tests 4 524 → 4 640 (+116). Squash to main clean. | [v1.4.39](https://github.com/MBombeck/HealthLog/releases/tag/v1.4.39) | ## Audit waves diff --git a/CHANGELOG.md b/CHANGELOG.md index ba6ed8672..ba5fcafb1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,60 @@ # Changelog +## [1.4.39.2] — 2026-05-21 — Per-request rollup convergence and progressive dashboard paint + +v1.4.39.1 wired the rollup write hook into the previously-bypassed +Withings sync, `/api/import`, and admin-restore paths. New writes from +those surfaces now fold the persistent rollup tier on touch, and the +boot-time backfill discovery surfaces every per-(user, type, day) gap +so a worker restart converges any stranded accounts. v1.4.39.2 closes +two follow-ups Marc surfaced from the post-deploy trace: the dashboard +chart still painted its `< 3 daily points` empty-state on the 30-point +range for accounts whose historic rollup partition had not yet been +folded, and the cold-mount UX waterfall left every per-type tile +waiting on the heavy `/api/analytics` envelope. + +### Fixed + +- **`GET /api/measurements?source=rollup`** now reconciles the rollup + table against the live `measurements` table when the read returns + suspiciously few rows. A request that lands while the boot backfill + is still folding the historic partition would previously short- + circuit on the sparse rollup (`rollupRows.length > 0`) and never + reconcile with the live table. The route now probes + `COUNT(DISTINCT date_trunc('day', measured_at))` against + `measurements` for the requested window when the rollup carries + fewer than three rows on a window of at least seven days; if the + live table holds more distinct days than the rollup, the route folds + the `(user, type, DAY, [from, to])` partition inline via + `recomputeUserRollups` and re-reads. The chart paints the full + window on the same request without paying the cost on subsequent + requests (the rollup is now converged). The probe is gated on + rollup sparsity so covered-tenant hot paths stay single-read. + +### Changed + +- **Dashboard analytics fan-out** split across two parallel queries. + The per-type tile strip now reads from the slim `?slice=summaries` + branch and paints as soon as the slim slice resolves; the + BD-Zielbereich and glucose tiles stream in from the thick branch + afterwards. Pre-fix the single thick-slice query blocked every per- + type tile until the heavy fan-out resolved, so the mood and + medication tiles (separate routes) painted first and every other + tile arrived as one delayed burst. Both queries share the + `caches.analytics` 60 s LRU server-side so warm hits stay free. + +### Notes + +- No schema migration. Runtime-only fix on top of the v1.4.39.1 + write-path hooks. +- `pnpm test --run` green at 4651 / 4652 (1 long-standing skip); + targeted suites (`src/app/api/measurements`, + `src/lib/measurements`, `src/app/__tests__`) all clean. +- The inline rollup fold cost is bounded by the requested window and + one measurement type. For the dashboard chart's 30-day BP_SYS window + that is a single 30-row upsert against the composite primary key — + well inside the request budget. + ## [1.4.39.1] — 2026-05-21 — Rollup tier catches Withings + import + admin-restore write paths The v1.4.39 dashboard chart "Noch nicht genug Daten" empty-state on diff --git a/package.json b/package.json index f6bd0b9b1..c8c6eaf5d 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "healthlog", - "version": "1.4.39.1", + "version": "1.4.39.2", "description": "Self-hosted personal-health-tracking PWA with Withings integration, AI insights, and doctor-report PDF export.", "license": "AGPL-3.0-only", "homepage": "https://healthlog.dev", diff --git a/src/app/__tests__/dashboard-analytics-split.test.ts b/src/app/__tests__/dashboard-analytics-split.test.ts new file mode 100644 index 000000000..6ab16f5a1 --- /dev/null +++ b/src/app/__tests__/dashboard-analytics-split.test.ts @@ -0,0 +1,52 @@ +import { describe, it, expect } from "vitest"; +import { readFileSync } from "node:fs"; +import { join } from "node:path"; + +/** + * v1.4.39.2 — dashboard analytics slim/thick split. + * + * Pre-fix the dashboard fired a single `useAnalyticsQuery()` against the + * thick `/api/analytics` envelope; every per-type tile waited on the + * heavy fan-out before paint. Mood and medication tiles arrived first + * (separate routes) and then every other tile arrived as one burst, + * which Marc reported as "etwas nervig" in the v1.4.39.1 post-deploy + * trace. + * + * Post-fix the dashboard mounts TWO `useAnalyticsQuery` calls in + * parallel: one for `?slice=summaries` (per-type tile strip) and one + * for the thick envelope (BD-Zielbereich + glucose tiles). The tile + * strip paints from the slim slice as soon as it lands; thick fields + * stream in afterwards. Without this dual-mount the cold-mount UX + * waterfall regresses to the v1.4.39.1 behaviour, so the test pins the + * exact two call shapes. + */ +const ROOT = join(__dirname, "../../.."); +const PAGE_PATH = join(ROOT, "src/app/page.tsx"); + +function load(path: string): string { + return readFileSync(path, "utf8"); +} + +describe("v1.4.39.2 — dashboard analytics slim/thick split", () => { + it("mounts both the slim slice and the thick slice in parallel", () => { + const src = load(PAGE_PATH); + // Slim slice — `?slice=summaries` paints the per-type tile strip. + expect(src).toMatch(/useAnalyticsQuery\(\{\s*slice:\s*"summaries"\s*\}\)/); + // Thick slice — the bare `useAnalyticsQuery()` (no slice) feeds + // the BD-Zielbereich + glucose tiles. Both must coexist. + expect(src).toMatch(/useAnalyticsQuery\(\)/); + }); + + it("merges slim and thick results so call-sites stay shape-stable", () => { + const src = load(PAGE_PATH); + // The merge contract: slim wins on overlapping fields, thick fills + // in `bpInTargetPct*` and `glucoseByContext`. The text anchors + // protect against a future refactor accidentally dropping one of + // the queries — the `data` object that downstream tile rendering + // consumes must stay shape-stable across both query states. + expect(src).toMatch(/analyticsSlimQuery/); + expect(src).toMatch(/analyticsThickQuery/); + expect(src).toMatch(/bpInTargetPct/); + expect(src).toMatch(/glucoseByContext/); + }); +}); diff --git a/src/app/api/measurements/__tests__/range-aggregation-route.test.ts b/src/app/api/measurements/__tests__/range-aggregation-route.test.ts index c2bb37dc8..8ead7e75e 100644 --- a/src/app/api/measurements/__tests__/range-aggregation-route.test.ts +++ b/src/app/api/measurements/__tests__/range-aggregation-route.test.ts @@ -27,6 +27,22 @@ vi.mock("@/lib/db", () => ({ }, })); +// v1.4.39.2 — the rollup branch's coverage-mismatch fallback may call +// `recomputeUserRollups` inline. Mock it so tests can assert whether +// the inline fold fired without exercising the real populator. +vi.mock("@/lib/measurements/rollups", async () => { + const actual = await vi.importActual< + typeof import("@/lib/measurements/rollups") + >("@/lib/measurements/rollups"); + return { + ...actual, + recomputeBucketsForMeasurement: vi.fn().mockResolvedValue(undefined), + recomputeUserRollups: vi + .fn() + .mockResolvedValue({ rowsUpserted: 0, durationMs: 0 }), + }; +}); + vi.mock("@/lib/auth/session", () => ({ getSession: vi.fn() })); vi.mock("@/lib/auth/audit", () => ({ auditLog: vi.fn() })); vi.mock("@/lib/logging/transports", () => ({ emitIfSampled: vi.fn() })); @@ -45,6 +61,7 @@ vi.mock("next/headers", () => ({ import { GET } from "../route"; import { prisma } from "@/lib/db"; import { getSession } from "@/lib/auth/session"; +import { recomputeUserRollups } from "@/lib/measurements/rollups"; const SESSION_OK = { session: { id: "sess-1", expiresAt: new Date(Date.now() + 3_600_000) }, @@ -320,6 +337,186 @@ describe("GET /api/measurements — all-time semantics (SD-H1)", () => { expect(json.data.measurements.length).toBe(7); }); + it("source=rollup folds the partition inline and re-reads when the rollup is under-converged (v1.4.39.2)", async () => { + // Marc's pre-fix production trace: 30-day BP_SYS window returned + // `total: 2` from the rollup branch even though `measurements` + // carried ~30 distinct days. The chart tripped its `< 3 daily + // points` empty-state because the route short-circuited on + // `rollupRows.length > 0` and never reconciled with the live + // table. The coverage-mismatch fallback runs an inline + // `recomputeUserRollups` for the (user, type, DAY, window) + // partition and re-reads so the request returns the converged + // rows. + const sparseRollup = [ + { + type: "BLOOD_PRESSURE_SYS", + bucketStart: new Date(Date.UTC(2026, 4, 1, 0, 0, 0)), + mean: 128, + count: 1, + sumValue: null, + }, + { + type: "BLOOD_PRESSURE_SYS", + bucketStart: new Date(Date.UTC(2026, 4, 10, 0, 0, 0)), + mean: 132, + count: 1, + sumValue: null, + }, + ]; + const convergedRollup = Array.from({ length: 28 }, (_, i) => ({ + type: "BLOOD_PRESSURE_SYS", + bucketStart: new Date(Date.UTC(2026, 4, 1 + i, 0, 0, 0)), + mean: 128 + (i % 5), + count: 2, + sumValue: null, + })); + // First read returns the sparse rows; second (post-fold) read + // returns the converged rows. + vi.mocked(prisma.measurementRollup.findMany) + .mockResolvedValueOnce(sparseRollup as never) + .mockResolvedValueOnce(convergedRollup as never); + // Coverage probe — live `measurements` has 28 distinct days. + vi.mocked(prisma.$queryRaw).mockResolvedValueOnce([ + { days: BigInt(28) }, + ] as never); + + const res = await GET( + getRequest( + "type=BLOOD_PRESSURE_SYS&from=2026-04-30T00:00:00Z&to=2026-05-30T00:00:00Z&aggregate=daily&source=rollup&limit=5000", + ), + ); + expect(res.status).toBe(200); + const json = (await res.json()) as { + data: { measurements: Array }; + }; + // The inline fold fired exactly once with the (type, DAY, window) + // bound so the partition write stays small. + expect(recomputeUserRollups).toHaveBeenCalledTimes(1); + expect(recomputeUserRollups).toHaveBeenCalledWith( + "user-1", + expect.objectContaining({ + types: ["BLOOD_PRESSURE_SYS"], + granularities: ["DAY"], + }), + ); + // The route re-read the rollup after the fold so the response + // carries the converged 28 buckets — well above the chart's + // `< 3 daily points` empty-state threshold. + expect(prisma.measurementRollup.findMany).toHaveBeenCalledTimes(2); + expect(json.data.measurements.length).toBe(28); + }); + + it("source=rollup skips the coverage probe on a covered window (v1.4.39.2 hot-path)", async () => { + // Happy case: 30-day window, 28 rollup rows present. The route + // must short-circuit on the first read and skip the probe so the + // covered-tenant hot path stays a single indexed read. + const buckets = Array.from({ length: 28 }, (_, i) => ({ + type: "WEIGHT", + bucketStart: new Date(Date.UTC(2026, 3, 16 + i, 0, 0, 0)), + mean: 81 + i * 0.05, + count: 2, + sumValue: null, + })); + vi.mocked(prisma.measurementRollup.findMany).mockResolvedValue( + buckets as never, + ); + + const res = await GET( + getRequest( + "type=WEIGHT&from=2026-04-15T00:00:00Z&to=2026-05-15T00:00:00Z&aggregate=daily&source=rollup&limit=5000", + ), + ); + expect(res.status).toBe(200); + expect(prisma.measurementRollup.findMany).toHaveBeenCalledTimes(1); + expect(recomputeUserRollups).not.toHaveBeenCalled(); + expect(prisma.$queryRaw).not.toHaveBeenCalled(); + }); + + it("source=rollup skips the coverage fallback on short windows (v1.4.39.2)", async () => { + // A 5-day window with 2 rollup rows is plausibly genuine sparsity + // for a user who measures BP every other day. The coverage + // fallback gates on `windowDays >= 7` so the probe + inline fold + // never fire on these short windows. + const sparseRollup = [ + { + type: "BLOOD_PRESSURE_SYS", + bucketStart: new Date(Date.UTC(2026, 4, 1, 0, 0, 0)), + mean: 128, + count: 1, + sumValue: null, + }, + { + type: "BLOOD_PRESSURE_SYS", + bucketStart: new Date(Date.UTC(2026, 4, 3, 0, 0, 0)), + mean: 132, + count: 1, + sumValue: null, + }, + ]; + vi.mocked(prisma.measurementRollup.findMany).mockResolvedValue( + sparseRollup as never, + ); + + const res = await GET( + getRequest( + "type=BLOOD_PRESSURE_SYS&from=2026-05-01T00:00:00Z&to=2026-05-06T00:00:00Z&aggregate=daily&source=rollup&limit=5000", + ), + ); + expect(res.status).toBe(200); + expect(recomputeUserRollups).not.toHaveBeenCalled(); + expect(prisma.$queryRaw).not.toHaveBeenCalled(); + const json = (await res.json()) as { + data: { measurements: Array }; + }; + expect(json.data.measurements.length).toBe(2); + }); + + it("source=rollup keeps the sparse rollup when live `measurements` agrees (v1.4.39.2)", async () => { + // Genuine sparsity: the user measured BP twice in 30 days. The + // probe finds the same 2 distinct days in `measurements` so the + // route returns the 2 rollup rows without firing the inline + // fold. The chart's `< 3 daily points` empty-state is the right + // surface for this case — the data really is sparse. + const sparseRollup = [ + { + type: "BLOOD_PRESSURE_SYS", + bucketStart: new Date(Date.UTC(2026, 4, 1, 0, 0, 0)), + mean: 128, + count: 1, + sumValue: null, + }, + { + type: "BLOOD_PRESSURE_SYS", + bucketStart: new Date(Date.UTC(2026, 4, 10, 0, 0, 0)), + mean: 132, + count: 1, + sumValue: null, + }, + ]; + vi.mocked(prisma.measurementRollup.findMany).mockResolvedValue( + sparseRollup as never, + ); + // Probe agrees — 2 distinct days in `measurements` too. + vi.mocked(prisma.$queryRaw).mockResolvedValueOnce([ + { days: BigInt(2) }, + ] as never); + + const res = await GET( + getRequest( + "type=BLOOD_PRESSURE_SYS&from=2026-04-30T00:00:00Z&to=2026-05-30T00:00:00Z&aggregate=daily&source=rollup&limit=5000", + ), + ); + expect(res.status).toBe(200); + // Probe fired (suspicious-low rollup) but the inline fold did + // not (live and rollup agree). + expect(prisma.$queryRaw).toHaveBeenCalledTimes(1); + expect(recomputeUserRollups).not.toHaveBeenCalled(); + const json = (await res.json()) as { + data: { measurements: Array }; + }; + expect(json.data.measurements.length).toBe(2); + }); + it("returns a weekly series when the all-time window is under two years", async () => { // Three months past one year → falls in the weekly grain band. const weeks = Array.from({ length: 60 }, (_, i) => ({ diff --git a/src/app/api/measurements/route.ts b/src/app/api/measurements/route.ts index 36ee85b15..2229d21ba 100644 --- a/src/app/api/measurements/route.ts +++ b/src/app/api/measurements/route.ts @@ -29,6 +29,7 @@ import { invalidateUserMeasurements } from "@/lib/cache/invalidate"; import { recomputeBucketsForMeasurement, collapseToTypeDayKeys, + recomputeUserRollups, } from "@/lib/measurements/rollups"; import { NextRequest } from "next/server"; import type { @@ -293,7 +294,113 @@ export const GET = apiHandler(async (request: NextRequest) => { sumValue: true, }, }); - if (rollupRows.length > 0) { + // v1.4.39.2 — coverage-mismatch fallback. v1.4.39.1 wired the + // rollup write hook into the previously-bypassed Withings sync, + // /api/import, and admin-restore paths, and widened the boot-time + // backfill discovery to per-(user, type, day). Together those + // changes ensure new writes and post-deploy restarts converge the + // rollup tier for every measurement type the user has logged. They + // do NOT, however, guarantee convergence inside a single request + // when the boot backfill has not yet finished folding the user's + // historic rows: a chart query lands first, the rollup table still + // carries 2 rows for a 30-day BP_SYS window, and the route's pre- + // v1.4.39.2 "rollup wins on length > 0" short-circuit returns the + // sparse rollup. The chart trips its `< 3 daily points` empty-state + // even though `measurements` holds the full 30 days. + // + // This branch closes that gap. When the rollup row count is + // suspiciously low for the requested window (≥ 7 days asked and + // < 3 rollup rows present), we probe `measurements` for the actual + // distinct-day count over the same window. If `measurements` + // carries strictly more days than the rollup, the rollup is + // under-converged: we fold the (user, type, DAY, [from, to]) + // partition inline, re-read the rollup, and return the refreshed + // rows. The inline fold is bounded by the window + type so it is + // small (~30 buckets × one type for the dashboard chart's 30-day + // window) and finishes well inside the request budget. + // + // The probe runs only on suspicious rollups so the covered-tenant + // happy path stays a single indexed read. Marc's tenant on a 30-day + // BP_SYS window pre-fix returned 2 rollup rows and hit `total: 2` + // in the live trace; post-fix the probe finds ~30 distinct days in + // `measurements`, folds them, and the chart paints all 30. + const windowMs = to.getTime() - from.getTime(); + const windowDays = Math.ceil(windowMs / 86_400_000); + const SUSPICIOUS_ROW_FLOOR = 3; + const SUSPICIOUS_WINDOW_DAYS = 7; + let effectiveRows = rollupRows; + let coverageFallbackFired = false; + if ( + rollupRows.length > 0 && + rollupRows.length < SUSPICIOUS_ROW_FLOOR && + windowDays >= SUSPICIOUS_WINDOW_DAYS + ) { + // Cheap probe — one indexed range scan over + // `(user_id, type, measured_at)`. Returns the count of distinct + // calendar days the live table holds for the window. + const liveDayCountRows = await prisma.$queryRaw< + Array<{ days: bigint }> + >` + SELECT COUNT(DISTINCT date_trunc('day', m."measured_at"))::bigint AS days + FROM measurements m + WHERE m."user_id" = ${user.id} + AND m."type" = ${type}::measurement_type + AND m."measured_at" >= ${from} + AND m."measured_at" <= ${to} + `; + const liveDays = Number(liveDayCountRows[0]?.days ?? BigInt(0)); + if (liveDays > rollupRows.length) { + coverageFallbackFired = true; + try { + // Inline fold of the (user, type, DAY, window) partition. + // Bounded by the window so it stays small even on a + // power-user account; the upsert is idempotent against the + // composite primary key. + await recomputeUserRollups(user.id, { + types: [type as MeasurementType], + granularities: ["DAY"], + from, + to, + }); + // Re-read so this request returns the converged rows. The + // next chart paint hits the same warm rollup without paying + // the fallback cost. + effectiveRows = await prisma.measurementRollup.findMany({ + where: { + userId: user.id, + type: type as MeasurementType, + granularity: "DAY", + bucketStart: { gte: from, lte: to }, + }, + orderBy: { bucketStart: "asc" }, + take: cap, + select: { + type: true, + bucketStart: true, + mean: true, + count: true, + sumValue: true, + }, + }); + } catch (err) { + // Best-effort — fall back to the legacy live aggregate path + // below if the fold itself failed so the chart paints + // something sensible regardless. `annotate` carries the + // surface so ops can spot a silent populator regression. + const message = err instanceof Error ? err.message : String(err); + annotate({ + meta: { + rollup_coverage_fallback_failed: true, + rollup_coverage_fallback_error: message, + type, + }, + }); + // Drop into the live `date_trunc` branch below. + effectiveRows = []; + } + } + } + if (effectiveRows.length > 0) { // Cumulative metrics (steps, active energy, distance, flights, // daylight) store `mean` per bucket but the chart needs the // daily SUM. v1.4.39 W-SUM — read `sumValue` directly; fall back @@ -303,7 +410,7 @@ export const GET = apiHandler(async (request: NextRequest) => { // SUM(value) for a single-source day because AVG = SUM / COUNT. const useSum = type != null && CUMULATIVE_HK_TYPES.has(type as MeasurementType); - const measurements = rollupRows.map((r) => ({ + const measurements = effectiveRows.map((r) => ({ type: r.type, value: useSum ? (r.sumValue ?? r.mean * r.count) @@ -318,6 +425,7 @@ export const GET = apiHandler(async (request: NextRequest) => { type, aggregate: "daily", source: "rollup", + coverage_fallback: coverageFallbackFired, }, }); return apiSuccess({ @@ -332,7 +440,8 @@ export const GET = apiHandler(async (request: NextRequest) => { } // Fall through to the live aggregate path when the rollup is // empty for the requested window — covers brand-new accounts + - // the race between a recent write hook and the read. + // the race between a recent write hook and the read, and the + // coverage-fallback inline-fold-failed branch above. } if (aggregate && aggregate !== "raw" && from && to) { diff --git a/src/app/page.tsx b/src/app/page.tsx index 91968ac27..39498c76a 100644 --- a/src/app/page.tsx +++ b/src/app/page.tsx @@ -1,6 +1,6 @@ "use client"; -import React, { useState } from "react"; +import React, { useMemo, useState } from "react"; import { useQuery } from "@tanstack/react-query"; import { useAuth } from "@/hooks/use-auth"; import { @@ -222,14 +222,55 @@ export default function DashboardPage() { refetchOnWindowFocus: false, } as const; - // v1.4.33 IW2 — the dashboard tile-strip reads `bpInTargetPct*` + - // `glucoseByContext` (thick-only fields) so it stays on the default - // thick slice. The shared hook still centralises the cache settings - // (60s staleTime, no refetch on mount / window focus) so the - // tile-strip dedups with the Insights mother page on the same - // queryKey + slice combination. - const analyticsQuery = useAnalyticsQuery(); - const data = analyticsQuery.data as AnalyticsData | undefined; + // v1.4.39.2 — split the dashboard's analytics consumption so the + // tile-strip paints from the slim slice and the BD-Zielbereich + + // glucose tiles stream in from the thick slice afterwards. + // + // Pre-fix: a single `useAnalyticsQuery()` against `/api/analytics` + // (thick slice) blocked every per-type tile until the heavy fan-out + // resolved. Mood and medication tiles paint from their own dedicated + // endpoints and arrived first; the per-type measurement tiles then + // arrived as one burst once the full slice landed — Marc reported + // this "blocked-then-burst" pattern as "etwas nervig". + // + // Post-fix: two parallel queries. The slim slice (`?slice=summaries`) + // resolves the per-type DataSummary headlines + `lastSeenByType` and + // typically returns in well under a second once the rollup tier is + // warm; that paints every per-type tile in the strip. The thick + // slice resolves `bpInTargetPct*` + `glucoseByContext` for the + // BD-Zielbereich and glucose tiles, which stream in independently. + // Both queries share `caches.analytics` server-side so warm hits + // stay free, and TanStack's parallel-mounting keeps the network fan- + // out flat. + const analyticsSlimQuery = useAnalyticsQuery({ slice: "summaries" }); + const analyticsThickQuery = useAnalyticsQuery(); + const data = useMemo(() => { + const slim = analyticsSlimQuery.data; + const thick = analyticsThickQuery.data; + if (!slim && !thick) return undefined; + // Slim wins on overlapping fields when both have resolved — slim + // and thick compute `summaries` / `lastSeenByType` from the same + // SQL helpers so the values agree, and reading from whichever + // resolves first keeps the tile strip painting as soon as either + // query lands. + const summaries = + slim?.summaries ?? thick?.summaries ?? ({} as Record); + const lastSeenByType = + slim?.lastSeenByType ?? thick?.lastSeenByType; + return { + summaries, + lastSeenByType, + bpInTargetPct: (thick?.bpInTargetPct ?? null) as number | null, + bpInTargetPct7d: thick?.bpInTargetPct7d ?? null, + bpInTargetPct30d: thick?.bpInTargetPct30d ?? null, + bpInTargetPctAllTime: thick?.bpInTargetPctAllTime ?? null, + bpInTargetPctPriorMonth: thick?.bpInTargetPctPriorMonth ?? null, + bpInTargetPctPriorYear: thick?.bpInTargetPctPriorYear ?? null, + glucoseByContext: thick?.glucoseByContext as + | Record + | undefined, + }; + }, [analyticsSlimQuery.data, analyticsThickQuery.data]); const { data: layoutData } = useQuery({ queryKey: queryKeys.dashboardWidgets(),