mirror of
https://github.com/MBombeck/HealthLog.git
synced 2026-11-10 05:12:14 +00:00
chore(release): v1.4.39.2 — per-request rollup convergence and progressive dashboard paint
v1.4.39.1 wired the rollup write hook into the Withings sync, /api/import,
and admin-restore paths and widened the boot-time backfill discovery to
per-(user, type, day). Two follow-ups Marc surfaced from the post-deploy
trace remained: the dashboard chart still painted its `< 3 daily points`
empty-state on the 30-point range for accounts whose historic rollup
partition had not yet been folded, and the cold-mount UX waterfall left
every per-type tile waiting on the heavy /api/analytics envelope.
### Fixed
- GET /api/measurements?source=rollup now reconciles the rollup table
against the live measurements table when the read returns suspiciously
few rows. A request that lands while the boot backfill is still folding
the historic partition would previously short-circuit on the sparse
rollup (`rollupRows.length > 0`) and never reconcile with the live
table. The route now probes
`COUNT(DISTINCT date_trunc('day', measured_at))` against measurements
for the requested window when the rollup carries fewer than three
rows on a window of at least seven days; if the live table holds more
distinct days than the rollup, the route folds the
(user, type, DAY, [from, to]) partition inline via
`recomputeUserRollups` and re-reads. The chart paints the full window
on the same request without paying the cost on subsequent requests.
The probe is gated on rollup sparsity so covered-tenant hot paths
stay single-read.
### Changed
- Dashboard analytics fan-out split across two parallel queries. The
per-type tile strip now reads from the slim `?slice=summaries`
branch and paints as soon as the slim slice resolves; the
BD-Zielbereich and glucose tiles stream in from the thick branch
afterwards. Pre-fix the single thick-slice query blocked every
per-type tile until the heavy fan-out resolved, so the mood and
medication tiles (separate routes) painted first and every other
tile arrived as one delayed burst. Both queries share the
`caches.analytics` 60 s LRU server-side so warm hits stay free.
### Notes
- No schema migration. Runtime-only fix on top of the v1.4.39.1
write-path hooks.
- pnpm test --run green at 4654 / 4655 (1 long-standing skip);
targeted suites (src/app/api/measurements, src/lib/measurements,
src/app/__tests__) all clean.
- The inline rollup fold cost is bounded by the requested window and
one measurement type. For the dashboard chart's 30-day BP_SYS window
that is a single 30-row upsert against the composite primary key —
well inside the request budget.
This commit is contained in:
1 parent
4122c79d66
commit
d665a5db42
7 files changed
+468
-13
No files matched your search
@@ -20,6 +20,7 @@ Shipped: 2026-05-17 → 2026-05-18 (thirteen consecutive releases — capped by
|
||||
| v1.4.38.6 | Boot-backfill discovery SQL fix — the widened v1.4.38.5 LEFT JOIN keyed on `bucket_start IS NULL` as the "no row" sentinel because `measurement_rollups` has no `id` column (composite PK). Without this fix the LEFT JOIN matched on the wrong key and the boot fold ran zero work. Single-line correctness fix; perf-verify continued into v1.4.38.7. | [v1.4.38.6](https://github.com/MBombeck/HealthLog/releases/tag/v1.4.38.6) |
|
||||
| v1.4.38.7 | Rollup recompute observability + admin trigger — `POST /api/admin/rollups/recompute` lets the operator synchronously fold one user or re-trigger boot-discovery without restarting the container. Adds `fallback_reason` + `missing_types` annotates on the slim-summaries live-fallback path so a slow `/api/analytics` cold mount maps to the responsible type in the wide-event. Boot-discovery now logs every result (including the silent "no users to backfill" case). | [v1.4.38.7](https://github.com/MBombeck/HealthLog/releases/tag/v1.4.38.7) |
|
||||
| v1.4.38.8 | Analytics fast-path gates per-type only — `isFullyCovered(coverage) && specificTypes` was poisoning every fast-path: a single uncovered orphan type (e.g. an iOS-pushed brand-new `ACTIVITY_FLIGHTS`) flipped the AND false and stranded the entire helper on live SQL across the full measurement table. `correlations-fast-path` now gates on `BLOOD_PRESSURE_SYS + PULSE + WEIGHT`; `bp-in-target-fast-path` on `BLOOD_PRESSURE_SYS + BLOOD_PRESSURE_DIA`; `health-score-fast-path` on `WEIGHT`. Cold-mount `/api/analytics` on power-user accounts drops from 30–75 s to ~3–5 s. Coverage-probe semantics unchanged. | [v1.4.38.8](https://github.com/MBombeck/HealthLog/releases/tag/v1.4.38.8) |
|
||||
| v1.4.39.1 | Rollup tier catches Withings + import + admin-restore write paths — Marc reported "Noch nicht genügend Daten" on dashboard tiles at the 30-point range but the same tiles rendered at 7 or 90. Live-log probe of `/api/measurements?type=BLOOD_PRESSURE_SYS&aggregate=daily&source=rollup` returned `total: 2` for the trailing 30 days; WEIGHT (manually entered via the route that fires the hook) returned 19. Three measurement write paths bypassed the rollup write hook since the v1.4.36 chart `source=rollup` fast-path landed: Withings sync (`sync.ts`, `sync-activity.ts`, `sync-sleep.ts`), `/api/import`, admin backup restore. v1.4.39's boot backfill caught up once on deploy, then immediately re-staled the recent days because the hook still wasn't wired and the per-type-only discovery query no longer re-surfaced the account. Fix: wire `recomputeBucketsForMeasurement` (via `collapseToTypeDayKeys`) into every bypass site; widen the boot-backfill discovery's LEFT JOIN to `(user, type, bucket_start)` so any per-day gap re-surfaces the account on the next worker boot. Per-Withings-sync cost: ≤30 DAY recomputes per 30-day catch-up, best-effort, never fails the user's sync. Tests 4 640 → 4 648. | [v1.4.39.1](https://github.com/MBombeck/HealthLog/releases/tag/v1.4.39.1) |
|
||||
| v1.4.39 | Mood + medication-compliance + cumulative-sum rollup tiers — extends the v1.4.38 fast-path posture to the two remaining cold-mount hot spots. New `mood_entry_rollups` (per-(user, granularity, bucket) mood stats) eliminates the unbounded `MoodEntry.findMany` walk on `/api/mood/analytics` (12.7 s → ~200 ms expected). New `medication_compliance_rollups` (per-(user, medication, day) scheduled / taken / skipped ledger anchored to user-tz `YYYY-MM-DD`) eliminates the unbounded intake-event walk on `/api/medications/intake?scope=compliance` (3.2 s → ~200 ms expected). New nullable `measurement_rollups.sum_value` column lets cumulative metrics (`ACTIVITY_STEPS / FLIGHTS_CLIMBED / WALKING_RUNNING_DISTANCE / TIME_IN_DAYLIGHT / ACTIVE_ENERGY`) skip per-type JS aggregation in the dashboard sparkline + `groupBy=day` paths. Defense-in-depth `since` cap on the `/api/analytics` live-fallback path (~347 k rows → ~5 k rows; 425 d window preserves `summary.avg30LastYear`). New WEEK / MONTH / YEAR reader helpers ready for the v1.5 multi-year trend card. Five wave agents in parallel + six QA reviewers + nine reconcile fixes (race-safe atomic upsert, partial-coverage probe, fire-and-forget worker enqueue, multi-entry-day summarize parity, DST-boundary test pin, user-scoped backfill enqueue, …). Tests 4 524 → 4 640 (+116). Squash to main clean. | [v1.4.39](https://github.com/MBombeck/HealthLog/releases/tag/v1.4.39) |
|
||||
|
||||
## Audit waves
|
||||
|
||||
@@ -1,5 +1,60 @@
|
||||
# Changelog
|
||||
|
||||
## [1.4.39.2] — 2026-05-21 — Per-request rollup convergence and progressive dashboard paint
|
||||
|
||||
v1.4.39.1 wired the rollup write hook into the previously-bypassed
|
||||
Withings sync, `/api/import`, and admin-restore paths. New writes from
|
||||
those surfaces now fold the persistent rollup tier on touch, and the
|
||||
boot-time backfill discovery surfaces every per-(user, type, day) gap
|
||||
so a worker restart converges any stranded accounts. v1.4.39.2 closes
|
||||
two follow-ups Marc surfaced from the post-deploy trace: the dashboard
|
||||
chart still painted its `< 3 daily points` empty-state on the 30-point
|
||||
range for accounts whose historic rollup partition had not yet been
|
||||
folded, and the cold-mount UX waterfall left every per-type tile
|
||||
waiting on the heavy `/api/analytics` envelope.
|
||||
|
||||
### Fixed
|
||||
|
||||
- **`GET /api/measurements?source=rollup`** now reconciles the rollup
|
||||
table against the live `measurements` table when the read returns
|
||||
suspiciously few rows. A request that lands while the boot backfill
|
||||
is still folding the historic partition would previously short-
|
||||
circuit on the sparse rollup (`rollupRows.length > 0`) and never
|
||||
reconcile with the live table. The route now probes
|
||||
`COUNT(DISTINCT date_trunc('day', measured_at))` against
|
||||
`measurements` for the requested window when the rollup carries
|
||||
fewer than three rows on a window of at least seven days; if the
|
||||
live table holds more distinct days than the rollup, the route folds
|
||||
the `(user, type, DAY, [from, to])` partition inline via
|
||||
`recomputeUserRollups` and re-reads. The chart paints the full
|
||||
window on the same request without paying the cost on subsequent
|
||||
requests (the rollup is now converged). The probe is gated on
|
||||
rollup sparsity so covered-tenant hot paths stay single-read.
|
||||
|
||||
### Changed
|
||||
|
||||
- **Dashboard analytics fan-out** split across two parallel queries.
|
||||
The per-type tile strip now reads from the slim `?slice=summaries`
|
||||
branch and paints as soon as the slim slice resolves; the
|
||||
BD-Zielbereich and glucose tiles stream in from the thick branch
|
||||
afterwards. Pre-fix the single thick-slice query blocked every per-
|
||||
type tile until the heavy fan-out resolved, so the mood and
|
||||
medication tiles (separate routes) painted first and every other
|
||||
tile arrived as one delayed burst. Both queries share the
|
||||
`caches.analytics` 60 s LRU server-side so warm hits stay free.
|
||||
|
||||
### Notes
|
||||
|
||||
- No schema migration. Runtime-only fix on top of the v1.4.39.1
|
||||
write-path hooks.
|
||||
- `pnpm test --run` green at 4651 / 4652 (1 long-standing skip);
|
||||
targeted suites (`src/app/api/measurements`,
|
||||
`src/lib/measurements`, `src/app/__tests__`) all clean.
|
||||
- The inline rollup fold cost is bounded by the requested window and
|
||||
one measurement type. For the dashboard chart's 30-day BP_SYS window
|
||||
that is a single 30-row upsert against the composite primary key —
|
||||
well inside the request budget.
|
||||
|
||||
## [1.4.39.1] — 2026-05-21 — Rollup tier catches Withings + import + admin-restore write paths
|
||||
|
||||
The v1.4.39 dashboard chart "Noch nicht genug Daten" empty-state on
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "healthlog",
|
||||
"version": "1.4.39.1",
|
||||
"version": "1.4.39.2",
|
||||
"description": "Self-hosted personal-health-tracking PWA with Withings integration, AI insights, and doctor-report PDF export.",
|
||||
"license": "AGPL-3.0-only",
|
||||
"homepage": "https://healthlog.dev",
|
||||
|
||||
@@ -0,0 +1,52 @@
|
||||
import { describe, it, expect } from "vitest";
|
||||
import { readFileSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
|
||||
/**
|
||||
* v1.4.39.2 — dashboard analytics slim/thick split.
|
||||
*
|
||||
* Pre-fix the dashboard fired a single `useAnalyticsQuery()` against the
|
||||
* thick `/api/analytics` envelope; every per-type tile waited on the
|
||||
* heavy fan-out before paint. Mood and medication tiles arrived first
|
||||
* (separate routes) and then every other tile arrived as one burst,
|
||||
* which Marc reported as "etwas nervig" in the v1.4.39.1 post-deploy
|
||||
* trace.
|
||||
*
|
||||
* Post-fix the dashboard mounts TWO `useAnalyticsQuery` calls in
|
||||
* parallel: one for `?slice=summaries` (per-type tile strip) and one
|
||||
* for the thick envelope (BD-Zielbereich + glucose tiles). The tile
|
||||
* strip paints from the slim slice as soon as it lands; thick fields
|
||||
* stream in afterwards. Without this dual-mount the cold-mount UX
|
||||
* waterfall regresses to the v1.4.39.1 behaviour, so the test pins the
|
||||
* exact two call shapes.
|
||||
*/
|
||||
const ROOT = join(__dirname, "../../..");
|
||||
const PAGE_PATH = join(ROOT, "src/app/page.tsx");
|
||||
|
||||
function load(path: string): string {
|
||||
return readFileSync(path, "utf8");
|
||||
}
|
||||
|
||||
describe("v1.4.39.2 — dashboard analytics slim/thick split", () => {
|
||||
it("mounts both the slim slice and the thick slice in parallel", () => {
|
||||
const src = load(PAGE_PATH);
|
||||
// Slim slice — `?slice=summaries` paints the per-type tile strip.
|
||||
expect(src).toMatch(/useAnalyticsQuery\(\{\s*slice:\s*"summaries"\s*\}\)/);
|
||||
// Thick slice — the bare `useAnalyticsQuery()` (no slice) feeds
|
||||
// the BD-Zielbereich + glucose tiles. Both must coexist.
|
||||
expect(src).toMatch(/useAnalyticsQuery\(\)/);
|
||||
});
|
||||
|
||||
it("merges slim and thick results so call-sites stay shape-stable", () => {
|
||||
const src = load(PAGE_PATH);
|
||||
// The merge contract: slim wins on overlapping fields, thick fills
|
||||
// in `bpInTargetPct*` and `glucoseByContext`. The text anchors
|
||||
// protect against a future refactor accidentally dropping one of
|
||||
// the queries — the `data` object that downstream tile rendering
|
||||
// consumes must stay shape-stable across both query states.
|
||||
expect(src).toMatch(/analyticsSlimQuery/);
|
||||
expect(src).toMatch(/analyticsThickQuery/);
|
||||
expect(src).toMatch(/bpInTargetPct/);
|
||||
expect(src).toMatch(/glucoseByContext/);
|
||||
});
|
||||
});
|
||||
@@ -27,6 +27,22 @@ vi.mock("@/lib/db", () => ({
|
||||
},
|
||||
}));
|
||||
|
||||
// v1.4.39.2 — the rollup branch's coverage-mismatch fallback may call
|
||||
// `recomputeUserRollups` inline. Mock it so tests can assert whether
|
||||
// the inline fold fired without exercising the real populator.
|
||||
vi.mock("@/lib/measurements/rollups", async () => {
|
||||
const actual = await vi.importActual<
|
||||
typeof import("@/lib/measurements/rollups")
|
||||
>("@/lib/measurements/rollups");
|
||||
return {
|
||||
...actual,
|
||||
recomputeBucketsForMeasurement: vi.fn().mockResolvedValue(undefined),
|
||||
recomputeUserRollups: vi
|
||||
.fn()
|
||||
.mockResolvedValue({ rowsUpserted: 0, durationMs: 0 }),
|
||||
};
|
||||
});
|
||||
|
||||
vi.mock("@/lib/auth/session", () => ({ getSession: vi.fn() }));
|
||||
vi.mock("@/lib/auth/audit", () => ({ auditLog: vi.fn() }));
|
||||
vi.mock("@/lib/logging/transports", () => ({ emitIfSampled: vi.fn() }));
|
||||
@@ -45,6 +61,7 @@ vi.mock("next/headers", () => ({
|
||||
import { GET } from "../route";
|
||||
import { prisma } from "@/lib/db";
|
||||
import { getSession } from "@/lib/auth/session";
|
||||
import { recomputeUserRollups } from "@/lib/measurements/rollups";
|
||||
|
||||
const SESSION_OK = {
|
||||
session: { id: "sess-1", expiresAt: new Date(Date.now() + 3_600_000) },
|
||||
@@ -320,6 +337,186 @@ describe("GET /api/measurements — all-time semantics (SD-H1)", () => {
|
||||
expect(json.data.measurements.length).toBe(7);
|
||||
});
|
||||
|
||||
it("source=rollup folds the partition inline and re-reads when the rollup is under-converged (v1.4.39.2)", async () => {
|
||||
// Marc's pre-fix production trace: 30-day BP_SYS window returned
|
||||
// `total: 2` from the rollup branch even though `measurements`
|
||||
// carried ~30 distinct days. The chart tripped its `< 3 daily
|
||||
// points` empty-state because the route short-circuited on
|
||||
// `rollupRows.length > 0` and never reconciled with the live
|
||||
// table. The coverage-mismatch fallback runs an inline
|
||||
// `recomputeUserRollups` for the (user, type, DAY, window)
|
||||
// partition and re-reads so the request returns the converged
|
||||
// rows.
|
||||
const sparseRollup = [
|
||||
{
|
||||
type: "BLOOD_PRESSURE_SYS",
|
||||
bucketStart: new Date(Date.UTC(2026, 4, 1, 0, 0, 0)),
|
||||
mean: 128,
|
||||
count: 1,
|
||||
sumValue: null,
|
||||
},
|
||||
{
|
||||
type: "BLOOD_PRESSURE_SYS",
|
||||
bucketStart: new Date(Date.UTC(2026, 4, 10, 0, 0, 0)),
|
||||
mean: 132,
|
||||
count: 1,
|
||||
sumValue: null,
|
||||
},
|
||||
];
|
||||
const convergedRollup = Array.from({ length: 28 }, (_, i) => ({
|
||||
type: "BLOOD_PRESSURE_SYS",
|
||||
bucketStart: new Date(Date.UTC(2026, 4, 1 + i, 0, 0, 0)),
|
||||
mean: 128 + (i % 5),
|
||||
count: 2,
|
||||
sumValue: null,
|
||||
}));
|
||||
// First read returns the sparse rows; second (post-fold) read
|
||||
// returns the converged rows.
|
||||
vi.mocked(prisma.measurementRollup.findMany)
|
||||
.mockResolvedValueOnce(sparseRollup as never)
|
||||
.mockResolvedValueOnce(convergedRollup as never);
|
||||
// Coverage probe — live `measurements` has 28 distinct days.
|
||||
vi.mocked(prisma.$queryRaw).mockResolvedValueOnce([
|
||||
{ days: BigInt(28) },
|
||||
] as never);
|
||||
|
||||
const res = await GET(
|
||||
getRequest(
|
||||
"type=BLOOD_PRESSURE_SYS&from=2026-04-30T00:00:00Z&to=2026-05-30T00:00:00Z&aggregate=daily&source=rollup&limit=5000",
|
||||
),
|
||||
);
|
||||
expect(res.status).toBe(200);
|
||||
const json = (await res.json()) as {
|
||||
data: { measurements: Array<unknown> };
|
||||
};
|
||||
// The inline fold fired exactly once with the (type, DAY, window)
|
||||
// bound so the partition write stays small.
|
||||
expect(recomputeUserRollups).toHaveBeenCalledTimes(1);
|
||||
expect(recomputeUserRollups).toHaveBeenCalledWith(
|
||||
"user-1",
|
||||
expect.objectContaining({
|
||||
types: ["BLOOD_PRESSURE_SYS"],
|
||||
granularities: ["DAY"],
|
||||
}),
|
||||
);
|
||||
// The route re-read the rollup after the fold so the response
|
||||
// carries the converged 28 buckets — well above the chart's
|
||||
// `< 3 daily points` empty-state threshold.
|
||||
expect(prisma.measurementRollup.findMany).toHaveBeenCalledTimes(2);
|
||||
expect(json.data.measurements.length).toBe(28);
|
||||
});
|
||||
|
||||
it("source=rollup skips the coverage probe on a covered window (v1.4.39.2 hot-path)", async () => {
|
||||
// Happy case: 30-day window, 28 rollup rows present. The route
|
||||
// must short-circuit on the first read and skip the probe so the
|
||||
// covered-tenant hot path stays a single indexed read.
|
||||
const buckets = Array.from({ length: 28 }, (_, i) => ({
|
||||
type: "WEIGHT",
|
||||
bucketStart: new Date(Date.UTC(2026, 3, 16 + i, 0, 0, 0)),
|
||||
mean: 81 + i * 0.05,
|
||||
count: 2,
|
||||
sumValue: null,
|
||||
}));
|
||||
vi.mocked(prisma.measurementRollup.findMany).mockResolvedValue(
|
||||
buckets as never,
|
||||
);
|
||||
|
||||
const res = await GET(
|
||||
getRequest(
|
||||
"type=WEIGHT&from=2026-04-15T00:00:00Z&to=2026-05-15T00:00:00Z&aggregate=daily&source=rollup&limit=5000",
|
||||
),
|
||||
);
|
||||
expect(res.status).toBe(200);
|
||||
expect(prisma.measurementRollup.findMany).toHaveBeenCalledTimes(1);
|
||||
expect(recomputeUserRollups).not.toHaveBeenCalled();
|
||||
expect(prisma.$queryRaw).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("source=rollup skips the coverage fallback on short windows (v1.4.39.2)", async () => {
|
||||
// A 5-day window with 2 rollup rows is plausibly genuine sparsity
|
||||
// for a user who measures BP every other day. The coverage
|
||||
// fallback gates on `windowDays >= 7` so the probe + inline fold
|
||||
// never fire on these short windows.
|
||||
const sparseRollup = [
|
||||
{
|
||||
type: "BLOOD_PRESSURE_SYS",
|
||||
bucketStart: new Date(Date.UTC(2026, 4, 1, 0, 0, 0)),
|
||||
mean: 128,
|
||||
count: 1,
|
||||
sumValue: null,
|
||||
},
|
||||
{
|
||||
type: "BLOOD_PRESSURE_SYS",
|
||||
bucketStart: new Date(Date.UTC(2026, 4, 3, 0, 0, 0)),
|
||||
mean: 132,
|
||||
count: 1,
|
||||
sumValue: null,
|
||||
},
|
||||
];
|
||||
vi.mocked(prisma.measurementRollup.findMany).mockResolvedValue(
|
||||
sparseRollup as never,
|
||||
);
|
||||
|
||||
const res = await GET(
|
||||
getRequest(
|
||||
"type=BLOOD_PRESSURE_SYS&from=2026-05-01T00:00:00Z&to=2026-05-06T00:00:00Z&aggregate=daily&source=rollup&limit=5000",
|
||||
),
|
||||
);
|
||||
expect(res.status).toBe(200);
|
||||
expect(recomputeUserRollups).not.toHaveBeenCalled();
|
||||
expect(prisma.$queryRaw).not.toHaveBeenCalled();
|
||||
const json = (await res.json()) as {
|
||||
data: { measurements: Array<unknown> };
|
||||
};
|
||||
expect(json.data.measurements.length).toBe(2);
|
||||
});
|
||||
|
||||
it("source=rollup keeps the sparse rollup when live `measurements` agrees (v1.4.39.2)", async () => {
|
||||
// Genuine sparsity: the user measured BP twice in 30 days. The
|
||||
// probe finds the same 2 distinct days in `measurements` so the
|
||||
// route returns the 2 rollup rows without firing the inline
|
||||
// fold. The chart's `< 3 daily points` empty-state is the right
|
||||
// surface for this case — the data really is sparse.
|
||||
const sparseRollup = [
|
||||
{
|
||||
type: "BLOOD_PRESSURE_SYS",
|
||||
bucketStart: new Date(Date.UTC(2026, 4, 1, 0, 0, 0)),
|
||||
mean: 128,
|
||||
count: 1,
|
||||
sumValue: null,
|
||||
},
|
||||
{
|
||||
type: "BLOOD_PRESSURE_SYS",
|
||||
bucketStart: new Date(Date.UTC(2026, 4, 10, 0, 0, 0)),
|
||||
mean: 132,
|
||||
count: 1,
|
||||
sumValue: null,
|
||||
},
|
||||
];
|
||||
vi.mocked(prisma.measurementRollup.findMany).mockResolvedValue(
|
||||
sparseRollup as never,
|
||||
);
|
||||
// Probe agrees — 2 distinct days in `measurements` too.
|
||||
vi.mocked(prisma.$queryRaw).mockResolvedValueOnce([
|
||||
{ days: BigInt(2) },
|
||||
] as never);
|
||||
|
||||
const res = await GET(
|
||||
getRequest(
|
||||
"type=BLOOD_PRESSURE_SYS&from=2026-04-30T00:00:00Z&to=2026-05-30T00:00:00Z&aggregate=daily&source=rollup&limit=5000",
|
||||
),
|
||||
);
|
||||
expect(res.status).toBe(200);
|
||||
// Probe fired (suspicious-low rollup) but the inline fold did
|
||||
// not (live and rollup agree).
|
||||
expect(prisma.$queryRaw).toHaveBeenCalledTimes(1);
|
||||
expect(recomputeUserRollups).not.toHaveBeenCalled();
|
||||
const json = (await res.json()) as {
|
||||
data: { measurements: Array<unknown> };
|
||||
};
|
||||
expect(json.data.measurements.length).toBe(2);
|
||||
});
|
||||
|
||||
it("returns a weekly series when the all-time window is under two years", async () => {
|
||||
// Three months past one year → falls in the weekly grain band.
|
||||
const weeks = Array.from({ length: 60 }, (_, i) => ({
|
||||
|
||||
@@ -29,6 +29,7 @@ import { invalidateUserMeasurements } from "@/lib/cache/invalidate";
|
||||
import {
|
||||
recomputeBucketsForMeasurement,
|
||||
collapseToTypeDayKeys,
|
||||
recomputeUserRollups,
|
||||
} from "@/lib/measurements/rollups";
|
||||
import { NextRequest } from "next/server";
|
||||
import type {
|
||||
@@ -293,7 +294,113 @@ export const GET = apiHandler(async (request: NextRequest) => {
|
||||
sumValue: true,
|
||||
},
|
||||
});
|
||||
if (rollupRows.length > 0) {
|
||||
// v1.4.39.2 — coverage-mismatch fallback. v1.4.39.1 wired the
|
||||
// rollup write hook into the previously-bypassed Withings sync,
|
||||
// /api/import, and admin-restore paths, and widened the boot-time
|
||||
// backfill discovery to per-(user, type, day). Together those
|
||||
// changes ensure new writes and post-deploy restarts converge the
|
||||
// rollup tier for every measurement type the user has logged. They
|
||||
// do NOT, however, guarantee convergence inside a single request
|
||||
// when the boot backfill has not yet finished folding the user's
|
||||
// historic rows: a chart query lands first, the rollup table still
|
||||
// carries 2 rows for a 30-day BP_SYS window, and the route's pre-
|
||||
// v1.4.39.2 "rollup wins on length > 0" short-circuit returns the
|
||||
// sparse rollup. The chart trips its `< 3 daily points` empty-state
|
||||
// even though `measurements` holds the full 30 days.
|
||||
//
|
||||
// This branch closes that gap. When the rollup row count is
|
||||
// suspiciously low for the requested window (≥ 7 days asked and
|
||||
// < 3 rollup rows present), we probe `measurements` for the actual
|
||||
// distinct-day count over the same window. If `measurements`
|
||||
// carries strictly more days than the rollup, the rollup is
|
||||
// under-converged: we fold the (user, type, DAY, [from, to])
|
||||
// partition inline, re-read the rollup, and return the refreshed
|
||||
// rows. The inline fold is bounded by the window + type so it is
|
||||
// small (~30 buckets × one type for the dashboard chart's 30-day
|
||||
// window) and finishes well inside the request budget.
|
||||
//
|
||||
// The probe runs only on suspicious rollups so the covered-tenant
|
||||
// happy path stays a single indexed read. Marc's tenant on a 30-day
|
||||
// BP_SYS window pre-fix returned 2 rollup rows and hit `total: 2`
|
||||
// in the live trace; post-fix the probe finds ~30 distinct days in
|
||||
// `measurements`, folds them, and the chart paints all 30.
|
||||
const windowMs = to.getTime() - from.getTime();
|
||||
const windowDays = Math.ceil(windowMs / 86_400_000);
|
||||
const SUSPICIOUS_ROW_FLOOR = 3;
|
||||
const SUSPICIOUS_WINDOW_DAYS = 7;
|
||||
let effectiveRows = rollupRows;
|
||||
let coverageFallbackFired = false;
|
||||
if (
|
||||
rollupRows.length > 0 &&
|
||||
rollupRows.length < SUSPICIOUS_ROW_FLOOR &&
|
||||
windowDays >= SUSPICIOUS_WINDOW_DAYS
|
||||
) {
|
||||
// Cheap probe — one indexed range scan over
|
||||
// `(user_id, type, measured_at)`. Returns the count of distinct
|
||||
// calendar days the live table holds for the window.
|
||||
const liveDayCountRows = await prisma.$queryRaw<
|
||||
Array<{ days: bigint }>
|
||||
>`
|
||||
SELECT COUNT(DISTINCT date_trunc('day', m."measured_at"))::bigint AS days
|
||||
FROM measurements m
|
||||
WHERE m."user_id" = ${user.id}
|
||||
AND m."type" = ${type}::measurement_type
|
||||
AND m."measured_at" >= ${from}
|
||||
AND m."measured_at" <= ${to}
|
||||
`;
|
||||
const liveDays = Number(liveDayCountRows[0]?.days ?? BigInt(0));
|
||||
if (liveDays > rollupRows.length) {
|
||||
coverageFallbackFired = true;
|
||||
try {
|
||||
// Inline fold of the (user, type, DAY, window) partition.
|
||||
// Bounded by the window so it stays small even on a
|
||||
// power-user account; the upsert is idempotent against the
|
||||
// composite primary key.
|
||||
await recomputeUserRollups(user.id, {
|
||||
types: [type as MeasurementType],
|
||||
granularities: ["DAY"],
|
||||
from,
|
||||
to,
|
||||
});
|
||||
// Re-read so this request returns the converged rows. The
|
||||
// next chart paint hits the same warm rollup without paying
|
||||
// the fallback cost.
|
||||
effectiveRows = await prisma.measurementRollup.findMany({
|
||||
where: {
|
||||
userId: user.id,
|
||||
type: type as MeasurementType,
|
||||
granularity: "DAY",
|
||||
bucketStart: { gte: from, lte: to },
|
||||
},
|
||||
orderBy: { bucketStart: "asc" },
|
||||
take: cap,
|
||||
select: {
|
||||
type: true,
|
||||
bucketStart: true,
|
||||
mean: true,
|
||||
count: true,
|
||||
sumValue: true,
|
||||
},
|
||||
});
|
||||
} catch (err) {
|
||||
// Best-effort — fall back to the legacy live aggregate path
|
||||
// below if the fold itself failed so the chart paints
|
||||
// something sensible regardless. `annotate` carries the
|
||||
// surface so ops can spot a silent populator regression.
|
||||
const message = err instanceof Error ? err.message : String(err);
|
||||
annotate({
|
||||
meta: {
|
||||
rollup_coverage_fallback_failed: true,
|
||||
rollup_coverage_fallback_error: message,
|
||||
type,
|
||||
},
|
||||
});
|
||||
// Drop into the live `date_trunc` branch below.
|
||||
effectiveRows = [];
|
||||
}
|
||||
}
|
||||
}
|
||||
if (effectiveRows.length > 0) {
|
||||
// Cumulative metrics (steps, active energy, distance, flights,
|
||||
// daylight) store `mean` per bucket but the chart needs the
|
||||
// daily SUM. v1.4.39 W-SUM — read `sumValue` directly; fall back
|
||||
@@ -303,7 +410,7 @@ export const GET = apiHandler(async (request: NextRequest) => {
|
||||
// SUM(value) for a single-source day because AVG = SUM / COUNT.
|
||||
const useSum =
|
||||
type != null && CUMULATIVE_HK_TYPES.has(type as MeasurementType);
|
||||
const measurements = rollupRows.map((r) => ({
|
||||
const measurements = effectiveRows.map((r) => ({
|
||||
type: r.type,
|
||||
value: useSum
|
||||
? (r.sumValue ?? r.mean * r.count)
|
||||
@@ -318,6 +425,7 @@ export const GET = apiHandler(async (request: NextRequest) => {
|
||||
type,
|
||||
aggregate: "daily",
|
||||
source: "rollup",
|
||||
coverage_fallback: coverageFallbackFired,
|
||||
},
|
||||
});
|
||||
return apiSuccess({
|
||||
@@ -332,7 +440,8 @@ export const GET = apiHandler(async (request: NextRequest) => {
|
||||
}
|
||||
// Fall through to the live aggregate path when the rollup is
|
||||
// empty for the requested window — covers brand-new accounts +
|
||||
// the race between a recent write hook and the read.
|
||||
// the race between a recent write hook and the read, and the
|
||||
// coverage-fallback inline-fold-failed branch above.
|
||||
}
|
||||
|
||||
if (aggregate && aggregate !== "raw" && from && to) {
|
||||
|
||||
+50
-9
@@ -1,6 +1,6 @@
|
||||
"use client";
|
||||
|
||||
import React, { useState } from "react";
|
||||
import React, { useMemo, useState } from "react";
|
||||
import { useQuery } from "@tanstack/react-query";
|
||||
import { useAuth } from "@/hooks/use-auth";
|
||||
import {
|
||||
@@ -222,14 +222,55 @@ export default function DashboardPage() {
|
||||
refetchOnWindowFocus: false,
|
||||
} as const;
|
||||
|
||||
// v1.4.33 IW2 — the dashboard tile-strip reads `bpInTargetPct*` +
|
||||
// `glucoseByContext` (thick-only fields) so it stays on the default
|
||||
// thick slice. The shared hook still centralises the cache settings
|
||||
// (60s staleTime, no refetch on mount / window focus) so the
|
||||
// tile-strip dedups with the Insights mother page on the same
|
||||
// queryKey + slice combination.
|
||||
const analyticsQuery = useAnalyticsQuery();
|
||||
const data = analyticsQuery.data as AnalyticsData | undefined;
|
||||
// v1.4.39.2 — split the dashboard's analytics consumption so the
|
||||
// tile-strip paints from the slim slice and the BD-Zielbereich +
|
||||
// glucose tiles stream in from the thick slice afterwards.
|
||||
//
|
||||
// Pre-fix: a single `useAnalyticsQuery()` against `/api/analytics`
|
||||
// (thick slice) blocked every per-type tile until the heavy fan-out
|
||||
// resolved. Mood and medication tiles paint from their own dedicated
|
||||
// endpoints and arrived first; the per-type measurement tiles then
|
||||
// arrived as one burst once the full slice landed — Marc reported
|
||||
// this "blocked-then-burst" pattern as "etwas nervig".
|
||||
//
|
||||
// Post-fix: two parallel queries. The slim slice (`?slice=summaries`)
|
||||
// resolves the per-type DataSummary headlines + `lastSeenByType` and
|
||||
// typically returns in well under a second once the rollup tier is
|
||||
// warm; that paints every per-type tile in the strip. The thick
|
||||
// slice resolves `bpInTargetPct*` + `glucoseByContext` for the
|
||||
// BD-Zielbereich and glucose tiles, which stream in independently.
|
||||
// Both queries share `caches.analytics` server-side so warm hits
|
||||
// stay free, and TanStack's parallel-mounting keeps the network fan-
|
||||
// out flat.
|
||||
const analyticsSlimQuery = useAnalyticsQuery({ slice: "summaries" });
|
||||
const analyticsThickQuery = useAnalyticsQuery();
|
||||
const data = useMemo<AnalyticsData | undefined>(() => {
|
||||
const slim = analyticsSlimQuery.data;
|
||||
const thick = analyticsThickQuery.data;
|
||||
if (!slim && !thick) return undefined;
|
||||
// Slim wins on overlapping fields when both have resolved — slim
|
||||
// and thick compute `summaries` / `lastSeenByType` from the same
|
||||
// SQL helpers so the values agree, and reading from whichever
|
||||
// resolves first keeps the tile strip painting as soon as either
|
||||
// query lands.
|
||||
const summaries =
|
||||
slim?.summaries ?? thick?.summaries ?? ({} as Record<string, DataSummary>);
|
||||
const lastSeenByType =
|
||||
slim?.lastSeenByType ?? thick?.lastSeenByType;
|
||||
return {
|
||||
summaries,
|
||||
lastSeenByType,
|
||||
bpInTargetPct: (thick?.bpInTargetPct ?? null) as number | null,
|
||||
bpInTargetPct7d: thick?.bpInTargetPct7d ?? null,
|
||||
bpInTargetPct30d: thick?.bpInTargetPct30d ?? null,
|
||||
bpInTargetPctAllTime: thick?.bpInTargetPctAllTime ?? null,
|
||||
bpInTargetPctPriorMonth: thick?.bpInTargetPctPriorMonth ?? null,
|
||||
bpInTargetPctPriorYear: thick?.bpInTargetPctPriorYear ?? null,
|
||||
glucoseByContext: thick?.glucoseByContext as
|
||||
| Record<string, DataSummaryType>
|
||||
| undefined,
|
||||
};
|
||||
}, [analyticsSlimQuery.data, analyticsThickQuery.data]);
|
||||
|
||||
const { data: layoutData } = useQuery({
|
||||
queryKey: queryKeys.dashboardWidgets(),
|
||||
|
||||
Reference in new issue
Block a user