chore(release): v1.4.39.1 — rollup tier catches Withings + import + admin-restore write paths

Hotfix on top of v1.4.39. Three measurement write paths (Withings
sync, `/api/import`, admin backup restore) were bypassing the
persistent rollup tier's write hook, so the dashboard chart's
`source=rollup` fast-path painted "Noch nicht genug Daten" on the
30 / 90-point range for accounts whose recent data only came through
those paths. The 7-day window kept rendering because the chart's raw
fetch never consults the rollup tier.

Closes the read-coverage gap by wiring `recomputeBucketsForMeasurement`
into every remaining ingest path and re-anchors the boot-time backfill
discovery to per-day coverage so already-deployed instances re-converge
on the next worker boot. The legacy `sum_value IS NULL` branch stays
on the same UNION so the v1.4.39 column-backfill continues working.

Also re-anchors one date-bound test in the medications-intake route
suite so the green CI run no longer depends on the wall clock.

Tests 4647 → 4648 passing; one long-standing skip unchanged.
This commit is contained in:
Marc-André Bombeck committed 2026-05-21 09:03:33 +02:00
1 parent d84922cdc3
commit 4122c79d66
15 files changed
+594 -20

No files matched your search

@@ -16,6 +16,11 @@ Shipped: 2026-05-17 → 2026-05-18 (thirteen consecutive releases — capped by
| v1.4.38.2 | Mood-reminder hotfix bundle + Settings toggle — six-axis QA on PR #190 surfaced enough real bugs that the feature could not have been used safely (locale resolver demoted 4 of 6 locales to English; ledger committed BEFORE delivery, so a transient APNs blip silently nuked the day's nudge AND blocked retries; DST arithmetic drifted the local-22:00 baseline by an hour twice a year; APNs payload forwarded Telegram `replyMarkup` to Apple + lockscreen; **no Settings UI to opt in** = dead opt-in). v1.4.38.2 closes all of the above plus adds `MoodReminderCard` Switch, `dispatchNotification` outcome bubble, `localHmAsUtc` DST-safe helper, APNs metadata allowlist, per-user `try` wrapper, 90-day retention sweep, dead-code drop, CHANGELOG `EVENT_DEFAULT_ENABLED` identifier-leak scrub. Unit suite 4565 → 4551 (mood-reminder.test.ts rewrite for new contract, not coverage loss). | [v1.4.38.2](https://github.com/MBombeck/HealthLog/releases/tag/v1.4.38.2) |
| v1.4.38.3 | CI green-up + chunk-load auto-recover — three pre-existing main reds closed (TODO marker on correlations `degraded` sentinel, rollup integration test racing the v1.4.37.1 fire-and-forget rollup populator, e2e drift after the v1.4.37 doctor-report hero refactor + the dashboard touch-target gate + `measurement-flow` mock missing `unit`/`source`). Plus new `AppError` branch detects the chunk-load error family and auto-reloads the SPA once per session — closes the stale-shell paper-cut after every deploy. Squash skipped: six atomic Marc-Voice commits landed directly on main, then back-merged to develop. | [v1.4.38.3](https://github.com/MBombeck/HealthLog/releases/tag/v1.4.38.3) |
| v1.4.38.4 | Pro-active stale-shell self-heal — Marc reported `healthlog.bombeck.io` "nicht erreichbar" on the v1.4.38.3 PWA; live curl was 100 % healthy (`/api/version` + `/api/health` both 200), so the symptom traced to a cached pre-v1.4.38.3 shell that never tripped the new `AppError` reload path. v1.4.38.4 layers three fixes so future deploys self-heal without user action: `<VersionPoller>` client component mounted in `<Providers>` polls `/api/version` every 60 s and on mismatch unregisters every SW, wipes CacheStorage, and `window.location.reload()`s (sessionStorage-gated to once per session); `next.config.ts` injects `NEXT_PUBLIC_APP_VERSION` from `package.json` at build time so the running shell knows which release it was built against; service worker `CACHE_VERSION` now keys to the release tag so the precached chunk graph is evicted on the next SW install. Tag re-pushed once after the editor-race dropped the `package.json` bump from the first release commit; the docker-publish workflow was cancelled mid-build and re-run from the correct commit. | [v1.4.38.4](https://github.com/MBombeck/HealthLog/releases/tag/v1.4.38.4) |
| v1.4.38.5 | Analytics fast-path restored on long-running accounts — boot-time backfill discovery widened to find every user that lacks DAY-bucket coverage for a measurement type they actually have, not only the empty-rollup case. The post-v1.4.38 backfill had only discovered users with zero rollup rows total, leaving every long-running account stranded on the live-SQL path. New discovery query is per-(user, type) so any single uncovered type triggers a full fold. | [v1.4.38.5](https://github.com/MBombeck/HealthLog/releases/tag/v1.4.38.5) |
| v1.4.38.6 | Boot-backfill discovery SQL fix — the widened v1.4.38.5 LEFT JOIN keyed on `bucket_start IS NULL` as the "no row" sentinel because `measurement_rollups` has no `id` column (composite PK). Without this fix the LEFT JOIN matched on the wrong key and the boot fold ran zero work. Single-line correctness fix; perf-verify continued into v1.4.38.7. | [v1.4.38.6](https://github.com/MBombeck/HealthLog/releases/tag/v1.4.38.6) |
| v1.4.38.7 | Rollup recompute observability + admin trigger — `POST /api/admin/rollups/recompute` lets the operator synchronously fold one user or re-trigger boot-discovery without restarting the container. Adds `fallback_reason` + `missing_types` annotates on the slim-summaries live-fallback path so a slow `/api/analytics` cold mount maps to the responsible type in the wide-event. Boot-discovery now logs every result (including the silent "no users to backfill" case). | [v1.4.38.7](https://github.com/MBombeck/HealthLog/releases/tag/v1.4.38.7) |
| v1.4.38.8 | Analytics fast-path gates per-type only — `isFullyCovered(coverage) && specificTypes` was poisoning every fast-path: a single uncovered orphan type (e.g. an iOS-pushed brand-new `ACTIVITY_FLIGHTS`) flipped the AND false and stranded the entire helper on live SQL across the full measurement table. `correlations-fast-path` now gates on `BLOOD_PRESSURE_SYS + PULSE + WEIGHT`; `bp-in-target-fast-path` on `BLOOD_PRESSURE_SYS + BLOOD_PRESSURE_DIA`; `health-score-fast-path` on `WEIGHT`. Cold-mount `/api/analytics` on power-user accounts drops from 30–75 s to ~3–5 s. Coverage-probe semantics unchanged. | [v1.4.38.8](https://github.com/MBombeck/HealthLog/releases/tag/v1.4.38.8) |
| v1.4.39 | Mood + medication-compliance + cumulative-sum rollup tiers — extends the v1.4.38 fast-path posture to the two remaining cold-mount hot spots. New `mood_entry_rollups` (per-(user, granularity, bucket) mood stats) eliminates the unbounded `MoodEntry.findMany` walk on `/api/mood/analytics` (12.7 s → ~200 ms expected). New `medication_compliance_rollups` (per-(user, medication, day) scheduled / taken / skipped ledger anchored to user-tz `YYYY-MM-DD`) eliminates the unbounded intake-event walk on `/api/medications/intake?scope=compliance` (3.2 s → ~200 ms expected). New nullable `measurement_rollups.sum_value` column lets cumulative metrics (`ACTIVITY_STEPS / FLIGHTS_CLIMBED / WALKING_RUNNING_DISTANCE / TIME_IN_DAYLIGHT / ACTIVE_ENERGY`) skip per-type JS aggregation in the dashboard sparkline + `groupBy=day` paths. Defense-in-depth `since` cap on the `/api/analytics` live-fallback path (~347 k rows → ~5 k rows; 425 d window preserves `summary.avg30LastYear`). New WEEK / MONTH / YEAR reader helpers ready for the v1.5 multi-year trend card. Five wave agents in parallel + six QA reviewers + nine reconcile fixes (race-safe atomic upsert, partial-coverage probe, fire-and-forget worker enqueue, multi-entry-day summarize parity, DST-boundary test pin, user-scoped backfill enqueue, …). Tests 4 524 → 4 640 (+116). Squash to main clean. | [v1.4.39](https://github.com/MBombeck/HealthLog/releases/tag/v1.4.39) |
## Audit waves
+68
View File
@@ -0,0 +1,68 @@
# v1.4.39 — Post-deploy perf verification
Live deploy: 2026-05-21T05:46:56Z — Coolify deploy uuid
`ujgsxpa46esyzybjldgxjeox`, commit hash
`d84922cdc3bfd118b10b74e6b8664a784837fe58`.
Migrations applied at boot:
```
Applying migration `0070_v1439_mood_rollups`
Applying migration `0071_v1439_medication_compliance_rollups`
Applying migration `0072_v1439_rollup_sum_value`
All migrations have been successfully applied.
```
## Unauthenticated probes (from this orchestrator)
| Endpoint | n | Median | Max |
| --- | --- | --- | --- |
| `GET /api/version` | 10 | **86 ms** | 104 ms |
| `GET /api/health` (Uptime-Kuma) | per-log | 18–78 ms | — |
The 60 s response cache wrap is not yet warm for the authenticated
heavy endpoints — perf-verify on `/api/mood/analytics`,
`/api/medications/intake?scope=compliance`,
`/api/dashboard/summary`, `/api/analytics` waits for Marc's next
browser session against the live deploy. The current open-internet
probes confirm the public-surface contract is fast.
## Coolify production log excerpt (boot)
- App container boots in ~1.5 s after migrations.
- Reminder worker registered.
- `host_metric_sample` background job running every 30 s without
pressure.
- No 4xx / 5xx in the first ~2 min of logs.
## What gets verified by Marc's first authenticated session
For each of the four read-path swaps, the wide-event annotates
`meta.<surface>.path = "rollup"` on the rollup-fast-path or
`path = "live"` on the fall-through. Marc loading the dashboard /
analytics page populates the 60 s LRU; a second mount within 60 s
hits warm cache (< 50 ms). The audit-predicted cold targets:
| Endpoint | Pre-v1.4.39 cold | Expected v1.4.39 cold |
| --- | --- | --- |
| `/api/mood/analytics` | 12.7 s (live) | ~200 ms (rollup) |
| `/api/medications/intake?scope=compliance` | 3.2 s (live) | ~200 ms (rollup) |
| `/api/dashboard/summary` cumulative sparkline | ~500 ms | ~300 ms |
| `/api/analytics` full slice live-fallback row cap | 347 k rows | ~5 k rows (425 d) |
## Self-heal verification
`/api/version` returns `1.4.39` end-to-end (curl + 10× burst). The
`VersionPoller` from v1.4.38.4 will pick up the new version-string at
the next 60 s tick on Marc's browser shell; the cached pre-v1.4.39
shell evicts CacheStorage + unregisters every SW + reloads
exactly once per session.
## Next perf-verify checkpoint
After Marc's first authenticated session, capture the wide-event
`path` annotates on each of the four endpoints from Coolify logs
and update this file with the live cold-mount numbers. v1.4.40
backlog already covers the carry-over consumers
(`health-score-fast-path` swap, `/api/analytics` A2 cumulative skip,
slope90 monthly bucket).
+47
View File
@@ -1,5 +1,52 @@
# Changelog
## [1.4.39.1] — 2026-05-21 — Rollup tier catches Withings + import + admin-restore write paths
The v1.4.39 dashboard chart "Noch nicht genug Daten" empty-state on
the 30-point range traced to three measurement write paths bypassing
the rollup table's per-write hook: Withings sync (BP / weight / pulse /
body-fat / activity / sleep), the JSON / CSV `/api/import` endpoint,
and the admin backup restore. Once the chart's `source=rollup` fast-
path landed in v1.4.36, those days never reached the chart at 30 / 90
days — the live raw fetch at 7 days kept showing data, the rollup
fast-path at 30 / 90 days under-counted distinct days, and the chart
tripped its `< 3 daily points` empty-state. v1.4.39.1 wires the rollup
write hook into every remaining ingest path and tightens the boot-time
backfill discovery so any stranded accounts re-converge on the next
worker boot.
### Fixed
- **Withings sync** (`sync.ts` / `sync-activity.ts` / `sync-sleep.ts`):
every (type, day) pair the sync touched is now handed to
`recomputeBucketsForMeasurement` after the row writes. Collapsed
via `collapseToTypeDayKeys` so a 30-day catch-up costs at most ~N
(type, day) recomputes rather than one per row. Best-effort: a
populator hiccup never fails the user's sync.
- **`/api/import` endpoint**: measurement creates now track touched
`(type, day)` pairs and fold the rollup tier at the end of the
batch. Mirrors the v1.4.39 mood-rollup hook on the same route.
- **Admin backup restore**: the transaction now wipes the user's
`measurement_rollups` partition alongside `mood_entry_rollups`, and
the post-transaction step kicks a full `recomputeUserRollups` so
the restored dataset paints fresh tiles instead of carrying the
previous owner's daily means forward.
- **Boot-time backfill discovery** (`enqueueBootTimeRollupBackfill`):
the missing-coverage join moved from per-type to per-day, so an
account whose Withings sync wrote 27 days of BP without ever firing
the rollup hook now re-surfaces on the next worker boot even when
one prior rollup row already existed for the same type. The legacy
`sum_value IS NULL` branch is preserved on the same UNION.
### Notes
- No schema migration. Additive write-hook plumbing only.
- `pnpm test --run` green at 4648 / 4649 (1 long-standing skip).
- The dashboard chart's read-side `source=rollup` short-circuit stays
put — once every write path folds the rollup tier on touch, the
short-circuit is always correct. A defensive coverage-mismatch
fallback is on the v1.4.40 backlog.
## [1.4.39] — 2026-05-21 — Mood, medication-compliance, and cumulative-sum rollup tiers
The v1.4.38 chain settled the measurement-rollup fast-path. v1.4.39
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "healthlog",
"version": "1.4.39",
"version": "1.4.39.1",
"description": "Self-hosted personal-health-tracking PWA with Withings integration, AI insights, and doctor-report PDF export.",
"license": "AGPL-3.0-only",
"homepage": "https://healthlog.dev",
@@ -48,6 +48,7 @@ import {
recomputeUserMedicationCompliance,
MEDICATION_COMPLIANCE_BACKFILL_DAYS,
} from "@/lib/medications/compliance-rollups";
import { recomputeUserRollups } from "@/lib/measurements/rollups";
export const dynamic = "force-dynamic";
@@ -291,6 +292,14 @@ const handler = apiHandler(
// pre-restore daily means. The fold below mints fresh rows
// from the restored mood entries.
await tx.moodEntryRollup.deleteMany({ where: { userId: ownerId } });
// v1.4.39.1 — wipe the persisted measurement rollup partition
// for the same reason. Pre-fix the partition kept the previous
// owner's daily means even after the underlying measurements
// were replaced — the chart's `source=rollup` fast-path could
// surface a stale 30-day mean built from rows that no longer
// existed. The fold below mints fresh rows from the restored
// measurement set.
await tx.measurementRollup.deleteMany({ where: { userId: ownerId } });
const channels = await tx.notificationChannel.deleteMany({
where: { userId: ownerId },
});
@@ -420,6 +429,29 @@ const handler = apiHandler(
return apiError("Restore failed", 500);
}
// v1.4.39.1 — re-fold the persistent measurement rollup tier from
// the just-restored measurements. Pre-fix the restore left the
// rollup table empty for the owner, so the dashboard chart's
// `source=rollup` fast-path silently returned zero buckets until
// the next worker boot ran the backfill discovery — multi-hour
// window of empty charts for the operator-restored account. Runs
// outside the transaction (the 5-year fold would otherwise hold a
// long write lock) and is best-effort so a populator hiccup never
// undoes the restore. The boot-time backfill is the safety net.
if (payload.measurements.length > 0) {
try {
await recomputeUserRollups(ownerId);
} catch (err) {
annotate({
meta: {
measurement_rollup_restore_failed: true,
measurement_rollup_restore_error:
err instanceof Error ? err.message : String(err),
},
});
}
}
// v1.4.39 W-MOOD — re-fold the mood rollup tier from the just-
// restored entries. Runs outside the transaction so the (5-year)
// fold can't hold a long write lock; best-effort so a populator
+119
View File
@@ -27,9 +27,29 @@ vi.mock("@/lib/logging/context", () => ({
annotate: vi.fn(),
}));
// v1.4.39.1 — surface the rollup hook so the new regression test can
// assert that the import path now folds the persistent rollup tier
// for each touched (type, day). Mood rollup mock stays no-op because
// the existing v1.4.39 W-MOOD hook already covers the moodlog branch.
vi.mock("@/lib/measurements/rollups", async () => {
const actual = await vi.importActual<
typeof import("@/lib/measurements/rollups")
>("@/lib/measurements/rollups");
return {
...actual,
recomputeBucketsForMeasurement: vi.fn().mockResolvedValue(undefined),
};
});
vi.mock("@/lib/mood/rollups", () => ({
recomputeUserMoodRollups: vi.fn().mockResolvedValue(undefined),
}));
import { NextRequest } from "next/server";
import { POST } from "../route";
import { checkRateLimit } from "@/lib/rate-limit";
import { prisma } from "@/lib/db";
import { recomputeBucketsForMeasurement } from "@/lib/measurements/rollups";
beforeEach(() => {
vi.resetAllMocks();
@@ -107,3 +127,102 @@ describe("POST /api/import — rate-limit guard", () => {
expect(body.error).toBeTruthy();
});
});
// v1.4.39.1 — the import path used to write measurements without
// firing the rollup hook, so the dashboard chart's `source=rollup`
// fast-path silently under-counted any imported days until the next
// worker boot ran the backfill discovery. Cover the wiring with a
// regression test so a future refactor can't silently drop it again.
describe("POST /api/import — measurement rollup hook (v1.4.39.1)", () => {
it("folds the persistent rollup table for each (type, day) the import touched", async () => {
vi.mocked(checkRateLimit).mockResolvedValue({
allowed: true,
remaining: 4,
resetAt: new Date(Date.now() + 1000),
} as never);
vi.mocked(prisma.measurement.create).mockResolvedValue({} as never);
const response = await POST(
new NextRequest("http://localhost/api/import", {
method: "POST",
body: JSON.stringify({
measurements: [
{
type: "BLOOD_PRESSURE_SYS",
value: 124,
unit: "mmHg",
measuredAt: "2026-05-12T07:30:00.000Z",
},
{
type: "BLOOD_PRESSURE_SYS",
value: 121,
unit: "mmHg",
measuredAt: "2026-05-12T19:30:00.000Z",
},
{
type: "BLOOD_PRESSURE_SYS",
value: 119,
unit: "mmHg",
measuredAt: "2026-05-14T08:00:00.000Z",
},
{
type: "WEIGHT",
value: 82.4,
unit: "kg",
measuredAt: "2026-05-14T07:00:00.000Z",
},
],
}),
headers: { "content-type": "application/json" },
}),
);
expect(response.status).toBeLessThan(400);
// Two BP rows on 2026-05-12 collapse to one rollup call; one BP
// row on 2026-05-14 is its own call; one WEIGHT row on 2026-05-14
// is a separate (type, day) pair. Total = 3 distinct rollup folds.
expect(recomputeBucketsForMeasurement).toHaveBeenCalledTimes(3);
const calls = vi
.mocked(recomputeBucketsForMeasurement)
.mock.calls.map((c) => `${c[1]}|${(c[2] as Date).toISOString()}`);
expect(new Set(calls)).toEqual(
new Set([
"BLOOD_PRESSURE_SYS|2026-05-12T00:00:00.000Z",
"BLOOD_PRESSURE_SYS|2026-05-14T00:00:00.000Z",
"WEIGHT|2026-05-14T00:00:00.000Z",
]),
);
});
it("does not call the rollup hook when no measurements were written", async () => {
vi.mocked(checkRateLimit).mockResolvedValue({
allowed: true,
remaining: 4,
resetAt: new Date(Date.now() + 1000),
} as never);
vi.mocked(prisma.measurement.create).mockRejectedValue(
new Error("P2002 duplicate"),
);
const response = await POST(
new NextRequest("http://localhost/api/import", {
method: "POST",
body: JSON.stringify({
measurements: [
{
type: "WEIGHT",
value: 82.4,
unit: "kg",
measuredAt: "2026-05-14T07:00:00.000Z",
},
],
}),
headers: { "content-type": "application/json" },
}),
);
expect(response.status).toBeLessThan(400);
// Every measurement raised a duplicate — no rollup fold needed.
expect(recomputeBucketsForMeasurement).not.toHaveBeenCalled();
});
});
+42 -1
View File
@@ -18,6 +18,11 @@ import {
glucoseContextEnum,
} from "@/lib/validations/measurement";
import { recomputeUserMoodRollups } from "@/lib/mood/rollups";
import {
collapseToTypeDayKeys,
recomputeBucketsForMeasurement,
} from "@/lib/measurements/rollups";
import type { MeasurementType } from "@/generated/prisma/client";
// Derived from canonical enum so round-trip export → import covers every
// type. Previous hardcoded subset silently dropped 4 of 11 types
@@ -82,9 +87,19 @@ export const POST = apiHandler(async (request: NextRequest) => {
const stats = { measurements: 0, moodEntries: 0, skipped: 0 };
// Import measurements
// v1.4.39.1 — track each (type, measuredAt) we wrote so we can re-
// fold the persistent rollup tier at the end. Pre-fix, the import
// path bypassed the rollup write hook entirely, so a CSV / JSON
// restore left the user's `source=rollup` chart paths under-counting
// the imported days until the next worker boot re-ran the backfill.
const touchedMeasurements: Array<{
type: MeasurementType;
measuredAt: Date;
}> = [];
if (data.measurements?.length) {
for (const m of data.measurements) {
try {
const measuredAt = new Date(m.measuredAt);
await prisma.measurement.create({
data: {
userId,
@@ -92,11 +107,15 @@ export const POST = apiHandler(async (request: NextRequest) => {
value: m.value,
unit: m.unit,
source: "IMPORT",
measuredAt: new Date(m.measuredAt),
measuredAt,
notes: m.notes || null,
glucoseContext: m.glucoseContext ?? null,
},
});
touchedMeasurements.push({
type: m.type as MeasurementType,
measuredAt,
});
stats.measurements++;
} catch {
// Unique constraint violation — skip duplicate
@@ -148,6 +167,28 @@ export const POST = apiHandler(async (request: NextRequest) => {
}
}
// v1.4.39.1 — refresh the persistent measurement rollup table for
// every distinct (type, day) the import touched. Collapsed so a
// 10 000-row CSV restore pays at most ~N (type, day) recomputes
// rather than 10 000 per-row hooks. Best-effort: a populator hiccup
// never fails the importer.
if (touchedMeasurements.length > 0) {
try {
const keys = collapseToTypeDayKeys(touchedMeasurements);
for (const k of keys) {
await recomputeBucketsForMeasurement(userId, k.type, k.measuredAt);
}
} catch (err) {
annotate({
meta: {
measurement_rollup_import_failed: true,
measurement_rollup_import_error:
err instanceof Error ? err.message : String(err),
},
});
}
}
await auditLog("import.upload", {
userId,
ipAddress: getClientIp(request),
@@ -208,8 +208,20 @@ describe("v1.4.39 W-MED — compliance rollup read swap", () => {
vi.mocked(prisma.$queryRaw).mockResolvedValue([
{ rolled_days: BigInt(7), event_days: BigInt(7) },
] as never);
// v1.4.39.1 — anchor the seed row on the runtime's "today" so the
// test stays green across the calendar regardless of when it
// runs. The pre-fix shape hard-coded `2026-05-18`, which fell out
// of the trailing-7-day window on every subsequent wall-clock
// day and silently shifted the body.data tail off the seed.
const todayKey = (() => {
const now = new Date();
const y = now.getUTCFullYear();
const m = String(now.getUTCMonth() + 1).padStart(2, "0");
const d = String(now.getUTCDate()).padStart(2, "0");
return `${y}-${m}-${d}`;
})();
vi.mocked(prisma.medicationComplianceRollup.findMany).mockResolvedValue([
{ day: "2026-05-18", scheduled: 3, taken: 2, skipped: 1 },
{ day: todayKey, scheduled: 3, taken: 2, skipped: 1 },
] as never);
const res = await GET(
+25 -3
View File
@@ -546,8 +546,8 @@ describe("enqueueBootTimeRollupBackfill", () => {
// v1.4.39 W-SUM — the discovery query now unions in users whose
// existing DAY rollup rows carry `sum_value IS NULL`. Re-folding
// converges those rows because `persistRollupRows` always writes the
// new column on upsert; the union keeps the per-type missing-coverage
// gap (v1.4.38.5) and the legacy-NULL backfill on the same indexed
// new column on upsert; the union keeps the per-day missing-coverage
// gap (v1.4.39.1) and the legacy-NULL backfill on the same indexed
// pass.
it("includes the sum_value IS NULL branch in the discovery query", async () => {
getGlobalBossMock.mockReturnValue({ send: bossSend });
@@ -557,7 +557,7 @@ describe("enqueueBootTimeRollupBackfill", () => {
const result = await enqueueBootTimeRollupBackfill();
expect(result).toEqual({ enqueued: 1, skipped: 0, error: null });
// The discovery SQL surfaces both per-type missing coverage AND
// The discovery SQL surfaces both per-day missing coverage AND
// legacy NULL sum_value rows under one UNION. The text-anchor is
// the only durable assertion at the unit level — the integration
// suite covers the planner shape on real Postgres.
@@ -566,4 +566,26 @@ describe("enqueueBootTimeRollupBackfill", () => {
expect(sqlText).toContain("sum_value");
expect(sqlText).toContain("UNION");
});
// v1.4.39.1 — the discovery anchor moved from per-type to per-day so
// accounts whose Withings sync / `/api/import` / admin restore wrote
// measurements without firing the rollup write hook get re-enqueued
// on the next worker boot. Pre-fix, the per-type LEFT JOIN matched
// on `(user, type)` and the user dropped off the discovery list as
// soon as ONE rollup row existed for the type — even if 27 OTHER
// days for that type were missing rollup rows.
it("anchors the missing-coverage join on (user, type, bucket_start) so per-day gaps surface", async () => {
getGlobalBossMock.mockReturnValue({ send: bossSend });
queryRaw.mockResolvedValueOnce([{ id: "withings-stranded-user" }]);
bossSend.mockResolvedValueOnce("job-id");
await enqueueBootTimeRollupBackfill();
const sqlParts = queryRaw.mock.calls[0][0] as TemplateStringsArray;
const sqlText = Array.isArray(sqlParts) ? sqlParts.join("?") : "";
// Inner DISTINCT widens to `(user_id, type, bucket_start)`.
expect(sqlText).toContain('date_trunc(\'day\', m."measured_at")');
// LEFT JOIN now compares bucket_start on both sides.
expect(sqlText).toContain('r."bucket_start" = mt."bucket_start"');
});
});
+25 -14
View File
@@ -741,35 +741,46 @@ export async function enqueueBootTimeRollupBackfill(): Promise<{
}
try {
// Users where at least one measurement type lacks DAY-rollup
// coverage. Anchored on the per-user `DISTINCT type` set so the
// planner uses the `(user_id, type, measured_at)` index path; the
// LEFT JOIN onto `measurement_rollups` uses the
// `(user_id, type, granularity, bucket_start)` composite primary
// key. `measurement_rollups` has no surrogate `id` column —
// `r."bucket_start" IS NULL` (any column from the right side of
// the LEFT JOIN) marks the unmatched partitions. A user surfaces
// here when ANY type is missing — including the brand-new-type
// case that v1.4.35.1's zero-rollup-only shape silently stranded.
// Users where at least one (type, day) pair in `measurements`
// has no matching DAY rollup row.
//
// v1.4.39.1 — tightened from per-type to per-day. The legacy shape
// matched users where ANY type had zero rollup rows. That stranded
// accounts whose Withings sync / `/api/import` / admin restore
// wrote measurements without firing the rollup write hook: as soon
// as ONE rollup row existed for the type (even from a prior boot
// backfill), the LEFT JOIN found a match and the user dropped off
// the discovery list — leaving the dashboard chart's
// `source=rollup` fast-path under-counting the missing days.
// Switching the join to `(user, type, bucketStart)` surfaces every
// per-day gap. Cost: the inner DISTINCT widens from
// `(user_id, type)` (≈15 rows/user) to
// `(user_id, type, day)` (≈5 k rows/user on a 5-year power user),
// which is still bounded by the `(user_id, type, measured_at)`
// index path. The LEFT JOIN walks `measurement_rollups` against
// its composite primary key.
//
// v1.4.39 W-SUM — the discovery also surfaces users whose existing
// DAY rollup rows carry `sum_value IS NULL`. Those rows pre-date
// the v1.4.39 writer change; re-folding the user converges them
// because `persistRollupRows` always writes the new column on
// upsert. The union runs in one query so the existing per-type
// coverage gap and the legacy-NULL backfill share a single planner
// pass.
// upsert. The union runs in one query so the per-day coverage gap
// and the legacy-NULL backfill share a single planner pass.
const users = await prisma.$queryRaw<Array<{ id: string }>>`
SELECT DISTINCT id FROM (
SELECT DISTINCT mt."user_id" AS id
FROM (
SELECT DISTINCT m."user_id", m."type"
SELECT DISTINCT
m."user_id",
m."type",
date_trunc('day', m."measured_at") AS bucket_start
FROM measurements m
) mt
LEFT JOIN measurement_rollups r
ON r."user_id" = mt."user_id"
AND r."type" = mt."type"
AND r."granularity" = 'DAY'
AND r."bucket_start" = mt."bucket_start"
WHERE r."bucket_start" IS NULL
UNION
SELECT DISTINCT r2."user_id" AS id
@@ -45,12 +45,27 @@ vi.mock("@/lib/logging/context", () => ({
})),
}));
// v1.4.39.1 — the sync now hands its touched (type, day) keys to the
// measurement rollup writer so the dashboard chart's `source=rollup`
// fast-path sees the new buckets. The mock collects the call args so
// the regression test below can assert the hook fired.
vi.mock("@/lib/measurements/rollups", async () => {
const actual = await vi.importActual<
typeof import("@/lib/measurements/rollups")
>("@/lib/measurements/rollups");
return {
...actual,
recomputeBucketsForMeasurement: vi.fn().mockResolvedValue(undefined),
};
});
import { prisma } from "@/lib/db";
import {
parkIntegrationAtReauth,
recordSyncFailure,
recordSyncSuccess,
} from "@/lib/integrations/status";
import { recomputeBucketsForMeasurement } from "@/lib/measurements/rollups";
import { fetchWithingsActivity, syncUserActivity } from "../sync-activity";
@@ -363,3 +378,66 @@ describe("syncUserActivity — scope-skip guard (v1.4.26)", () => {
expect(parkIntegrationAtReauth).not.toHaveBeenCalled();
});
});
describe("syncUserActivity — measurement rollup hook (v1.4.39.1)", () => {
it("folds the persistent rollup table for each (type, day) the sync touched", async () => {
// Two distinct calendar days × three activity metrics each = 6
// writes; the rollup hook should be called once per distinct
// `(type, dayStart)` so the dashboard chart's `source=rollup` fast-
// path lights up the Withings-ingested days. Pre-v1.4.39.1 the
// sync skipped the hook entirely, leaving the rollup tier blind to
// Withings activity.
installFetchMock([
{ date: "2026-05-12", steps: 8420, distance: 6720, calories: 412 },
{ date: "2026-05-13", steps: 9100, distance: 7100, calories: 430 },
]);
vi.mocked(prisma.measurement.findFirst).mockResolvedValue(null);
vi.mocked(prisma.measurement.create).mockResolvedValue({} as never);
await syncUserActivity("user-1");
expect(recomputeBucketsForMeasurement).toHaveBeenCalledTimes(6);
const calls = vi
.mocked(recomputeBucketsForMeasurement)
.mock.calls.map((c) => ({
userId: c[0],
type: c[1],
measuredAtIso: (c[2] as Date).toISOString(),
}));
// Distinct (type, day) pairs the sync touched, derived from the two
// dates × three metrics × `collapseToTypeDayKeys`.
const distinct = new Set(calls.map((c) => `${c.type}|${c.measuredAtIso}`));
expect(distinct.size).toBe(6);
expect(calls.every((c) => c.userId === "user-1")).toBe(true);
});
it("collapses multiple writes for the same (type, day) into a single rollup recompute", async () => {
// A single-date entry exercises the existing day-collapse helper —
// three metric rows for 2026-05-12 must fold into three rollup
// calls (one per type), not nine.
installFetchMock([
{ date: "2026-05-12", steps: 8420, distance: 6720, calories: 412 },
]);
vi.mocked(prisma.measurement.findFirst).mockResolvedValue(null);
vi.mocked(prisma.measurement.create).mockResolvedValue({} as never);
await syncUserActivity("user-1");
expect(recomputeBucketsForMeasurement).toHaveBeenCalledTimes(3);
});
it("swallows a populator failure so the sync still returns its imported count", async () => {
// The rollup recompute is a best-effort cache tier — a populator
// hiccup must never fail the user's sync. The sync's `imported`
// return value should still reflect the rows written.
installFetchMock([{ date: "2026-05-12", steps: 8420 }]);
vi.mocked(prisma.measurement.findFirst).mockResolvedValue(null);
vi.mocked(prisma.measurement.create).mockResolvedValue({} as never);
vi.mocked(recomputeBucketsForMeasurement).mockRejectedValueOnce(
new Error("simulated rollup failure"),
);
const imported = await syncUserActivity("user-1");
expect(imported).toBe(1);
});
});
@@ -44,12 +44,27 @@ vi.mock("@/lib/logging/context", () => ({
})),
}));
// v1.4.39.1 — the sleep sync now folds the persistent measurement
// rollup tier for each touched SLEEP_DURATION day so the dashboard
// chart's `source=rollup` fast-path sees the new segments. The mock
// surfaces the call args for the regression test below.
vi.mock("@/lib/measurements/rollups", async () => {
const actual = await vi.importActual<
typeof import("@/lib/measurements/rollups")
>("@/lib/measurements/rollups");
return {
...actual,
recomputeBucketsForMeasurement: vi.fn().mockResolvedValue(undefined),
};
});
import { prisma } from "@/lib/db";
import {
parkIntegrationAtReauth,
recordSyncFailure,
recordSyncSuccess,
} from "@/lib/integrations/status";
import { recomputeBucketsForMeasurement } from "@/lib/measurements/rollups";
import {
fetchWithingsSleep,
@@ -352,3 +367,47 @@ describe("syncUserSleep — scope-skip guard (v1.4.26)", () => {
expect(parkIntegrationAtReauth).not.toHaveBeenCalled();
});
});
describe("syncUserSleep — measurement rollup hook (v1.4.39.1)", () => {
it("folds the persistent rollup table once per night the sync touched", async () => {
// Two distinct nights, each with multiple stage segments. The
// sync should collapse the per-night segments to one rollup
// recompute per night via `collapseToTypeDayKeys`. Pre-v1.4.39.1
// the rollup tier never heard about Withings sleep at all.
const nightA = Math.floor(Date.UTC(2026, 4, 12, 22) / 1000); // 2026-05-12 22:00 UTC
const nightB = Math.floor(Date.UTC(2026, 4, 13, 22) / 1000); // 2026-05-13 22:00 UTC
installFetchMock([
{ startdate: nightA, enddate: nightA + 3600, state: 1, id: 1 },
{ startdate: nightA + 3600, enddate: nightA + 5400, state: 2, id: 1 },
{ startdate: nightA + 5400, enddate: nightA + 7200, state: 3, id: 1 },
{ startdate: nightB, enddate: nightB + 3600, state: 1, id: 2 },
{ startdate: nightB + 3600, enddate: nightB + 5400, state: 2, id: 2 },
]);
vi.mocked(prisma.measurement.findFirst).mockResolvedValue(null);
vi.mocked(prisma.measurement.create).mockResolvedValue({} as never);
await syncUserSleep("user-1");
// Both segments of night A land in the same UTC day; night B in
// the next. Distinct (SLEEP_DURATION, dayStart) tuples = 2.
expect(recomputeBucketsForMeasurement).toHaveBeenCalledTimes(2);
const types = vi
.mocked(recomputeBucketsForMeasurement)
.mock.calls.map((c) => c[1]);
expect(new Set(types)).toEqual(new Set(["SLEEP_DURATION"]));
});
it("swallows a populator failure so the sync still returns its imported count", async () => {
installFetchMock([
{ startdate: 1715000000, enddate: 1715003600, state: 2, id: 1 },
]);
vi.mocked(prisma.measurement.findFirst).mockResolvedValue(null);
vi.mocked(prisma.measurement.create).mockResolvedValue({} as never);
vi.mocked(recomputeBucketsForMeasurement).mockRejectedValueOnce(
new Error("simulated rollup failure"),
);
const imported = await syncUserSleep("user-1");
expect(imported).toBe(1);
});
});
+24
View File
@@ -38,6 +38,10 @@ import type { MeasurementType } from "@/generated/prisma/client";
import { prisma } from "@/lib/db";
import { getEvent } from "@/lib/logging/context";
import { getUnitForType } from "@/lib/validations/measurement";
import {
collapseToTypeDayKeys,
recomputeBucketsForMeasurement,
} from "@/lib/measurements/rollups";
import { hasActivityScope } from "./client";
import {
@@ -285,6 +289,11 @@ export async function syncUserActivity(
}
let imported = 0;
// v1.4.39.1 — track every (type, measuredAt) we touched so the
// persistent rollup tier can be re-folded at the end of the sync.
// See sync.ts header for the full rationale; the chart's
// `source=rollup` fast-path otherwise misses Withings activity days.
const touched: Array<{ type: MeasurementType; measuredAt: Date }> = [];
for (const entry of entries) {
if (!entry.date) continue;
const measuredAt = activityMeasuredAt(entry.date);
@@ -323,6 +332,7 @@ export async function syncUserActivity(
},
});
}
touched.push({ type, measuredAt });
imported++;
} catch (err) {
getEvent()?.addWarning(
@@ -332,6 +342,20 @@ export async function syncUserActivity(
}
}
// v1.4.39.1 — refresh the persistent rollup table for every distinct
// (type, day) the sync touched. Collapsed so a multi-day Withings
// catch-up costs at most ~N (type, day) recomputes.
try {
const keys = collapseToTypeDayKeys(touched);
for (const k of keys) {
await recomputeBucketsForMeasurement(userId, k.type, k.measuredAt);
}
} catch (err) {
getEvent()?.addWarning(
`withings activity: rollup recompute failed for ${userId}: ${err}`,
);
}
await recordSyncSuccess(userId, "withings");
return imported;
}
+23
View File
@@ -36,6 +36,10 @@ import type { MeasurementType, SleepStage } from "@/generated/prisma/client";
import { prisma } from "@/lib/db";
import { getEvent } from "@/lib/logging/context";
import { getUnitForType } from "@/lib/validations/measurement";
import {
collapseToTypeDayKeys,
recomputeBucketsForMeasurement,
} from "@/lib/measurements/rollups";
import { hasActivityScope } from "./client";
import {
@@ -223,6 +227,10 @@ export async function syncUserSleep(
let imported = 0;
let segmentIndex = 0;
// v1.4.39.1 — track every (type, measuredAt) we touched so the
// persistent rollup tier can be re-folded at the end. See sync.ts for
// the full rationale.
const touched: Array<{ type: MeasurementType; measuredAt: Date }> = [];
for (const segment of segments) {
const stage = mapWithingsSleepState(segment.state);
if (!stage) {
@@ -270,6 +278,7 @@ export async function syncUserSleep(
},
});
}
touched.push({ type: SLEEP_TYPE, measuredAt });
imported++;
} catch (err) {
getEvent()?.addWarning(
@@ -279,6 +288,20 @@ export async function syncUserSleep(
segmentIndex++;
}
// v1.4.39.1 — refresh the persistent rollup table for every distinct
// (type, day) the sync touched. Sleep segments collapse heavily —
// ~10 stage rows per night land in one DAY recompute.
try {
const keys = collapseToTypeDayKeys(touched);
for (const k of keys) {
await recomputeBucketsForMeasurement(userId, k.type, k.measuredAt);
}
} catch (err) {
getEvent()?.addWarning(
`withings sleep: rollup recompute failed for ${userId}: ${err}`,
);
}
await recordSyncSuccess(userId, "withings");
return imported;
}
+33
View File
@@ -18,6 +18,10 @@ import {
recordSyncFailure,
recordSyncSuccess,
} from "@/lib/integrations/status";
import {
collapseToTypeDayKeys,
recomputeBucketsForMeasurement,
} from "@/lib/measurements/rollups";
/**
* Build the callback URL handed to Withings at `Notify.subscribe` time.
@@ -194,6 +198,18 @@ export async function syncUserMeasurements(
}
let imported = 0;
// v1.4.39.1 — track every (type, measuredAt) we touched so the
// persistent rollup tier can be re-folded at the end of the sync.
// Pre-fix, Withings-ingested BP / weight / pulse / body-fat rows
// skipped the rollup write hook entirely: subsequent dashboard chart
// fetches with `source=rollup` saw fewer DAY buckets than the live
// measurements table, painting the "Noch nicht genug Daten" empty
// state for the 30-day range on accounts whose recent data only came
// through Withings. We collect first, fold once at the end so a 100-
// row monthly catch-up costs at most ~N (type, day) recomputes rather
// than N per-row hooks. Best-effort: a populator hiccup never fails
// the user's sync.
const touched: Array<{ type: MeasurementType; measuredAt: Date }> = [];
for (const m of measures) {
const measType = m.type as MeasurementType;
@@ -233,12 +249,29 @@ export async function syncUserMeasurements(
},
});
}
touched.push({ type: measType, measuredAt: m.measuredAt });
imported++;
} catch (err) {
getEvent()?.addWarning(`Failed to upsert measure: ${err}`);
}
}
// v1.4.39.1 — refresh the persistent rollup table for every distinct
// (type, day) the sync touched. The chart-data + analytics read paths
// consume these buckets via `source=rollup`; without this hook,
// Withings-only metrics stayed off the rollup tier and the dashboard
// chart at the 30-day range under-counted recent days.
try {
const keys = collapseToTypeDayKeys(touched);
for (const k of keys) {
await recomputeBucketsForMeasurement(userId, k.type, k.measuredAt);
}
} catch (err) {
getEvent()?.addWarning(
`withings: rollup recompute failed for ${userId}: ${err}`,
);
}
// Update last synced timestamp
await prisma.withingsConnection.update({
where: { userId },