Merge remote-tracking branch 'origin/main' into develop

CHANGELOG.md

@@ -1,5 +1,99 @@
 # Changelog
 
+## [1.4.38.8] — 2026-05-18 — Analytics fast-path gates per-type only
+
+The v1.4.38.5–.7 chain confirmed the rollup-fast-path was bouncing
+to the live SQL aggregator even when the helper's own types were
+fully covered — every fast-path stacked `isFullyCovered(coverage)
+&& specificTypes` and a single unrelated uncovered type (e.g. an
+iOS-pushed brand-new ACTIVITY_FLIGHTS reading) flipped the AND
+false and stranded the whole helper on live SQL across the full
+347 k-row measurement table.
+
+### Fixed
+
+- **`correlations-fast-path` gates only on `BLOOD_PRESSURE_SYS +
+  PULSE + WEIGHT`** — the three types the helper actually reads.
+  `isFullyCovered(coverage)` AND-term dropped.
+- **`bp-in-target-fast-path` gates only on `BLOOD_PRESSURE_SYS +
+  BLOOD_PRESSURE_DIA`**. Same fix.
+- **`health-score-fast-path` gates only on `WEIGHT`**. Same fix.
+
+Coverage-probe semantics unchanged. Each helper now consults the
+per-type entries directly; a brand-new uncovered metric type no
+longer poisons unrelated cards.
+
+### Operator notes
+
+- No new migration. No env-var change.
+- Expected: cold-mount `/api/analytics` on Marc-sized accounts
+  drops from 30-75 s to ~3-5 s.
+
+## [1.4.38.7] — 2026-05-18 — Rollup recompute observability + admin trigger
+
+The v1.4.38.5 / v1.4.38.6 chain promised a fast-path recovery for
+power-user accounts but gave the operator no way to confirm whether
+the boot-time discovery had actually found a stranded user — the
+success log only fired when `enqueued > 0 || skipped > 0`, so the
+silent "found nothing" case looked identical to the silent "boot hook
+never ran" case. And once the worker was up there was no way to
+re-trigger discovery short of bouncing the container.
+
+### Added
+
+- **`POST /api/admin/rollups/recompute`** — operator-triggered ad-hoc
+  rollup recompute. Body `{ userId: string }` synchronously folds
+  one user (`recomputeUserRollups` awaited inside the request). Body
+  `{}` re-runs `enqueueBootTimeRollupBackfill()` to kick the
+  boot-discovery loop across every user. Admin gate via
+  `requireAdmin()` (cookie-only, never Bearer).
+- **`fallback_reason` + `missing_types` annotate** on the
+  `slim_summaries` live-fallback path. When the rollup-fast-path
+  declines, the wide-event now reports exactly which measurement
+  types are missing DAY-bucket coverage so the operator can match
+  a slow `/api/analytics` cold-mount to the responsible type without
+  touching the DB.
+
+### Changed
+
+- **Boot-backfill discovery now logs every result, including the
+  silent "no users to backfill" case.** The line cost is one row per
+  worker boot; the operator gain is the ability to tell discovery
+  ran cleanly from discovery silently no-op'd.
+
+### Operator notes
+
+- No new migration. No env-var change.
+
+## [1.4.38.6] — 2026-05-18 — Boot-backfill discovery SQL fix
+
+The v1.4.38.5 discovery rewrite filtered the LEFT JOIN with
+`WHERE r."id" IS NULL` — but `measurement_rollups` has a composite
+primary key `(user_id, type, granularity, bucket_start)` and no
+surrogate `id` column. Postgres rejected the query with
+`column r.id does not exist` (SQL state 42703), the worker boot
+swallowed the error per the helper's best-effort contract, and the
+queue stayed empty. Net effect: v1.4.38.5 deployed cleanly but
+delivered none of its promised fast-path recovery.
+
+### Fixed
+
+- **Boot-backfill discovery now filters on `r."bucket_start" IS NULL`**
+  — any column from the right side of the LEFT JOIN serves as the
+  "no matching row" sentinel; `bucket_start` is part of the composite
+  primary key so the planner already touches the column. Verified
+  against the live schema: `42703` no longer raised on worker boot,
+  and the discovery query returns the expected per-type-missing
+  candidate set.
+
+### Operator notes
+
+- No new migration. No env-var change.
+- On first boot after deploy the rollup-full-backfill queue picks
+  up users with any missing type-coverage and folds them. Power-
+  user accounts may see one slow `/api/analytics` cold-mount before
+  the next request lands on the restored fast path.
+
 ## [1.4.38.5] — 2026-05-18 — Analytics fast-path restored on long-running accounts
 
 Hotfix on top of v1.4.38.4. `/api/analytics` and the dashboard

package.json

@@ -1,6 +1,6 @@
 {
   "name": "healthlog",
-  "version": "1.4.38.5",
+  "version": "1.4.38.8",
   "description": "Self-hosted personal-health-tracking PWA with Withings integration, AI insights, and doctor-report PDF export.",
   "license": "AGPL-3.0-only",
   "homepage": "https://healthlog.dev",

src/app/api/admin/rollups/recompute/route.ts

@@ -0,0 +1,111 @@
+/**
+ * `POST /api/admin/rollups/recompute` — operator-triggered ad-hoc
+ * rollup recompute.
+ *
+ * Surface added in v1.4.38.7 after the boot-time backfill could not
+ * be re-triggered without a deploy. A power-user account whose
+ * `isFullyCovered(coverage)` probe returned `false` (typically
+ * because a brand-new measurement type landed between the last fold
+ * and the next analytics read) gets stranded on the live SQL
+ * aggregator until the next worker boot. This endpoint lets the
+ * operator force a fold for one user or kick the boot-discovery loop
+ * across every user without bouncing the worker container.
+ *
+ * Body:
+ *   { userId?: string }
+ *
+ *   - `userId` — single user mode. Synchronously calls
+ *     `recomputeUserRollups(userId)` and returns the row count when
+ *     the fold completes. The call is awaited inside the request, so
+ *     the operator sees the duration in the response.
+ *   - `userId` omitted — calls `enqueueBootTimeRollupBackfill()` to
+ *     re-run the worker-boot discovery (`{ enqueued, skipped }` shape).
+ *
+ * Admin gate via `requireAdmin()` (cookie-only — Bearer never elevates
+ * to admin per the v1.4.25 security boundary).
+ */
+import type { NextRequest } from "next/server";
+
+import { apiHandler, requireAdmin } from "@/lib/api-handler";
+import { auditLog } from "@/lib/auth/audit";
+import {
+  apiError,
+  apiSuccess,
+  getClientIp,
+  safeJson,
+} from "@/lib/api-response";
+import { annotate } from "@/lib/logging/context";
+import {
+  enqueueBootTimeRollupBackfill,
+  recomputeUserRollups,
+} from "@/lib/measurements/rollups";
+
+interface RecomputeBody {
+  userId?: string;
+}
+
+export const POST = apiHandler(async (request: NextRequest) => {
+  const { user } = await requireAdmin();
+  annotate({ action: { name: "admin.rollups.recompute" } });
+
+  const { data: rawBody, error: jsonError } =
+    await safeJson<RecomputeBody>(request);
+  if (jsonError) return jsonError;
+
+  const body: RecomputeBody =
+    rawBody && typeof rawBody === "object" ? rawBody : {};
+  const userId =
+    typeof body.userId === "string" && body.userId.length > 0
+      ? body.userId
+      : undefined;
+
+  if (userId) {
+    await auditLog("admin.rollups.recompute.start", {
+      userId: user.id,
+      ipAddress: getClientIp(request),
+      details: { targetUserId: userId, mode: "single" },
+    });
+
+    let result: Awaited<ReturnType<typeof recomputeUserRollups>>;
+    try {
+      result = await recomputeUserRollups(userId);
+    } catch (err) {
+      const message = err instanceof Error ? err.message : String(err);
+      return apiError(message, 500);
+    }
+
+    await auditLog("admin.rollups.recompute.complete", {
+      userId: user.id,
+      ipAddress: getClientIp(request),
+      details: { targetUserId: userId, ...result },
+    });
+
+    return apiSuccess({
+      mode: "single",
+      targetUserId: userId,
+      rowsUpserted: result.rowsUpserted,
+      durationMs: result.durationMs,
+    });
+  }
+
+  await auditLog("admin.rollups.recompute.start", {
+    userId: user.id,
+    ipAddress: getClientIp(request),
+    details: { mode: "boot-discovery" },
+  });
+
+  const discovery = await enqueueBootTimeRollupBackfill();
+
+  await auditLog("admin.rollups.recompute.complete", {
+    userId: user.id,
+    ipAddress: getClientIp(request),
+    details: { mode: "boot-discovery", ...discovery },
+  });
+
+  return apiSuccess({
+    mode: "boot-discovery",
+    enqueued: discovery.enqueued,
+    skipped: discovery.skipped,
+    error: discovery.error,
+  });
+});

src/lib/analytics/bp-in-target-fast-path.ts

@@ -54,7 +54,6 @@ import { prisma } from "@/lib/db";
 import { annotate } from "@/lib/logging/context";
 import { readRollupBuckets } from "@/lib/measurements/rollups";
 import {
-  isFullyCovered,
   probeRollupCoverage,
   type RollupCoverageMap,
 } from "@/lib/measurements/rollup-coverage";
@@ -149,14 +148,16 @@ export async function computeBpInTargetFastPath(input: {
     ? "near-utc"
     : "non-utc-live-fallback";
 
-  // Belt-and-braces: only take the rollup path when EVERY logged type
-  // is covered AND both BP types are present in the map AND the
-  // cross-tz guard cleared. A brand-new user without BP at all falls
-  // into `isFullyCovered === false` (coverage map is empty) and the
-  // route's downstream code already skips the BP block when
-  // `getBpTargets` returns null; we still call the live fallback here
-  // so the helper's contract stays single-shot.
-  if (userNearUtc && isFullyCovered(coverage) && sysCovered && diaCovered) {
+  // v1.4.38.8 — gate only on the types the helper actually reads.
+  // The previous `isFullyCovered(coverage) && sysCovered && diaCovered`
+  // shape poisoned the BP fast-path with any unrelated brand-new
+  // measurement type the user had logged (e.g. iOS pushed a new
+  // ACTIVITY_FLIGHTS reading without a rollup row yet) — even though
+  // SYS + DIA are fully covered. The cold-mount then fanned out to the
+  // live aggregator over the full measurement table. Coverage probe is
+  // already keyed per type, so trusting the per-type entries directly
+  // is the correct gate.
+  if (userNearUtc && sysCovered && diaCovered) {
     return computeFromRollups(userId, targets, now, tzGuard);
   }
   return computeFromLive(userId, targets, now, tzGuard);

src/lib/analytics/correlations-fast-path.ts

@@ -47,7 +47,6 @@ import { prisma } from "@/lib/db";
 import { annotate } from "@/lib/logging/context";
 import { readRollupBuckets } from "@/lib/measurements/rollups";
 import {
-  isFullyCovered,
   probeRollupCoverage,
   type RollupCoverageMap,
 } from "@/lib/measurements/rollup-coverage";
@@ -140,14 +139,13 @@ export async function computeCorrelationHypothesesFastPath(
   // local-day buckets and this guard can come back down.
   const userNearUtc = isNearUtc(userTz, now);
 
-  // The three measurement-derived streams ride the rollup-fast-path
-  // only when EVERY type the user has logged is covered AND each of
-  // SYS / PULSE / WEIGHT is in the coverage map. Partial coverage
-  // falls back to the live path so a brand-new metric type doesn't
-  // make the correlation card vanish.
+  // v1.4.38.8 — gate only on the three types this helper actually
+  // reads (SYS / PULSE / WEIGHT). The prior `isFullyCovered &&` AND
+  // poisoned the correlations fast-path with any unrelated brand-new
+  // type the user had logged, even when SYS + PULSE + WEIGHT were
+  // fully covered. Per-type coverage is the correct gate.
   const measurementsOnRollups =
     userNearUtc &&
-    isFullyCovered(coverage) &&
     coverage.get("BLOOD_PRESSURE_SYS") === true &&
     coverage.get("PULSE") === true &&
     coverage.get("WEIGHT") === true;

src/lib/analytics/health-score-fast-path.ts

@@ -42,7 +42,6 @@ import { prisma } from "@/lib/db";
 import { annotate } from "@/lib/logging/context";
 import { readRollupBuckets } from "@/lib/measurements/rollups";
 import {
-  isFullyCovered,
   probeRollupCoverage,
   type RollupCoverageMap,
 } from "@/lib/measurements/rollup-coverage";
@@ -146,8 +145,11 @@ export async function computeUserHealthScoreFastPath(
   const prevUntil = new Date(now.getTime() - 7 * DAY_MS);
 
   const coverage = input.coverage ?? (await probeRollupCoverage(userId));
-  const weightCovered =
-    isFullyCovered(coverage) && coverage.get("WEIGHT") === true;
+  // v1.4.38.8 — gate only on the WEIGHT type this helper reads. The
+  // prior `isFullyCovered &&` AND poisoned the score fast-path with
+  // any unrelated brand-new type that lacked rollup coverage. Per-
+  // type coverage is the correct gate.
+  const weightCovered = coverage.get("WEIGHT") === true;
 
   // Weight series — rollup-fast-path when fully covered, raw findMany
   // otherwise. The raw read selects only the columns the slope helper

src/lib/analytics/summaries-slice.ts

@@ -217,6 +217,26 @@ export async function computeSummariesSlice(
   if (isFullyCovered(coverage)) {
     return computeFromRollups(userId);
   }
+  // v1.4.38.7 — annotate the live fallback with the per-type
+  // coverage map so the operator can see WHICH type stranded the
+  // request on the live aggregator. Without this the wide-event only
+  // says `path:"live"`, and "why" requires shell access to the DB.
+  const missing: string[] = [];
+  for (const [type, hasBuckets] of coverage.entries()) {
+    if (!hasBuckets) missing.push(type);
+  }
+  annotate({
+    meta: {
+      analytics: {
+        slim_summaries: {
+          fallback_reason: "partial_rollup_coverage",
+          missing_types: missing,
+          covered_count: coverage.size - missing.length,
+          total_types: coverage.size,
+        },
+      },
+    },
+  });
   return computeFromLiveAggregate(userId);
 }
 

src/lib/jobs/reminder-worker.ts

@@ -1960,7 +1960,13 @@ export async function startReminderWorker() {
         "error",
         `[rollup-full-backfill] boot discovery failed: ${error}`,
       );
-    } else if (enqueued > 0 || skipped > 0) {
+    } else {
+      // v1.4.38.7 — log the discovery result on every boot, including
+      // the silent `enqueued=0 skipped=0` case. Without this, an
+      // operator chasing "analytics is slow" cannot tell whether the
+      // discovery query ran successfully (and found nothing to fold)
+      // vs. silently no-op'd. The line is one row per worker boot, so
+      // the log cost is negligible.
       workerLog(
         "info",
         `[rollup-full-backfill] boot discovery: enqueued=${enqueued} skipped=${skipped}`,

src/lib/measurements/rollups.ts

@@ -736,10 +736,11 @@ export async function enqueueBootTimeRollupBackfill(): Promise<{
     // planner uses the `(user_id, type, measured_at)` index path; the
     // LEFT JOIN onto `measurement_rollups` uses the
     // `(user_id, type, granularity, bucket_start)` composite primary
-    // key. `r."id" IS NULL` filters the unmatched partitions. A user
-    // surfaces here when ANY type is missing — including the brand-
-    // new-type case that v1.4.35.1's zero-rollup-only shape silently
-    // stranded.
+    // key. `measurement_rollups` has no surrogate `id` column —
+    // `r."bucket_start" IS NULL` (any column from the right side of
+    // the LEFT JOIN) marks the unmatched partitions. A user surfaces
+    // here when ANY type is missing — including the brand-new-type
+    // case that v1.4.35.1's zero-rollup-only shape silently stranded.
     const users = await prisma.$queryRaw<Array<{ id: string }>>`
       SELECT DISTINCT mt."user_id" AS id
       FROM (
@@ -750,7 +751,7 @@ export async function enqueueBootTimeRollupBackfill(): Promise<{
         ON  r."user_id"     = mt."user_id"
         AND r."type"        = mt."type"
         AND r."granularity" = 'DAY'
-      WHERE r."id" IS NULL
+      WHERE r."bucket_start" IS NULL
     `;
 
     if (users.length === 0) {