perf(insights): fold the all-time extremes from the DAY rollup tier

readAllTimeExtremes read every raw reading of weight, blood pressure and
pulse twice on each call, a sequential pass over the measurements table
that made it the largest read on the database. The DAY tier already
holds the same per-source day fold except for the extremes of the
readings themselves: min_value and max_value widen to a row's own
intra-hour range.

Migration 0417 adds reading_min_value and reading_max_value to
measurement_rollups. The writer fills them from the same aggregate, and
the boot re-fold refills existing rows through the arm that already
watches sum_xy. The reader folds the tier for whole UTC days inside the
fold window and the raw rows for the rest: older days, the partial day
a lookback floor falls in, and any type whose tier rows lack the new
columns. The raw arm reads narrow columns once instead of twice.

An integration test compares the result with the previous query on a
seeded history with two sources per day, hourly pulse statistics with
their own range, readings older than the window, midnight edges and
deletions, and shows the tier is the path that answers.

src/lib/insights/feature-blocks.ts

@@ -17,11 +17,17 @@ import {
   foldMeanSql,
   isHourlyMeanTypeSql,
 } from "@/lib/measurements/day-mean";
-import { loadUserSourcePriority } from "@/lib/rollups/measurement-read";
+import {
+  loadUserSourcePriority,
+  utcBucketEnd,
+} from "@/lib/rollups/measurement-read";
 import { classifyAgainstEffectiveRange } from "@/lib/labs/reference-range";
 import { calendarDaysUntil } from "@/lib/measurement-reminders/due-day";
 import { resolveLabFields } from "@/lib/labs/serialise";
-import { readRollupBuckets } from "@/lib/rollups/measurement-rollups";
+import {
+  readRollupBuckets,
+  ROLLUP_FOLD_WINDOW_MS,
+} from "@/lib/rollups/measurement-rollups";
 import { deriveBucketedTypes } from "@/lib/signals/adapters/correlation";
 import type {
   AggregatedFeatures,
@@ -40,11 +46,10 @@ interface AllTimeExtremes {
 }
 
 /**
- * v1.18.11 — full-history min / max / mean per measurement type via ONE
- * grouped SQL aggregation, with NO row materialisation in JS. Fills the
- * `allTime*` feature fields; the windowed `summarize()` covers trends + recent
- * windows, and this covers the long-horizon figures the prompt labels
- * "allTime".
+ * v1.18.11 — full-history min / max / mean per measurement type, folded in
+ * SQL with NO row materialisation in JS. Fills the `allTime*` feature fields;
+ * the windowed `summarize()` covers trends + recent windows, and this covers
+ * the long-horizon figures the prompt labels "allTime".
  *
  * One source per (type, day): each source is folded per day first, each day
  * keeps its ladder-canonical source, then the days fold into the figures —
@@ -52,6 +57,25 @@ interface AllTimeExtremes {
  * row let a device that samples densely (a watch's pulse) outweigh the
  * readings the source ladder prefers on the same days.
  *
+ * Where the days come from. A day is the same per-source fold either way —
+ * count, sum, the extremes of the readings and, for an hourly-mean type, the
+ * day value — and the DAY rollup tier already holds exactly that per (type,
+ * source, day), recomputed on every write. Reading the raw rows instead meant
+ * two passes over every reading of these types the account ever recorded on
+ * every call, the largest read on the database. So the figures fold:
+ *
+ *   - the DAY tier, for every whole UTC day from `tierFrom` on, for each type
+ *     whose tier rows in that range all carry the reading extremes
+ *     (migration 0417; the boot re-fold fills older rows);
+ *   - the raw rows for the rest: the days before `tierFrom` (older than the
+ *     fold window, or the partial day the `floor` falls in), and every day of
+ *     a type the tier does not cover yet.
+ *
+ * The split is at a UTC midnight, so no day is folded from both, and the
+ * canonical pick per day is the same on both sides. The tier's pulse day
+ * value follows the stored day-mean method; the raw fold follows this
+ * process's mirror of it, which trails a change by at most a minute.
+ *
  * Only the four types that expose `allTime*` fields are aggregated (weight,
  * systolic, diastolic, pulse). Returns a map keyed by `MeasurementType`; a type
  * with no rows is simply absent.
@@ -67,15 +91,88 @@ export async function readAllTimeExtremes(
 ): Promise<Map<MeasurementType, AllTimeExtremes>> {
   const out = new Map<MeasurementType, AllTimeExtremes>();
   if (types.length === 0) return out;
+  // The first whole UTC day the tier may answer: after the fold window's
+  // edge (the fold writes nothing older, and the edge day may be partial)
+  // and after the floor's own day (only part of it is inside the floor).
+  const windowEdge = new Date(Date.now() - ROLLUP_FOLD_WINDOW_MS);
+  const tierFrom = utcBucketEnd(
+    floor !== null && floor > windowEdge ? floor : windowEdge,
+    "DAY",
+  );
+  const typeList = (list: readonly MeasurementType[]) =>
+    Prisma.join(list.map((t) => Prisma.sql`${t}::measurement_type`));
+
+  const [priorityJson, tierRows] = await Promise.all([
+    loadUserSourcePriority(userId),
+    prisma.$queryRaw<Array<{ type: string }>>`
+      SELECT r."type"::text AS type
+      FROM measurement_rollups r
+      WHERE r."user_id" = ${userId}
+        AND r."granularity" = 'DAY'
+        AND r."type" IN (${typeList(types)})
+        AND r."bucket_start" >= ${tierFrom}
+      GROUP BY r."type"
+      HAVING bool_and(
+        r."reading_min_value" IS NOT NULL AND r."reading_max_value" IS NOT NULL
+      )
+    `,
+  ]);
+  const tierTypes = new Set(tierRows.map((r) => r.type));
+  const fromTier = types.filter((t) => tierTypes.has(t));
+  const rawOnly = types.filter((t) => !tierTypes.has(t));
+
   const since =
     floor === null ? Prisma.empty : Prisma.sql`AND m."measured_at" >= ${floor}`;
-  const priorityJson = await loadUserSourcePriority(userId);
-  const rank = Prisma.raw(
-    buildSourceRankCase(priorityJson, 'p."type"', 'p."source"'),
-  );
-  const typeList = Prisma.join(
-    types.map((t) => Prisma.sql`${t}::measurement_type`),
-  );
+  const rawArms: Prisma.Sql[] = [];
+  if (rawOnly.length > 0) {
+    rawArms.push(Prisma.sql`
+      SELECT m."type", m."source", m."measured_at", m."value"
+      FROM measurements m
+      WHERE m."user_id" = ${userId}
+        AND m."deleted_at" IS NULL
+        AND m."type" IN (${typeList(rawOnly)})
+        ${since}`);
+  }
+  if (fromTier.length > 0) {
+    rawArms.push(Prisma.sql`
+      SELECT m."type", m."source", m."measured_at", m."value"
+      FROM measurements m
+      WHERE m."user_id" = ${userId}
+        AND m."deleted_at" IS NULL
+        AND m."type" IN (${typeList(fromTier)})
+        ${since}
+        AND m."measured_at" < ${tierFrom}`);
+  }
+  const rank = (alias: string) =>
+    Prisma.raw(
+      buildSourceRankCase(priorityJson, `${alias}."type"`, `${alias}."source"`),
+    );
+  // The tier's canonical days, in the shape of the raw fold's: the day's
+  // reading-weighted sum and count, its reading extremes, and its day value
+  // with a weight of one (a DAY row's `mean` is the day value for an
+  // hourly-mean type).
+  const tierCanon =
+    fromTier.length === 0
+      ? Prisma.empty
+      : Prisma.sql`
+    UNION ALL
+    SELECT * FROM (
+      SELECT DISTINCT ON (r."type", r."bucket_start")
+        r."type",
+        r."count"                                                AS cnt,
+        COALESCE(r."sum_value", r."count" * r."mean")::double precision
+                                                                 AS total,
+        r."reading_min_value"                                    AS min_value,
+        r."reading_max_value"                                    AS max_value,
+        r."mean"                                                 AS wsum,
+        1.0::double precision                                    AS wdays
+      FROM measurement_rollups r
+      WHERE r."user_id" = ${userId}
+        AND r."granularity" = 'DAY'
+        AND r."type" IN (${typeList(fromTier)})
+        AND r."bucket_start" >= ${tierFrom}
+      ORDER BY r."type", r."bucket_start", (${rank("r")}), r."source"::text
+    ) tier`;
   // A day of pulse is the mean of its hours' means and the full history the
   // mean of its days (`day-mean.ts`). `weighted` carries, per source and day,
   // the day-weighted sum and the day's weight (one) for those types only;
@@ -92,14 +189,10 @@ export async function readAllTimeExtremes(
   const rows = await prisma.$queryRaw<
     Array<{ type: string; mean: number; min: number; max: number }>
   >`
-    WITH hourly_src AS (
-      SELECT m.*
-      FROM measurements m
-      WHERE m."user_id" = ${userId}
-        AND m."deleted_at" IS NULL
-        AND m."type" IN (${typeList})
-        ${since}
-        AND ${Prisma.raw(isHourlyMeanTypeSql('m."type"'))}
+    WITH live AS (${Prisma.join(rawArms, " UNION ALL ")}),
+    hourly_src AS (
+      SELECT * FROM live l
+      WHERE ${Prisma.raw(isHourlyMeanTypeSql('l."type"'))}
     ),
     weighted AS (
       SELECT
@@ -113,34 +206,32 @@ export async function readAllTimeExtremes(
     ),
     per_source AS (
       SELECT
-        m."type",
-        m."source",
-        date_trunc('day', m."measured_at") AS day,
+        l."type",
+        l."source",
+        date_trunc('day', l."measured_at") AS day,
         COUNT(*)::int                      AS cnt,
-        SUM(m."value")::double precision   AS total,
-        MIN(m."value")::double precision   AS min_value,
-        MAX(m."value")::double precision   AS max_value
-      FROM measurements m
-      WHERE m."user_id" = ${userId}
-        AND m."deleted_at" IS NULL
-        AND m."type" IN (${typeList})
-        ${since}
-      GROUP BY m."type", m."source", 3
-    ),
-    canon AS (
-      SELECT DISTINCT ON (p."type", p.day)
-        p."type", p.cnt, p.total, p.min_value, p.max_value, w.wsum, w.wdays
-      FROM per_source p
-      LEFT JOIN weighted w
-        ON w."type" = p."type" AND w."source" = p."source" AND w.day = p.day
-      ORDER BY p."type", p.day, (${rank}), p."source"::text
+        SUM(l."value")::double precision   AS total,
+        MIN(l."value")::double precision   AS min_value,
+        MAX(l."value")::double precision   AS max_value
+      FROM live l
+      GROUP BY l."type", l."source", 3
     )
     SELECT
       c."type"::text                                  AS type,
       ${mean}::double precision                       AS mean,
       MIN(c.min_value)::double precision              AS min,
       MAX(c.max_value)::double precision              AS max
-    FROM canon c
+    FROM (
+      SELECT * FROM (
+        SELECT DISTINCT ON (p."type", p.day)
+          p."type", p.cnt, p.total, p.min_value, p.max_value, w.wsum, w.wdays
+        FROM per_source p
+        LEFT JOIN weighted w
+          ON w."type" = p."type" AND w."source" = p."source" AND w.day = p.day
+        ORDER BY p."type", p.day, (${rank("p")}), p."source"::text
+      ) raw
+      ${tierCanon}
+    ) c
     GROUP BY c."type"
   `;
   for (const r of rows) {

src/lib/rollups/__tests__/measurement-rollups.test.ts

@@ -188,6 +188,8 @@ describe("recomputeBucketsForMeasurement", () => {
         sd: 2.0,
         slope: -0.1,
         r2: 0.9,
+        reading_min_value: 80.5,
+        reading_max_value: 84.5,
       },
     ]);
     getGlobalBossMock.mockReturnValue({ send: bossSend });
@@ -210,14 +212,14 @@ describe("recomputeBucketsForMeasurement", () => {
     // instead of raising (or being dropped). Param layout per row:
     // [userId, type, granularity, bucketStart, source, count, mean,
     //  min, max, sum, sd, slope, r2, sumX, sumXy, sumXx, sumYy,
-    //  computedAt].
+    //  readingMin, readingMax, computedAt].
     expect(txExecuteRawUnsafe).toHaveBeenCalledTimes(1);
     const [sql, ...params] = txExecuteRawUnsafe.mock.calls[0];
     expect(sql).toContain(
       "ON CONFLICT (user_id, type, granularity, bucket_start, source)",
     );
     expect(sql).toContain("DO UPDATE SET");
-    expect(params).toHaveLength(18);
+    expect(params).toHaveLength(20);
     expect(params[0]).toBe("user-1");
     expect(params[1]).toBe("WEIGHT");
     expect(params[2]).toBe("DAY");
@@ -227,6 +229,10 @@ describe("recomputeBucketsForMeasurement", () => {
     // v1.4.39 — sumValue flows through. Cumulative read paths
     // (steps, flights, distance, daylight, active-energy) consume it.
     expect(params[9]).toBe(247.5);
+    // The reading extremes, beside the range-widened min / max.
+    expect(params[7]).toBe(80.0);
+    expect(params[17]).toBe(80.5);
+    expect(params[18]).toBe(84.5);
 
     // WEEK / MONTH / YEAR — three enqueues against the worker queue.
     expect(bossSend).toHaveBeenCalledTimes(3);
@@ -453,13 +459,13 @@ describe("persistRollupRows — large path (via recomputeUserRollups)", () => {
     expect(deleteMany.mock.calls[0][0].where.userId).toBe("user-1");
     expect(deleteMany.mock.calls[0][0].where.granularity).toBe("DAY");
 
-    // 3 chunks of 18 params per row, every one an ON CONFLICT DO UPDATE
+    // 3 chunks of 20 params per row, every one an ON CONFLICT DO UPDATE
     // upsert — tolerant of a mid-transaction hook insert AND of a
     // concurrent fold on the same partition (v1.28.33, issue #486).
     expect(txExecuteRawUnsafe).toHaveBeenCalledTimes(3);
-    expect(txExecuteRawUnsafe.mock.calls[0]).toHaveLength(1 + 500 * 18);
-    expect(txExecuteRawUnsafe.mock.calls[1]).toHaveLength(1 + 500 * 18);
-    expect(txExecuteRawUnsafe.mock.calls[2]).toHaveLength(1 + 200 * 18);
+    expect(txExecuteRawUnsafe.mock.calls[0]).toHaveLength(1 + 500 * 20);
+    expect(txExecuteRawUnsafe.mock.calls[1]).toHaveLength(1 + 500 * 20);
+    expect(txExecuteRawUnsafe.mock.calls[2]).toHaveLength(1 + 200 * 20);
     for (const call of txExecuteRawUnsafe.mock.calls) {
       expect(call[0]).toContain("ON CONFLICT");
       expect(call[0]).toContain("DO UPDATE SET");
@@ -575,7 +581,7 @@ describe("persistRollupRows — large path (via recomputeUserRollups)", () => {
     expect(transaction).toHaveBeenCalledTimes(1);
     expect(typeof transaction.mock.calls[0][0]).toBe("function");
     expect(txExecuteRawUnsafe).toHaveBeenCalledTimes(1);
-    expect(txExecuteRawUnsafe.mock.calls[0]).toHaveLength(1 + 2 * 18);
+    expect(txExecuteRawUnsafe.mock.calls[0]).toHaveLength(1 + 2 * 20);
     expect(result.rowsUpserted).toBe(2);
   });
 });

src/lib/rollups/measurement-rollups.ts

@@ -184,6 +184,10 @@ interface RollupRow {
   sum_xy: number | null;
   sum_xx: number | null;
   sum_yy: number | null;
+  // The extremes of the readings' own `value`, without the per-row range
+  // `min_value` / `max_value` widen to. See `MeasurementRollup.readingMinValue`.
+  reading_min_value: number | null;
+  reading_max_value: number | null;
 }
 
 /**
@@ -546,7 +550,9 @@ async function runRollupAggregate(
         SUM(${REBASED_X})::double precision                       AS sum_x,
         SUM(${REBASED_X} * m."value")::double precision           AS sum_xy,
         SUM(${REBASED_X} * ${REBASED_X})::double precision        AS sum_xx,
-        SUM(m."value" * m."value")::double precision              AS sum_yy
+        SUM(m."value" * m."value")::double precision              AS sum_yy,
+        MIN(m."value")::double precision                          AS reading_min_value,
+        MAX(m."value")::double precision                          AS reading_max_value
       FROM measurements m
       WHERE m."user_id" = $1
         AND m."type" IN (${typeList})
@@ -594,7 +600,9 @@ async function runRollupAggregate(
       SUM(${REBASED_X})::double precision                       AS sum_x,
       SUM(${REBASED_X} * m."value")::double precision           AS sum_xy,
       SUM(${REBASED_X} * ${REBASED_X})::double precision        AS sum_xx,
-      SUM(m."value" * m."value")::double precision              AS sum_yy
+      SUM(m."value" * m."value")::double precision              AS sum_yy,
+      MIN(m."value")::double precision                          AS reading_min_value,
+      MAX(m."value")::double precision                          AS reading_max_value
     FROM measurements m
     WHERE m."user_id" = $1
       AND m."measured_at" >= $2
@@ -921,20 +929,24 @@ function buildRollupUpsert(
       row.sum_xy ?? null,
       row.sum_xx ?? null,
       row.sum_yy ?? null,
+      row.reading_min_value ?? null,
+      row.reading_max_value ?? null,
       computedAt,
     );
     tuples.push(
       `($${base + 1}, $${base + 2}::"measurement_type", $${base + 3}::"measurement_rollup_granularity", ` +
         `$${base + 4}::timestamptz, $${base + 5}::"measurement_source", $${base + 6}::int, ` +
         `$${base + 7}, $${base + 8}, $${base + 9}, $${base + 10}, $${base + 11}, $${base + 12}, ` +
-        `$${base + 13}, $${base + 14}, $${base + 15}, $${base + 16}, $${base + 17}, $${base + 18}::timestamptz)`,
+        `$${base + 13}, $${base + 14}, $${base + 15}, $${base + 16}, $${base + 17}, $${base + 18}, ` +
+        `$${base + 19}, $${base + 20}::timestamptz)`,
     );
   }
   const sql = `
     INSERT INTO measurement_rollups
       (user_id, type, granularity, bucket_start, source, count, mean,
        min_value, max_value, sum_value, sd, slope, r2,
-       sum_x, sum_xy, sum_xx, sum_yy, computed_at)
+       sum_x, sum_xy, sum_xx, sum_yy, reading_min_value, reading_max_value,
+       computed_at)
     VALUES ${tuples.join(", ")}
     ON CONFLICT (user_id, type, granularity, bucket_start, source)
     DO UPDATE SET
@@ -950,6 +962,8 @@ function buildRollupUpsert(
       sum_xy      = EXCLUDED.sum_xy,
       sum_xx      = EXCLUDED.sum_xx,
       sum_yy      = EXCLUDED.sum_yy,
+      reading_min_value = EXCLUDED.reading_min_value,
+      reading_max_value = EXCLUDED.reading_max_value,
       computed_at = EXCLUDED.computed_at
   `;
   return { sql, params };
@@ -1201,7 +1215,9 @@ export async function enqueueBootTimeRollupBackfill(
     //      catches up — mirrors the v1.4.39 sum_value rollout, which the
     //      v1.4.41 note confirms converged via this same boot queue. Bounded
     //      to the fold window so days outside it (un-foldable by design)
-    //      never re-flag a user on every boot.
+    //      never re-flag a user on every boot. The same arm refills the
+    //      reading extremes of migration 0417 (`reading_min_value IS NULL`);
+    //      the extremes reader treats a NULL as uncovered until it lands.
     const users = await prisma.$queryRaw<Array<{ id: string }>>`
       SELECT DISTINCT id FROM (
         SELECT DISTINCT mt."user_id" AS id
@@ -1227,7 +1243,7 @@ export async function enqueueBootTimeRollupBackfill(
         FROM measurement_rollups r
         WHERE r."granularity" = 'DAY'
           AND r."bucket_start" >= ${foldWindowStart}
-          AND r."sum_xy" IS NULL
+          AND (r."sum_xy" IS NULL OR r."reading_min_value" IS NULL)
       ) discovered
     `;