perf(rollups): boot-backfill discovery widens to per-type missing coverage

Marc's account hit the slow-path live SQL aggregator on every read
after every deploy: `/api/analytics` cold-mounted in ~75 s,
`/api/mood/analytics` in ~13 s, `slim_summaries` annotated
`path:"live", row_count:347114`. Root cause traced to
`enqueueBootTimeRollupBackfill`'s discovery query:

  WHERE NOT EXISTS (
    SELECT 1 FROM measurement_rollups r WHERE r."user_id" = u."id"
  )

The clause only matched users with ZERO rollup rows. Any account
that had ever been folded (every long-running user) silently
stranded once a brand-new measurement type landed: the existing
WEIGHT bucket kept the user invisible to discovery, but the new
type's empty DAY partition kept `isFullyCovered(coverage)` at
`false` forever. Every analytics / dashboard read then fanned out
to the live aggregator across the full measurement table.

Discovery now joins `(user, distinct type)` against
`measurement_rollups` on the `(user, type, granularity=DAY)`
partition and surfaces users where any partition is unmatched. The
v1.4.35.1 zero-rollup case is still covered (the LEFT JOIN of an
empty rollup table emits a row for every type the user has logged).

Integration test pins the partial-coverage regression: a user with
WEIGHT rolled up + a brand-new PULSE row now lands on the queue.
This commit is contained in:
Marc-André Bombeck committed 2026-05-18 06:06:35 +02:00
1 parent e1528952bc
commit 93d639c746
2 files changed
+88 -15

No files matched your search

+29 -15
View File
@@ -703,9 +703,16 @@ async function ensureUserRollupsFreshImpl(
* persistent rollup tier without operator intervention.
*
* Idempotent across reboots:
* - the discovery query only matches accounts with no rollup rows at
* all, so once the fold completes the user drops off the list and
* subsequent boots skip them
* - the discovery query matches accounts that either have zero
* rollup coverage at all OR have at least one measurement type
* whose DAY-rollup partition is empty. Once every type the user
* has logged has at least one DAY bucket, the user drops off the
* list and subsequent boots skip them. The v1.4.35.1 shape (zero-
* rollups only) silently stranded users who logged a brand-new
* type after the initial fold — their `isFullyCovered` probe stays
* `false`, every read fans out to the live SQL aggregator, and
* `/api/analytics` cold-mounts in tens of seconds instead of
* 1.5-3 s. v1.4.38.5 widens discovery to per-type missing coverage.
* - pg-boss `singletonKey` coalesces duplicate sends within the
* queue, so a fast restart while a backfill is queued doesn't
* double up
@@ -724,19 +731,26 @@ export async function enqueueBootTimeRollupBackfill(): Promise<{
}
try {
// Users with at least one measurement but no rollup row yet. The
// `LIMIT 1` inside each `EXISTS` keeps the planner from materialising
// the full child relation just to answer existence — important on
// power-user accounts with 100k+ measurements.
// Users where at least one measurement type lacks DAY-rollup
// coverage. Anchored on the per-user `DISTINCT type` set so the
// planner uses the `(user_id, type, measured_at)` index path; the
// LEFT JOIN onto `measurement_rollups` uses the
// `(user_id, type, granularity, bucket_start)` composite primary
// key. `r."id" IS NULL` filters the unmatched partitions. A user
// surfaces here when ANY type is missing — including the brand-
// new-type case that v1.4.35.1's zero-rollup-only shape silently
// stranded.
const users = await prisma.$queryRaw<Array<{ id: string }>>`
SELECT u."id"
FROM users u
WHERE EXISTS (
SELECT 1 FROM measurements m WHERE m."user_id" = u."id" LIMIT 1
)
AND NOT EXISTS (
SELECT 1 FROM measurement_rollups r WHERE r."user_id" = u."id" LIMIT 1
)
SELECT DISTINCT mt."user_id" AS id
FROM (
SELECT DISTINCT m."user_id", m."type"
FROM measurements m
) mt
LEFT JOIN measurement_rollups r
ON r."user_id" = mt."user_id"
AND r."type" = mt."type"
AND r."granularity" = 'DAY'
WHERE r."id" IS NULL
`;
if (users.length === 0) {
@@ -638,6 +638,65 @@ describe("measurement rollups — integration", () => {
vi.mocked(getGlobalBoss).mockReturnValue(null as never);
});
// ─── v1.4.38.5 regression — partial coverage discovery ──────
//
// The v1.4.35.1 shape only matched users with ZERO rollup rows
// anywhere. That silently stranded users who logged a brand-new
// type after the initial fold: their existing WEIGHT bucket made
// them invisible to discovery, but the brand-new PULSE type still
// had no DAY bucket → `isFullyCovered` stayed `false` → every read
// fanned out to the live aggregator (10s of seconds on power-user
// accounts). v1.4.38.5 widens discovery to per-type-missing.
it("v1.4.38.5 — enqueues a user with rollups for one type but a brand-new uncovered type", async () => {
const prisma = getPrismaClient();
const user = await prisma.user.create({
data: {
username: "boot-backfill-partial",
email: "boot-backfill-partial@example.test",
role: "USER",
},
});
await prisma.measurement.create({
data: {
userId: user.id,
type: "WEIGHT",
value: 80,
unit: "kg",
source: "MANUAL",
measuredAt: new Date("2026-04-01T08:00:00.000Z"),
},
});
// Fold WEIGHT — this is the state a long-running account is in
// after the initial v1.4.35.1 boot-time backfill landed.
await recomputeUserRollups(user.id, { granularities: ["DAY"] });
// Now log a brand-new type. The v1.4.35.1 discovery saw "user
// has at least one rollup row" and skipped them; v1.4.38.5
// discovery checks per-type coverage and surfaces them.
await prisma.measurement.create({
data: {
userId: user.id,
type: "PULSE",
value: 65,
unit: "bpm",
source: "MANUAL",
measuredAt: new Date("2026-04-02T08:00:00.000Z"),
},
});
bossSend.mockResolvedValue("job-id");
vi.mocked(getGlobalBoss).mockReturnValue({ send: bossSend } as never);
const result = await enqueueBootTimeRollupBackfill();
expect(result.enqueued).toBe(1);
expect(bossSend).toHaveBeenCalledTimes(1);
expect(bossSend.mock.calls[0][1]).toMatchObject({ userId: user.id });
vi.mocked(getGlobalBoss).mockReturnValue(null as never);
});
// ─── v1.4.36 rollup-fresh skip-live contract ──────────────────
//
// The v1.4.35 read-swap kept the heavy live aggregate running in