fix(rollups): annotate when ensureUserRollupsFresh swallows an error

The read-path warm-up's bare `catch {}` hid populator regressions —
a deadlock or pool exhaustion inside `recomputeUserRollups` would
return `{ recomputed: false }` silently and the read fall-through
to the live aggregator masked the failure until performance bled
through.

The catch now annotates `rollup_refresh_failed` plus the error
message on the wide-event pipeline (no-op when there's no request
context) and emits a `console.error` with the same shape so the
worker log path surfaces the failure regardless of caller surface.
Return shape unchanged — the read path still must not fail.

Unit test simulates a deadlock inside the inner recompute and asserts
both the annotate meta and the console fallback fire while the outer
call still returns `{ recomputed: false }`.
This commit is contained in:
Marc-André Bombeck committed 2026-05-17 13:05:11 +02:00
1 parent d7ad2c0be0
commit 234fb7235e
2 files changed
+58 -1

No files matched your search

@@ -52,6 +52,15 @@ vi.mock("@/lib/jobs/boss-instance", () => ({
getGlobalBoss: () => mocks.getGlobalBossMock(),
}));
// v1.4.36 QA H3 — `ensureUserRollupsFresh` now annotates + console-
// errors on populator failures so silent regressions show up in ops.
// Mock the annotate boundary so the test can assert the meta payload.
vi.mock("@/lib/logging/context", () => ({
annotate: vi.fn(),
}));
import { annotate } from "@/lib/logging/context";
const {
queryRaw,
queryRawUnsafe,
@@ -281,6 +290,40 @@ describe("ensureUserRollupsFresh", () => {
const result = await ensureUserRollupsFresh("user-1");
expect(result.recomputed).toBe(false);
});
it("annotates the failure when the inner recompute throws (H3)", async () => {
// Stale watermark — rollup older than the newest measurement so
// the inner recompute branch fires.
const rollupAt = new Date("2026-05-10T10:00:00.000Z");
const measurementAt = new Date("2026-05-10T12:00:00.000Z");
findFirst.mockResolvedValueOnce({ computedAt: rollupAt });
findFirstMeasurement.mockResolvedValueOnce({
updatedAt: measurementAt,
measuredAt: measurementAt,
});
// The recompute aggregate query throws — simulates a populator
// regression (pool exhausted, deadlock, etc).
queryRawUnsafe.mockRejectedValueOnce(new Error("deadlock detected"));
const consoleSpy = vi.spyOn(console, "error").mockImplementation(() => {});
const result = await ensureUserRollupsFresh("user-1");
// The read path still gets a clean `{ recomputed: false }`.
expect(result.recomputed).toBe(false);
// The annotate event fires with the documented shape so ops can
// spot the silent populator regression in the wide-event pipeline.
expect(annotate).toHaveBeenCalledWith({
meta: {
rollup_refresh_failed: true,
rollup_refresh_error: "deadlock detected",
},
});
// And the console.error fallback fires so the worker log path
// surfaces the failure even when there's no request context.
expect(consoleSpy).toHaveBeenCalled();
consoleSpy.mockRestore();
});
});
describe("enqueueBootTimeRollupBackfill", () => {
+15 -1
View File
@@ -30,6 +30,7 @@
*/
import { prisma } from "@/lib/db";
import { getGlobalBoss } from "@/lib/jobs/boss-instance";
import { annotate } from "@/lib/logging/context";
import type {
MeasurementType,
RollupGranularity,
@@ -616,8 +617,21 @@ export async function ensureUserRollupsFresh(
to: new Date(),
});
return { recomputed: true };
} catch {
} catch (err) {
// Best-effort — never block the read path on a populator failure.
// v1.4.36 QA H3 — surface the failure so ops can spot a silent
// populator regression. `annotate` is a no-op when there's no
// request context (worker calls, tests); `console.error` carries
// the same shape through the worker log pipeline so the regression
// is visible regardless of caller surface.
const message = err instanceof Error ? err.message : String(err);
annotate({
meta: {
rollup_refresh_failed: true,
rollup_refresh_error: message,
},
});
console.error("[rollups] ensureUserRollupsFresh failed:", message);
return { recomputed: false };
}
}