perf: bound heavy-tenant memory on feature, consolidation, backup and auth paths

Cap and page the unbounded reads that let a multi-year tenant pull its
whole measurement history into memory on hot paths.

- extractFeatures: read newest-first with a 6000-row cap (mirrors the
  Coach snapshot) and reverse to ascending, so a 365-400 day window no
  longer materialises hundreds of thousands of PULSE / glucose rows per
  Coach turn and nightly briefing. Full-history extremes still come from
  readAllTimeExtremes, so the cap never relabels an extreme.
- Add a (userId, measuredAt) index on measurements (migration 0210) for
  the type-less range scans that extractFeatures and the consolidation
  passes drive; the existing (userId, type, measuredAt) cannot serve them
  as a prefix.
- runConsolidation: keyset-paginate the per-(user, type) source scan on
  (measuredAt, id) instead of one unbounded findMany, removing the giant
  single result-set allocation that drove the boot storm. Output is
  identical; covered by a paging unit test.
- Stagger the boot-time backfill enqueues with increasing startAfter
  offsets so the heavy full-history loads no longer all drain in parallel
  onto one tenant at startup. Non-boot callers keep their immediate send.
- Backups: replace load-all-then-stringify with paged narrow-select reads
  for the weekly and off-host writers, bounding transient heap. Off-host
  still includes soft-deleted rows. Covered by a serialize/restore
  round-trip test.
- Drop the per-request auth.bearer.success audit insert; the wide event
  already records auth_method and user_id, so the polling client no longer
  drives a write + pool checkout on every request. Failure-path audit
  stays.
- Coach conversation detail: cap the message include to the newest 200 so
  a long thread no longer decrypts every message on each open; response
  shape unchanged.
- Admin audit-log: floor the scan at the retention horizon so the listing
  and its count stop sweeping open-ended over the substring filters.
This commit is contained in:
Marc-André Bombeck committed 2026-06-28 03:07:15 +02:00
1 parent bc617adc2c
commit 84683f4ce9
24 files changed
+971 -78

No files matched your search

+22 -9
View File
@@ -3,6 +3,7 @@ import { apiHandler, requireAdmin } from "@/lib/api-handler";
import { apiSuccess, returnAllZodIssues } from "@/lib/api-response";
import { annotate } from "@/lib/logging/context";
import { redactSecrets } from "@/lib/logging/redact";
import { getAuditLogRetentionDays } from "@/lib/jobs/audit-log-cleanup";
import { NextRequest } from "next/server";
import { z } from "zod/v4";
@@ -39,7 +40,9 @@ export const dynamic = "force-dynamic";
* - actor — substring match against `userId` OR `user.username`
* - action — exact match (e.g. `auth.login.failed`)
* - target — substring match against the JSON-encoded `details` field
* - since — ISO-8601 lower bound on `createdAt`
* - since — ISO-8601 lower bound on `createdAt`. Absent, the scan
* floors at the audit-log retention horizon so an
* open-ended sweep can't seqscan the table unbounded.
* - until — ISO-8601 upper bound on `createdAt`
* - filter — legacy shortcut: `auth` restricts to `auth.*` actions
*
@@ -117,15 +120,25 @@ export const GET = apiHandler(async (request: NextRequest) => {
ands.push({ details: { contains: parsed.target } });
}
if (parsed.since || parsed.until) {
const range: { gte?: Date; lte?: Date } = {};
if (parsed.since) range.gte = new Date(parsed.since);
if (parsed.until) range.lte = new Date(parsed.until);
ands.push({ createdAt: range });
}
// Bound the lower edge of the scan. `actor`/`target` are substring
// matches (seqscans) and the exact `count` below runs a second pass over
// the same predicate, so an open-ended history sweep on this admin-only,
// low-frequency surface would scan the table twice with no floor. The
// cleanup job already prunes rows past the retention horizon, so aligning
// the default floor to that same horizon bounds both passes without ever
// hiding a row the deployment still keeps. An explicit `since` overrides.
const lowerBound = parsed.since
? new Date(parsed.since)
: new Date(Date.now() - getAuditLogRetentionDays() * 86_400_000);
const createdAt: { gte: Date; lte?: Date } = { gte: lowerBound };
if (parsed.until) createdAt.lte = new Date(parsed.until);
const where: Record<string, unknown> | undefined =
ands.length === 0 ? undefined : ands.length === 1 ? ands[0] : { AND: ands };
// Keep `createdAt` a top-level sibling of the optional filter clause so a
// single substring/exact filter stays flat (no `AND` wrapper); the
// collapse below only folds the independent filter `ands` entries.
const filterWhere: Record<string, unknown> =
ands.length === 0 ? {} : ands.length === 1 ? ands[0] : { AND: ands };
const where: Record<string, unknown> = { ...filterWhere, createdAt };
const [entries, total] = await Promise.all([
prisma.auditLog.findMany({