mirror of
https://github.com/MBombeck/HealthLog.git
synced 2026-11-07 04:12:04 +00:00
perf(measurements): cover soft-delete reads and cap the all-time live scan
Add a covering partial index over (user_id, type, measured_at DESC) WHERE deleted_at IS NULL (migration 0183). The hot reads all filter deleted_at and order measured_at DESC, but the base index is ASC and excludes deleted_at, so every type not served by the two existing partial indexes pays a heap re-check of deleted_at over its partition. The covering partial removes the re-check for the DISTINCT-ON-latest, list and series reads. Cap the cold-fallback all-time aggregate in summaries-slice at a 15-year window. It sits far beyond any real history, so it never changes a real all-time count/min/max/mean; it exists only so a coverage-miss cold read can't trigger an unbounded full-partition scan.
This commit is contained in:
1 parent
56d8cf19c5
commit
3fb1f3989c
4 files changed
+72
-8
No files matched your search
@@ -0,0 +1,40 @@
|
||||
-- v1.18.10 P-5 — covering partial index for soft-delete-filtered reads.
|
||||
--
|
||||
-- The hot reads on `measurements` all filter `deleted_at IS NULL` and order
|
||||
-- `measured_at DESC` per type — the DISTINCT-ON-latest pass, the management
|
||||
-- list, the series reads, and the per-type count. The base
|
||||
-- `(user_id, type, measured_at)` index is ASC and does NOT carry
|
||||
-- `deleted_at`, so those reads pay a heap re-check of `deleted_at` over the
|
||||
-- type partition for every candidate row.
|
||||
--
|
||||
-- Two partial indexes already cover the highest-volume types
|
||||
-- (`measurements_dense_intraday_retention_idx` for HRV/PULSE, migration 0108;
|
||||
-- `measurements_legacy_step_consolidation_idx` for ACTIVITY_STEPS, migration
|
||||
-- 0087). This index closes the gap for every OTHER type: a covering partial
|
||||
-- index keyed `(user_id, type, measured_at DESC)` over live rows only serves
|
||||
-- the latest / list / series reads without the heap re-check, and — being
|
||||
-- partial on `deleted_at IS NULL` — it carries no entries for tombstoned rows.
|
||||
--
|
||||
-- Additive + order-safe: a pure `CREATE INDEX IF NOT EXISTS`, no column add,
|
||||
-- no backfill, no rewrite of `measurements`.
|
||||
--
|
||||
-- Prisma can't express a partial-predicate index, so it lives here in raw
|
||||
-- migration SQL and is intentionally omitted from `schema.prisma` (the same
|
||||
-- treatment migrations 0087 + 0108 use; a comment on the model documents it).
|
||||
--
|
||||
-- Idempotent guard (`IF NOT EXISTS`) makes reruns safe. Forward-only.
|
||||
--
|
||||
-- Lock note (conscious trade-off): a plain, non-`CONCURRENTLY` build, so it
|
||||
-- runs inside Prisma's migration transaction and holds an ACCESS EXCLUSIVE
|
||||
-- lock on `measurements` for the build. On the largest multi-year HealthKit
|
||||
-- tenants the live row set is large, so deploy at a low-traffic window. A
|
||||
-- `CONCURRENTLY` build cannot run inside the migration transaction and is not
|
||||
-- used here to keep the migration atomic with the rest of the release.
|
||||
--
|
||||
-- Reversibility:
|
||||
-- DROP INDEX IF EXISTS "measurements_live_covering_idx";
|
||||
-- A roll-back loses only the scan acceleration; no data is touched.
|
||||
|
||||
CREATE INDEX IF NOT EXISTS "measurements_live_covering_idx"
|
||||
ON "measurements" ("user_id", "type", "measured_at" DESC)
|
||||
WHERE "deleted_at" IS NULL;
|
||||
@@ -1113,6 +1113,12 @@ model Measurement {
|
||||
// — see migration 0108. It accelerates the v1.10.0 dense intra-day
|
||||
// retention drain's discovery + per-user scan. Same Prisma-can't-
|
||||
// express-a-partial-predicate reason it is omitted here.
|
||||
// `measurements_live_covering_idx` is a covering partial index over
|
||||
// `(user_id, type, measured_at DESC) WHERE deleted_at IS NULL` — see
|
||||
// migration 0183 (v1.18.10 P-5). It serves the DISTINCT-ON-latest, list,
|
||||
// and series reads for every type NOT covered by the two partials above
|
||||
// and removes the `deleted_at` heap re-check. Same Prisma-can't-express-a-
|
||||
// partial-predicate reason it is omitted here.
|
||||
@@map("measurements")
|
||||
}
|
||||
|
||||
|
||||
@@ -592,15 +592,18 @@ describe("computeSummariesSlice", () => {
|
||||
// 90-day cap to `canonicalMeasurementsFrom(rank, "90 days")`. The
|
||||
// helper lives in `@/lib/analytics/source-rank-sql` (stubbed
|
||||
// above), so pin the contract at the call boundary: the slice
|
||||
// must ask for the 90-day window. The `allTime` aggregate
|
||||
// deliberately calls it WITHOUT an interval (no cap — all-time
|
||||
// count/min/max/mean must scan every row).
|
||||
// must ask for the 90-day window.
|
||||
expect(canonicalMeasurementsFrom).toHaveBeenCalledWith(
|
||||
expect.any(String),
|
||||
"90 days",
|
||||
);
|
||||
// v1.18.10 P-7 — the `allTime` aggregate is now capped at a generous
|
||||
// 15-year scan-DoS floor (far beyond any real history) so a
|
||||
// coverage-miss cold read can't trigger an unbounded full-partition
|
||||
// scan. It no longer calls the helper without an interval.
|
||||
expect(canonicalMeasurementsFrom).toHaveBeenCalledWith(
|
||||
expect.any(String),
|
||||
"15 years",
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -81,11 +81,22 @@ import { resolveUserTimezone } from "@/lib/tz/resolver";
|
||||
|
||||
const DAY_MS = 24 * 60 * 60 * 1000;
|
||||
|
||||
/**
|
||||
* v1.18.10 P-7 — upper bound on the cold-fallback `allTime` aggregate's
|
||||
* scan window. Sits far beyond any real account's history (the earliest prod
|
||||
* measurements are < 10 years old) so it never changes a real all-time
|
||||
* count / min / max / mean; it exists purely so a coverage-miss cold read
|
||||
* can't trigger an unbounded full-partition scan. Whitelisted by
|
||||
* `canonicalMeasurementsFrom`'s interval regex (`<N> years`).
|
||||
*/
|
||||
const ALL_TIME_LIVE_CAP_INTERVAL = "15 years" as const;
|
||||
|
||||
/**
|
||||
* v1.4.48 — all-time aggregate row used on the cold-mount fallback
|
||||
* path. Holds the linearly composable columns (`count / min / max /
|
||||
* mean`) that must reflect every row the user has ever logged for
|
||||
* that type, so this query intentionally has no `measured_at` cap.
|
||||
* that type, capped at `ALL_TIME_LIVE_CAP_INTERVAL` (v1.18.10 P-7) as a
|
||||
* scan-DoS floor.
|
||||
*/
|
||||
interface AllTimeAggregateRow {
|
||||
type: string;
|
||||
@@ -608,9 +619,13 @@ async function computeFromRollups(userId: string): Promise<SummariesSlice> {
|
||||
* v1.4.48 — split into two parallel queries:
|
||||
*
|
||||
* 1. `allTime` — the linearly composable columns (`count / min /
|
||||
* max / mean`) keep the full-partition scan because they must
|
||||
* reflect every row the user has ever logged. No `measured_at`
|
||||
* cap.
|
||||
* max / mean`). v1.18.10 P-7 caps this at `ALL_TIME_LIVE_CAP_INTERVAL`
|
||||
* (15 years) so an un-windowed cold-fallback read on a coverage-miss
|
||||
* tenant can never scan an unbounded partition. The cap is a scan-DoS
|
||||
* floor, not a semantic change: it sits far beyond any real account's
|
||||
* history (the earliest prod rows are < 10 years old) and the steady
|
||||
* state is rollup-backed anyway, so it only ever bounds the cold path.
|
||||
*
|
||||
*
|
||||
* 2. `windowed` — `avg7 / avg30` and the slope/r² tuples are all
|
||||
* already filtered to a 7/30/90 day window inside the SELECT.
|
||||
@@ -648,7 +663,7 @@ async function computeFromLiveAggregate(
|
||||
MIN(m."value")::double precision AS min_value,
|
||||
MAX(m."value")::double precision AS max_value,
|
||||
AVG(m."value")::double precision AS mean_value
|
||||
FROM ${canonicalMeasurementsFrom(rankUnqualified)}
|
||||
FROM ${canonicalMeasurementsFrom(rankUnqualified, ALL_TIME_LIVE_CAP_INTERVAL)}
|
||||
GROUP BY m."type"
|
||||
`,
|
||||
userId,
|
||||
|
||||
Reference in new issue
Block a user