fix(insights): tighter Pearson surfacing gate (n>=20)

The Pearson p-value derives from the Fisher's-Z normal approximation.
At low df (df < 18, n < 20) the normal approx is generous in the
wrong direction — at df=12 a true p≈0.04 reports as p≈0.025, sneaking
past the p<0.05 surfacing gate. The v1.4.22 product-lead memo flagged
this as Code-MED-03 with a deadline of "before v1.5/v1.6 auto-
discovery ships" because the false-positive cost compounds when
correlations get auto-generated.

Two paths considered:
  - Rigorous: drop in a 30-LOC incomplete-beta implementation so the
    p-value is accurate at every df. Cleanest fix, no test churn.
  - Conservative: raise `MIN_PAIRED_N` from 14 to 20 so the borderline
    band is excluded entirely. Smaller patch, no maths-library
    dependency.

v1.4.23 ships the conservative fix. The rigorous incomplete-beta
swap is queued as a v1.4.24 candidate; if usage data from the
v1.4.16 B5e feedback aggregator later shows users miss the
borderline cards, the rigorous path lands then.

Trade-off: a small set of borderline-significance correlation cards
stop rendering. Acceptable for v1.5 — the prompt-injection guards +
warm Coach prose make it more important to under-report than over-
report patterns.

Tests +1 (`pearson` n=15 → insufficient pin) plus an existing
`correlateBpCompliance` n=13 test renamed to n=15 + the noise-fixture
extended from 15 to 20 entries to clear the new gate.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Marc-André BombeckandClaude Opus 4.7 committed 2026-05-11 01:38:54 +02:00
1 parent 0eda1de16f
commit 1faee95c84
3 files changed
+64 -18

No files matched your search

+4 -3
View File
@@ -183,9 +183,10 @@ export const GET = apiHandler(async () => {
// v1.4.20 phase B3 — three pre-defined correlation hypotheses.
// All three run on the trailing 30 days so a sparse account doesn't
// burn the n >= 14 gate on a stale window. Each runner gates on
// n >= 14 + p < 0.05; below the bar the result.status === "insufficient"
// and the UI paints an EmptyState.
// burn the n >= 20 gate on a stale window (v1.4.23 H6 raised the
// floor from 14 → 20). Each runner gates on n >= 20 + p < 0.05;
// below the bar the result.status === "insufficient" and the UI
// paints an EmptyState.
const correlations = await computeCorrelationHypotheses(user.id);
// v1.4.20 phase B5 — Personal Health Score. Server-deterministic