fix(insights): tighter Pearson surfacing gate (n>=20)

The Pearson p-value derives from the Fisher's-Z normal approximation.
At low df (df < 18, n < 20) the normal approx is generous in the
wrong direction — at df=12 a true p≈0.04 reports as p≈0.025, sneaking
past the p<0.05 surfacing gate. The v1.4.22 product-lead memo flagged
this as Code-MED-03 with a deadline of "before v1.5/v1.6 auto-
discovery ships" because the false-positive cost compounds when
correlations get auto-generated.

Two paths considered:
  - Rigorous: drop in a 30-LOC incomplete-beta implementation so the
    p-value is accurate at every df. Cleanest fix, no test churn.
  - Conservative: raise `MIN_PAIRED_N` from 14 to 20 so the borderline
    band is excluded entirely. Smaller patch, no maths-library
    dependency.

v1.4.23 ships the conservative fix. The rigorous incomplete-beta
swap is queued as a v1.4.24 candidate; if usage data from the
v1.4.16 B5e feedback aggregator later shows users miss the
borderline cards, the rigorous path lands then.

Trade-off: a small set of borderline-significance correlation cards
stop rendering. Acceptable for v1.5 — the prompt-injection guards +
warm Coach prose make it more important to under-report than over-
report patterns.

Tests +1 (`pearson` n=15 → insufficient pin) plus an existing
`correlateBpCompliance` n=13 test renamed to n=15 + the noise-fixture
extended from 15 to 20 entries to clear the new gate.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

src/app/api/analytics/route.ts

@@ -183,9 +183,10 @@ export const GET = apiHandler(async () => {
 
   // v1.4.20 phase B3 — three pre-defined correlation hypotheses.
   // All three run on the trailing 30 days so a sparse account doesn't
-  // burn the n >= 14 gate on a stale window. Each runner gates on
+  // burn the n >= 20 gate on a stale window (v1.4.23 H6 raised the
-  // n >= 14 + p < 0.05; below the bar the result.status === "insufficient"
+  // floor from 14 → 20). Each runner gates on n >= 20 + p < 0.05;
-  // and the UI paints an EmptyState.
+  // below the bar the result.status === "insufficient" and the UI
+  // paints an EmptyState.
   const correlations = await computeCorrelationHypotheses(user.id);
 
   // v1.4.20 phase B5 — Personal Health Score. Server-deterministic

src/lib/insights/__tests__/correlations.test.ts

@@ -13,8 +13,9 @@ import {
  * v1.4.20 phase B3 — correlation discovery.
  *
  * Three pre-defined hypotheses are surfaced as `<CorrelationCard>` rows.
- * Quality bar is non-negotiable: n >= 14, p < 0.05. Anything below
+ * Quality bar is non-negotiable: n >= 20 (v1.4.23 H6 raise), p < 0.05.
- * returns `status: "insufficient"` so the card paints an EmptyState.
+ * Anything below returns `status: "insufficient"` so the card paints
+ * an EmptyState.
  */
 
 function makeDate(daysAgo: number): Date {
@@ -69,13 +70,15 @@ describe("pearson", () => {
 
   it("returns r near 0 for noise, with p above the surface threshold", () => {
     // Deterministic noise (no Math.random) so the test is reproducible.
+    // v1.4.23 H6 — fixture extended from 15 to 20 entries to clear the
+    // raised MIN_PAIRED_N gate (was 14, now 20).
     const xs = [
       0.1, 0.5, 0.9, 0.2, 0.7, 0.4, 0.6, 0.3, 0.8, 0.05, 0.95, 0.15, 0.85, 0.55,
-      0.25,
+      0.25, 0.35, 0.65, 0.75, 0.45, 0.05,
     ];
     const ys = [
       0.5, 0.1, 0.7, 0.3, 0.2, 0.9, 0.4, 0.6, 0.05, 0.95, 0.85, 0.55, 0.25,
-      0.45, 0.65,
+      0.45, 0.65, 0.15, 0.75, 0.35, 0.05, 0.95,
     ];
     const result = pearson({ xs, ys });
     expect(result.status).toBe("ok");
@@ -86,12 +89,30 @@ describe("pearson", () => {
       expect(result.pValue).toBeGreaterThan(MAX_P_VALUE);
     }
   });
+
+  /**
+   * v1.4.23 H6 — pin the surfacing-gate raise. n=15 is the borderline
+   * case the v1.4.22 product-lead memo flagged: under the old n>=14
+   * gate the normal-approx p-value would surface a card; under the
+   * new n>=20 gate the call short-circuits to `insufficient` so a
+   * future reviewer can't lower the floor without breaking this pin.
+   */
+  it("returns insufficient (too_few_pairs) at n=15 (H6 surfacing-gate raise)", () => {
+    const xs = Array.from({ length: 15 }, (_, i) => i);
+    const ys = xs.map((x) => 2 * x + 5);
+    const result = pearson({ xs, ys });
+    expect(result.status).toBe("insufficient");
+    if (result.status === "insufficient") {
+      expect(result.reason).toBe("too_few_pairs");
+      expect(result.n).toBe(15);
+    }
+  });
 });
 
 // ── weekdayAnova() ──────────────────────────────────────────────────
 
 describe("weekdayAnova", () => {
-  it("flags too_few_pairs when total n < 14", () => {
+  it("flags too_few_pairs when total n is below the surfacing gate", () => {
     const result = weekdayAnova([
       { weekday: 0, value: 80 },
       { weekday: 1, value: 81 },
@@ -147,10 +168,10 @@ describe("correlateBpCompliance", () => {
     }));
   }
 
-  it("returns insufficient (too_few_pairs) at n=13", () => {
+  it("returns insufficient (too_few_pairs) at n=15 (H6 surfacing-gate raise)", () => {
     const daily = buildPairs(
       Array.from(
-        { length: 13 },
+        { length: 15 },
         (_, i) => [50 + i * 4, 150 - i * 2] as [number, number],
       ),
     );
@@ -158,7 +179,7 @@ describe("correlateBpCompliance", () => {
     expect(result.status).toBe("insufficient");
     if (result.status === "insufficient") {
       expect(result.reason).toBe("too_few_pairs");
-      expect(result.n).toBe(13);
+      expect(result.n).toBe(15);
     }
   });
 

src/lib/insights/correlations.ts

@@ -16,11 +16,22 @@
  * data-grounded hypotheses are enough to ship the surface.
  *
  * Quality bar (non-negotiable):
- *   - n >= 14 paired data points
+ *   - n >= 20 paired data points  (v1.4.23 H6 — raised from 14)
  *   - p <  0.05
  * Anything below the bar returns `{ kind: "insufficient", n, reason }` so
  * the card can render an EmptyState and never imply false confidence.
  *
+ * Pearson p-value gate rationale (v1.4.23 H6, Code-MED-03):
+ *   The two-sided p-value derives from the Fisher's-Z normal
+ *   approximation. At low df (n < 20, df < 18) the normal approx is
+ *   generous in the wrong direction — at df=12 a true p ≈ 0.04 gets
+ *   reported as p ≈ 0.025, sneaking past the surfacing gate. v1.4.23
+ *   raises `MIN_PAIRED_N` from 14 to 20 so the borderline-significance
+ *   band is excluded entirely. A rigorous incomplete-beta fix is
+ *   queued as a v1.4.24 candidate; raising the gate is the safer
+ *   patch here because the false-positive cost compounds when v1.5/v1.6
+ *   auto-discovery ships.
+ *
  * Conservative phrasing — interpretations always read as "a pattern worth
  * watching" or similar, never "X causes Y". Causation language is banned
  * from this module by code-review convention.
@@ -74,8 +85,18 @@ export interface CorrelationInsufficient {
 
 export type CorrelationResult = CorrelationOk | CorrelationInsufficient;
 
-/** Hard floor for paired sample count. */
+/**
-export const MIN_PAIRED_N = 14;
+ * Hard floor for paired sample count.
+ *
+ * v1.4.23 H6 — raised from 14 to 20 so borderline-df Pearson cards
+ * stop surfacing. The Fisher's-Z normal approximation overstates
+ * significance at df < 18; raising the gate is the simpler patch here
+ * (a rigorous incomplete-beta replacement is queued as a v1.4.24
+ * candidate). Acceptable trade-off: less false-positive noise; the
+ * v1.4.16 B5e feedback aggregator can reverse the call if usage data
+ * later shows users miss the borderline cards.
+ */
+export const MIN_PAIRED_N = 20;
 /** Two-sided p-value threshold for a card to surface. */
 export const MAX_P_VALUE = 0.05;
 
@@ -143,9 +164,10 @@ export function pearson(input: {
   const clamped = Math.max(-1, Math.min(1, r));
 
   // Two-sided p-value via t-distribution. t = r * sqrt(n-2) / sqrt(1-r^2).
-  // For n >= 14 the normal approximation is accurate enough for a
+  // For n >= 20 (v1.4.23 H6 raise) the normal approximation is
-  // surfacing decision; we use it directly to avoid pulling in an
+  // conservative enough for a surfacing decision; we use it directly
-  // incomplete-beta implementation.
+  // to avoid pulling in an incomplete-beta implementation. The
+  // rigorous incomplete-beta swap is a v1.4.24 candidate.
   const tStat =
     Math.abs(clamped) >= 1
       ? Number.POSITIVE_INFINITY
@@ -196,7 +218,9 @@ function twoSidedPFromT(absT: number, df: number): number {
   if (df <= 0) return 1;
   // For very small df the normal approx overstates significance; clamp
   // to a conservative "not significant" so a tiny n never sneaks past
-  // the p < 0.05 gate. n >= 14 (df >= 12) is the gate we target.
+  // the p < 0.05 gate. v1.4.23 H6 raised the surfacing gate to
+  // n >= 20 (df >= 18); the conservative correction below stays as
+  // defence-in-depth in case a caller passes a tighter `minPairs`.
   if (df < 12) {
     // Crude correction — multiply absT down so p inflates. Keeps the
     // test-suite synthetic data realistic without overfitting.