Commit Graph
3 Commits
Author SHA1 Message Date
Marc-André Bombeck 4183202ed3 Delete the citation-enforcement chain that never ran
generateInsight() parsed against the strict aiInsightResponseSchema,
cross-checked that every recommendation's metricSource appeared in
citations[], and threw InsightSchemaError on a second failure. Its only
caller was runWithFallback(), which had no production caller — every
live surface runs runRawCompletionWithFallback. So none of it executed.

Wiring it into the live path was the alternative and it is not viable:
the live insightResultSchema has no citations[] field at all and treats
metricSource as optional, so the cross-check would have rejected
essentially every generation rather than tightening anything.

Removed: runWithFallback and its param/result types, generateInsight
and its private helpers, findUncitedRecommendations,
findRecommendationsMissingRationale, the never-thrown InsightSchemaError
and the isHardProviderFailure branch that excluded it, and
computeConfidence — a deterministic override that only ran inside the
dead wrapper.

Kept: buildRetryCorrectionMessage, which comprehensive-generate sends on
its own retry leg; the file it lived in is renamed to match what it now
holds. aiInsightResponseSchema stays as the shape that message
describes. Note the retry prompt asks for a stricter payload than the
live validator requires — that mismatch predates this and is left alone.

computeCitationCoverage is untouched and still live: it is a metric, not
a gate, and v1.30.24 wired it into the comprehensive post-parse step.

Chain-runner coverage the deleted tests carried (last-working cache,
empty-input guard) is ported onto runRawCompletionWithFallback, which
shares the machinery.
2026-07-19 05:50:57 +02:00
Marc-André BombeckandClaude Opus 4.7 3048dd6637 style(prettier): sweep formatting across the tree
Pre-existing format drift accumulated across v1.4.x; running
prettier brings the whole tree back to baseline so future diffs
read clean. No behaviour change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 11:05:40 +02:00
Marc-André BombeckandClaude Opus 4.7 0cb0373e59 feat(ai): deterministic confidence-score computation per recommendation
Each recommendation now carries an optional `confidence: 0..100`
integer in `aiRecommendationSchema`. New `src/lib/ai/confidence.ts`
exposes `computeConfidence({ n, recencyDays, deviationStdRatio })`,
the deterministic formula the wrapper will use to override whatever
number the model picks.

Score breakdown (sum, max 100):
  - n contribution: log10-saturating curve, capped at 40. n<3 hard-
    caps the whole score at 5*n (max 15) so a three-sample window
    never produces a confident-looking meter.
  - recency contribution: 30 at <2d, decays linearly to 0 at 30d.
  - signal contribution: 30 at |z|>=1.5, scaled linearly to 0 at
    z=0; null ratio (no baseline yet) returns a neutral 15.

Schema is `.optional()` so:
  - The wrapper can fill it post-validation regardless of what the
    LLM emitted — the model's calibrated probability is unreliable.
  - Legacy cached payloads from before B5d landed still parse.

Integer-only and bounded so the meter component renders without
clamping; a renegade provider that emits 101 or 67.5 fails parse and
triggers the wrapper's corrective retry.

15 fixtures cover the input matrix: n hard-cap (n=0..2), n=3 floor,
mid-range (n=14, n=20), saturation (n=100 plateau, n=10000 cap),
recency clamp (45d -> 0), null-ratio neutral, negative-z magnitude,
plus monotonicity-in-n + integer-in-bounds invariants.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 01:40:29 +02:00