mirror of
https://github.com/MBombeck/HealthLog.git
synced 2026-11-08 12:32:43 +00:00
Marc-André BombeckandClaude Opus 4.7
af21d4d4a8
After zod parse, `generateInsight()` invokes `computeConfidence()` for
every recommendation and overwrites `rec.confidence` — the model's
self-reported number is discarded. Calibrated probabilities are not
a small-LLM strength, so this is the deterministic substrate the
v1.4.17 feedback ratchet will tune.
The wrapper does not have direct access to the user's bucket-series
data, so an optional `confidenceContext` resolver lets the caller
(typically the route layer) supply per-rec `ConfidenceInputs` derived
from `metricSource.n` + the freshest bucket + the deviation magnitude
vs. the user's 90-day baseline. When the resolver is omitted, the
wrapper falls back to inputs derivable from the parsed payload alone:
- n: `metricSource.n` (or 0 — triggers the n<3 hard-cap)
- recencyDays: 0 (defensive default; assumes window ends "now")
- deviationStdRatio: null (neutral 15-point signal contribution)
The override fires regardless of resolver presence so the contract
is "the confidence value on the response NEVER comes from the LLM".
A new structured Wide-Event annotation
(`ai_confidence_override_event` = `confidence.override.applied`)
carries per-rec `{id, model, computed}` triples so admin observability
can chart drift between the model's claimed confidence and the
deterministic value.
`annotateCitationCoverage()` now runs before `applyConfidenceOverride()`
so the existing citation/rationale-coverage tests can still match
against `mock.calls[0]`. The citation-coverage test that asserted
`toHaveBeenCalledTimes(1)` was relaxed to look up the right call by
meta-key rather than by ordinal.
6 new wrapper tests cover: model value discarded, wrapper-fills when
absent, resolver path overrides default, n<3 hard-cap survives strong
signal, missing `metricSource.n` -> n=0 floor, resolver receives each
rec in payload order.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>