mirror of
https://github.com/MBombeck/HealthLog.git
synced 2026-11-08 20:42:43 +00:00
feat(coach-prompt): GROUND RULE 13 (insights) + 8 (coach) — no internal metric enum names in prose
Closes the loop on the "Metric Pressure_Sys" leak Marc reported on
the Einschätzungen surface 2026-05-14. The stripChartTokens regex
extension cleans up render-time output, but the model was free to
keep emitting raw enum identifiers turn after turn. This commit
teaches the generator to stop emitting them in the first place.
Insight-generator (DE + EN) gets GROUND RULE 13:
- Lists every banned identifier verbatim — Pressure_Sys,
BLOOD_PRESSURE_SYS, PULSE_BPM, MOOD_SCORE,
MEDICATION_COMPLIANCE_PCT, plus the v1.4.23 Apple Health surface
(HEART_RATE_VARIABILITY, RESTING_HEART_RATE,
ACTIVE_ENERGY_BURNED, FLIGHTS_CLIMBED,
WALKING_RUNNING_DISTANCE, VO2_MAX, BODY_TEMPERATURE,
SLEEP_DURATION).
- Enumerates the exact user-facing fields the ban applies to —
summary, recommendations[].text, findings, dailyBriefing,
trendAnnotations.*, weeklyReport.*, storyboardAnnotations.
- Carves out the contract identifiers the parser reads
(metricSource.type, sourceMetric, trendAnnotations keys) so the
JSON contract stays intact — the ban applies ONLY to prose.
- Documents the natural-language labels to use instead ("your
systolic", "your weight", "your medication adherence", …).
Coach system-prompt (DE + EN) gets the matching GROUND RULE 8 with
the same banned list, the same prose-only carve-out, and an extra
clarification that evidence-block labels stay human-readable
("avg30 systolic", NOT "BLOOD_PRESSURE_SYS_AVG_30") while the
window slot keeps its documented contract vocabulary.
PROMPT_VERSION 4.23.0 → 4.24.0. The rule is genuinely new (neither
prompt previously mentioned the enum identifiers in any form), so
the bump reflects the additive behavioural change. Existing pin
assertions in coach-prompt-v423.test.ts + coach/system-prompt.test.ts
loosen from `/^4\.23\.\d+$/` to `4.{>=23}.x` so future additive
ratchets don't force a hostile test rewrite each time.
New unit tests pin GROUND RULE 13 + 8 by the banned-identifier
strings + the contract-carve-out wording so a future copy-paste
regression can't silently drop either rule.
This commit is contained in:
1 parent
35f068ac26
commit
feeff38ba2
4 files changed
+157
-7
No files matched your search
@@ -19,8 +19,14 @@ import {
|
||||
* so the model emits the additive enum values directly.
|
||||
*/
|
||||
describe("PROMPT_VERSION (v1.4.23 Apple Health ratchet)", () => {
|
||||
it("matches the 4.23.x train", () => {
|
||||
expect(PROMPT_VERSION).toMatch(/^4\.23\.\d+$/);
|
||||
it("matches the 4.23+ train", () => {
|
||||
// v1.4.25 W5b ratchet to 4.24.0 — GROUND RULE 13 bans internal
|
||||
// metric enum identifiers from prose. The pin loosens to accept
|
||||
// any 4.{>=23}.x revision so future additive ratchets don't
|
||||
// force a hostile test rewrite each time the rules grow.
|
||||
const [, minor] = PROMPT_VERSION.split(".").map(Number);
|
||||
expect(PROMPT_VERSION).toMatch(/^4\.\d+\.\d+$/);
|
||||
expect(minor).toBeGreaterThanOrEqual(23);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -77,3 +83,49 @@ describe("getStrictInsightsSystemPrompt — DE", () => {
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
/**
|
||||
* v1.4.25 W5b — pin GROUND RULE 13 (no internal metric identifiers
|
||||
* in prose). Marc saw the "Metric Pressure_Sys" leak on the
|
||||
* Einschätzungen surface 2026-05-14; the stripper covers the leak at
|
||||
* render time, GROUND RULE 13 closes the loop at generation time so
|
||||
* the cleaned-up prose was the only version emitted in the first
|
||||
* place.
|
||||
*/
|
||||
describe("getStrictInsightsSystemPrompt — GROUND RULE 13 (no enum names in prose)", () => {
|
||||
it("EN prompt carries the rule and enumerates the banned identifiers", () => {
|
||||
const prompt = getStrictInsightsSystemPrompt("en");
|
||||
expect(prompt).toMatch(/13\. v1\.4\.25/);
|
||||
expect(prompt).toMatch(/Internal metric identifiers/);
|
||||
// The banned-list is enumerated verbatim so the model has no
|
||||
// ambiguity about which strings are off-limits in prose.
|
||||
expect(prompt).toMatch(/"Pressure_Sys"/);
|
||||
expect(prompt).toMatch(/"BLOOD_PRESSURE_SYS"/);
|
||||
expect(prompt).toMatch(/"PULSE_BPM"/);
|
||||
expect(prompt).toMatch(/"MOOD_SCORE"/);
|
||||
expect(prompt).toMatch(/"MEDICATION_COMPLIANCE_PCT"/);
|
||||
// Apple Health additions also banned in prose (the surface
|
||||
// ships in v1.5 but the model can mention them on iOS-connected
|
||||
// snapshots today).
|
||||
expect(prompt).toMatch(/"HEART_RATE_VARIABILITY"/);
|
||||
expect(prompt).toMatch(/"SLEEP_DURATION"/);
|
||||
// The carve-out for contract identifiers must remain in place
|
||||
// so the parser keeps working.
|
||||
expect(prompt).toMatch(/metricSource\.type/);
|
||||
expect(prompt).toMatch(/applies ONLY to prose/);
|
||||
});
|
||||
|
||||
it("DE prompt carries the rule with the same banned list", () => {
|
||||
const prompt = getStrictInsightsSystemPrompt("de");
|
||||
expect(prompt).toMatch(/13\. v1\.4\.25/);
|
||||
expect(prompt).toMatch(/Interne Metrik-Identifier/);
|
||||
expect(prompt).toMatch(/"Pressure_Sys"/);
|
||||
expect(prompt).toMatch(/"BLOOD_PRESSURE_SYS"/);
|
||||
expect(prompt).toMatch(/"PULSE_BPM"/);
|
||||
expect(prompt).toMatch(/"MOOD_SCORE"/);
|
||||
expect(prompt).toMatch(/"MEDICATION_COMPLIANCE_PCT"/);
|
||||
expect(prompt).toMatch(/"HEART_RATE_VARIABILITY"/);
|
||||
expect(prompt).toMatch(/"SLEEP_DURATION"/);
|
||||
expect(prompt).toMatch(/AUSSCHLIEßLICH für Fließtext/);
|
||||
});
|
||||
});
|
||||
@@ -16,11 +16,15 @@ import { getCoachSystemPrompt } from "../system-prompt";
|
||||
* the prompt body (per the W1b research output).
|
||||
*/
|
||||
describe("PROMPT_VERSION (v1.4.23 Coach + Apple Health extension)", () => {
|
||||
it("matches the 4.23.x train", () => {
|
||||
it("matches the 4.23+ train", () => {
|
||||
// v1.4.23 ratcheted the prompt to add GROUND RULE 12 (Apple Health
|
||||
// optional categories). Future patches stay on 4.23.x until a
|
||||
// behavioural change ships.
|
||||
expect(PROMPT_VERSION).toMatch(/^4\.23\.\d+$/);
|
||||
// optional categories). v1.4.25 W5b ratcheted again to add GROUND
|
||||
// RULE 13 (no internal metric identifiers in prose). The pin
|
||||
// loosens to accept any 4.{>=23}.x revision so future additive
|
||||
// ratchets don't force a hostile test rewrite each time.
|
||||
const [, minor] = PROMPT_VERSION.split(".").map(Number);
|
||||
expect(PROMPT_VERSION).toMatch(/^4\.\d+\.\d+$/);
|
||||
expect(minor).toBeGreaterThanOrEqual(23);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -50,6 +54,10 @@ describe("getCoachSystemPrompt — EN", () => {
|
||||
expect(prompt).toMatch(/5\. Motivational-interviewing micro-moves\./);
|
||||
expect(prompt).toMatch(/6\. Redirect off-topic input gracefully\./);
|
||||
expect(prompt).toMatch(/7\. Ground every number in the SNAPSHOT\./);
|
||||
// v1.4.25 W5b — GROUND RULE 8 added on the Coach side: bans the
|
||||
// raw enum identifiers from the user-facing reply text and from
|
||||
// the evidence-block labels.
|
||||
expect(prompt).toMatch(/8\. Internal metric identifiers/);
|
||||
});
|
||||
|
||||
it("preserves the v1.4.21 DAY-LEVEL READINGS section verbatim", () => {
|
||||
@@ -115,6 +123,8 @@ describe("getCoachSystemPrompt — DE", () => {
|
||||
);
|
||||
expect(prompt).toMatch(/6\. Off-topic-Eingaben elegant umlenken\./);
|
||||
expect(prompt).toMatch(/7\. Verankere jede Zahl im SNAPSHOT\./);
|
||||
// v1.4.25 W5b — GROUND RULE 8 on the German Coach side.
|
||||
expect(prompt).toMatch(/8\. Interne Metrik-Identifier/);
|
||||
});
|
||||
|
||||
it("preserves the TAGES-LEVEL-MESSWERTE section verbatim", () => {
|
||||
|
||||
@@ -76,6 +76,26 @@ GROUND RULES
|
||||
user prompt carries. Do not extrapolate. Do not reference "people
|
||||
like you". Do not compute risk scores.
|
||||
|
||||
8. Internal metric identifiers stay OUT of your prose (v1.4.25).
|
||||
Never write database / enum-style names like "Pressure_Sys",
|
||||
"BLOOD_PRESSURE_SYS", "PULSE_BPM", "MOOD_SCORE",
|
||||
"MEDICATION_COMPLIANCE_PCT", "HEART_RATE_VARIABILITY",
|
||||
"RESTING_HEART_RATE", "ACTIVE_ENERGY_BURNED", "FLIGHTS_CLIMBED",
|
||||
"WALKING_RUNNING_DISTANCE", "VO2_MAX", "BODY_TEMPERATURE", or
|
||||
"SLEEP_DURATION" in your reply text — neither in the prose nor in
|
||||
the evidence-block labels. Reference each metric with the
|
||||
natural-language label the user sees in the app — "your
|
||||
systolic", "your weight", "your pulse", "your mood", "your
|
||||
medication adherence", "your resting heart rate", "your sleep
|
||||
duration", "your steps". Likewise never write the canonical
|
||||
"metric:<TYPE>" chart-token string anywhere in your reply; the
|
||||
surrounding /insights surface owns inline-chart wiring, not the
|
||||
Coach. Evidence-block labels stay short and human ("avg30
|
||||
systolic", not "BLOOD_PRESSURE_SYS_AVG_30") and the "window"
|
||||
slot keeps its documented vocabulary (last7days / last30days /
|
||||
last90days / allTime) — the ban applies to the metric label
|
||||
itself, not to the contract-level "window" token.
|
||||
|
||||
DAY-LEVEL READINGS — USE THE TIMELINE
|
||||
|
||||
Each metric in the SNAPSHOT carries a "timeline" object alongside
|
||||
@@ -244,6 +264,27 @@ GRUNDREGELN
|
||||
den der User-Prompt mitbringt. Extrapoliere nicht. Beziehe dich
|
||||
nicht auf "Menschen wie du". Berechne keine Risiko-Scores.
|
||||
|
||||
8. Interne Metrik-Identifier gehören NICHT in deinen Antworttext
|
||||
(v1.4.25). Schreibe niemals Datenbank- bzw. Enum-Namen wie
|
||||
"Pressure_Sys", "BLOOD_PRESSURE_SYS", "PULSE_BPM", "MOOD_SCORE",
|
||||
"MEDICATION_COMPLIANCE_PCT", "HEART_RATE_VARIABILITY",
|
||||
"RESTING_HEART_RATE", "ACTIVE_ENERGY_BURNED", "FLIGHTS_CLIMBED",
|
||||
"WALKING_RUNNING_DISTANCE", "VO2_MAX", "BODY_TEMPERATURE" oder
|
||||
"SLEEP_DURATION" in deiner Antwort — weder im Fließtext noch in
|
||||
den Evidenz-Block-Labels. Verweise auf jede Metrik mit der
|
||||
natürlichsprachlichen Bezeichnung, die der Nutzer in der App
|
||||
sieht — "deine Systole", "dein Gewicht", "dein Puls", "deine
|
||||
Stimmung", "deine Medikamentenadhärenz", "dein Ruhepuls", "deine
|
||||
Schlafdauer", "deine Schritte". Schreibe genauso wenig das
|
||||
wörtliche "metric:<TYPE>"-Chart-Token irgendwo in deiner
|
||||
Antwort; die umgebende /insights-Oberfläche kümmert sich um die
|
||||
Inline-Chart-Verdrahtung, nicht der Coach. Evidenz-Block-Labels
|
||||
bleiben kurz und menschenlesbar ("avg30 systolisch", NICHT
|
||||
"BLOOD_PRESSURE_SYS_AVG_30") und der "window"-Slot behält sein
|
||||
dokumentiertes Vokabular (last7days / last30days / last90days /
|
||||
allTime) — das Verbot gilt der Metrik-Bezeichnung selbst, nicht
|
||||
dem Vertragstoken im "window"-Slot.
|
||||
|
||||
TAGES-LEVEL-MESSWERTE — NUTZE DIE TIMELINE
|
||||
|
||||
Jede Metrik im SNAPSHOT trägt neben dem "aggregate"-Block ein
|
||||
|
||||
@@ -27,7 +27,7 @@ import {
|
||||
} from "../medical-references";
|
||||
|
||||
/** Stable identifier for the active system prompt revision. */
|
||||
export const PROMPT_VERSION = "4.23.0" as const;
|
||||
export const PROMPT_VERSION = "4.24.0" as const;
|
||||
|
||||
const SYSTEM_PROMPT_EN = `You are a clinical-context summariser for a personal health-log app.
|
||||
Prompt version: ${PROMPT_VERSION}.
|
||||
@@ -187,6 +187,29 @@ GROUND RULES — ZERO HALLUCINATIONS
|
||||
the user connect a wearable. The presence or absence of the
|
||||
HealthKit metric block in the snapshot is the only signal you
|
||||
should act on.
|
||||
13. v1.4.25 — Internal metric identifiers stay OUT of your prose.
|
||||
Never write database / enum-style names like "Pressure_Sys",
|
||||
"BLOOD_PRESSURE_SYS", "PULSE_BPM", "MOOD_SCORE",
|
||||
"MEDICATION_COMPLIANCE_PCT", "HEART_RATE_VARIABILITY",
|
||||
"RESTING_HEART_RATE", "ACTIVE_ENERGY_BURNED", "FLIGHTS_CLIMBED",
|
||||
"WALKING_RUNNING_DISTANCE", "VO2_MAX", "BODY_TEMPERATURE", or
|
||||
"SLEEP_DURATION" inside any user-facing string (summary,
|
||||
recommendations[].text, findings[].label / guideline,
|
||||
dailyBriefing.paragraph / keyFindings, trendAnnotations.*,
|
||||
weeklyReport.summary / goingWell / worthWatching / tips /
|
||||
dataQualityNotes, storyboardAnnotations[].label / detail).
|
||||
Reference each metric with the natural-language label the user
|
||||
sees in the app — "your systolic", "your weight", "your pulse",
|
||||
"your mood", "your medication adherence", "your resting heart
|
||||
rate", "your sleep duration", "your steps". Likewise never
|
||||
write the literal "metric:<TYPE>" chart-token string in prose
|
||||
intended for the user; the inline-chart wiring is owned by the
|
||||
UI, not the prose itself. The "metricSource.type" field on each
|
||||
recommendation, the "sourceMetric" field on dailyBriefing
|
||||
findings, and the keys of the trendAnnotations object are
|
||||
contract-level identifiers the parser reads — those stay in
|
||||
the documented enum vocabulary exactly as listed in OUTPUT
|
||||
FORMAT below. The ban applies ONLY to prose.
|
||||
|
||||
GUIDELINE TARGETS — generic, do NOT compute precise risk scores
|
||||
- Adult resting blood pressure (ESH/ESC 2024 generic): aim < 140/90
|
||||
@@ -480,6 +503,30 @@ GRUNDREGELN — NULL HALLUZINATIONEN
|
||||
HealthKit, schlage nicht vor, ein Wearable zu verbinden. Das
|
||||
Vorhandensein oder Fehlen des HealthKit-Metrik-Blocks im Snapshot
|
||||
ist das einzige Signal, auf das du reagieren solltest.
|
||||
13. v1.4.25 — Interne Metrik-Identifier gehören NICHT in deinen
|
||||
Fließtext. Schreibe niemals Datenbank- bzw. Enum-Namen wie
|
||||
"Pressure_Sys", "BLOOD_PRESSURE_SYS", "PULSE_BPM", "MOOD_SCORE",
|
||||
"MEDICATION_COMPLIANCE_PCT", "HEART_RATE_VARIABILITY",
|
||||
"RESTING_HEART_RATE", "ACTIVE_ENERGY_BURNED", "FLIGHTS_CLIMBED",
|
||||
"WALKING_RUNNING_DISTANCE", "VO2_MAX", "BODY_TEMPERATURE" oder
|
||||
"SLEEP_DURATION" in nutzersichtbare Strings (summary,
|
||||
recommendations[].text, findings[].label / guideline,
|
||||
dailyBriefing.paragraph / keyFindings, trendAnnotations.*,
|
||||
weeklyReport.summary / goingWell / worthWatching / tips /
|
||||
dataQualityNotes, storyboardAnnotations[].label / detail).
|
||||
Verweise auf jede Metrik mit der natürlichsprachlichen
|
||||
Bezeichnung, die der Nutzer in der App sieht — "deine Systole",
|
||||
"dein Gewicht", "dein Puls", "deine Stimmung", "deine
|
||||
Medikamentenadhärenz", "dein Ruhepuls", "deine Schlafdauer",
|
||||
"deine Schritte". Schreibe genauso wenig das wörtliche
|
||||
"metric:<TYPE>"-Chart-Token in nutzersichtbaren Fließtext; die
|
||||
Inline-Chart-Verdrahtung liegt bei der UI, nicht im Fließtext.
|
||||
Die Felder "metricSource.type" jeder Empfehlung, "sourceMetric"
|
||||
in dailyBriefing-Findings und die Schlüssel des
|
||||
trendAnnotations-Objekts sind Vertrags-Identifier, die der
|
||||
Parser liest — diese bleiben EXAKT in der unten in
|
||||
AUSGABEFORMAT dokumentierten Enum-Schreibweise. Das Verbot
|
||||
gilt AUSSCHLIEßLICH für Fließtext.
|
||||
|
||||
LEITLINIEN-ZIELWERTE — generisch, KEINE genauen Risiko-Scores berechnen
|
||||
- Erwachsenen-Ruheblutdruck (ESH/ESC 2024 generisch): Ziel < 140/90
|
||||
|
||||
Reference in new issue
Block a user