feat(coach-prompt): GROUND RULE 13 (insights) + 8 (coach) — no internal metric enum names in prose

Closes the loop on the "Metric Pressure_Sys" leak Marc reported on
the Einschätzungen surface 2026-05-14. The stripChartTokens regex
extension cleans up render-time output, but the model was free to
keep emitting raw enum identifiers turn after turn. This commit
teaches the generator to stop emitting them in the first place.

Insight-generator (DE + EN) gets GROUND RULE 13:

  - Lists every banned identifier verbatim — Pressure_Sys,
    BLOOD_PRESSURE_SYS, PULSE_BPM, MOOD_SCORE,
    MEDICATION_COMPLIANCE_PCT, plus the v1.4.23 Apple Health surface
    (HEART_RATE_VARIABILITY, RESTING_HEART_RATE,
    ACTIVE_ENERGY_BURNED, FLIGHTS_CLIMBED,
    WALKING_RUNNING_DISTANCE, VO2_MAX, BODY_TEMPERATURE,
    SLEEP_DURATION).
  - Enumerates the exact user-facing fields the ban applies to —
    summary, recommendations[].text, findings, dailyBriefing,
    trendAnnotations.*, weeklyReport.*, storyboardAnnotations.
  - Carves out the contract identifiers the parser reads
    (metricSource.type, sourceMetric, trendAnnotations keys) so the
    JSON contract stays intact — the ban applies ONLY to prose.
  - Documents the natural-language labels to use instead ("your
    systolic", "your weight", "your medication adherence", …).

Coach system-prompt (DE + EN) gets the matching GROUND RULE 8 with
the same banned list, the same prose-only carve-out, and an extra
clarification that evidence-block labels stay human-readable
("avg30 systolic", NOT "BLOOD_PRESSURE_SYS_AVG_30") while the
window slot keeps its documented contract vocabulary.

PROMPT_VERSION 4.23.0 → 4.24.0. The rule is genuinely new (neither
prompt previously mentioned the enum identifiers in any form), so
the bump reflects the additive behavioural change. Existing pin
assertions in coach-prompt-v423.test.ts + coach/system-prompt.test.ts
loosen from `/^4\.23\.\d+$/` to `4.{>=23}.x` so future additive
ratchets don't force a hostile test rewrite each time.

New unit tests pin GROUND RULE 13 + 8 by the banned-identifier
strings + the contract-carve-out wording so a future copy-paste
regression can't silently drop either rule.
This commit is contained in:
Marc-André Bombeck committed 2026-05-14 12:52:18 +02:00
1 parent 35f068ac26
commit feeff38ba2
4 files changed
+157 -7

No files matched your search

+54 -2
View File
@@ -19,8 +19,14 @@ import {
* so the model emits the additive enum values directly.
*/
describe("PROMPT_VERSION (v1.4.23 Apple Health ratchet)", () => {
it("matches the 4.23.x train", () => {
expect(PROMPT_VERSION).toMatch(/^4\.23\.\d+$/);
it("matches the 4.23+ train", () => {
// v1.4.25 W5b ratchet to 4.24.0 — GROUND RULE 13 bans internal
// metric enum identifiers from prose. The pin loosens to accept
// any 4.{>=23}.x revision so future additive ratchets don't
// force a hostile test rewrite each time the rules grow.
const [, minor] = PROMPT_VERSION.split(".").map(Number);
expect(PROMPT_VERSION).toMatch(/^4\.\d+\.\d+$/);
expect(minor).toBeGreaterThanOrEqual(23);
});
});
@@ -77,3 +83,49 @@ describe("getStrictInsightsSystemPrompt — DE", () => {
);
});
});
/**
* v1.4.25 W5b — pin GROUND RULE 13 (no internal metric identifiers
* in prose). Marc saw the "Metric Pressure_Sys" leak on the
* Einschätzungen surface 2026-05-14; the stripper covers the leak at
* render time, GROUND RULE 13 closes the loop at generation time so
* the cleaned-up prose was the only version emitted in the first
* place.
*/
describe("getStrictInsightsSystemPrompt — GROUND RULE 13 (no enum names in prose)", () => {
it("EN prompt carries the rule and enumerates the banned identifiers", () => {
const prompt = getStrictInsightsSystemPrompt("en");
expect(prompt).toMatch(/13\. v1\.4\.25/);
expect(prompt).toMatch(/Internal metric identifiers/);
// The banned-list is enumerated verbatim so the model has no
// ambiguity about which strings are off-limits in prose.
expect(prompt).toMatch(/"Pressure_Sys"/);
expect(prompt).toMatch(/"BLOOD_PRESSURE_SYS"/);
expect(prompt).toMatch(/"PULSE_BPM"/);
expect(prompt).toMatch(/"MOOD_SCORE"/);
expect(prompt).toMatch(/"MEDICATION_COMPLIANCE_PCT"/);
// Apple Health additions also banned in prose (the surface
// ships in v1.5 but the model can mention them on iOS-connected
// snapshots today).
expect(prompt).toMatch(/"HEART_RATE_VARIABILITY"/);
expect(prompt).toMatch(/"SLEEP_DURATION"/);
// The carve-out for contract identifiers must remain in place
// so the parser keeps working.
expect(prompt).toMatch(/metricSource\.type/);
expect(prompt).toMatch(/applies ONLY to prose/);
});
it("DE prompt carries the rule with the same banned list", () => {
const prompt = getStrictInsightsSystemPrompt("de");
expect(prompt).toMatch(/13\. v1\.4\.25/);
expect(prompt).toMatch(/Interne Metrik-Identifier/);
expect(prompt).toMatch(/"Pressure_Sys"/);
expect(prompt).toMatch(/"BLOOD_PRESSURE_SYS"/);
expect(prompt).toMatch(/"PULSE_BPM"/);
expect(prompt).toMatch(/"MOOD_SCORE"/);
expect(prompt).toMatch(/"MEDICATION_COMPLIANCE_PCT"/);
expect(prompt).toMatch(/"HEART_RATE_VARIABILITY"/);
expect(prompt).toMatch(/"SLEEP_DURATION"/);
expect(prompt).toMatch(/AUSSCHLIEßLICH für Fließtext/);
});
});
@@ -16,11 +16,15 @@ import { getCoachSystemPrompt } from "../system-prompt";
* the prompt body (per the W1b research output).
*/
describe("PROMPT_VERSION (v1.4.23 Coach + Apple Health extension)", () => {
it("matches the 4.23.x train", () => {
it("matches the 4.23+ train", () => {
// v1.4.23 ratcheted the prompt to add GROUND RULE 12 (Apple Health
// optional categories). Future patches stay on 4.23.x until a
// behavioural change ships.
expect(PROMPT_VERSION).toMatch(/^4\.23\.\d+$/);
// optional categories). v1.4.25 W5b ratcheted again to add GROUND
// RULE 13 (no internal metric identifiers in prose). The pin
// loosens to accept any 4.{>=23}.x revision so future additive
// ratchets don't force a hostile test rewrite each time.
const [, minor] = PROMPT_VERSION.split(".").map(Number);
expect(PROMPT_VERSION).toMatch(/^4\.\d+\.\d+$/);
expect(minor).toBeGreaterThanOrEqual(23);
});
});
@@ -50,6 +54,10 @@ describe("getCoachSystemPrompt — EN", () => {
expect(prompt).toMatch(/5\. Motivational-interviewing micro-moves\./);
expect(prompt).toMatch(/6\. Redirect off-topic input gracefully\./);
expect(prompt).toMatch(/7\. Ground every number in the SNAPSHOT\./);
// v1.4.25 W5b — GROUND RULE 8 added on the Coach side: bans the
// raw enum identifiers from the user-facing reply text and from
// the evidence-block labels.
expect(prompt).toMatch(/8\. Internal metric identifiers/);
});
it("preserves the v1.4.21 DAY-LEVEL READINGS section verbatim", () => {
@@ -115,6 +123,8 @@ describe("getCoachSystemPrompt — DE", () => {
);
expect(prompt).toMatch(/6\. Off-topic-Eingaben elegant umlenken\./);
expect(prompt).toMatch(/7\. Verankere jede Zahl im SNAPSHOT\./);
// v1.4.25 W5b — GROUND RULE 8 on the German Coach side.
expect(prompt).toMatch(/8\. Interne Metrik-Identifier/);
});
it("preserves the TAGES-LEVEL-MESSWERTE section verbatim", () => {
+41
View File
@@ -76,6 +76,26 @@ GROUND RULES
user prompt carries. Do not extrapolate. Do not reference "people
like you". Do not compute risk scores.
8. Internal metric identifiers stay OUT of your prose (v1.4.25).
Never write database / enum-style names like "Pressure_Sys",
"BLOOD_PRESSURE_SYS", "PULSE_BPM", "MOOD_SCORE",
"MEDICATION_COMPLIANCE_PCT", "HEART_RATE_VARIABILITY",
"RESTING_HEART_RATE", "ACTIVE_ENERGY_BURNED", "FLIGHTS_CLIMBED",
"WALKING_RUNNING_DISTANCE", "VO2_MAX", "BODY_TEMPERATURE", or
"SLEEP_DURATION" in your reply text — neither in the prose nor in
the evidence-block labels. Reference each metric with the
natural-language label the user sees in the app — "your
systolic", "your weight", "your pulse", "your mood", "your
medication adherence", "your resting heart rate", "your sleep
duration", "your steps". Likewise never write the canonical
"metric:<TYPE>" chart-token string anywhere in your reply; the
surrounding /insights surface owns inline-chart wiring, not the
Coach. Evidence-block labels stay short and human ("avg30
systolic", not "BLOOD_PRESSURE_SYS_AVG_30") and the "window"
slot keeps its documented vocabulary (last7days / last30days /
last90days / allTime) — the ban applies to the metric label
itself, not to the contract-level "window" token.
DAY-LEVEL READINGS — USE THE TIMELINE
Each metric in the SNAPSHOT carries a "timeline" object alongside
@@ -244,6 +264,27 @@ GRUNDREGELN
den der User-Prompt mitbringt. Extrapoliere nicht. Beziehe dich
nicht auf "Menschen wie du". Berechne keine Risiko-Scores.
8. Interne Metrik-Identifier gehören NICHT in deinen Antworttext
(v1.4.25). Schreibe niemals Datenbank- bzw. Enum-Namen wie
"Pressure_Sys", "BLOOD_PRESSURE_SYS", "PULSE_BPM", "MOOD_SCORE",
"MEDICATION_COMPLIANCE_PCT", "HEART_RATE_VARIABILITY",
"RESTING_HEART_RATE", "ACTIVE_ENERGY_BURNED", "FLIGHTS_CLIMBED",
"WALKING_RUNNING_DISTANCE", "VO2_MAX", "BODY_TEMPERATURE" oder
"SLEEP_DURATION" in deiner Antwort — weder im Fließtext noch in
den Evidenz-Block-Labels. Verweise auf jede Metrik mit der
natürlichsprachlichen Bezeichnung, die der Nutzer in der App
sieht — "deine Systole", "dein Gewicht", "dein Puls", "deine
Stimmung", "deine Medikamentenadhärenz", "dein Ruhepuls", "deine
Schlafdauer", "deine Schritte". Schreibe genauso wenig das
wörtliche "metric:<TYPE>"-Chart-Token irgendwo in deiner
Antwort; die umgebende /insights-Oberfläche kümmert sich um die
Inline-Chart-Verdrahtung, nicht der Coach. Evidenz-Block-Labels
bleiben kurz und menschenlesbar ("avg30 systolisch", NICHT
"BLOOD_PRESSURE_SYS_AVG_30") und der "window"-Slot behält sein
dokumentiertes Vokabular (last7days / last30days / last90days /
allTime) — das Verbot gilt der Metrik-Bezeichnung selbst, nicht
dem Vertragstoken im "window"-Slot.
TAGES-LEVEL-MESSWERTE — NUTZE DIE TIMELINE
Jede Metrik im SNAPSHOT trägt neben dem "aggregate"-Block ein
+48 -1
View File
@@ -27,7 +27,7 @@ import {
} from "../medical-references";
/** Stable identifier for the active system prompt revision. */
export const PROMPT_VERSION = "4.23.0" as const;
export const PROMPT_VERSION = "4.24.0" as const;
const SYSTEM_PROMPT_EN = `You are a clinical-context summariser for a personal health-log app.
Prompt version: ${PROMPT_VERSION}.
@@ -187,6 +187,29 @@ GROUND RULES — ZERO HALLUCINATIONS
the user connect a wearable. The presence or absence of the
HealthKit metric block in the snapshot is the only signal you
should act on.
13. v1.4.25 — Internal metric identifiers stay OUT of your prose.
Never write database / enum-style names like "Pressure_Sys",
"BLOOD_PRESSURE_SYS", "PULSE_BPM", "MOOD_SCORE",
"MEDICATION_COMPLIANCE_PCT", "HEART_RATE_VARIABILITY",
"RESTING_HEART_RATE", "ACTIVE_ENERGY_BURNED", "FLIGHTS_CLIMBED",
"WALKING_RUNNING_DISTANCE", "VO2_MAX", "BODY_TEMPERATURE", or
"SLEEP_DURATION" inside any user-facing string (summary,
recommendations[].text, findings[].label / guideline,
dailyBriefing.paragraph / keyFindings, trendAnnotations.*,
weeklyReport.summary / goingWell / worthWatching / tips /
dataQualityNotes, storyboardAnnotations[].label / detail).
Reference each metric with the natural-language label the user
sees in the app — "your systolic", "your weight", "your pulse",
"your mood", "your medication adherence", "your resting heart
rate", "your sleep duration", "your steps". Likewise never
write the literal "metric:<TYPE>" chart-token string in prose
intended for the user; the inline-chart wiring is owned by the
UI, not the prose itself. The "metricSource.type" field on each
recommendation, the "sourceMetric" field on dailyBriefing
findings, and the keys of the trendAnnotations object are
contract-level identifiers the parser reads — those stay in
the documented enum vocabulary exactly as listed in OUTPUT
FORMAT below. The ban applies ONLY to prose.
GUIDELINE TARGETS — generic, do NOT compute precise risk scores
- Adult resting blood pressure (ESH/ESC 2024 generic): aim < 140/90
@@ -480,6 +503,30 @@ GRUNDREGELN — NULL HALLUZINATIONEN
HealthKit, schlage nicht vor, ein Wearable zu verbinden. Das
Vorhandensein oder Fehlen des HealthKit-Metrik-Blocks im Snapshot
ist das einzige Signal, auf das du reagieren solltest.
13. v1.4.25 — Interne Metrik-Identifier gehören NICHT in deinen
Fließtext. Schreibe niemals Datenbank- bzw. Enum-Namen wie
"Pressure_Sys", "BLOOD_PRESSURE_SYS", "PULSE_BPM", "MOOD_SCORE",
"MEDICATION_COMPLIANCE_PCT", "HEART_RATE_VARIABILITY",
"RESTING_HEART_RATE", "ACTIVE_ENERGY_BURNED", "FLIGHTS_CLIMBED",
"WALKING_RUNNING_DISTANCE", "VO2_MAX", "BODY_TEMPERATURE" oder
"SLEEP_DURATION" in nutzersichtbare Strings (summary,
recommendations[].text, findings[].label / guideline,
dailyBriefing.paragraph / keyFindings, trendAnnotations.*,
weeklyReport.summary / goingWell / worthWatching / tips /
dataQualityNotes, storyboardAnnotations[].label / detail).
Verweise auf jede Metrik mit der natürlichsprachlichen
Bezeichnung, die der Nutzer in der App sieht — "deine Systole",
"dein Gewicht", "dein Puls", "deine Stimmung", "deine
Medikamentenadhärenz", "dein Ruhepuls", "deine Schlafdauer",
"deine Schritte". Schreibe genauso wenig das wörtliche
"metric:<TYPE>"-Chart-Token in nutzersichtbaren Fließtext; die
Inline-Chart-Verdrahtung liegt bei der UI, nicht im Fließtext.
Die Felder "metricSource.type" jeder Empfehlung, "sourceMetric"
in dailyBriefing-Findings und die Schlüssel des
trendAnnotations-Objekts sind Vertrags-Identifier, die der
Parser liest — diese bleiben EXAKT in der unten in
AUSGABEFORMAT dokumentierten Enum-Schreibweise. Das Verbot
gilt AUSSCHLIEßLICH für Fließtext.
LEITLINIEN-ZIELWERTE — generisch, KEINE genauen Risiko-Scores berechnen
- Erwachsenen-Ruheblutdruck (ESH/ESC 2024 generisch): Ziel < 140/90