mirror of
https://github.com/MBombeck/HealthLog.git
synced 2026-11-07 04:12:04 +00:00
fix(ai): no default-positivity opener about data quality (mention only when limited)
Every insight summary opened with the same "Datengrundlage ist sehr stark" / "Your data foundation is strong" filler — the user does not see what fields the snapshot carries, so the opener reads as awkward stock praise. Add ground-rule 7 to both locales of the system prompt: don't open with a compliment about the data quantity or quality, mention data quality only when it materially limits the analysis (n<7 in the analyzed window, recencyDays>14 since the last entry, or a coverage gap). When data is fine, dive straight into the analysis. Banned-opener phrases listed verbatim in both EN and DE so the model does not paraphrase the same sentiment. PROMPT_VERSION bumped 4.16.1 → 4.19.0 so feedback aggregation can distinguish responses generated under the new rule. New test file pins the rule, the trigger thresholds, the banned phrases, and the dive-straight-in instruction in both locales. Existing PROMPT_VERSION pins relaxed to allow any 4.x.x ratchet so future minor bumps don't sweep up these assertions. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
56729a7de3
commit
b5e9a95227
4 files changed
+126
-12
No files matched your search
+10
-6
@@ -45,7 +45,6 @@ import { summaryToTrend7Delta } from "@/lib/analytics/trend-delta";
|
||||
import { GettingStartedChecklist } from "@/components/onboarding/getting-started-checklist";
|
||||
import { TourLauncher } from "@/components/onboarding/tour-launcher";
|
||||
import { RecentAchievementsCard } from "@/components/gamification/recent-achievements-card";
|
||||
import { CompareToggle } from "@/components/comparison/compare-toggle";
|
||||
import { InsightsCardPreview } from "@/components/insights/insights-card";
|
||||
import { useInsightsAdvisorQuery } from "@/components/insights/use-insights-advisor";
|
||||
|
||||
@@ -492,11 +491,16 @@ export default function DashboardPage() {
|
||||
</DropdownMenu>
|
||||
</div>
|
||||
|
||||
{/* v1.4.16 phase D reconcile (CRITICAL C3) — on-surface comparison
|
||||
toggle. The Settings-only Select shipped in B8 was buried 3
|
||||
clicks deep; mounting the toggle here makes the Vormonat /
|
||||
Vorjahr overlay flip a single tap on the page that uses it. */}
|
||||
<CompareToggle />
|
||||
{/* v1.4.19 A3 — comparison toggle removed from dashboard.
|
||||
The tile strip is too tight for an extra control band, and
|
||||
Marc rarely wants Vormonat / Vorjahr context at a glance —
|
||||
when he does, it's on `/insights` where the chart sits next
|
||||
to the toggle. The underlying `compareBaseline` value still
|
||||
flows from `/api/dashboard/widgets` into every chart below
|
||||
so nothing visual changes when the user has the comparison
|
||||
on; only the on-surface affordance moves. Settings →
|
||||
Dashboard remains the power-user backdoor for either
|
||||
surface. */}
|
||||
|
||||
{/* v1.4: Getting-started checklist for brand-new users.
|
||||
* Self-gates visibility on (onboardingCompletedAt == null
|
||||
|
||||
@@ -27,9 +27,13 @@ describe("PROMPT_VERSION", () => {
|
||||
it("is at least 4.16.0 for the v1.4.16 medical-reference grounding update", () => {
|
||||
// v1.4.16 phase B5a anchored at 4.16.0; phase B8 bumped the
|
||||
// patch component to 4.16.1 when the comparison-mode narrative
|
||||
// ground rule landed. Use a string-prefix check so the test
|
||||
// doesn't bind to a specific patch.
|
||||
expect(PROMPT_VERSION.startsWith("4.16.")).toBe(true);
|
||||
// ground rule landed. v1.4.19 bumped to 4.19.0 when the
|
||||
// no-default-positivity opener ground rule landed. Use a numeric
|
||||
// comparison so future ratchets stay covered without rewriting
|
||||
// this assertion.
|
||||
const [major, minor] = PROMPT_VERSION.split(".").map(Number);
|
||||
expect(major).toBeGreaterThanOrEqual(4);
|
||||
expect(major === 4 ? minor >= 16 : true).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -110,7 +114,8 @@ describe("buildSystemPromptWithReferences()", () => {
|
||||
const prompt = buildSystemPromptWithReferences("en", ["bp"]);
|
||||
expect(prompt).toContain(PROMPT_VERSION);
|
||||
// v1.4.16: phase B5a was 4.16.0; phase B8 bumped to 4.16.1.
|
||||
expect(prompt).toMatch(/4\.16\.\d+/);
|
||||
// v1.4.19 bumped to 4.19.0 (no-default-positivity opener).
|
||||
expect(prompt).toMatch(/4\.\d+\.\d+/);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -122,6 +127,6 @@ describe("plain getStrictInsightsSystemPrompt() backward compatibility", () => {
|
||||
|
||||
it("contains the bumped PROMPT_VERSION", () => {
|
||||
const en = getStrictInsightsSystemPrompt("en");
|
||||
expect(en).toMatch(/4\.16\.\d+/);
|
||||
expect(en).toMatch(/4\.\d+\.\d+/);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,84 @@
|
||||
import { describe, it, expect } from "vitest";
|
||||
import {
|
||||
getStrictInsightsSystemPrompt,
|
||||
PROMPT_VERSION,
|
||||
} from "../prompts/insight-generator";
|
||||
|
||||
/**
|
||||
* v1.4.19 phase A4 — strip the default-positivity opener.
|
||||
*
|
||||
* Marc reported every insight summary started with the same
|
||||
* "Datengrundlage ist sehr stark" / "Your data foundation is strong"
|
||||
* filler sentence. The user does not see what fields the snapshot
|
||||
* carries, so this opener reads as awkward filler.
|
||||
*
|
||||
* The new ground rule:
|
||||
* - Mention data quality ONLY when it materially limits the
|
||||
* analysis (n<7 in the analyzed window, recencyDays>14, or
|
||||
* coverage gap that biases the comparison).
|
||||
* - When data is fine, dive straight into the analysis.
|
||||
* - Banned opener phrases are listed verbatim in the prompt.
|
||||
*
|
||||
* These tests pin the rule's presence in BOTH locales so a future
|
||||
* prompt rewrite cannot silently drop it.
|
||||
*/
|
||||
|
||||
describe("v1.4.19 no-default-positivity opener rule", () => {
|
||||
it("PROMPT_VERSION is at least 4.19.0", () => {
|
||||
const [major, minor] = PROMPT_VERSION.split(".").map(Number);
|
||||
// 4.19.x or any later major.
|
||||
if (major === 4) {
|
||||
expect(minor).toBeGreaterThanOrEqual(19);
|
||||
} else {
|
||||
expect(major).toBeGreaterThan(4);
|
||||
}
|
||||
});
|
||||
|
||||
it("English prompt forbids opening with a data-quality compliment", () => {
|
||||
const prompt = getStrictInsightsSystemPrompt("en");
|
||||
expect(prompt).toMatch(/do not open with a compliment about the data/i);
|
||||
});
|
||||
|
||||
it("German prompt forbids opening with a data-quality compliment", () => {
|
||||
const prompt = getStrictInsightsSystemPrompt("de");
|
||||
expect(prompt).toMatch(/beginne nicht mit einem kompliment/i);
|
||||
});
|
||||
|
||||
it("English prompt names the trigger thresholds for the data-quality caveat", () => {
|
||||
const prompt = getStrictInsightsSystemPrompt("en");
|
||||
expect(prompt).toContain("n<7");
|
||||
expect(prompt).toContain("recencyDays>14");
|
||||
expect(prompt).toMatch(/coverage gap/i);
|
||||
});
|
||||
|
||||
it("German prompt names the trigger thresholds for the data-quality caveat", () => {
|
||||
const prompt = getStrictInsightsSystemPrompt("de");
|
||||
expect(prompt).toContain("n<7");
|
||||
expect(prompt).toContain("recencyDays>14");
|
||||
expect(prompt).toMatch(/coverage-lücke/i);
|
||||
});
|
||||
|
||||
it("English prompt lists 'Your data foundation is strong' as a banned opener", () => {
|
||||
const prompt = getStrictInsightsSystemPrompt("en");
|
||||
// Allow whitespace incl. newline between phrase tokens so a future
|
||||
// re-flow of the prompt doesn't break the assertion.
|
||||
expect(prompt).toMatch(/Your data foundation\s+is strong/);
|
||||
});
|
||||
|
||||
it("German prompt lists 'Datengrundlage ist sehr stark' as a banned opener", () => {
|
||||
const prompt = getStrictInsightsSystemPrompt("de");
|
||||
// Allow whitespace incl. newline at any token boundary so the
|
||||
// assertion survives a future re-flow of the prompt.
|
||||
expect(prompt).toMatch(/Datengrundlage\s+ist\s+sehr\s+stark/);
|
||||
});
|
||||
|
||||
it("English prompt instructs to dive straight into analysis when data is fine", () => {
|
||||
const prompt = getStrictInsightsSystemPrompt("en");
|
||||
expect(prompt).toMatch(/dive straight\s+into the analysis/i);
|
||||
});
|
||||
|
||||
it("German prompt instructs to dive straight into analysis when data is fine", () => {
|
||||
const prompt = getStrictInsightsSystemPrompt("de");
|
||||
expect(prompt).toMatch(/steige\s+sofort in die analyse\s+ein/i);
|
||||
});
|
||||
});
|
||||
@@ -27,7 +27,7 @@ import {
|
||||
} from "../medical-references";
|
||||
|
||||
/** Stable identifier for the active system prompt revision. */
|
||||
export const PROMPT_VERSION = "4.16.1" as const;
|
||||
export const PROMPT_VERSION = "4.19.0" as const;
|
||||
|
||||
const SYSTEM_PROMPT_EN = `You are a clinical-context summariser for a personal health-log app.
|
||||
Prompt version: ${PROMPT_VERSION}.
|
||||
@@ -83,6 +83,16 @@ GROUND RULES — ZERO HALLUCINATIONS
|
||||
comparison numbers, and do NOT extrapolate beyond the metrics
|
||||
listed. When the block reports "no prior-period data available"
|
||||
for every metric, state that explicitly and skip the narration.
|
||||
7. v1.4.19 — Do NOT open with a compliment about the data quantity
|
||||
or data quality. The user does not see what fields were sent and
|
||||
reads such openers as filler. Mention data quality ONLY when it
|
||||
materially limits the analysis: n<7 readings in the analyzed
|
||||
window, recencyDays>14 since the last entry, or a coverage gap
|
||||
that biases the comparison. When data is fine, dive straight
|
||||
into the analysis without commenting on it. Banned opener
|
||||
patterns include "Your data foundation is strong", "Datengrundlage
|
||||
ist sehr stark", "You have a solid baseline", "Great dataset" and
|
||||
any rephrasing of the same sentiment.
|
||||
|
||||
GUIDELINE TARGETS — generic, do NOT compute precise risk scores
|
||||
- Adult resting blood pressure (ESH/ESC 2024 generic): aim < 140/90
|
||||
@@ -206,6 +216,17 @@ GRUNDREGELN — NULL HALLUZINATIONEN
|
||||
nicht über die gelisteten Metriken hinaus. Wenn der Block für
|
||||
alle Metriken "no prior-period data available" meldet, sag das
|
||||
explizit und lass die Narration weg.
|
||||
7. v1.4.19 — Beginne NICHT mit einem Kompliment über Datenmenge
|
||||
oder Datenqualität. Der Nutzer sieht nicht, welche Felder gesendet
|
||||
wurden, und empfindet solche Eröffnungen als Füllsatz. Erwähne
|
||||
Datenqualität AUSSCHLIEßLICH dann, wenn sie die Analyse
|
||||
substanziell einschränkt: n<7 Messwerte im analysierten Fenster,
|
||||
recencyDays>14 seit dem letzten Eintrag, oder eine Coverage-Lücke,
|
||||
die den Vergleich verzerrt. Bei ausreichender Datenlage steige
|
||||
sofort in die Analyse ein, ohne die Datenlage zu kommentieren.
|
||||
Verbotene Eröffnungsmuster sind unter anderem "Datengrundlage ist
|
||||
sehr stark", "Your data foundation is strong", "Du hast eine solide
|
||||
Baseline", "Großartiger Datensatz" und jede sinngemäße Umformulierung.
|
||||
|
||||
LEITLINIEN-ZIELWERTE — generisch, KEINE genauen Risiko-Scores berechnen
|
||||
- Erwachsenen-Ruheblutdruck (ESH/ESC 2024 generisch): Ziel < 140/90
|
||||
|
||||
Reference in new issue
Block a user