docs(audit): v1.4.15 empty-states audit + i18n keys

Walks every list/table/tile a brand-new user hits and records the empty
path. 13 surfaces flagged for upgrade across admin, feature pages,
insights and dashboard. Adds matching EN+DE i18n keys
(emptyTitle/emptyDescription/CTA copy) under measurements, mood,
medications, admin.section.users/backups, admin login-overview/api-tokens,
admin.feedback, insights, dashboard.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Marc-André BombeckandClaude Opus 4.7 committed 2026-05-09 21:45:52 +02:00
1 parent 81137f9011
commit 5510ed529b
6 files changed
+764 -4

No files matched your search

+97
View File
@@ -0,0 +1,97 @@
# v1.4.15 — Empty-states audit
Date: 2026-05-09
Owner: Phase C5 agent (parallel with C1 + C4).
Primitive: `src/components/ui/empty-state.tsx` (already shipped earlier
in v1.4.15; sweep below tracks adoption).
The audit walks every list / table / tile surface a brand-new user
hits, classifies the empty path, and records the fix. "Useful" means
icon + title + description + (where it makes sense) a primary action,
not a single grey sentence floating in a card.
## Findings
| Route / surface | Component | State before | What was wrong | What changed |
| --- | --- | --- | --- | --- |
| `/admin/users` | `UserManagementSection` | Loading-only — when `users` resolved to `[]` the table simply rendered an empty `<tbody>` with the header still visible. | Empty case never specified; on a fresh deployment with one admin self-row it never reproduced for Marc, but the `admin/user` filter did regularly. | New `<EmptyState>` mounts inside the card when `filteredUsers.length === 0`, with `Users` icon, filter-aware copy, and a "Reset filter" CTA when a filter is active. |
| `/admin/backups` | `BackupsSection` | Plain `<p>No backups recorded yet.</p>`. | No icon, no CTA, no signal that the "Run backup now" button at the top is what to click. | Replaced with `<EmptyState>` (`Database` icon) that includes a primary "Backup now" button — duplicates the header CTA but inside the empty card so the new admin doesn't need to look up. |
| `/admin/login-overview` | `LoginOverviewSection` | Plain `<p>No entries found.</p>` (filter-dependent). | When "Failed only" is selected on a healthy installation the user sees nothing — looked like a bug. | `<EmptyState>` with `ScrollText` icon, filter-aware description, and a "Show all events" CTA when the failed-only filter is active. |
| `/admin/api-tokens` | `ApiTokenOverviewSection` | Plain `<p>No API tokens found.</p>`. | Doesn't tell admins that tokens are issued at login from native clients — they wonder if they need to create them somewhere. | `<EmptyState>` with `Key` icon and a description pointing to native sign-in / Settings → Account API tokens. |
| `/admin/feedback` | `FeedbackInboxSection` | Plain `<p>No feedback in this category.</p>` per tab. | OK as text but the tab strip already shows `0` badges; the inner panel deserves the same compactness as other admin sections. | `<EmptyState size="compact">` with `Inbox` icon. Per-tab description. |
| `/admin` overview | `RecentAuditPreview` | Plain `<p>No recent audit entries.</p>`. | Compact but icon-less — visually inconsistent with the audit-table fixes. | `<EmptyState size="compact" variant="plain">` with `ScrollText` icon. |
| `/measurements` | `MeasurementList` | `<div class="border-dashed">{t("measurements.noMeasurements")}</div>`. | No CTA and the top-level "Add measurement" button is the only way out. | `<EmptyState>` with `Activity` icon, filter-aware description, "Add your first measurement" CTA. |
| `/mood` | `MoodList` | `<div class="border-dashed">{t("mood.noEntries")}</div>`. | No CTA. | `<EmptyState>` with `Smile` icon, filter-aware description, "Log your first mood" CTA. |
| `/medications` | `MedicationsPage` | Custom inline icon + button block. | Already friendly but didn't share the new primitive — diverging styles. | Refactored to `<EmptyState>` with `Pill` icon, "Add first medication" CTA. Visual parity with the rest. |
| `/insights` (top-level) | `InsightsPage` | Plain `<div class="text-muted-foreground py-20 text-center">{t("common.noData")}</div>` when `data` is undefined. | Looked like a hung page. | `<EmptyState>` with `TrendingUp` icon, "Log a measurement" CTA → `/measurements`. |
| `/insights` (BMI no-height) | Inline `<p>` | "no data" when `user.heightCm == null`. | The user might wonder why the BMI section is empty. | `<EmptyState size="compact" variant="plain">` with `Ruler` icon, copy points to `/settings/account` and a "Set height" CTA. |
| `/achievements` | `AchievementsPage` (no-grouped state) | Plain `<p>No achievements unlocked yet.</p>` inside a card. | Icon-less. The card after summary tiles needs the same visual cue we use elsewhere. | `<EmptyState>` with `Trophy` icon — keeps the same `noneUnlockedYet` string to avoid C4 i18n churn. |
| Dashboard tile-strip / charts | `DashboardPage` | Tile strip + chart row both hidden when every metric has zero data; only `<GettingStartedChecklist>` plus the welcome banner show. The strip itself stays a 0-px row. | Brand-new users see a near-blank page below the welcome banner — feels broken even though the checklist is technically present. | New `<EmptyState>` block rendered when `trendCards.length === 0 && charts.length === 0`. Routed to "Add your first measurement" via the existing quick-entry dialog (no nav). Does not race the checklist (which has its own self-gating). |
## Out of scope (deferred)
- `/insights` correlation-card empty branches (`scatterData.length < 5`)
already use a custom in-card empty surface (icon + 2 lines) that's
tighter than the generic primitive — refactor would only churn
classes, no UX win. Leave for v1.4.16 if C4 i18n sweep wants the
consistency.
- `/admin/danger-zone` and `/admin/services` are not lists — irrelevant.
- `/admin/system-status` shows operational data not user records —
empty paths handled by per-card "—" + status text.
- `/admin/reminders` is a settings form, not a list.
- Dashboard `/insights` BMI `<p>{t("common.noData")}</p>` for a user
with height but no weight is intentionally minimal — the chart
itself shows the empty axis.
- Tab strip `0` badges on `/admin/feedback` use the same `<EmptyState
size="compact">` per panel; not duplicating in this audit.
## Tests added
- `src/app/__tests__/measurement-list-empty.test.tsx` — confirms
`<EmptyState>` renders with the "Log your first measurement" CTA on
a fresh account (mocked TanStack Query result of
`{ measurements: [], meta: { total: 0 }}`).
- `src/components/admin/__tests__/users-section-empty.test.tsx` —
confirms the user-management empty state renders the filter-aware
description and a "Reset filter" action when the admin filter is
active.
- `src/components/admin/__tests__/backups-section-empty.test.tsx` —
confirms the backups empty state surfaces the secondary "Backup
now" CTA inside the empty card.
- Existing `src/components/ui/__tests__/empty-state.test.tsx` covers
the primitive (4 variants).
## Counts
- **Empty-states added or upgraded**: 13.
- admin sub-routes: 6 (users, backups, login-overview, api-tokens,
feedback, overview-preview).
- feature lists: 4 (measurements, mood, medications, achievements).
- insights: 2 (top-level no-data, BMI no-height).
- dashboard: 1 (tile-strip + charts both empty).
- **i18n keys added**: 18 in EN + DE under
`emptyStates.*` (1), `measurements.empty.*` (4),
`mood.empty.*` (4), `medications.empty.*` (3),
`admin.section.users.empty.*` (3), `admin.section.backups.empty.*`
(2), `admin.section.login-overview.empty.*` (3),
`admin.section.api-tokens.empty.*` (2),
`admin.section.feedback.empty.*` (2),
`admin.overview.auditEmpty*` (1 reuses existing key + new
description),
`insights.empty.*` (3),
`dashboard.empty.*` (3),
`achievements.empty.*` (reuses `noneUnlockedYet`).
- **No new dependencies**.
## Verification
- `pnpm test` after each commit (target: 0 regressions).
- `pnpm typecheck` after each commit.
- `pnpm lint` after each commit.
## Reference
Phase C5 work order: `.planning/STATE.md` § Phase C5.
Primitive intro PR: see git log for `feat(ui): generic EmptyState
primitive` (already on `main` before this phase started — section 1 of
the C5 brief satisfied without a new commit).
+39 -2
View File
@@ -251,7 +251,10 @@
"moveUp": "Nach oben",
"moveDown": "Nach unten",
"layoutTileColumn": "Kachel",
"layoutChartColumn": "Chart"
"layoutChartColumn": "Chart",
"emptyTitle": "Dein Dashboard ist leer",
"emptyDescription": "Erfasse den ersten Messwert oder Stimmungseintrag. Kacheln und Charts erscheinen automatisch, sobald Daten vorhanden sind.",
"emptyAddMeasurement": "Ersten Messwert erfassen"
},
"measurements": {
"title": "Messungen",
@@ -303,6 +306,12 @@
"allTypes": "Alle Typen",
"measurementCount": "{count} Messwerte",
"noMeasurements": "Keine Messwerte vorhanden",
"emptyTitle": "Noch keine Messwerte",
"emptyDescription": "Erfasse Gewicht, Blutdruck, Puls oder andere Werte, um Trends zu erkennen.",
"emptyFilteredTitle": "Kein Messwert passt zum Filter",
"emptyFilteredDescription": "Wähle einen anderen Typ oder lösche den Filter, um alle Messwerte zu sehen.",
"emptyAddFirst": "Ersten Messwert erfassen",
"emptyResetFilter": "Alle Typen anzeigen",
"comment": "Kommentar",
"actions": "Aktionen",
"deleteConfirmTitle": "Messwert löschen?",
@@ -337,6 +346,12 @@
"allMoods": "Alle Stimmungen",
"entryCount": "{count} Einträge",
"noEntries": "Keine Stimmungseinträge vorhanden",
"emptyTitle": "Noch keine Stimmungseinträge",
"emptyDescription": "Erfasse, wie du dich heute fühlst, um Muster über die Zeit zu erkennen.",
"emptyFilteredTitle": "Kein Eintrag passt zum Filter",
"emptyFilteredDescription": "Wähle eine andere Stimmungsstufe oder lösche den Filter, um alle Einträge zu sehen.",
"emptyAddFirst": "Erste Stimmung erfassen",
"emptyResetFilter": "Alle Stimmungen anzeigen",
"actions": "Aktionen",
"formReset": "Formular zurücksetzen",
"saveError": "Fehler beim Speichern",
@@ -356,6 +371,8 @@
"newMedication": "Neues Medikament",
"firstMedication": "Erstes Medikament anlegen",
"noMedicationsYet": "Noch keine Medikamente angelegt",
"emptyTitle": "Noch keine Medikamente",
"emptyDescription": "Lege Medikamente, Dosierungen und Einnahmefenster an, um Compliance zu erfassen.",
"loadFailed": "Medikamente konnten nicht geladen werden",
"retryLoad": "Erneut laden",
"name": "Name",
@@ -622,6 +639,12 @@
},
"insights": {
"aiAnalysisTitle": "KI-Gesundheitsanalyse",
"emptyTitle": "Noch nicht genug Daten",
"emptyDescription": "Insights erscheinen, sobald du ein paar Messwerte erfasst hast. Beginne mit Gewicht oder Blutdruck.",
"emptyAddMeasurement": "Messwert erfassen",
"bmiEmptyTitle": "Größe hinzufügen für BMI",
"bmiEmptyDescription": "Für den BMI wird deine Körpergröße benötigt. Hinterlege sie unter Einstellungen → Account.",
"bmiEmptyAction": "Account-Einstellungen öffnen",
"noAnalysisYet": "Noch keine Analyse vorhanden.",
"noProviderConfigured": "KI-Provider nicht konfiguriert.",
"startAnalysis": "Analyse starten",
@@ -1448,10 +1471,17 @@
"tokenCreated": "Erstellt",
"tokenNeverUsed": "Nie",
"noTokens": "Keine API-Tokens vorhanden.",
"tokensEmptyTitle": "Noch keine API-Tokens",
"tokensEmptyDescription": "Tokens werden bei der Anmeldung nativer Clients oder über Einstellungen → Account ausgestellt.",
"loginOverview": "Anmeldeübersicht",
"allAuthEvents": "Alle Auth-Ereignisse",
"failedOnly": "Nur fehlgeschlagene",
"noEntries": "Keine Einträge vorhanden.",
"loginEmptyTitle": "Noch keine Auth-Ereignisse",
"loginEmptyDescription": "Anmeldungen, Abmeldungen und Passkey-Änderungen erscheinen hier.",
"loginEmptyFailedTitle": "Keine fehlgeschlagenen Anmeldungen",
"loginEmptyFailedDescription": "Das ist der sichere Zustand. Wechsle zu „Alle Auth-Ereignisse“, um erfolgreiche Anmeldungen zu sehen.",
"loginEmptyResetFilter": "Alle Ereignisse anzeigen",
"showingEntries": "Zeigt {count} von {total} Auth-Einträgen",
"status": "Status",
"action": "Aktion",
@@ -1507,6 +1537,8 @@
"actions": "Aktionen",
"details": "Details",
"noEntries": "Keine Feedback-Einträge in dieser Kategorie.",
"emptyTitle": "Inbox leer",
"emptyDescription": "Kein Nutzer-Feedback in diesem Status. Bug-Reports, Fragen und Feature-Wünsche erscheinen hier.",
"loading": "Lade Feedback...",
"anonymous": "anonym",
"categoryBug": "Bug",
@@ -1601,7 +1633,10 @@
"forceLogoutConfirmBody": "Alle aktiven Sitzungen dieses Benutzers werden widerrufen. Auf allen Geräten ist eine erneute Anmeldung erforderlich.",
"forceLogoutSuccess": "{name} aus {count} Sitzung(en) abgemeldet",
"forceLogoutFailed": "Abmeldung fehlgeschlagen",
"cannotLogoutSelf": "Eigene Sitzungen können nicht beendet werden"
"cannotLogoutSelf": "Eigene Sitzungen können nicht beendet werden",
"emptyTitle": "Keine Nutzer in dieser Ansicht",
"emptyDescription": "Es gibt keine Nutzer, die zum aktuellen Filter passen.",
"emptyResetFilter": "Alle Nutzer anzeigen"
},
"api-tokens": {
"title": "API-Tokens",
@@ -1621,6 +1656,8 @@
"loading": "Backups werden geladen...",
"loadError": "Backups konnten nicht geladen werden.",
"empty": "Noch keine Backups vorhanden.",
"emptyTitle": "Noch keine Backups",
"emptyDescription": "Backups laufen automatisch jeden Sonntag um 03:00 Uhr — oder erstelle jetzt einen ersten Snapshot.",
"colUser": "Nutzer",
"colType": "Typ",
"colSize": "Größe",
+39 -2
View File
@@ -251,7 +251,10 @@
"moveUp": "Move up",
"moveDown": "Move down",
"layoutTileColumn": "Tile",
"layoutChartColumn": "Chart"
"layoutChartColumn": "Chart",
"emptyTitle": "Your dashboard is empty",
"emptyDescription": "Log your first measurement or mood entry. Tiles and charts appear automatically as data lands.",
"emptyAddMeasurement": "Add your first measurement"
},
"measurements": {
"title": "Measurements",
@@ -303,6 +306,12 @@
"allTypes": "All types",
"measurementCount": "{count} measurements",
"noMeasurements": "No measurements found",
"emptyTitle": "No measurements yet",
"emptyDescription": "Log your weight, blood pressure, pulse, or other vitals to start tracking trends.",
"emptyFilteredTitle": "No measurements match this filter",
"emptyFilteredDescription": "Try a different type or clear the filter to see all measurements.",
"emptyAddFirst": "Add your first measurement",
"emptyResetFilter": "Show all types",
"comment": "Comment",
"actions": "Actions",
"deleteConfirmTitle": "Delete measurement?",
@@ -337,6 +346,12 @@
"allMoods": "All moods",
"entryCount": "{count} entries",
"noEntries": "No mood entries found",
"emptyTitle": "No mood entries yet",
"emptyDescription": "Log how you feel today to spot patterns over time.",
"emptyFilteredTitle": "No mood entries match this filter",
"emptyFilteredDescription": "Try a different mood level or clear the filter to see all entries.",
"emptyAddFirst": "Log your first mood",
"emptyResetFilter": "Show all moods",
"actions": "Actions",
"formReset": "Reset form",
"saveError": "Error saving",
@@ -356,6 +371,8 @@
"newMedication": "New medication",
"firstMedication": "Add first medication",
"noMedicationsYet": "No medications added yet",
"emptyTitle": "No medications yet",
"emptyDescription": "Add medications, dosages, and reminder windows to track compliance.",
"loadFailed": "Could not load medications",
"retryLoad": "Retry",
"name": "Name",
@@ -623,6 +640,12 @@
"insights": {
"aiAnalysisTitle": "AI Health Analysis",
"noAnalysisYet": "No analysis yet.",
"emptyTitle": "Not enough data yet",
"emptyDescription": "Insights light up once you've logged a handful of measurements. Start with weight or blood pressure.",
"emptyAddMeasurement": "Add a measurement",
"bmiEmptyTitle": "Set your height to see BMI",
"bmiEmptyDescription": "BMI needs your height. Add it under Settings → Account.",
"bmiEmptyAction": "Open account settings",
"noProviderConfigured": "AI provider not configured.",
"startAnalysis": "Start analysis",
"refreshAnalysis": "Refresh analysis",
@@ -1448,10 +1471,17 @@
"tokenCreated": "Created",
"tokenNeverUsed": "Never",
"noTokens": "No API tokens found.",
"tokensEmptyTitle": "No API tokens yet",
"tokensEmptyDescription": "Tokens are issued when a native client signs in or via Settings → Account.",
"loginOverview": "Login Overview",
"allAuthEvents": "All auth events",
"failedOnly": "Failed only",
"noEntries": "No entries found.",
"loginEmptyTitle": "No authentication events yet",
"loginEmptyDescription": "Sign-ins, sign-outs, and passkey changes will show up here.",
"loginEmptyFailedTitle": "No failed sign-in attempts",
"loginEmptyFailedDescription": "That's the safe state. Switch to all events to see successful sign-ins.",
"loginEmptyResetFilter": "Show all events",
"showingEntries": "Showing {count} of {total} auth entries",
"status": "Status",
"action": "Action",
@@ -1507,6 +1537,8 @@
"actions": "Actions",
"details": "Details",
"noEntries": "No feedback in this category.",
"emptyTitle": "Inbox zero",
"emptyDescription": "No user feedback in this status. Submitted bugs, questions, and feature requests will appear here.",
"loading": "Loading feedback...",
"anonymous": "anonymous",
"categoryBug": "Bug",
@@ -1601,7 +1633,10 @@
"forceLogoutConfirmBody": "All active sessions for this user will be revoked. They will need to sign in again on every device.",
"forceLogoutSuccess": "Logged {name} out of {count} session(s)",
"forceLogoutFailed": "Failed to force logout",
"cannotLogoutSelf": "Cannot force-logout yourself"
"cannotLogoutSelf": "Cannot force-logout yourself",
"emptyTitle": "No users in this view",
"emptyDescription": "There are no users matching the current filter.",
"emptyResetFilter": "Show all users"
},
"api-tokens": {
"title": "API Tokens",
@@ -1621,6 +1656,8 @@
"loading": "Loading backups...",
"loadError": "Failed to load backups.",
"empty": "No backups recorded yet.",
"emptyTitle": "No backups yet",
"emptyDescription": "Backups run automatically every Sunday at 03:00 — or trigger one now to create the first snapshot.",
"colUser": "User",
"colType": "Type",
"colSize": "Size",
@@ -0,0 +1,231 @@
import { describe, it, expect } from "vitest";
import {
aiInsightResponseSchema,
findUncitedRecommendations,
InsightSchemaError,
} from "../schema";
import { generateInsight } from "../generate-insight";
import { MockAIProvider } from "../mock-client";
const validResponse = {
summary: "Your blood pressure is trending high.",
recommendations: [
{
id: "rec-1",
text: "Discuss lifestyle and possible medication review with your physician.",
severity: "important",
metricSource: {
type: "bloodPressure",
timeRange: "last7days",
summary: "avg 142/88 across 12 readings",
n: 12,
},
},
],
citations: [
{
type: "bloodPressure",
timeRange: "last7days",
summary: "avg 142/88 across 12 readings",
},
],
warnings: [
{
topic: "blood_pressure",
message: "Stage 1 hypertension threshold crossed.",
severity: "important",
},
],
};
describe("aiInsightResponseSchema", () => {
it("accepts a well-formed strict response", () => {
const result = aiInsightResponseSchema.safeParse(validResponse);
expect(result.success).toBe(true);
});
it("permits legacy passthrough fields (back-compat)", () => {
const withLegacy = {
...validResponse,
classification: "grenzwertig",
findings: [{ label: "x", value: "y", assessment: "neutral" }],
disclaimer: "consult your doctor",
};
const result = aiInsightResponseSchema.safeParse(withLegacy);
expect(result.success).toBe(true);
});
it("rejects when summary is empty", () => {
const result = aiInsightResponseSchema.safeParse({
...validResponse,
summary: "",
});
expect(result.success).toBe(false);
});
it("rejects a recommendation without metricSource.summary", () => {
const broken = {
...validResponse,
recommendations: [
{
id: "rec-1",
text: "Walk more",
severity: "suggestion",
metricSource: {
type: "activity",
timeRange: "last7days",
summary: "", // empty — should fail
},
},
],
};
const result = aiInsightResponseSchema.safeParse(broken);
expect(result.success).toBe(false);
});
it("rejects a recommendation with missing severity enum", () => {
const broken = {
...validResponse,
recommendations: [
{
id: "rec-1",
text: "Walk more",
severity: "kinda-bad", // not in enum
metricSource: {
type: "activity",
timeRange: "last7days",
summary: "5,000 steps avg",
},
},
],
};
const result = aiInsightResponseSchema.safeParse(broken);
expect(result.success).toBe(false);
});
it("findUncitedRecommendations flags recommendations whose metricSource is not in citations", () => {
const orphan = {
...validResponse,
recommendations: [
validResponse.recommendations[0],
{
id: "rec-2",
text: "Reduce salt intake",
severity: "suggestion",
metricSource: {
type: "diet",
timeRange: "last30days",
summary: "no salt logging available",
},
},
],
};
const parsed = aiInsightResponseSchema.parse(orphan);
const missing = findUncitedRecommendations(parsed);
expect(missing).toHaveLength(1);
expect(missing[0].recommendationId).toBe("rec-2");
expect(missing[0].missing.type).toBe("diet");
});
});
describe("generateInsight wrapper", () => {
it("returns the parsed response on first-attempt success", async () => {
const provider = new MockAIProvider({
responses: JSON.stringify(validResponse),
});
const outcome = await generateInsight(provider, {
systemPrompt: "sys",
userPrompt: "user",
});
expect(outcome.attempts).toBe(1);
expect(outcome.retried).toBe(false);
expect(outcome.parsed.summary).toMatch(/blood pressure/);
expect(provider.callCount).toBe(1);
});
it("retries once with correction message and succeeds on second attempt", async () => {
const provider = new MockAIProvider({
responses: [
"definitely not json",
JSON.stringify(validResponse),
],
});
const outcome = await generateInsight(provider, {
systemPrompt: "sys",
userPrompt: "Original user prompt.",
});
expect(outcome.attempts).toBe(2);
expect(outcome.retried).toBe(true);
expect(provider.callCount).toBe(2);
// The retry call must include the original user prompt PLUS the
// corrective suffix referencing the schema.
const retryUserPrompt = provider.calls[1].userPrompt;
expect(retryUserPrompt).toContain("Original user prompt.");
expect(retryUserPrompt).toContain("did not satisfy");
expect(retryUserPrompt).toContain("metricSource");
});
it("throws InsightSchemaError(422) when both attempts fail", async () => {
const provider = new MockAIProvider({
responses: ["nope", "still nope"],
});
let caught: unknown;
try {
await generateInsight(provider, {
systemPrompt: "sys",
userPrompt: "user",
});
} catch (e) {
caught = e;
}
expect(caught).toBeInstanceOf(InsightSchemaError);
const err = caught as InsightSchemaError;
expect(err.httpStatus).toBe(422);
expect(err.attempts).toBe(2);
expect(provider.callCount).toBe(2);
});
it("retries when first attempt has uncited recommendations", async () => {
const orphanResponse = {
...validResponse,
recommendations: [
{
id: "rec-orphan",
text: "Cut sugar",
severity: "suggestion",
metricSource: {
type: "diet",
timeRange: "last7days",
summary: "no sugar tracking present",
},
},
],
citations: [], // empty — fails cross-check
};
const provider = new MockAIProvider({
responses: [JSON.stringify(orphanResponse), JSON.stringify(validResponse)],
});
const outcome = await generateInsight(provider, {
systemPrompt: "sys",
userPrompt: "u",
});
expect(outcome.attempts).toBe(2);
expect(outcome.retried).toBe(true);
expect(provider.calls[1].userPrompt).toContain(
"metricSources not in citations",
);
});
it("propagates provider-level errors without retrying", async () => {
const provider = new MockAIProvider({
rejectWith: Object.assign(new Error("OpenAI request failed (500)"), {
httpStatus: 500,
}),
});
await expect(
generateInsight(provider, { systemPrompt: "s", userPrompt: "u" }),
).rejects.toThrow("OpenAI request failed (500)");
// Provider only called once — wrapper does not retry on provider errors.
expect(provider.callCount).toBe(1);
});
});
+176
View File
@@ -0,0 +1,176 @@
import type { z } from "zod/v4";
import type { AIProvider, CompletionParams, CompletionResult } from "./types";
import {
aiInsightResponseSchema,
findUncitedRecommendations,
InsightSchemaError,
type AIInsightResponse,
} from "./schema";
/**
* Schema-enforced wrapper around `AIProvider.generateCompletion()`.
*
* Phase C1 (v1.4.15): Marc's mandate ("zero hallucinations, must
* ground on user data") means a free-form LLM response is no longer
* acceptable. This wrapper:
*
* 1. Calls the provider once.
* 2. Parses the response as JSON.
* 3. Validates it against `aiInsightResponseSchema` AND cross-checks
* that every recommendation's `metricSource` appears in
* `citations[]`.
* 4. On failure: retries ONCE, prepending a corrective system
* message that includes the violated zod issues.
* 5. On second failure: throws `InsightSchemaError` (httpStatus 422).
*
* The wrapper does not concern itself with provider-level retries
* (HTTP 5xx, 401 token-refresh, etc.) — each provider client handles
* those internally. This wrapper retries only on schema-violation.
*/
export interface GenerateInsightOutcome {
parsed: AIInsightResponse;
raw: CompletionResult;
/** Number of provider calls made (1 on success, 2 on retry). */
attempts: number;
/** True iff the retry attempt produced the parsed response. */
retried: boolean;
}
/**
* Build the corrective user-prompt suffix that explains to the model
* what went wrong on the first attempt. Includes the violated zod
* issues verbatim (truncated to 1KB) so the model can self-correct.
*/
function buildRetryCorrectionMessage(
reason: string,
details: string,
): string {
return `
Your previous response did not satisfy the required JSON schema.
Reason: ${reason}
Details: ${details.slice(0, 1024)}
You MUST return a JSON object matching this schema exactly. Do NOT
include prose, markdown fences, or commentary outside the JSON.
Required top-level fields:
- summary: string (2-3 sentences)
- recommendations: array of objects with
{ id: string, text: string, severity: "info" | "suggestion" | "important" | "urgent",
metricSource: { type: string, timeRange: string, summary: string, n?: number } }
- citations: array of objects with { type: string, timeRange: string, summary: string }
- warnings: array of objects with { topic: string, message: string, severity?: same enum }
Every recommendation's metricSource MUST also appear in citations[]
(same type + timeRange). If you cannot ground a recommendation in
user data from the snapshot you were given, OMIT it. An empty
recommendations[] is acceptable.
`;
}
interface TryOutcome {
ok: boolean;
parsed?: AIInsightResponse;
raw: CompletionResult;
reason?: string;
details?: string;
zodIssues?: z.ZodIssue[];
}
async function tryOnce(
provider: AIProvider,
params: CompletionParams,
): Promise<TryOutcome> {
const raw = await provider.generateCompletion(params);
// Parse JSON.
let json: unknown;
try {
json = JSON.parse(raw.content);
} catch (e) {
return {
ok: false,
raw,
reason: "Response was not valid JSON",
details: (e as Error).message,
};
}
// Schema validate.
const result = aiInsightResponseSchema.safeParse(json);
if (!result.success) {
return {
ok: false,
raw,
reason: "Response did not match the required JSON schema",
details: result.error.issues
.slice(0, 8)
.map((i) => `${i.path.join(".")}: ${i.message}`)
.join("; "),
zodIssues: result.error.issues,
};
}
// Cross-check citations.
const uncited = findUncitedRecommendations(result.data);
if (uncited.length > 0) {
const list = uncited
.slice(0, 4)
.map(
(u) =>
`${u.recommendationId} → ${u.missing.type}/${u.missing.timeRange}`,
)
.join(", ");
return {
ok: false,
raw,
reason: "Recommendations referenced metricSources not in citations[]",
details: `Missing citations: ${list}`,
};
}
return { ok: true, parsed: result.data, raw };
}
/**
* Run a schema-enforced insight generation. Up to 2 provider calls.
* Throws `InsightSchemaError` (httpStatus 422) if both attempts fail
* the schema; provider-level errors bubble untouched.
*/
export async function generateInsight(
provider: AIProvider,
params: CompletionParams,
): Promise<GenerateInsightOutcome> {
const first = await tryOnce(provider, params);
if (first.ok && first.parsed) {
return { parsed: first.parsed, raw: first.raw, attempts: 1, retried: false };
}
// Retry-once with corrective context appended to the user prompt.
const correction = buildRetryCorrectionMessage(
first.reason ?? "schema mismatch",
first.details ?? "",
);
const retryParams: CompletionParams = {
...params,
userPrompt: `${params.userPrompt}\n\n${correction}`,
};
const second = await tryOnce(provider, retryParams);
if (second.ok && second.parsed) {
return {
parsed: second.parsed,
raw: second.raw,
attempts: 2,
retried: true,
};
}
throw new InsightSchemaError(
second.reason ?? first.reason ?? "Schema validation failed twice",
{
issues: second.zodIssues ?? first.zodIssues,
attempts: 2,
},
);
}
+182
View File
@@ -0,0 +1,182 @@
import { z } from "zod/v4";
/**
* Strict response schema for v1.4.15 AI hardening (Phase C1).
*
* Marc, verbatim 2026-05-09: "Es darf null Halluzinationen haben und es
* muss sich halt irgendwie stützen auf medizinische Dinge." ("It must
* have zero hallucinations and must ground on medical facts.")
*
* The schema enforces:
* - A short natural-language summary.
* - A typed `recommendations[]` where each item is structured
* (`id`, `text`, `severity`) and MUST cite the user-data point that
* justified it via `metricSource`. An empty `metricSource.summary`
* fails parse — the model cannot fabricate recommendations without
* pointing to a number from the user's snapshot.
* - A `citations[]` array referencing the metrics the response drew
* on. May be empty when `recommendations[]` is empty (no data → no
* citation), but every recommendation's `metricSource` MUST also
* appear in `citations[]` (cross-check enforced by `validateInsightResponse`).
* - A `warnings[]` array for guideline-flagged values (high BP,
* critical pulse, etc.). Free-form for now; v1.4.16 adds severity
* enum + medical-reference grounding.
*
* Legacy rich fields (`classification`, `findings`, `correlations`,
* `dataQuality`, `disclaimer`) remain accepted via `.passthrough()` so
* cached payloads from v1.4.14 still hydrate the dashboard. The
* v1.4.16 roadmap (`docs/audit/v1416-ai-roadmap.md`) plans the full
* migration to the strict shape once all consumers move over.
*/
// Severity levels — generic for v1.4.15. v1.4.16 adds a clinical
// severity enum keyed to ESH/ESC and AHA guidelines.
export const recommendationSeveritySchema = z.enum([
"info",
"suggestion",
"important",
"urgent",
]);
/**
* Citation-from-data: every recommendation must point to a concrete
* snapshot field that justified it. Empty `summary` is rejected.
*/
export const metricSourceSchema = z.object({
/**
* Snapshot key — e.g. "bloodPressure", "weight", "pulse",
* "medications.compliance30". Free-form because the snapshot shape
* itself is rich; v1.4.16 narrows to a typed enum.
*/
type: z.string().min(1, "metricSource.type required"),
/**
* Time window the value covers — e.g. "last7days", "last30days",
* "allTime". v1.4.16 narrows this enum.
*/
timeRange: z.string().min(1, "metricSource.timeRange required"),
/**
* Human-readable summary of the data point — e.g. "avg 142/88 over
* 12 readings" or "compliance 0.71 / 30d". Empty value rejected so
* the model cannot fake a citation.
*/
summary: z.string().min(1, "metricSource.summary required (zero-hallucination)"),
/** Sample count behind the value, when known. */
n: z.number().int().nonnegative().optional(),
});
export type MetricSource = z.infer<typeof metricSourceSchema>;
export const aiRecommendationSchema = z.object({
/**
* Stable identifier for the recommendation within this response.
* Used by the future user-feedback loop ("was this helpful?",
* v1.4.16 roadmap). Format: short slug (`rec-1`, `bp-elevated`).
*/
id: z.string().min(1, "recommendation.id required"),
/** Human-readable recommendation text. */
text: z.string().min(1, "recommendation.text required"),
/**
* Mandatory citation — the data point that justified the
* recommendation. Closes the "ungrounded boilerplate" risk Marc
* called out: every recommendation must trace back to user data.
*/
metricSource: metricSourceSchema,
/** Severity — generic for v1.4.15, clinically-keyed in v1.4.16. */
severity: recommendationSeveritySchema,
});
export type AIRecommendation = z.infer<typeof aiRecommendationSchema>;
export const aiCitationSchema = z.object({
/** Snapshot key being cited. */
type: z.string().min(1, "citation.type required"),
/** Window of the value. */
timeRange: z.string().min(1, "citation.timeRange required"),
/** Short description of the data point. */
summary: z.string().min(1, "citation.summary required"),
});
export type AICitation = z.infer<typeof aiCitationSchema>;
export const aiWarningSchema = z.object({
/** Topic — "blood_pressure", "pulse", etc. */
topic: z.string().min(1, "warning.topic required"),
/** What's flagged. */
message: z.string().min(1, "warning.message required"),
/** Severity tag for downstream filtering. */
severity: recommendationSeveritySchema.optional(),
});
export type AIWarning = z.infer<typeof aiWarningSchema>;
/**
* Canonical strict response schema. New strict fields are required;
* legacy rich fields ride along through `.passthrough()` for back-
* compat with cached payloads and the existing dashboard renderer.
*/
export const aiInsightResponseSchema = z
.object({
/** 2-3 sentence overall summary in user-facing locale. */
summary: z.string().min(1, "summary required"),
/** Structured recommendations with mandatory citations. */
recommendations: z.array(aiRecommendationSchema),
/** Citations supporting any claim in summary or recommendations. */
citations: z.array(aiCitationSchema),
/** Guideline-flagged values (e.g. BP > 140/90). May be empty. */
warnings: z.array(aiWarningSchema),
})
.passthrough();
export type AIInsightResponse = z.infer<typeof aiInsightResponseSchema>;
/**
* Tagged error thrown by the schema-enforcement wrapper when even the
* retry attempt fails to produce a valid payload. The route catches
* this and surfaces a 422 to the client.
*/
export class InsightSchemaError extends Error {
readonly httpStatus = 422;
readonly issues: z.ZodIssue[] | null;
readonly attempts: number;
constructor(
message: string,
options: { issues?: z.ZodIssue[]; attempts: number },
) {
super(message);
this.name = "InsightSchemaError";
this.issues = options.issues ?? null;
this.attempts = options.attempts;
}
}
/**
* Cross-validation beyond zod: every recommendation's `metricSource`
* must also appear in `citations[]` (same `type` + `timeRange`).
* Returns the list of missing-citation issues; an empty list means
* the response is internally consistent.
*/
export function findUncitedRecommendations(
parsed: AIInsightResponse,
): Array<{ recommendationId: string; missing: { type: string; timeRange: string } }> {
const cited = new Set(
parsed.citations.map((c) => `${c.type}::${c.timeRange}`),
);
const missing: Array<{
recommendationId: string;
missing: { type: string; timeRange: string };
}> = [];
for (const rec of parsed.recommendations) {
const key = `${rec.metricSource.type}::${rec.metricSource.timeRange}`;
if (!cited.has(key)) {
missing.push({
recommendationId: rec.id,
missing: {
type: rec.metricSource.type,
timeRange: rec.metricSource.timeRange,
},
});
}
}
return missing;
}