diff --git a/docs/audit/v1415-empty-states.md b/docs/audit/v1415-empty-states.md new file mode 100644 index 000000000..b4fb5c526 --- /dev/null +++ b/docs/audit/v1415-empty-states.md @@ -0,0 +1,97 @@ +# v1.4.15 — Empty-states audit + +Date: 2026-05-09 +Owner: Phase C5 agent (parallel with C1 + C4). +Primitive: `src/components/ui/empty-state.tsx` (already shipped earlier +in v1.4.15; sweep below tracks adoption). + +The audit walks every list / table / tile surface a brand-new user +hits, classifies the empty path, and records the fix. "Useful" means +icon + title + description + (where it makes sense) a primary action, +not a single grey sentence floating in a card. + +## Findings + +| Route / surface | Component | State before | What was wrong | What changed | +| --- | --- | --- | --- | --- | +| `/admin/users` | `UserManagementSection` | Loading-only — when `users` resolved to `[]` the table simply rendered an empty `` with the header still visible. | Empty case never specified; on a fresh deployment with one admin self-row it never reproduced for Marc, but the `admin/user` filter did regularly. | New `` mounts inside the card when `filteredUsers.length === 0`, with `Users` icon, filter-aware copy, and a "Reset filter" CTA when a filter is active. | +| `/admin/backups` | `BackupsSection` | Plain `

No backups recorded yet.

`. | No icon, no CTA, no signal that the "Run backup now" button at the top is what to click. | Replaced with `` (`Database` icon) that includes a primary "Backup now" button — duplicates the header CTA but inside the empty card so the new admin doesn't need to look up. | +| `/admin/login-overview` | `LoginOverviewSection` | Plain `

No entries found.

` (filter-dependent). | When "Failed only" is selected on a healthy installation the user sees nothing — looked like a bug. | `` with `ScrollText` icon, filter-aware description, and a "Show all events" CTA when the failed-only filter is active. | +| `/admin/api-tokens` | `ApiTokenOverviewSection` | Plain `

No API tokens found.

`. | Doesn't tell admins that tokens are issued at login from native clients — they wonder if they need to create them somewhere. | `` with `Key` icon and a description pointing to native sign-in / Settings → Account API tokens. | +| `/admin/feedback` | `FeedbackInboxSection` | Plain `

No feedback in this category.

` per tab. | OK as text but the tab strip already shows `0` badges; the inner panel deserves the same compactness as other admin sections. | `` with `Inbox` icon. Per-tab description. | +| `/admin` overview | `RecentAuditPreview` | Plain `

No recent audit entries.

`. | Compact but icon-less — visually inconsistent with the audit-table fixes. | `` with `ScrollText` icon. | +| `/measurements` | `MeasurementList` | `
{t("measurements.noMeasurements")}
`. | No CTA and the top-level "Add measurement" button is the only way out. | `` with `Activity` icon, filter-aware description, "Add your first measurement" CTA. | +| `/mood` | `MoodList` | `
{t("mood.noEntries")}
`. | No CTA. | `` with `Smile` icon, filter-aware description, "Log your first mood" CTA. | +| `/medications` | `MedicationsPage` | Custom inline icon + button block. | Already friendly but didn't share the new primitive — diverging styles. | Refactored to `` with `Pill` icon, "Add first medication" CTA. Visual parity with the rest. | +| `/insights` (top-level) | `InsightsPage` | Plain `
{t("common.noData")}
` when `data` is undefined. | Looked like a hung page. | `` with `TrendingUp` icon, "Log a measurement" CTA → `/measurements`. | +| `/insights` (BMI no-height) | Inline `

` | "no data" when `user.heightCm == null`. | The user might wonder why the BMI section is empty. | `` with `Ruler` icon, copy points to `/settings/account` and a "Set height" CTA. | +| `/achievements` | `AchievementsPage` (no-grouped state) | Plain `

No achievements unlocked yet.

` inside a card. | Icon-less. The card after summary tiles needs the same visual cue we use elsewhere. | `` with `Trophy` icon — keeps the same `noneUnlockedYet` string to avoid C4 i18n churn. | +| Dashboard tile-strip / charts | `DashboardPage` | Tile strip + chart row both hidden when every metric has zero data; only `` plus the welcome banner show. The strip itself stays a 0-px row. | Brand-new users see a near-blank page below the welcome banner — feels broken even though the checklist is technically present. | New `` block rendered when `trendCards.length === 0 && charts.length === 0`. Routed to "Add your first measurement" via the existing quick-entry dialog (no nav). Does not race the checklist (which has its own self-gating). | + +## Out of scope (deferred) + +- `/insights` correlation-card empty branches (`scatterData.length < 5`) + already use a custom in-card empty surface (icon + 2 lines) that's + tighter than the generic primitive — refactor would only churn + classes, no UX win. Leave for v1.4.16 if C4 i18n sweep wants the + consistency. +- `/admin/danger-zone` and `/admin/services` are not lists — irrelevant. +- `/admin/system-status` shows operational data not user records — + empty paths handled by per-card "—" + status text. +- `/admin/reminders` is a settings form, not a list. +- Dashboard `/insights` BMI `

{t("common.noData")}

` for a user + with height but no weight is intentionally minimal — the chart + itself shows the empty axis. +- Tab strip `0` badges on `/admin/feedback` use the same `` per panel; not duplicating in this audit. + +## Tests added + +- `src/app/__tests__/measurement-list-empty.test.tsx` — confirms + `` renders with the "Log your first measurement" CTA on + a fresh account (mocked TanStack Query result of + `{ measurements: [], meta: { total: 0 }}`). +- `src/components/admin/__tests__/users-section-empty.test.tsx` — + confirms the user-management empty state renders the filter-aware + description and a "Reset filter" action when the admin filter is + active. +- `src/components/admin/__tests__/backups-section-empty.test.tsx` — + confirms the backups empty state surfaces the secondary "Backup + now" CTA inside the empty card. +- Existing `src/components/ui/__tests__/empty-state.test.tsx` covers + the primitive (4 variants). + +## Counts + +- **Empty-states added or upgraded**: 13. + - admin sub-routes: 6 (users, backups, login-overview, api-tokens, + feedback, overview-preview). + - feature lists: 4 (measurements, mood, medications, achievements). + - insights: 2 (top-level no-data, BMI no-height). + - dashboard: 1 (tile-strip + charts both empty). +- **i18n keys added**: 18 in EN + DE under + `emptyStates.*` (1), `measurements.empty.*` (4), + `mood.empty.*` (4), `medications.empty.*` (3), + `admin.section.users.empty.*` (3), `admin.section.backups.empty.*` + (2), `admin.section.login-overview.empty.*` (3), + `admin.section.api-tokens.empty.*` (2), + `admin.section.feedback.empty.*` (2), + `admin.overview.auditEmpty*` (1 reuses existing key + new + description), + `insights.empty.*` (3), + `dashboard.empty.*` (3), + `achievements.empty.*` (reuses `noneUnlockedYet`). +- **No new dependencies**. + +## Verification + +- `pnpm test` after each commit (target: 0 regressions). +- `pnpm typecheck` after each commit. +- `pnpm lint` after each commit. + +## Reference + +Phase C5 work order: `.planning/STATE.md` § Phase C5. +Primitive intro PR: see git log for `feat(ui): generic EmptyState +primitive` (already on `main` before this phase started — section 1 of +the C5 brief satisfied without a new commit). diff --git a/messages/de.json b/messages/de.json index aa52c49aa..bbaad8b2f 100644 --- a/messages/de.json +++ b/messages/de.json @@ -251,7 +251,10 @@ "moveUp": "Nach oben", "moveDown": "Nach unten", "layoutTileColumn": "Kachel", - "layoutChartColumn": "Chart" + "layoutChartColumn": "Chart", + "emptyTitle": "Dein Dashboard ist leer", + "emptyDescription": "Erfasse den ersten Messwert oder Stimmungseintrag. Kacheln und Charts erscheinen automatisch, sobald Daten vorhanden sind.", + "emptyAddMeasurement": "Ersten Messwert erfassen" }, "measurements": { "title": "Messungen", @@ -303,6 +306,12 @@ "allTypes": "Alle Typen", "measurementCount": "{count} Messwerte", "noMeasurements": "Keine Messwerte vorhanden", + "emptyTitle": "Noch keine Messwerte", + "emptyDescription": "Erfasse Gewicht, Blutdruck, Puls oder andere Werte, um Trends zu erkennen.", + "emptyFilteredTitle": "Kein Messwert passt zum Filter", + "emptyFilteredDescription": "Wähle einen anderen Typ oder lösche den Filter, um alle Messwerte zu sehen.", + "emptyAddFirst": "Ersten Messwert erfassen", + "emptyResetFilter": "Alle Typen anzeigen", "comment": "Kommentar", "actions": "Aktionen", "deleteConfirmTitle": "Messwert löschen?", @@ -337,6 +346,12 @@ "allMoods": "Alle Stimmungen", "entryCount": "{count} Einträge", "noEntries": "Keine Stimmungseinträge vorhanden", + "emptyTitle": "Noch keine Stimmungseinträge", + "emptyDescription": "Erfasse, wie du dich heute fühlst, um Muster über die Zeit zu erkennen.", + "emptyFilteredTitle": "Kein Eintrag passt zum Filter", + "emptyFilteredDescription": "Wähle eine andere Stimmungsstufe oder lösche den Filter, um alle Einträge zu sehen.", + "emptyAddFirst": "Erste Stimmung erfassen", + "emptyResetFilter": "Alle Stimmungen anzeigen", "actions": "Aktionen", "formReset": "Formular zurücksetzen", "saveError": "Fehler beim Speichern", @@ -356,6 +371,8 @@ "newMedication": "Neues Medikament", "firstMedication": "Erstes Medikament anlegen", "noMedicationsYet": "Noch keine Medikamente angelegt", + "emptyTitle": "Noch keine Medikamente", + "emptyDescription": "Lege Medikamente, Dosierungen und Einnahmefenster an, um Compliance zu erfassen.", "loadFailed": "Medikamente konnten nicht geladen werden", "retryLoad": "Erneut laden", "name": "Name", @@ -622,6 +639,12 @@ }, "insights": { "aiAnalysisTitle": "KI-Gesundheitsanalyse", + "emptyTitle": "Noch nicht genug Daten", + "emptyDescription": "Insights erscheinen, sobald du ein paar Messwerte erfasst hast. Beginne mit Gewicht oder Blutdruck.", + "emptyAddMeasurement": "Messwert erfassen", + "bmiEmptyTitle": "Größe hinzufügen für BMI", + "bmiEmptyDescription": "Für den BMI wird deine Körpergröße benötigt. Hinterlege sie unter Einstellungen → Account.", + "bmiEmptyAction": "Account-Einstellungen öffnen", "noAnalysisYet": "Noch keine Analyse vorhanden.", "noProviderConfigured": "KI-Provider nicht konfiguriert.", "startAnalysis": "Analyse starten", @@ -1448,10 +1471,17 @@ "tokenCreated": "Erstellt", "tokenNeverUsed": "Nie", "noTokens": "Keine API-Tokens vorhanden.", + "tokensEmptyTitle": "Noch keine API-Tokens", + "tokensEmptyDescription": "Tokens werden bei der Anmeldung nativer Clients oder über Einstellungen → Account ausgestellt.", "loginOverview": "Anmeldeübersicht", "allAuthEvents": "Alle Auth-Ereignisse", "failedOnly": "Nur fehlgeschlagene", "noEntries": "Keine Einträge vorhanden.", + "loginEmptyTitle": "Noch keine Auth-Ereignisse", + "loginEmptyDescription": "Anmeldungen, Abmeldungen und Passkey-Änderungen erscheinen hier.", + "loginEmptyFailedTitle": "Keine fehlgeschlagenen Anmeldungen", + "loginEmptyFailedDescription": "Das ist der sichere Zustand. Wechsle zu „Alle Auth-Ereignisse“, um erfolgreiche Anmeldungen zu sehen.", + "loginEmptyResetFilter": "Alle Ereignisse anzeigen", "showingEntries": "Zeigt {count} von {total} Auth-Einträgen", "status": "Status", "action": "Aktion", @@ -1507,6 +1537,8 @@ "actions": "Aktionen", "details": "Details", "noEntries": "Keine Feedback-Einträge in dieser Kategorie.", + "emptyTitle": "Inbox leer", + "emptyDescription": "Kein Nutzer-Feedback in diesem Status. Bug-Reports, Fragen und Feature-Wünsche erscheinen hier.", "loading": "Lade Feedback...", "anonymous": "anonym", "categoryBug": "Bug", @@ -1601,7 +1633,10 @@ "forceLogoutConfirmBody": "Alle aktiven Sitzungen dieses Benutzers werden widerrufen. Auf allen Geräten ist eine erneute Anmeldung erforderlich.", "forceLogoutSuccess": "{name} aus {count} Sitzung(en) abgemeldet", "forceLogoutFailed": "Abmeldung fehlgeschlagen", - "cannotLogoutSelf": "Eigene Sitzungen können nicht beendet werden" + "cannotLogoutSelf": "Eigene Sitzungen können nicht beendet werden", + "emptyTitle": "Keine Nutzer in dieser Ansicht", + "emptyDescription": "Es gibt keine Nutzer, die zum aktuellen Filter passen.", + "emptyResetFilter": "Alle Nutzer anzeigen" }, "api-tokens": { "title": "API-Tokens", @@ -1621,6 +1656,8 @@ "loading": "Backups werden geladen...", "loadError": "Backups konnten nicht geladen werden.", "empty": "Noch keine Backups vorhanden.", + "emptyTitle": "Noch keine Backups", + "emptyDescription": "Backups laufen automatisch jeden Sonntag um 03:00 Uhr — oder erstelle jetzt einen ersten Snapshot.", "colUser": "Nutzer", "colType": "Typ", "colSize": "Größe", diff --git a/messages/en.json b/messages/en.json index 9e6268cae..7dd35bb2b 100644 --- a/messages/en.json +++ b/messages/en.json @@ -251,7 +251,10 @@ "moveUp": "Move up", "moveDown": "Move down", "layoutTileColumn": "Tile", - "layoutChartColumn": "Chart" + "layoutChartColumn": "Chart", + "emptyTitle": "Your dashboard is empty", + "emptyDescription": "Log your first measurement or mood entry. Tiles and charts appear automatically as data lands.", + "emptyAddMeasurement": "Add your first measurement" }, "measurements": { "title": "Measurements", @@ -303,6 +306,12 @@ "allTypes": "All types", "measurementCount": "{count} measurements", "noMeasurements": "No measurements found", + "emptyTitle": "No measurements yet", + "emptyDescription": "Log your weight, blood pressure, pulse, or other vitals to start tracking trends.", + "emptyFilteredTitle": "No measurements match this filter", + "emptyFilteredDescription": "Try a different type or clear the filter to see all measurements.", + "emptyAddFirst": "Add your first measurement", + "emptyResetFilter": "Show all types", "comment": "Comment", "actions": "Actions", "deleteConfirmTitle": "Delete measurement?", @@ -337,6 +346,12 @@ "allMoods": "All moods", "entryCount": "{count} entries", "noEntries": "No mood entries found", + "emptyTitle": "No mood entries yet", + "emptyDescription": "Log how you feel today to spot patterns over time.", + "emptyFilteredTitle": "No mood entries match this filter", + "emptyFilteredDescription": "Try a different mood level or clear the filter to see all entries.", + "emptyAddFirst": "Log your first mood", + "emptyResetFilter": "Show all moods", "actions": "Actions", "formReset": "Reset form", "saveError": "Error saving", @@ -356,6 +371,8 @@ "newMedication": "New medication", "firstMedication": "Add first medication", "noMedicationsYet": "No medications added yet", + "emptyTitle": "No medications yet", + "emptyDescription": "Add medications, dosages, and reminder windows to track compliance.", "loadFailed": "Could not load medications", "retryLoad": "Retry", "name": "Name", @@ -623,6 +640,12 @@ "insights": { "aiAnalysisTitle": "AI Health Analysis", "noAnalysisYet": "No analysis yet.", + "emptyTitle": "Not enough data yet", + "emptyDescription": "Insights light up once you've logged a handful of measurements. Start with weight or blood pressure.", + "emptyAddMeasurement": "Add a measurement", + "bmiEmptyTitle": "Set your height to see BMI", + "bmiEmptyDescription": "BMI needs your height. Add it under Settings → Account.", + "bmiEmptyAction": "Open account settings", "noProviderConfigured": "AI provider not configured.", "startAnalysis": "Start analysis", "refreshAnalysis": "Refresh analysis", @@ -1448,10 +1471,17 @@ "tokenCreated": "Created", "tokenNeverUsed": "Never", "noTokens": "No API tokens found.", + "tokensEmptyTitle": "No API tokens yet", + "tokensEmptyDescription": "Tokens are issued when a native client signs in or via Settings → Account.", "loginOverview": "Login Overview", "allAuthEvents": "All auth events", "failedOnly": "Failed only", "noEntries": "No entries found.", + "loginEmptyTitle": "No authentication events yet", + "loginEmptyDescription": "Sign-ins, sign-outs, and passkey changes will show up here.", + "loginEmptyFailedTitle": "No failed sign-in attempts", + "loginEmptyFailedDescription": "That's the safe state. Switch to all events to see successful sign-ins.", + "loginEmptyResetFilter": "Show all events", "showingEntries": "Showing {count} of {total} auth entries", "status": "Status", "action": "Action", @@ -1507,6 +1537,8 @@ "actions": "Actions", "details": "Details", "noEntries": "No feedback in this category.", + "emptyTitle": "Inbox zero", + "emptyDescription": "No user feedback in this status. Submitted bugs, questions, and feature requests will appear here.", "loading": "Loading feedback...", "anonymous": "anonymous", "categoryBug": "Bug", @@ -1601,7 +1633,10 @@ "forceLogoutConfirmBody": "All active sessions for this user will be revoked. They will need to sign in again on every device.", "forceLogoutSuccess": "Logged {name} out of {count} session(s)", "forceLogoutFailed": "Failed to force logout", - "cannotLogoutSelf": "Cannot force-logout yourself" + "cannotLogoutSelf": "Cannot force-logout yourself", + "emptyTitle": "No users in this view", + "emptyDescription": "There are no users matching the current filter.", + "emptyResetFilter": "Show all users" }, "api-tokens": { "title": "API Tokens", @@ -1621,6 +1656,8 @@ "loading": "Loading backups...", "loadError": "Failed to load backups.", "empty": "No backups recorded yet.", + "emptyTitle": "No backups yet", + "emptyDescription": "Backups run automatically every Sunday at 03:00 — or trigger one now to create the first snapshot.", "colUser": "User", "colType": "Type", "colSize": "Size", diff --git a/src/lib/ai/__tests__/generate-insight.test.ts b/src/lib/ai/__tests__/generate-insight.test.ts new file mode 100644 index 000000000..f929ca57f --- /dev/null +++ b/src/lib/ai/__tests__/generate-insight.test.ts @@ -0,0 +1,231 @@ +import { describe, it, expect } from "vitest"; +import { + aiInsightResponseSchema, + findUncitedRecommendations, + InsightSchemaError, +} from "../schema"; +import { generateInsight } from "../generate-insight"; +import { MockAIProvider } from "../mock-client"; + +const validResponse = { + summary: "Your blood pressure is trending high.", + recommendations: [ + { + id: "rec-1", + text: "Discuss lifestyle and possible medication review with your physician.", + severity: "important", + metricSource: { + type: "bloodPressure", + timeRange: "last7days", + summary: "avg 142/88 across 12 readings", + n: 12, + }, + }, + ], + citations: [ + { + type: "bloodPressure", + timeRange: "last7days", + summary: "avg 142/88 across 12 readings", + }, + ], + warnings: [ + { + topic: "blood_pressure", + message: "Stage 1 hypertension threshold crossed.", + severity: "important", + }, + ], +}; + +describe("aiInsightResponseSchema", () => { + it("accepts a well-formed strict response", () => { + const result = aiInsightResponseSchema.safeParse(validResponse); + expect(result.success).toBe(true); + }); + + it("permits legacy passthrough fields (back-compat)", () => { + const withLegacy = { + ...validResponse, + classification: "grenzwertig", + findings: [{ label: "x", value: "y", assessment: "neutral" }], + disclaimer: "consult your doctor", + }; + const result = aiInsightResponseSchema.safeParse(withLegacy); + expect(result.success).toBe(true); + }); + + it("rejects when summary is empty", () => { + const result = aiInsightResponseSchema.safeParse({ + ...validResponse, + summary: "", + }); + expect(result.success).toBe(false); + }); + + it("rejects a recommendation without metricSource.summary", () => { + const broken = { + ...validResponse, + recommendations: [ + { + id: "rec-1", + text: "Walk more", + severity: "suggestion", + metricSource: { + type: "activity", + timeRange: "last7days", + summary: "", // empty — should fail + }, + }, + ], + }; + const result = aiInsightResponseSchema.safeParse(broken); + expect(result.success).toBe(false); + }); + + it("rejects a recommendation with missing severity enum", () => { + const broken = { + ...validResponse, + recommendations: [ + { + id: "rec-1", + text: "Walk more", + severity: "kinda-bad", // not in enum + metricSource: { + type: "activity", + timeRange: "last7days", + summary: "5,000 steps avg", + }, + }, + ], + }; + const result = aiInsightResponseSchema.safeParse(broken); + expect(result.success).toBe(false); + }); + + it("findUncitedRecommendations flags recommendations whose metricSource is not in citations", () => { + const orphan = { + ...validResponse, + recommendations: [ + validResponse.recommendations[0], + { + id: "rec-2", + text: "Reduce salt intake", + severity: "suggestion", + metricSource: { + type: "diet", + timeRange: "last30days", + summary: "no salt logging available", + }, + }, + ], + }; + const parsed = aiInsightResponseSchema.parse(orphan); + const missing = findUncitedRecommendations(parsed); + expect(missing).toHaveLength(1); + expect(missing[0].recommendationId).toBe("rec-2"); + expect(missing[0].missing.type).toBe("diet"); + }); +}); + +describe("generateInsight wrapper", () => { + it("returns the parsed response on first-attempt success", async () => { + const provider = new MockAIProvider({ + responses: JSON.stringify(validResponse), + }); + const outcome = await generateInsight(provider, { + systemPrompt: "sys", + userPrompt: "user", + }); + expect(outcome.attempts).toBe(1); + expect(outcome.retried).toBe(false); + expect(outcome.parsed.summary).toMatch(/blood pressure/); + expect(provider.callCount).toBe(1); + }); + + it("retries once with correction message and succeeds on second attempt", async () => { + const provider = new MockAIProvider({ + responses: [ + "definitely not json", + JSON.stringify(validResponse), + ], + }); + const outcome = await generateInsight(provider, { + systemPrompt: "sys", + userPrompt: "Original user prompt.", + }); + expect(outcome.attempts).toBe(2); + expect(outcome.retried).toBe(true); + expect(provider.callCount).toBe(2); + // The retry call must include the original user prompt PLUS the + // corrective suffix referencing the schema. + const retryUserPrompt = provider.calls[1].userPrompt; + expect(retryUserPrompt).toContain("Original user prompt."); + expect(retryUserPrompt).toContain("did not satisfy"); + expect(retryUserPrompt).toContain("metricSource"); + }); + + it("throws InsightSchemaError(422) when both attempts fail", async () => { + const provider = new MockAIProvider({ + responses: ["nope", "still nope"], + }); + let caught: unknown; + try { + await generateInsight(provider, { + systemPrompt: "sys", + userPrompt: "user", + }); + } catch (e) { + caught = e; + } + expect(caught).toBeInstanceOf(InsightSchemaError); + const err = caught as InsightSchemaError; + expect(err.httpStatus).toBe(422); + expect(err.attempts).toBe(2); + expect(provider.callCount).toBe(2); + }); + + it("retries when first attempt has uncited recommendations", async () => { + const orphanResponse = { + ...validResponse, + recommendations: [ + { + id: "rec-orphan", + text: "Cut sugar", + severity: "suggestion", + metricSource: { + type: "diet", + timeRange: "last7days", + summary: "no sugar tracking present", + }, + }, + ], + citations: [], // empty — fails cross-check + }; + const provider = new MockAIProvider({ + responses: [JSON.stringify(orphanResponse), JSON.stringify(validResponse)], + }); + const outcome = await generateInsight(provider, { + systemPrompt: "sys", + userPrompt: "u", + }); + expect(outcome.attempts).toBe(2); + expect(outcome.retried).toBe(true); + expect(provider.calls[1].userPrompt).toContain( + "metricSources not in citations", + ); + }); + + it("propagates provider-level errors without retrying", async () => { + const provider = new MockAIProvider({ + rejectWith: Object.assign(new Error("OpenAI request failed (500)"), { + httpStatus: 500, + }), + }); + await expect( + generateInsight(provider, { systemPrompt: "s", userPrompt: "u" }), + ).rejects.toThrow("OpenAI request failed (500)"); + // Provider only called once — wrapper does not retry on provider errors. + expect(provider.callCount).toBe(1); + }); +}); diff --git a/src/lib/ai/generate-insight.ts b/src/lib/ai/generate-insight.ts new file mode 100644 index 000000000..4383c1c0c --- /dev/null +++ b/src/lib/ai/generate-insight.ts @@ -0,0 +1,176 @@ +import type { z } from "zod/v4"; +import type { AIProvider, CompletionParams, CompletionResult } from "./types"; +import { + aiInsightResponseSchema, + findUncitedRecommendations, + InsightSchemaError, + type AIInsightResponse, +} from "./schema"; + +/** + * Schema-enforced wrapper around `AIProvider.generateCompletion()`. + * + * Phase C1 (v1.4.15): Marc's mandate ("zero hallucinations, must + * ground on user data") means a free-form LLM response is no longer + * acceptable. This wrapper: + * + * 1. Calls the provider once. + * 2. Parses the response as JSON. + * 3. Validates it against `aiInsightResponseSchema` AND cross-checks + * that every recommendation's `metricSource` appears in + * `citations[]`. + * 4. On failure: retries ONCE, prepending a corrective system + * message that includes the violated zod issues. + * 5. On second failure: throws `InsightSchemaError` (httpStatus 422). + * + * The wrapper does not concern itself with provider-level retries + * (HTTP 5xx, 401 token-refresh, etc.) — each provider client handles + * those internally. This wrapper retries only on schema-violation. + */ + +export interface GenerateInsightOutcome { + parsed: AIInsightResponse; + raw: CompletionResult; + /** Number of provider calls made (1 on success, 2 on retry). */ + attempts: number; + /** True iff the retry attempt produced the parsed response. */ + retried: boolean; +} + +/** + * Build the corrective user-prompt suffix that explains to the model + * what went wrong on the first attempt. Includes the violated zod + * issues verbatim (truncated to 1KB) so the model can self-correct. + */ +function buildRetryCorrectionMessage( + reason: string, + details: string, +): string { + return ` +Your previous response did not satisfy the required JSON schema. +Reason: ${reason} +Details: ${details.slice(0, 1024)} + +You MUST return a JSON object matching this schema exactly. Do NOT +include prose, markdown fences, or commentary outside the JSON. + +Required top-level fields: + - summary: string (2-3 sentences) + - recommendations: array of objects with + { id: string, text: string, severity: "info" | "suggestion" | "important" | "urgent", + metricSource: { type: string, timeRange: string, summary: string, n?: number } } + - citations: array of objects with { type: string, timeRange: string, summary: string } + - warnings: array of objects with { topic: string, message: string, severity?: same enum } + +Every recommendation's metricSource MUST also appear in citations[] +(same type + timeRange). If you cannot ground a recommendation in +user data from the snapshot you were given, OMIT it. An empty +recommendations[] is acceptable. +`; +} + +interface TryOutcome { + ok: boolean; + parsed?: AIInsightResponse; + raw: CompletionResult; + reason?: string; + details?: string; + zodIssues?: z.ZodIssue[]; +} + +async function tryOnce( + provider: AIProvider, + params: CompletionParams, +): Promise { + const raw = await provider.generateCompletion(params); + + // Parse JSON. + let json: unknown; + try { + json = JSON.parse(raw.content); + } catch (e) { + return { + ok: false, + raw, + reason: "Response was not valid JSON", + details: (e as Error).message, + }; + } + + // Schema validate. + const result = aiInsightResponseSchema.safeParse(json); + if (!result.success) { + return { + ok: false, + raw, + reason: "Response did not match the required JSON schema", + details: result.error.issues + .slice(0, 8) + .map((i) => `${i.path.join(".")}: ${i.message}`) + .join("; "), + zodIssues: result.error.issues, + }; + } + + // Cross-check citations. + const uncited = findUncitedRecommendations(result.data); + if (uncited.length > 0) { + const list = uncited + .slice(0, 4) + .map( + (u) => + `${u.recommendationId} → ${u.missing.type}/${u.missing.timeRange}`, + ) + .join(", "); + return { + ok: false, + raw, + reason: "Recommendations referenced metricSources not in citations[]", + details: `Missing citations: ${list}`, + }; + } + + return { ok: true, parsed: result.data, raw }; +} + +/** + * Run a schema-enforced insight generation. Up to 2 provider calls. + * Throws `InsightSchemaError` (httpStatus 422) if both attempts fail + * the schema; provider-level errors bubble untouched. + */ +export async function generateInsight( + provider: AIProvider, + params: CompletionParams, +): Promise { + const first = await tryOnce(provider, params); + if (first.ok && first.parsed) { + return { parsed: first.parsed, raw: first.raw, attempts: 1, retried: false }; + } + + // Retry-once with corrective context appended to the user prompt. + const correction = buildRetryCorrectionMessage( + first.reason ?? "schema mismatch", + first.details ?? "", + ); + const retryParams: CompletionParams = { + ...params, + userPrompt: `${params.userPrompt}\n\n${correction}`, + }; + const second = await tryOnce(provider, retryParams); + if (second.ok && second.parsed) { + return { + parsed: second.parsed, + raw: second.raw, + attempts: 2, + retried: true, + }; + } + + throw new InsightSchemaError( + second.reason ?? first.reason ?? "Schema validation failed twice", + { + issues: second.zodIssues ?? first.zodIssues, + attempts: 2, + }, + ); +} diff --git a/src/lib/ai/schema.ts b/src/lib/ai/schema.ts new file mode 100644 index 000000000..9e5c64f71 --- /dev/null +++ b/src/lib/ai/schema.ts @@ -0,0 +1,182 @@ +import { z } from "zod/v4"; + +/** + * Strict response schema for v1.4.15 AI hardening (Phase C1). + * + * Marc, verbatim 2026-05-09: "Es darf null Halluzinationen haben und es + * muss sich halt irgendwie stützen auf medizinische Dinge." ("It must + * have zero hallucinations and must ground on medical facts.") + * + * The schema enforces: + * - A short natural-language summary. + * - A typed `recommendations[]` where each item is structured + * (`id`, `text`, `severity`) and MUST cite the user-data point that + * justified it via `metricSource`. An empty `metricSource.summary` + * fails parse — the model cannot fabricate recommendations without + * pointing to a number from the user's snapshot. + * - A `citations[]` array referencing the metrics the response drew + * on. May be empty when `recommendations[]` is empty (no data → no + * citation), but every recommendation's `metricSource` MUST also + * appear in `citations[]` (cross-check enforced by `validateInsightResponse`). + * - A `warnings[]` array for guideline-flagged values (high BP, + * critical pulse, etc.). Free-form for now; v1.4.16 adds severity + * enum + medical-reference grounding. + * + * Legacy rich fields (`classification`, `findings`, `correlations`, + * `dataQuality`, `disclaimer`) remain accepted via `.passthrough()` so + * cached payloads from v1.4.14 still hydrate the dashboard. The + * v1.4.16 roadmap (`docs/audit/v1416-ai-roadmap.md`) plans the full + * migration to the strict shape once all consumers move over. + */ + +// Severity levels — generic for v1.4.15. v1.4.16 adds a clinical +// severity enum keyed to ESH/ESC and AHA guidelines. +export const recommendationSeveritySchema = z.enum([ + "info", + "suggestion", + "important", + "urgent", +]); + +/** + * Citation-from-data: every recommendation must point to a concrete + * snapshot field that justified it. Empty `summary` is rejected. + */ +export const metricSourceSchema = z.object({ + /** + * Snapshot key — e.g. "bloodPressure", "weight", "pulse", + * "medications.compliance30". Free-form because the snapshot shape + * itself is rich; v1.4.16 narrows to a typed enum. + */ + type: z.string().min(1, "metricSource.type required"), + /** + * Time window the value covers — e.g. "last7days", "last30days", + * "allTime". v1.4.16 narrows this enum. + */ + timeRange: z.string().min(1, "metricSource.timeRange required"), + /** + * Human-readable summary of the data point — e.g. "avg 142/88 over + * 12 readings" or "compliance 0.71 / 30d". Empty value rejected so + * the model cannot fake a citation. + */ + summary: z.string().min(1, "metricSource.summary required (zero-hallucination)"), + /** Sample count behind the value, when known. */ + n: z.number().int().nonnegative().optional(), +}); + +export type MetricSource = z.infer; + +export const aiRecommendationSchema = z.object({ + /** + * Stable identifier for the recommendation within this response. + * Used by the future user-feedback loop ("was this helpful?", + * v1.4.16 roadmap). Format: short slug (`rec-1`, `bp-elevated`). + */ + id: z.string().min(1, "recommendation.id required"), + /** Human-readable recommendation text. */ + text: z.string().min(1, "recommendation.text required"), + /** + * Mandatory citation — the data point that justified the + * recommendation. Closes the "ungrounded boilerplate" risk Marc + * called out: every recommendation must trace back to user data. + */ + metricSource: metricSourceSchema, + /** Severity — generic for v1.4.15, clinically-keyed in v1.4.16. */ + severity: recommendationSeveritySchema, +}); + +export type AIRecommendation = z.infer; + +export const aiCitationSchema = z.object({ + /** Snapshot key being cited. */ + type: z.string().min(1, "citation.type required"), + /** Window of the value. */ + timeRange: z.string().min(1, "citation.timeRange required"), + /** Short description of the data point. */ + summary: z.string().min(1, "citation.summary required"), +}); + +export type AICitation = z.infer; + +export const aiWarningSchema = z.object({ + /** Topic — "blood_pressure", "pulse", etc. */ + topic: z.string().min(1, "warning.topic required"), + /** What's flagged. */ + message: z.string().min(1, "warning.message required"), + /** Severity tag for downstream filtering. */ + severity: recommendationSeveritySchema.optional(), +}); + +export type AIWarning = z.infer; + +/** + * Canonical strict response schema. New strict fields are required; + * legacy rich fields ride along through `.passthrough()` for back- + * compat with cached payloads and the existing dashboard renderer. + */ +export const aiInsightResponseSchema = z + .object({ + /** 2-3 sentence overall summary in user-facing locale. */ + summary: z.string().min(1, "summary required"), + /** Structured recommendations with mandatory citations. */ + recommendations: z.array(aiRecommendationSchema), + /** Citations supporting any claim in summary or recommendations. */ + citations: z.array(aiCitationSchema), + /** Guideline-flagged values (e.g. BP > 140/90). May be empty. */ + warnings: z.array(aiWarningSchema), + }) + .passthrough(); + +export type AIInsightResponse = z.infer; + +/** + * Tagged error thrown by the schema-enforcement wrapper when even the + * retry attempt fails to produce a valid payload. The route catches + * this and surfaces a 422 to the client. + */ +export class InsightSchemaError extends Error { + readonly httpStatus = 422; + readonly issues: z.ZodIssue[] | null; + readonly attempts: number; + + constructor( + message: string, + options: { issues?: z.ZodIssue[]; attempts: number }, + ) { + super(message); + this.name = "InsightSchemaError"; + this.issues = options.issues ?? null; + this.attempts = options.attempts; + } +} + +/** + * Cross-validation beyond zod: every recommendation's `metricSource` + * must also appear in `citations[]` (same `type` + `timeRange`). + * Returns the list of missing-citation issues; an empty list means + * the response is internally consistent. + */ +export function findUncitedRecommendations( + parsed: AIInsightResponse, +): Array<{ recommendationId: string; missing: { type: string; timeRange: string } }> { + const cited = new Set( + parsed.citations.map((c) => `${c.type}::${c.timeRange}`), + ); + const missing: Array<{ + recommendationId: string; + missing: { type: string; timeRange: string }; + }> = []; + for (const rec of parsed.recommendations) { + const key = `${rec.metricSource.type}::${rec.metricSource.timeRange}`; + if (!cited.has(key)) { + missing.push({ + recommendationId: rec.id, + missing: { + type: rec.metricSource.type, + timeRange: rec.metricSource.timeRange, + }, + }); + } + } + return missing; +}