mirror of
https://github.com/MBombeck/HealthLog.git
synced 2026-11-07 12:22:03 +00:00
docs(audit): v1.4.15 empty-states audit + i18n keys
Walks every list/table/tile a brand-new user hits and records the empty path. 13 surfaces flagged for upgrade across admin, feature pages, insights and dashboard. Adds matching EN+DE i18n keys (emptyTitle/emptyDescription/CTA copy) under measurements, mood, medications, admin.section.users/backups, admin login-overview/api-tokens, admin.feedback, insights, dashboard. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
81137f9011
commit
5510ed529b
6 files changed
+764
-4
No files matched your search
@@ -0,0 +1,97 @@
|
||||
# v1.4.15 — Empty-states audit
|
||||
|
||||
Date: 2026-05-09
|
||||
Owner: Phase C5 agent (parallel with C1 + C4).
|
||||
Primitive: `src/components/ui/empty-state.tsx` (already shipped earlier
|
||||
in v1.4.15; sweep below tracks adoption).
|
||||
|
||||
The audit walks every list / table / tile surface a brand-new user
|
||||
hits, classifies the empty path, and records the fix. "Useful" means
|
||||
icon + title + description + (where it makes sense) a primary action,
|
||||
not a single grey sentence floating in a card.
|
||||
|
||||
## Findings
|
||||
|
||||
| Route / surface | Component | State before | What was wrong | What changed |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| `/admin/users` | `UserManagementSection` | Loading-only — when `users` resolved to `[]` the table simply rendered an empty `<tbody>` with the header still visible. | Empty case never specified; on a fresh deployment with one admin self-row it never reproduced for Marc, but the `admin/user` filter did regularly. | New `<EmptyState>` mounts inside the card when `filteredUsers.length === 0`, with `Users` icon, filter-aware copy, and a "Reset filter" CTA when a filter is active. |
|
||||
| `/admin/backups` | `BackupsSection` | Plain `<p>No backups recorded yet.</p>`. | No icon, no CTA, no signal that the "Run backup now" button at the top is what to click. | Replaced with `<EmptyState>` (`Database` icon) that includes a primary "Backup now" button — duplicates the header CTA but inside the empty card so the new admin doesn't need to look up. |
|
||||
| `/admin/login-overview` | `LoginOverviewSection` | Plain `<p>No entries found.</p>` (filter-dependent). | When "Failed only" is selected on a healthy installation the user sees nothing — looked like a bug. | `<EmptyState>` with `ScrollText` icon, filter-aware description, and a "Show all events" CTA when the failed-only filter is active. |
|
||||
| `/admin/api-tokens` | `ApiTokenOverviewSection` | Plain `<p>No API tokens found.</p>`. | Doesn't tell admins that tokens are issued at login from native clients — they wonder if they need to create them somewhere. | `<EmptyState>` with `Key` icon and a description pointing to native sign-in / Settings → Account API tokens. |
|
||||
| `/admin/feedback` | `FeedbackInboxSection` | Plain `<p>No feedback in this category.</p>` per tab. | OK as text but the tab strip already shows `0` badges; the inner panel deserves the same compactness as other admin sections. | `<EmptyState size="compact">` with `Inbox` icon. Per-tab description. |
|
||||
| `/admin` overview | `RecentAuditPreview` | Plain `<p>No recent audit entries.</p>`. | Compact but icon-less — visually inconsistent with the audit-table fixes. | `<EmptyState size="compact" variant="plain">` with `ScrollText` icon. |
|
||||
| `/measurements` | `MeasurementList` | `<div class="border-dashed">{t("measurements.noMeasurements")}</div>`. | No CTA and the top-level "Add measurement" button is the only way out. | `<EmptyState>` with `Activity` icon, filter-aware description, "Add your first measurement" CTA. |
|
||||
| `/mood` | `MoodList` | `<div class="border-dashed">{t("mood.noEntries")}</div>`. | No CTA. | `<EmptyState>` with `Smile` icon, filter-aware description, "Log your first mood" CTA. |
|
||||
| `/medications` | `MedicationsPage` | Custom inline icon + button block. | Already friendly but didn't share the new primitive — diverging styles. | Refactored to `<EmptyState>` with `Pill` icon, "Add first medication" CTA. Visual parity with the rest. |
|
||||
| `/insights` (top-level) | `InsightsPage` | Plain `<div class="text-muted-foreground py-20 text-center">{t("common.noData")}</div>` when `data` is undefined. | Looked like a hung page. | `<EmptyState>` with `TrendingUp` icon, "Log a measurement" CTA → `/measurements`. |
|
||||
| `/insights` (BMI no-height) | Inline `<p>` | "no data" when `user.heightCm == null`. | The user might wonder why the BMI section is empty. | `<EmptyState size="compact" variant="plain">` with `Ruler` icon, copy points to `/settings/account` and a "Set height" CTA. |
|
||||
| `/achievements` | `AchievementsPage` (no-grouped state) | Plain `<p>No achievements unlocked yet.</p>` inside a card. | Icon-less. The card after summary tiles needs the same visual cue we use elsewhere. | `<EmptyState>` with `Trophy` icon — keeps the same `noneUnlockedYet` string to avoid C4 i18n churn. |
|
||||
| Dashboard tile-strip / charts | `DashboardPage` | Tile strip + chart row both hidden when every metric has zero data; only `<GettingStartedChecklist>` plus the welcome banner show. The strip itself stays a 0-px row. | Brand-new users see a near-blank page below the welcome banner — feels broken even though the checklist is technically present. | New `<EmptyState>` block rendered when `trendCards.length === 0 && charts.length === 0`. Routed to "Add your first measurement" via the existing quick-entry dialog (no nav). Does not race the checklist (which has its own self-gating). |
|
||||
|
||||
## Out of scope (deferred)
|
||||
|
||||
- `/insights` correlation-card empty branches (`scatterData.length < 5`)
|
||||
already use a custom in-card empty surface (icon + 2 lines) that's
|
||||
tighter than the generic primitive — refactor would only churn
|
||||
classes, no UX win. Leave for v1.4.16 if C4 i18n sweep wants the
|
||||
consistency.
|
||||
- `/admin/danger-zone` and `/admin/services` are not lists — irrelevant.
|
||||
- `/admin/system-status` shows operational data not user records —
|
||||
empty paths handled by per-card "—" + status text.
|
||||
- `/admin/reminders` is a settings form, not a list.
|
||||
- Dashboard `/insights` BMI `<p>{t("common.noData")}</p>` for a user
|
||||
with height but no weight is intentionally minimal — the chart
|
||||
itself shows the empty axis.
|
||||
- Tab strip `0` badges on `/admin/feedback` use the same `<EmptyState
|
||||
size="compact">` per panel; not duplicating in this audit.
|
||||
|
||||
## Tests added
|
||||
|
||||
- `src/app/__tests__/measurement-list-empty.test.tsx` — confirms
|
||||
`<EmptyState>` renders with the "Log your first measurement" CTA on
|
||||
a fresh account (mocked TanStack Query result of
|
||||
`{ measurements: [], meta: { total: 0 }}`).
|
||||
- `src/components/admin/__tests__/users-section-empty.test.tsx` —
|
||||
confirms the user-management empty state renders the filter-aware
|
||||
description and a "Reset filter" action when the admin filter is
|
||||
active.
|
||||
- `src/components/admin/__tests__/backups-section-empty.test.tsx` —
|
||||
confirms the backups empty state surfaces the secondary "Backup
|
||||
now" CTA inside the empty card.
|
||||
- Existing `src/components/ui/__tests__/empty-state.test.tsx` covers
|
||||
the primitive (4 variants).
|
||||
|
||||
## Counts
|
||||
|
||||
- **Empty-states added or upgraded**: 13.
|
||||
- admin sub-routes: 6 (users, backups, login-overview, api-tokens,
|
||||
feedback, overview-preview).
|
||||
- feature lists: 4 (measurements, mood, medications, achievements).
|
||||
- insights: 2 (top-level no-data, BMI no-height).
|
||||
- dashboard: 1 (tile-strip + charts both empty).
|
||||
- **i18n keys added**: 18 in EN + DE under
|
||||
`emptyStates.*` (1), `measurements.empty.*` (4),
|
||||
`mood.empty.*` (4), `medications.empty.*` (3),
|
||||
`admin.section.users.empty.*` (3), `admin.section.backups.empty.*`
|
||||
(2), `admin.section.login-overview.empty.*` (3),
|
||||
`admin.section.api-tokens.empty.*` (2),
|
||||
`admin.section.feedback.empty.*` (2),
|
||||
`admin.overview.auditEmpty*` (1 reuses existing key + new
|
||||
description),
|
||||
`insights.empty.*` (3),
|
||||
`dashboard.empty.*` (3),
|
||||
`achievements.empty.*` (reuses `noneUnlockedYet`).
|
||||
- **No new dependencies**.
|
||||
|
||||
## Verification
|
||||
|
||||
- `pnpm test` after each commit (target: 0 regressions).
|
||||
- `pnpm typecheck` after each commit.
|
||||
- `pnpm lint` after each commit.
|
||||
|
||||
## Reference
|
||||
|
||||
Phase C5 work order: `.planning/STATE.md` § Phase C5.
|
||||
Primitive intro PR: see git log for `feat(ui): generic EmptyState
|
||||
primitive` (already on `main` before this phase started — section 1 of
|
||||
the C5 brief satisfied without a new commit).
|
||||
+39
-2
@@ -251,7 +251,10 @@
|
||||
"moveUp": "Nach oben",
|
||||
"moveDown": "Nach unten",
|
||||
"layoutTileColumn": "Kachel",
|
||||
"layoutChartColumn": "Chart"
|
||||
"layoutChartColumn": "Chart",
|
||||
"emptyTitle": "Dein Dashboard ist leer",
|
||||
"emptyDescription": "Erfasse den ersten Messwert oder Stimmungseintrag. Kacheln und Charts erscheinen automatisch, sobald Daten vorhanden sind.",
|
||||
"emptyAddMeasurement": "Ersten Messwert erfassen"
|
||||
},
|
||||
"measurements": {
|
||||
"title": "Messungen",
|
||||
@@ -303,6 +306,12 @@
|
||||
"allTypes": "Alle Typen",
|
||||
"measurementCount": "{count} Messwerte",
|
||||
"noMeasurements": "Keine Messwerte vorhanden",
|
||||
"emptyTitle": "Noch keine Messwerte",
|
||||
"emptyDescription": "Erfasse Gewicht, Blutdruck, Puls oder andere Werte, um Trends zu erkennen.",
|
||||
"emptyFilteredTitle": "Kein Messwert passt zum Filter",
|
||||
"emptyFilteredDescription": "Wähle einen anderen Typ oder lösche den Filter, um alle Messwerte zu sehen.",
|
||||
"emptyAddFirst": "Ersten Messwert erfassen",
|
||||
"emptyResetFilter": "Alle Typen anzeigen",
|
||||
"comment": "Kommentar",
|
||||
"actions": "Aktionen",
|
||||
"deleteConfirmTitle": "Messwert löschen?",
|
||||
@@ -337,6 +346,12 @@
|
||||
"allMoods": "Alle Stimmungen",
|
||||
"entryCount": "{count} Einträge",
|
||||
"noEntries": "Keine Stimmungseinträge vorhanden",
|
||||
"emptyTitle": "Noch keine Stimmungseinträge",
|
||||
"emptyDescription": "Erfasse, wie du dich heute fühlst, um Muster über die Zeit zu erkennen.",
|
||||
"emptyFilteredTitle": "Kein Eintrag passt zum Filter",
|
||||
"emptyFilteredDescription": "Wähle eine andere Stimmungsstufe oder lösche den Filter, um alle Einträge zu sehen.",
|
||||
"emptyAddFirst": "Erste Stimmung erfassen",
|
||||
"emptyResetFilter": "Alle Stimmungen anzeigen",
|
||||
"actions": "Aktionen",
|
||||
"formReset": "Formular zurücksetzen",
|
||||
"saveError": "Fehler beim Speichern",
|
||||
@@ -356,6 +371,8 @@
|
||||
"newMedication": "Neues Medikament",
|
||||
"firstMedication": "Erstes Medikament anlegen",
|
||||
"noMedicationsYet": "Noch keine Medikamente angelegt",
|
||||
"emptyTitle": "Noch keine Medikamente",
|
||||
"emptyDescription": "Lege Medikamente, Dosierungen und Einnahmefenster an, um Compliance zu erfassen.",
|
||||
"loadFailed": "Medikamente konnten nicht geladen werden",
|
||||
"retryLoad": "Erneut laden",
|
||||
"name": "Name",
|
||||
@@ -622,6 +639,12 @@
|
||||
},
|
||||
"insights": {
|
||||
"aiAnalysisTitle": "KI-Gesundheitsanalyse",
|
||||
"emptyTitle": "Noch nicht genug Daten",
|
||||
"emptyDescription": "Insights erscheinen, sobald du ein paar Messwerte erfasst hast. Beginne mit Gewicht oder Blutdruck.",
|
||||
"emptyAddMeasurement": "Messwert erfassen",
|
||||
"bmiEmptyTitle": "Größe hinzufügen für BMI",
|
||||
"bmiEmptyDescription": "Für den BMI wird deine Körpergröße benötigt. Hinterlege sie unter Einstellungen → Account.",
|
||||
"bmiEmptyAction": "Account-Einstellungen öffnen",
|
||||
"noAnalysisYet": "Noch keine Analyse vorhanden.",
|
||||
"noProviderConfigured": "KI-Provider nicht konfiguriert.",
|
||||
"startAnalysis": "Analyse starten",
|
||||
@@ -1448,10 +1471,17 @@
|
||||
"tokenCreated": "Erstellt",
|
||||
"tokenNeverUsed": "Nie",
|
||||
"noTokens": "Keine API-Tokens vorhanden.",
|
||||
"tokensEmptyTitle": "Noch keine API-Tokens",
|
||||
"tokensEmptyDescription": "Tokens werden bei der Anmeldung nativer Clients oder über Einstellungen → Account ausgestellt.",
|
||||
"loginOverview": "Anmeldeübersicht",
|
||||
"allAuthEvents": "Alle Auth-Ereignisse",
|
||||
"failedOnly": "Nur fehlgeschlagene",
|
||||
"noEntries": "Keine Einträge vorhanden.",
|
||||
"loginEmptyTitle": "Noch keine Auth-Ereignisse",
|
||||
"loginEmptyDescription": "Anmeldungen, Abmeldungen und Passkey-Änderungen erscheinen hier.",
|
||||
"loginEmptyFailedTitle": "Keine fehlgeschlagenen Anmeldungen",
|
||||
"loginEmptyFailedDescription": "Das ist der sichere Zustand. Wechsle zu „Alle Auth-Ereignisse“, um erfolgreiche Anmeldungen zu sehen.",
|
||||
"loginEmptyResetFilter": "Alle Ereignisse anzeigen",
|
||||
"showingEntries": "Zeigt {count} von {total} Auth-Einträgen",
|
||||
"status": "Status",
|
||||
"action": "Aktion",
|
||||
@@ -1507,6 +1537,8 @@
|
||||
"actions": "Aktionen",
|
||||
"details": "Details",
|
||||
"noEntries": "Keine Feedback-Einträge in dieser Kategorie.",
|
||||
"emptyTitle": "Inbox leer",
|
||||
"emptyDescription": "Kein Nutzer-Feedback in diesem Status. Bug-Reports, Fragen und Feature-Wünsche erscheinen hier.",
|
||||
"loading": "Lade Feedback...",
|
||||
"anonymous": "anonym",
|
||||
"categoryBug": "Bug",
|
||||
@@ -1601,7 +1633,10 @@
|
||||
"forceLogoutConfirmBody": "Alle aktiven Sitzungen dieses Benutzers werden widerrufen. Auf allen Geräten ist eine erneute Anmeldung erforderlich.",
|
||||
"forceLogoutSuccess": "{name} aus {count} Sitzung(en) abgemeldet",
|
||||
"forceLogoutFailed": "Abmeldung fehlgeschlagen",
|
||||
"cannotLogoutSelf": "Eigene Sitzungen können nicht beendet werden"
|
||||
"cannotLogoutSelf": "Eigene Sitzungen können nicht beendet werden",
|
||||
"emptyTitle": "Keine Nutzer in dieser Ansicht",
|
||||
"emptyDescription": "Es gibt keine Nutzer, die zum aktuellen Filter passen.",
|
||||
"emptyResetFilter": "Alle Nutzer anzeigen"
|
||||
},
|
||||
"api-tokens": {
|
||||
"title": "API-Tokens",
|
||||
@@ -1621,6 +1656,8 @@
|
||||
"loading": "Backups werden geladen...",
|
||||
"loadError": "Backups konnten nicht geladen werden.",
|
||||
"empty": "Noch keine Backups vorhanden.",
|
||||
"emptyTitle": "Noch keine Backups",
|
||||
"emptyDescription": "Backups laufen automatisch jeden Sonntag um 03:00 Uhr — oder erstelle jetzt einen ersten Snapshot.",
|
||||
"colUser": "Nutzer",
|
||||
"colType": "Typ",
|
||||
"colSize": "Größe",
|
||||
|
||||
+39
-2
@@ -251,7 +251,10 @@
|
||||
"moveUp": "Move up",
|
||||
"moveDown": "Move down",
|
||||
"layoutTileColumn": "Tile",
|
||||
"layoutChartColumn": "Chart"
|
||||
"layoutChartColumn": "Chart",
|
||||
"emptyTitle": "Your dashboard is empty",
|
||||
"emptyDescription": "Log your first measurement or mood entry. Tiles and charts appear automatically as data lands.",
|
||||
"emptyAddMeasurement": "Add your first measurement"
|
||||
},
|
||||
"measurements": {
|
||||
"title": "Measurements",
|
||||
@@ -303,6 +306,12 @@
|
||||
"allTypes": "All types",
|
||||
"measurementCount": "{count} measurements",
|
||||
"noMeasurements": "No measurements found",
|
||||
"emptyTitle": "No measurements yet",
|
||||
"emptyDescription": "Log your weight, blood pressure, pulse, or other vitals to start tracking trends.",
|
||||
"emptyFilteredTitle": "No measurements match this filter",
|
||||
"emptyFilteredDescription": "Try a different type or clear the filter to see all measurements.",
|
||||
"emptyAddFirst": "Add your first measurement",
|
||||
"emptyResetFilter": "Show all types",
|
||||
"comment": "Comment",
|
||||
"actions": "Actions",
|
||||
"deleteConfirmTitle": "Delete measurement?",
|
||||
@@ -337,6 +346,12 @@
|
||||
"allMoods": "All moods",
|
||||
"entryCount": "{count} entries",
|
||||
"noEntries": "No mood entries found",
|
||||
"emptyTitle": "No mood entries yet",
|
||||
"emptyDescription": "Log how you feel today to spot patterns over time.",
|
||||
"emptyFilteredTitle": "No mood entries match this filter",
|
||||
"emptyFilteredDescription": "Try a different mood level or clear the filter to see all entries.",
|
||||
"emptyAddFirst": "Log your first mood",
|
||||
"emptyResetFilter": "Show all moods",
|
||||
"actions": "Actions",
|
||||
"formReset": "Reset form",
|
||||
"saveError": "Error saving",
|
||||
@@ -356,6 +371,8 @@
|
||||
"newMedication": "New medication",
|
||||
"firstMedication": "Add first medication",
|
||||
"noMedicationsYet": "No medications added yet",
|
||||
"emptyTitle": "No medications yet",
|
||||
"emptyDescription": "Add medications, dosages, and reminder windows to track compliance.",
|
||||
"loadFailed": "Could not load medications",
|
||||
"retryLoad": "Retry",
|
||||
"name": "Name",
|
||||
@@ -623,6 +640,12 @@
|
||||
"insights": {
|
||||
"aiAnalysisTitle": "AI Health Analysis",
|
||||
"noAnalysisYet": "No analysis yet.",
|
||||
"emptyTitle": "Not enough data yet",
|
||||
"emptyDescription": "Insights light up once you've logged a handful of measurements. Start with weight or blood pressure.",
|
||||
"emptyAddMeasurement": "Add a measurement",
|
||||
"bmiEmptyTitle": "Set your height to see BMI",
|
||||
"bmiEmptyDescription": "BMI needs your height. Add it under Settings → Account.",
|
||||
"bmiEmptyAction": "Open account settings",
|
||||
"noProviderConfigured": "AI provider not configured.",
|
||||
"startAnalysis": "Start analysis",
|
||||
"refreshAnalysis": "Refresh analysis",
|
||||
@@ -1448,10 +1471,17 @@
|
||||
"tokenCreated": "Created",
|
||||
"tokenNeverUsed": "Never",
|
||||
"noTokens": "No API tokens found.",
|
||||
"tokensEmptyTitle": "No API tokens yet",
|
||||
"tokensEmptyDescription": "Tokens are issued when a native client signs in or via Settings → Account.",
|
||||
"loginOverview": "Login Overview",
|
||||
"allAuthEvents": "All auth events",
|
||||
"failedOnly": "Failed only",
|
||||
"noEntries": "No entries found.",
|
||||
"loginEmptyTitle": "No authentication events yet",
|
||||
"loginEmptyDescription": "Sign-ins, sign-outs, and passkey changes will show up here.",
|
||||
"loginEmptyFailedTitle": "No failed sign-in attempts",
|
||||
"loginEmptyFailedDescription": "That's the safe state. Switch to all events to see successful sign-ins.",
|
||||
"loginEmptyResetFilter": "Show all events",
|
||||
"showingEntries": "Showing {count} of {total} auth entries",
|
||||
"status": "Status",
|
||||
"action": "Action",
|
||||
@@ -1507,6 +1537,8 @@
|
||||
"actions": "Actions",
|
||||
"details": "Details",
|
||||
"noEntries": "No feedback in this category.",
|
||||
"emptyTitle": "Inbox zero",
|
||||
"emptyDescription": "No user feedback in this status. Submitted bugs, questions, and feature requests will appear here.",
|
||||
"loading": "Loading feedback...",
|
||||
"anonymous": "anonymous",
|
||||
"categoryBug": "Bug",
|
||||
@@ -1601,7 +1633,10 @@
|
||||
"forceLogoutConfirmBody": "All active sessions for this user will be revoked. They will need to sign in again on every device.",
|
||||
"forceLogoutSuccess": "Logged {name} out of {count} session(s)",
|
||||
"forceLogoutFailed": "Failed to force logout",
|
||||
"cannotLogoutSelf": "Cannot force-logout yourself"
|
||||
"cannotLogoutSelf": "Cannot force-logout yourself",
|
||||
"emptyTitle": "No users in this view",
|
||||
"emptyDescription": "There are no users matching the current filter.",
|
||||
"emptyResetFilter": "Show all users"
|
||||
},
|
||||
"api-tokens": {
|
||||
"title": "API Tokens",
|
||||
@@ -1621,6 +1656,8 @@
|
||||
"loading": "Loading backups...",
|
||||
"loadError": "Failed to load backups.",
|
||||
"empty": "No backups recorded yet.",
|
||||
"emptyTitle": "No backups yet",
|
||||
"emptyDescription": "Backups run automatically every Sunday at 03:00 — or trigger one now to create the first snapshot.",
|
||||
"colUser": "User",
|
||||
"colType": "Type",
|
||||
"colSize": "Size",
|
||||
|
||||
@@ -0,0 +1,231 @@
|
||||
import { describe, it, expect } from "vitest";
|
||||
import {
|
||||
aiInsightResponseSchema,
|
||||
findUncitedRecommendations,
|
||||
InsightSchemaError,
|
||||
} from "../schema";
|
||||
import { generateInsight } from "../generate-insight";
|
||||
import { MockAIProvider } from "../mock-client";
|
||||
|
||||
const validResponse = {
|
||||
summary: "Your blood pressure is trending high.",
|
||||
recommendations: [
|
||||
{
|
||||
id: "rec-1",
|
||||
text: "Discuss lifestyle and possible medication review with your physician.",
|
||||
severity: "important",
|
||||
metricSource: {
|
||||
type: "bloodPressure",
|
||||
timeRange: "last7days",
|
||||
summary: "avg 142/88 across 12 readings",
|
||||
n: 12,
|
||||
},
|
||||
},
|
||||
],
|
||||
citations: [
|
||||
{
|
||||
type: "bloodPressure",
|
||||
timeRange: "last7days",
|
||||
summary: "avg 142/88 across 12 readings",
|
||||
},
|
||||
],
|
||||
warnings: [
|
||||
{
|
||||
topic: "blood_pressure",
|
||||
message: "Stage 1 hypertension threshold crossed.",
|
||||
severity: "important",
|
||||
},
|
||||
],
|
||||
};
|
||||
|
||||
describe("aiInsightResponseSchema", () => {
|
||||
it("accepts a well-formed strict response", () => {
|
||||
const result = aiInsightResponseSchema.safeParse(validResponse);
|
||||
expect(result.success).toBe(true);
|
||||
});
|
||||
|
||||
it("permits legacy passthrough fields (back-compat)", () => {
|
||||
const withLegacy = {
|
||||
...validResponse,
|
||||
classification: "grenzwertig",
|
||||
findings: [{ label: "x", value: "y", assessment: "neutral" }],
|
||||
disclaimer: "consult your doctor",
|
||||
};
|
||||
const result = aiInsightResponseSchema.safeParse(withLegacy);
|
||||
expect(result.success).toBe(true);
|
||||
});
|
||||
|
||||
it("rejects when summary is empty", () => {
|
||||
const result = aiInsightResponseSchema.safeParse({
|
||||
...validResponse,
|
||||
summary: "",
|
||||
});
|
||||
expect(result.success).toBe(false);
|
||||
});
|
||||
|
||||
it("rejects a recommendation without metricSource.summary", () => {
|
||||
const broken = {
|
||||
...validResponse,
|
||||
recommendations: [
|
||||
{
|
||||
id: "rec-1",
|
||||
text: "Walk more",
|
||||
severity: "suggestion",
|
||||
metricSource: {
|
||||
type: "activity",
|
||||
timeRange: "last7days",
|
||||
summary: "", // empty — should fail
|
||||
},
|
||||
},
|
||||
],
|
||||
};
|
||||
const result = aiInsightResponseSchema.safeParse(broken);
|
||||
expect(result.success).toBe(false);
|
||||
});
|
||||
|
||||
it("rejects a recommendation with missing severity enum", () => {
|
||||
const broken = {
|
||||
...validResponse,
|
||||
recommendations: [
|
||||
{
|
||||
id: "rec-1",
|
||||
text: "Walk more",
|
||||
severity: "kinda-bad", // not in enum
|
||||
metricSource: {
|
||||
type: "activity",
|
||||
timeRange: "last7days",
|
||||
summary: "5,000 steps avg",
|
||||
},
|
||||
},
|
||||
],
|
||||
};
|
||||
const result = aiInsightResponseSchema.safeParse(broken);
|
||||
expect(result.success).toBe(false);
|
||||
});
|
||||
|
||||
it("findUncitedRecommendations flags recommendations whose metricSource is not in citations", () => {
|
||||
const orphan = {
|
||||
...validResponse,
|
||||
recommendations: [
|
||||
validResponse.recommendations[0],
|
||||
{
|
||||
id: "rec-2",
|
||||
text: "Reduce salt intake",
|
||||
severity: "suggestion",
|
||||
metricSource: {
|
||||
type: "diet",
|
||||
timeRange: "last30days",
|
||||
summary: "no salt logging available",
|
||||
},
|
||||
},
|
||||
],
|
||||
};
|
||||
const parsed = aiInsightResponseSchema.parse(orphan);
|
||||
const missing = findUncitedRecommendations(parsed);
|
||||
expect(missing).toHaveLength(1);
|
||||
expect(missing[0].recommendationId).toBe("rec-2");
|
||||
expect(missing[0].missing.type).toBe("diet");
|
||||
});
|
||||
});
|
||||
|
||||
describe("generateInsight wrapper", () => {
|
||||
it("returns the parsed response on first-attempt success", async () => {
|
||||
const provider = new MockAIProvider({
|
||||
responses: JSON.stringify(validResponse),
|
||||
});
|
||||
const outcome = await generateInsight(provider, {
|
||||
systemPrompt: "sys",
|
||||
userPrompt: "user",
|
||||
});
|
||||
expect(outcome.attempts).toBe(1);
|
||||
expect(outcome.retried).toBe(false);
|
||||
expect(outcome.parsed.summary).toMatch(/blood pressure/);
|
||||
expect(provider.callCount).toBe(1);
|
||||
});
|
||||
|
||||
it("retries once with correction message and succeeds on second attempt", async () => {
|
||||
const provider = new MockAIProvider({
|
||||
responses: [
|
||||
"definitely not json",
|
||||
JSON.stringify(validResponse),
|
||||
],
|
||||
});
|
||||
const outcome = await generateInsight(provider, {
|
||||
systemPrompt: "sys",
|
||||
userPrompt: "Original user prompt.",
|
||||
});
|
||||
expect(outcome.attempts).toBe(2);
|
||||
expect(outcome.retried).toBe(true);
|
||||
expect(provider.callCount).toBe(2);
|
||||
// The retry call must include the original user prompt PLUS the
|
||||
// corrective suffix referencing the schema.
|
||||
const retryUserPrompt = provider.calls[1].userPrompt;
|
||||
expect(retryUserPrompt).toContain("Original user prompt.");
|
||||
expect(retryUserPrompt).toContain("did not satisfy");
|
||||
expect(retryUserPrompt).toContain("metricSource");
|
||||
});
|
||||
|
||||
it("throws InsightSchemaError(422) when both attempts fail", async () => {
|
||||
const provider = new MockAIProvider({
|
||||
responses: ["nope", "still nope"],
|
||||
});
|
||||
let caught: unknown;
|
||||
try {
|
||||
await generateInsight(provider, {
|
||||
systemPrompt: "sys",
|
||||
userPrompt: "user",
|
||||
});
|
||||
} catch (e) {
|
||||
caught = e;
|
||||
}
|
||||
expect(caught).toBeInstanceOf(InsightSchemaError);
|
||||
const err = caught as InsightSchemaError;
|
||||
expect(err.httpStatus).toBe(422);
|
||||
expect(err.attempts).toBe(2);
|
||||
expect(provider.callCount).toBe(2);
|
||||
});
|
||||
|
||||
it("retries when first attempt has uncited recommendations", async () => {
|
||||
const orphanResponse = {
|
||||
...validResponse,
|
||||
recommendations: [
|
||||
{
|
||||
id: "rec-orphan",
|
||||
text: "Cut sugar",
|
||||
severity: "suggestion",
|
||||
metricSource: {
|
||||
type: "diet",
|
||||
timeRange: "last7days",
|
||||
summary: "no sugar tracking present",
|
||||
},
|
||||
},
|
||||
],
|
||||
citations: [], // empty — fails cross-check
|
||||
};
|
||||
const provider = new MockAIProvider({
|
||||
responses: [JSON.stringify(orphanResponse), JSON.stringify(validResponse)],
|
||||
});
|
||||
const outcome = await generateInsight(provider, {
|
||||
systemPrompt: "sys",
|
||||
userPrompt: "u",
|
||||
});
|
||||
expect(outcome.attempts).toBe(2);
|
||||
expect(outcome.retried).toBe(true);
|
||||
expect(provider.calls[1].userPrompt).toContain(
|
||||
"metricSources not in citations",
|
||||
);
|
||||
});
|
||||
|
||||
it("propagates provider-level errors without retrying", async () => {
|
||||
const provider = new MockAIProvider({
|
||||
rejectWith: Object.assign(new Error("OpenAI request failed (500)"), {
|
||||
httpStatus: 500,
|
||||
}),
|
||||
});
|
||||
await expect(
|
||||
generateInsight(provider, { systemPrompt: "s", userPrompt: "u" }),
|
||||
).rejects.toThrow("OpenAI request failed (500)");
|
||||
// Provider only called once — wrapper does not retry on provider errors.
|
||||
expect(provider.callCount).toBe(1);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,176 @@
|
||||
import type { z } from "zod/v4";
|
||||
import type { AIProvider, CompletionParams, CompletionResult } from "./types";
|
||||
import {
|
||||
aiInsightResponseSchema,
|
||||
findUncitedRecommendations,
|
||||
InsightSchemaError,
|
||||
type AIInsightResponse,
|
||||
} from "./schema";
|
||||
|
||||
/**
|
||||
* Schema-enforced wrapper around `AIProvider.generateCompletion()`.
|
||||
*
|
||||
* Phase C1 (v1.4.15): Marc's mandate ("zero hallucinations, must
|
||||
* ground on user data") means a free-form LLM response is no longer
|
||||
* acceptable. This wrapper:
|
||||
*
|
||||
* 1. Calls the provider once.
|
||||
* 2. Parses the response as JSON.
|
||||
* 3. Validates it against `aiInsightResponseSchema` AND cross-checks
|
||||
* that every recommendation's `metricSource` appears in
|
||||
* `citations[]`.
|
||||
* 4. On failure: retries ONCE, prepending a corrective system
|
||||
* message that includes the violated zod issues.
|
||||
* 5. On second failure: throws `InsightSchemaError` (httpStatus 422).
|
||||
*
|
||||
* The wrapper does not concern itself with provider-level retries
|
||||
* (HTTP 5xx, 401 token-refresh, etc.) — each provider client handles
|
||||
* those internally. This wrapper retries only on schema-violation.
|
||||
*/
|
||||
|
||||
export interface GenerateInsightOutcome {
|
||||
parsed: AIInsightResponse;
|
||||
raw: CompletionResult;
|
||||
/** Number of provider calls made (1 on success, 2 on retry). */
|
||||
attempts: number;
|
||||
/** True iff the retry attempt produced the parsed response. */
|
||||
retried: boolean;
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the corrective user-prompt suffix that explains to the model
|
||||
* what went wrong on the first attempt. Includes the violated zod
|
||||
* issues verbatim (truncated to 1KB) so the model can self-correct.
|
||||
*/
|
||||
function buildRetryCorrectionMessage(
|
||||
reason: string,
|
||||
details: string,
|
||||
): string {
|
||||
return `
|
||||
Your previous response did not satisfy the required JSON schema.
|
||||
Reason: ${reason}
|
||||
Details: ${details.slice(0, 1024)}
|
||||
|
||||
You MUST return a JSON object matching this schema exactly. Do NOT
|
||||
include prose, markdown fences, or commentary outside the JSON.
|
||||
|
||||
Required top-level fields:
|
||||
- summary: string (2-3 sentences)
|
||||
- recommendations: array of objects with
|
||||
{ id: string, text: string, severity: "info" | "suggestion" | "important" | "urgent",
|
||||
metricSource: { type: string, timeRange: string, summary: string, n?: number } }
|
||||
- citations: array of objects with { type: string, timeRange: string, summary: string }
|
||||
- warnings: array of objects with { topic: string, message: string, severity?: same enum }
|
||||
|
||||
Every recommendation's metricSource MUST also appear in citations[]
|
||||
(same type + timeRange). If you cannot ground a recommendation in
|
||||
user data from the snapshot you were given, OMIT it. An empty
|
||||
recommendations[] is acceptable.
|
||||
`;
|
||||
}
|
||||
|
||||
interface TryOutcome {
|
||||
ok: boolean;
|
||||
parsed?: AIInsightResponse;
|
||||
raw: CompletionResult;
|
||||
reason?: string;
|
||||
details?: string;
|
||||
zodIssues?: z.ZodIssue[];
|
||||
}
|
||||
|
||||
async function tryOnce(
|
||||
provider: AIProvider,
|
||||
params: CompletionParams,
|
||||
): Promise<TryOutcome> {
|
||||
const raw = await provider.generateCompletion(params);
|
||||
|
||||
// Parse JSON.
|
||||
let json: unknown;
|
||||
try {
|
||||
json = JSON.parse(raw.content);
|
||||
} catch (e) {
|
||||
return {
|
||||
ok: false,
|
||||
raw,
|
||||
reason: "Response was not valid JSON",
|
||||
details: (e as Error).message,
|
||||
};
|
||||
}
|
||||
|
||||
// Schema validate.
|
||||
const result = aiInsightResponseSchema.safeParse(json);
|
||||
if (!result.success) {
|
||||
return {
|
||||
ok: false,
|
||||
raw,
|
||||
reason: "Response did not match the required JSON schema",
|
||||
details: result.error.issues
|
||||
.slice(0, 8)
|
||||
.map((i) => `${i.path.join(".")}: ${i.message}`)
|
||||
.join("; "),
|
||||
zodIssues: result.error.issues,
|
||||
};
|
||||
}
|
||||
|
||||
// Cross-check citations.
|
||||
const uncited = findUncitedRecommendations(result.data);
|
||||
if (uncited.length > 0) {
|
||||
const list = uncited
|
||||
.slice(0, 4)
|
||||
.map(
|
||||
(u) =>
|
||||
`${u.recommendationId} → ${u.missing.type}/${u.missing.timeRange}`,
|
||||
)
|
||||
.join(", ");
|
||||
return {
|
||||
ok: false,
|
||||
raw,
|
||||
reason: "Recommendations referenced metricSources not in citations[]",
|
||||
details: `Missing citations: ${list}`,
|
||||
};
|
||||
}
|
||||
|
||||
return { ok: true, parsed: result.data, raw };
|
||||
}
|
||||
|
||||
/**
|
||||
* Run a schema-enforced insight generation. Up to 2 provider calls.
|
||||
* Throws `InsightSchemaError` (httpStatus 422) if both attempts fail
|
||||
* the schema; provider-level errors bubble untouched.
|
||||
*/
|
||||
export async function generateInsight(
|
||||
provider: AIProvider,
|
||||
params: CompletionParams,
|
||||
): Promise<GenerateInsightOutcome> {
|
||||
const first = await tryOnce(provider, params);
|
||||
if (first.ok && first.parsed) {
|
||||
return { parsed: first.parsed, raw: first.raw, attempts: 1, retried: false };
|
||||
}
|
||||
|
||||
// Retry-once with corrective context appended to the user prompt.
|
||||
const correction = buildRetryCorrectionMessage(
|
||||
first.reason ?? "schema mismatch",
|
||||
first.details ?? "",
|
||||
);
|
||||
const retryParams: CompletionParams = {
|
||||
...params,
|
||||
userPrompt: `${params.userPrompt}\n\n${correction}`,
|
||||
};
|
||||
const second = await tryOnce(provider, retryParams);
|
||||
if (second.ok && second.parsed) {
|
||||
return {
|
||||
parsed: second.parsed,
|
||||
raw: second.raw,
|
||||
attempts: 2,
|
||||
retried: true,
|
||||
};
|
||||
}
|
||||
|
||||
throw new InsightSchemaError(
|
||||
second.reason ?? first.reason ?? "Schema validation failed twice",
|
||||
{
|
||||
issues: second.zodIssues ?? first.zodIssues,
|
||||
attempts: 2,
|
||||
},
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,182 @@
|
||||
import { z } from "zod/v4";
|
||||
|
||||
/**
|
||||
* Strict response schema for v1.4.15 AI hardening (Phase C1).
|
||||
*
|
||||
* Marc, verbatim 2026-05-09: "Es darf null Halluzinationen haben und es
|
||||
* muss sich halt irgendwie stützen auf medizinische Dinge." ("It must
|
||||
* have zero hallucinations and must ground on medical facts.")
|
||||
*
|
||||
* The schema enforces:
|
||||
* - A short natural-language summary.
|
||||
* - A typed `recommendations[]` where each item is structured
|
||||
* (`id`, `text`, `severity`) and MUST cite the user-data point that
|
||||
* justified it via `metricSource`. An empty `metricSource.summary`
|
||||
* fails parse — the model cannot fabricate recommendations without
|
||||
* pointing to a number from the user's snapshot.
|
||||
* - A `citations[]` array referencing the metrics the response drew
|
||||
* on. May be empty when `recommendations[]` is empty (no data → no
|
||||
* citation), but every recommendation's `metricSource` MUST also
|
||||
* appear in `citations[]` (cross-check enforced by `validateInsightResponse`).
|
||||
* - A `warnings[]` array for guideline-flagged values (high BP,
|
||||
* critical pulse, etc.). Free-form for now; v1.4.16 adds severity
|
||||
* enum + medical-reference grounding.
|
||||
*
|
||||
* Legacy rich fields (`classification`, `findings`, `correlations`,
|
||||
* `dataQuality`, `disclaimer`) remain accepted via `.passthrough()` so
|
||||
* cached payloads from v1.4.14 still hydrate the dashboard. The
|
||||
* v1.4.16 roadmap (`docs/audit/v1416-ai-roadmap.md`) plans the full
|
||||
* migration to the strict shape once all consumers move over.
|
||||
*/
|
||||
|
||||
// Severity levels — generic for v1.4.15. v1.4.16 adds a clinical
|
||||
// severity enum keyed to ESH/ESC and AHA guidelines.
|
||||
export const recommendationSeveritySchema = z.enum([
|
||||
"info",
|
||||
"suggestion",
|
||||
"important",
|
||||
"urgent",
|
||||
]);
|
||||
|
||||
/**
|
||||
* Citation-from-data: every recommendation must point to a concrete
|
||||
* snapshot field that justified it. Empty `summary` is rejected.
|
||||
*/
|
||||
export const metricSourceSchema = z.object({
|
||||
/**
|
||||
* Snapshot key — e.g. "bloodPressure", "weight", "pulse",
|
||||
* "medications.compliance30". Free-form because the snapshot shape
|
||||
* itself is rich; v1.4.16 narrows to a typed enum.
|
||||
*/
|
||||
type: z.string().min(1, "metricSource.type required"),
|
||||
/**
|
||||
* Time window the value covers — e.g. "last7days", "last30days",
|
||||
* "allTime". v1.4.16 narrows this enum.
|
||||
*/
|
||||
timeRange: z.string().min(1, "metricSource.timeRange required"),
|
||||
/**
|
||||
* Human-readable summary of the data point — e.g. "avg 142/88 over
|
||||
* 12 readings" or "compliance 0.71 / 30d". Empty value rejected so
|
||||
* the model cannot fake a citation.
|
||||
*/
|
||||
summary: z.string().min(1, "metricSource.summary required (zero-hallucination)"),
|
||||
/** Sample count behind the value, when known. */
|
||||
n: z.number().int().nonnegative().optional(),
|
||||
});
|
||||
|
||||
export type MetricSource = z.infer<typeof metricSourceSchema>;
|
||||
|
||||
export const aiRecommendationSchema = z.object({
|
||||
/**
|
||||
* Stable identifier for the recommendation within this response.
|
||||
* Used by the future user-feedback loop ("was this helpful?",
|
||||
* v1.4.16 roadmap). Format: short slug (`rec-1`, `bp-elevated`).
|
||||
*/
|
||||
id: z.string().min(1, "recommendation.id required"),
|
||||
/** Human-readable recommendation text. */
|
||||
text: z.string().min(1, "recommendation.text required"),
|
||||
/**
|
||||
* Mandatory citation — the data point that justified the
|
||||
* recommendation. Closes the "ungrounded boilerplate" risk Marc
|
||||
* called out: every recommendation must trace back to user data.
|
||||
*/
|
||||
metricSource: metricSourceSchema,
|
||||
/** Severity — generic for v1.4.15, clinically-keyed in v1.4.16. */
|
||||
severity: recommendationSeveritySchema,
|
||||
});
|
||||
|
||||
export type AIRecommendation = z.infer<typeof aiRecommendationSchema>;
|
||||
|
||||
export const aiCitationSchema = z.object({
|
||||
/** Snapshot key being cited. */
|
||||
type: z.string().min(1, "citation.type required"),
|
||||
/** Window of the value. */
|
||||
timeRange: z.string().min(1, "citation.timeRange required"),
|
||||
/** Short description of the data point. */
|
||||
summary: z.string().min(1, "citation.summary required"),
|
||||
});
|
||||
|
||||
export type AICitation = z.infer<typeof aiCitationSchema>;
|
||||
|
||||
export const aiWarningSchema = z.object({
|
||||
/** Topic — "blood_pressure", "pulse", etc. */
|
||||
topic: z.string().min(1, "warning.topic required"),
|
||||
/** What's flagged. */
|
||||
message: z.string().min(1, "warning.message required"),
|
||||
/** Severity tag for downstream filtering. */
|
||||
severity: recommendationSeveritySchema.optional(),
|
||||
});
|
||||
|
||||
export type AIWarning = z.infer<typeof aiWarningSchema>;
|
||||
|
||||
/**
|
||||
* Canonical strict response schema. New strict fields are required;
|
||||
* legacy rich fields ride along through `.passthrough()` for back-
|
||||
* compat with cached payloads and the existing dashboard renderer.
|
||||
*/
|
||||
export const aiInsightResponseSchema = z
|
||||
.object({
|
||||
/** 2-3 sentence overall summary in user-facing locale. */
|
||||
summary: z.string().min(1, "summary required"),
|
||||
/** Structured recommendations with mandatory citations. */
|
||||
recommendations: z.array(aiRecommendationSchema),
|
||||
/** Citations supporting any claim in summary or recommendations. */
|
||||
citations: z.array(aiCitationSchema),
|
||||
/** Guideline-flagged values (e.g. BP > 140/90). May be empty. */
|
||||
warnings: z.array(aiWarningSchema),
|
||||
})
|
||||
.passthrough();
|
||||
|
||||
export type AIInsightResponse = z.infer<typeof aiInsightResponseSchema>;
|
||||
|
||||
/**
|
||||
* Tagged error thrown by the schema-enforcement wrapper when even the
|
||||
* retry attempt fails to produce a valid payload. The route catches
|
||||
* this and surfaces a 422 to the client.
|
||||
*/
|
||||
export class InsightSchemaError extends Error {
|
||||
readonly httpStatus = 422;
|
||||
readonly issues: z.ZodIssue[] | null;
|
||||
readonly attempts: number;
|
||||
|
||||
constructor(
|
||||
message: string,
|
||||
options: { issues?: z.ZodIssue[]; attempts: number },
|
||||
) {
|
||||
super(message);
|
||||
this.name = "InsightSchemaError";
|
||||
this.issues = options.issues ?? null;
|
||||
this.attempts = options.attempts;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Cross-validation beyond zod: every recommendation's `metricSource`
|
||||
* must also appear in `citations[]` (same `type` + `timeRange`).
|
||||
* Returns the list of missing-citation issues; an empty list means
|
||||
* the response is internally consistent.
|
||||
*/
|
||||
export function findUncitedRecommendations(
|
||||
parsed: AIInsightResponse,
|
||||
): Array<{ recommendationId: string; missing: { type: string; timeRange: string } }> {
|
||||
const cited = new Set(
|
||||
parsed.citations.map((c) => `${c.type}::${c.timeRange}`),
|
||||
);
|
||||
const missing: Array<{
|
||||
recommendationId: string;
|
||||
missing: { type: string; timeRange: string };
|
||||
}> = [];
|
||||
for (const rec of parsed.recommendations) {
|
||||
const key = `${rec.metricSource.type}::${rec.metricSource.timeRange}`;
|
||||
if (!cited.has(key)) {
|
||||
missing.push({
|
||||
recommendationId: rec.id,
|
||||
missing: {
|
||||
type: rec.metricSource.type,
|
||||
timeRange: rec.metricSource.timeRange,
|
||||
},
|
||||
});
|
||||
}
|
||||
}
|
||||
return missing;
|
||||
}
|
||||
Reference in new issue
Block a user