docs(planning): mark Wave-B B5e (user-feedback loop) complete

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Marc-André BombeckandClaude Opus 4.7 committed 2026-05-10 01:47:40 +02:00
1 parent 887928211b
commit c8e76399ba
2 files changed
+146 -3

No files matched your search

+30 -3
View File
@@ -251,10 +251,37 @@ Last update: 2026-05-09T23:12:52+02:00
### B5e — User-Feedback Loop
- [ ] "War das hilfreich?" thumbs-up/down on each rec, persisted as RecommendationFeedback (new table)
- [ ] Aggregated feedback feeds into prompt-tuning: low-helpful patterns get a system-prompt addendum next generation
- [ ] Privacy: feedback is per-user; no cross-user training (offline only, opt-in for any aggregation)
- [x] "War das hilfreich?" thumbs-up/down on each rec, persisted as RecommendationFeedback (new table)
- [x] Aggregated feedback infrastructure (daily worker writes per-(severity x metricSourceType x provider x prompt-version) summary to AppSettings) — prompt-mutation itself deferred to v1.4.17 ratchet per research §3
- [x] Privacy: feedback is per-user; cross-user aggregate gated behind /admin/ai-quality (requireAdmin); single-user-default-on per research §3
- Detailed report: `.planning/phase-B5e-report.md`
- Commits on origin/main:
- `badc893 feat(db): RecommendationFeedback model with GDPR cascade + provider attribution`
- `8255e3e feat(api): /api/insights/feedback persists thumbs with provider attribution`
- `ca84fb7 feat(insights): thumbs-up/down feedback component with optimistic updates`
- `47ca1e4 feat(insights): RecommendationCard renders feedback thumbs in named slot`
- `71ffe30 feat(jobs): daily feedback aggregator writes per-(severity x provider) summary to AppSettings`
- `8879282 feat(admin): AI quality preview showing per-(severity x provider) helpful-rate`
- Status block — 2026-05-10T01:45+02:00: B5e complete on origin/main.
6 atomic TDD-first commits; +20 net direct tests on top of B5d's
parallel +56. New Prisma model `RecommendationFeedback` with the
`(userId, recommendationId, recommendationText)` dedup key; server-
side provider attribution via `resolveFeedbackAttribution()` (reads
latest `insights.generate` audit row, prefers chainProviderType >
providerType > "unknown"). New `<RecommendationFeedback>` component
fills B5c's `data-slot="rec-feedback-slot"` with TanStack Query
optimistic updates + localStorage refresh-defence; `asFeedbackTime
Range()` defence-in-depth gate keeps the slot empty for legacy/
partial recs so a thumbs-click can never produce a 422. New
pg-boss `feedback-aggregator` queue (daily 04:00 Europe/Berlin)
writes per-(severity x provider) helpful-rate summary to
`AppSettings.adminAiInsightsFeedbackSummary`. New `/admin/ai-quality`
route surfaces the table tinted by helpful-rate band. EN+DE i18n:
18 new keys. `pnpm test` 1464/1464, `pnpm test:integration` 59/59,
`pnpm typecheck` 0 errors, `pnpm lint` 12 pre-existing warnings.
Worktree `agent/b5e-feedback-loop` shipped without colliding with
B5d's parallel ConfidenceMeter wiring on the same rec-card surface
(named-slot pattern delivered as designed).
### B6 — Settings naming-audit
+116
View File
@@ -0,0 +1,116 @@
# Phase B5e — User-feedback loop (v1.4.16)
Completed 2026-05-10 ~01:45 CEST.
Worktree-isolated under `agent/b5e-feedback-loop` so the parallel B5d
worker on `agent/b5d-confidence` couldn't bleed in. Six atomic
commits on origin/main, each TDD-first (failing test then fix).
## What landed
| # | Commit | What it ships |
|---|--------|---------------|
| 1 | `badc893 feat(db): RecommendationFeedback model with GDPR cascade + provider attribution` | Prisma model + migration `0034_recommendation_feedback`. `(userId, recommendationId, recommendationText)` unique constraint as the dedup key; `(userId, createdAt)` and `(severity, helpful, createdAt)` indexes for the per-user timeline + aggregator slice. Adds `AppSettings.adminAiInsightsFeedbackSummary Json?` for the daily aggregator's output. Cascade-delete integration test seeds + asserts a feedback row. |
| 2 | `8255e3e feat(api): /api/insights/feedback persists thumbs with provider attribution` | New `POST /api/insights/feedback` route: zod-validated body, `withIdempotency()` retry-replay, `requireAuth()` gate, server-fills providerType + promptVersion via `resolveFeedbackAttribution()` (reads latest `insights.generate` audit row, prefers chainProviderType > providerType > "unknown"). 201 on insert / 409 on dedup / 422 on validation. Audit row `insights.recommendation.feedback`. 7 schema tests + 5 attribution tests + 4 integration tests. |
| 3 | `ca84fb7 feat(insights): thumbs-up/down feedback component with optimistic updates` | `<RecommendationFeedback>` component — TanStack Query mutation, two icon buttons (thumb-up / thumb-down), state machine (default → submitting → submitted/already-rated/error). localStorage cache keyed `(userId, recId, truncatedRecText)`. Auth-resolution race handled via setState-during-render (not useEffect — honours `react-hooks/set-state-in-effect`). EN+DE i18n: 5 new keys under `insights.recommendation.{feedbackHelpful,feedbackNotHelpful,feedbackThanks,feedbackConfirmed,feedbackAlreadyRated}`. 6 component tests. |
| 4 | `47ca1e4 feat(insights): RecommendationCard renders feedback thumbs in named slot` | Fills the `data-slot="rec-feedback-slot"` reserved by B5c. Defence-in-depth `asFeedbackTimeRange()` + attribute completeness check leaves the slot empty for legacy/partial recs (no id, missing severity, out-of-vocabulary timeRange) so a thumbs-click can never produce a 422. 3 new rec-card tests + retrofit `useMutation` + `useAuth` mocks across the existing rec-card / advisor-card / integration test files. |
| 5 | `71ffe30 feat(jobs): daily feedback aggregator writes per-(severity x provider) summary to AppSettings` | New pg-boss `feedback-aggregator` queue running daily 04:00 Europe/Berlin (after the rate-limit / idempotency / audit-log cleanups). Pure `buildFeedbackBuckets()` helper for in-memory aggregation; `aggregateRecommendationFeedback()` does the Prisma round-trip + AppSettings upsert. Bucket sort is deterministic (severity → metricSourceType → provider → prompt). Worker handler degrades gracefully on failure (warn, don't throw). 5 unit tests cover bucketing + 2 aggregator scenarios. |
| 6 | `8879282 feat(admin): AI quality preview showing per-(severity x provider) helpful-rate` | New `/admin/ai-quality` route + `<AiQualitySection>` component reading `GET /api/admin/ai-quality` (admin-only). 7-column table tinted by severity + helpful-rate band (green ≥ 0.8 / yellow ≥ 0.5 / orange below). New `ai-quality` slug threaded through ADMIN_SECTION_SLUGS + admin-shell + dynamic [section] renderer. Empty-state copy distinguishes "no feedback yet" from "all-zeros". EN+DE i18n: 13 new keys (`admin.section.ai-quality.*` + `admin.aiQuality.*`). 5 dedicated component tests + 1 sections-smoke + 1 slug-parity test. |
## Verification
- `pnpm test` — 1464/1464 unit tests pass (was 1396 before B5e ramp;
B5e contributes +20 net direct tests; the rest is B5d landing in
parallel during the rebase window). 183 test files (was 175).
- `pnpm test:integration` — 59/59 integration tests pass (was 55
before; +4 for `insights-feedback.test.ts` covering happy path,
dedup-409, validation-422, and unknown-provider fallback).
- `pnpm typecheck` — 0 errors.
- `pnpm lint` — 12 pre-existing warnings / 0 errors.
## Architecture decisions worth surfacing
**Server-fills attribution** — the brief spec said "Server fills
providerType + promptVersion from the user's most recent cached
payload". `User.insightsCachedText` doesn't carry provider attribution
itself (it stores the AI response, not its provenance), so the
helper reads the latest `insights.generate` audit row whose `details`
JSON carries `chainProviderType` (B5b's runner) + `providerType`
(legacy fallback). PROMPT_VERSION snapshots the current constant.
This is correctly tamper-resistant — the client never supplies any
of these fields.
**Idempotency-cache safety** — the response body `{ id, createdAt }`
contains no secrets, so it's safely cacheable by `withIdempotency()`.
The `hlk_/hlr_/sk-` defence-in-depth filter in `idempotency.ts` is a
no-op for this endpoint; no extra wiring needed.
**Defence-in-depth at the rec-card slot** — `asFeedbackTimeRange()`
gates the slot on a four-window vocabulary so a future provider
that emits `metricSource.timeRange = "lastFortnight"` can't render
a thumbs-row that would 422 on submit. Same gate handles legacy
recs (no id, no severity) by leaving the slot DOM-stable but empty.
**Non-API-driven aggregator wiring** — the worker handler receives
a `Job<FeedbackAggregatorPayload>[]` but ignores the payload (cron-
scheduled). The `void jobs;` pattern matches every other cron-only
queue in this file (host-metric-sample, audit-log-cleanup, etc.).
**Admin route placement** — research §3.B suggested `/admin/ai-
quality` OR a slot inside `/admin/system-status`. Chose the dedicated
route per the brief's "your call" — system-status is already crowded
with the host-load chart, facts grid, and worker-status panel, and
the AI-quality table needs the full content width to surface 7
columns cleanly on mobile.
## Cross-agent race notes
The rebase before commit 6 brought in 7 commits from B5d (confidence
score + ConfidenceMeter component + low-confidence caption + rationale
schema confidence-required flip). B5d's edits to
`recommendation-card.tsx` extended the file in non-conflicting ways:
B5d added the `confidence` field to `NormalisedRec`, threaded it
through `<RationaleCard>`, and rendered `<ConfidenceMeter>` in
`data-slot="rec-confidence-slot"`. My commit 4 had filled
`data-slot="rec-feedback-slot"` and added `feedbackProps` threading.
The post-rebase diff shows both B5c slots populated by their owners
(B5d ConfidenceMeter, B5e RecommendationFeedback) with zero overlap
or conflict — exactly what the named-slot pattern was meant to
deliver. Zero collisions on commits 1, 2, 3, 5, 6.
## What v1.4.17 inherits
- **Aggregator output is the source of truth for prompt-tuning.**
`AppSettings.adminAiInsightsFeedbackSummary.buckets` is the input
the v1.4.17 ratchet reads to append per-bucket "OMIT" or "REPHRASE"
rules to the system prompt. Schema is shipped; mutation is
deferred.
- **Single-user-default-on policy honoured** — only Marc's ratings
shape Marc's future prompts; cross-user aggregate is admin-only
and surfaces only on `/admin/ai-quality`. No opt-out flag added in
v1.4.16 (research §3 default-on stance).
- **Confidence ratchet hook is wired** — research §5.2 noted the
aggregator's tunable threshold table will need
`(severity x confidence_band)` slicing so low-confidence-and-low-
helpful-rate buckets ratchet faster than high-confidence ones.
`RecommendationFeedback` doesn't yet carry the confidence value
(research §5.2 flagged adding a `recommendationConfidence Int?`
column as cheap insurance — that's a v1.4.17 follow-up; the
current schema migration is forward-compatible with adding it).
## Files touched
- prisma/schema.prisma + prisma/migrations/0034_recommendation_feedback/
- src/lib/validations/recommendation-feedback.ts (+ tests)
- src/lib/ai/feedback-attribution.ts (+ tests)
- src/app/api/insights/feedback/route.ts
- src/components/insights/recommendation-feedback.tsx (+ tests)
- src/components/insights/recommendation-card.tsx (slot fill)
- src/lib/jobs/feedback-aggregator.ts (+ tests)
- src/lib/jobs/reminder-worker.ts (queue + handler wiring)
- src/app/api/admin/ai-quality/route.ts
- src/components/admin/ai-quality-section.tsx (+ tests)
- src/components/admin/{section-slugs.ts,admin-shell.tsx}
- src/app/admin/[section]/renderer.tsx
- messages/{en,de}.json (+ 18 new keys total: 5 feedback + 13 admin)
- tests/integration/{cascade-delete,insights-feedback}.test.ts