mirror of
https://github.com/MBombeck/HealthLog.git
synced 2026-11-10 05:12:14 +00:00
Merge pull request #789 from MBombeck/release/v1.37.15
Release v1.37.15
This commit is contained in:
49 files changed
+2318
-82
No files matched your search
@@ -1,5 +1,15 @@
|
||||
# Changelog
|
||||
|
||||
## [1.37.15] — 2026-08-13
|
||||
|
||||
### Fixed
|
||||
|
||||
- Re-running the assessment no longer reports a failure while the server is still writing the result. When a refresh takes longer than the browser is willing to wait, the page now keeps watching quietly until the new assessment lands, then swaps it in and confirms. Only a refresh that provably failed says so. Before, a slow refresh showed an error toast and the page held on to the old assessment until the next day.
|
||||
- Document previews show again. The security policy the app sends to the browser did not cover the way previews load since the sharing fence, so PDF previews were blocked outright and image thumbnails quietly fell back to their icons. The policy now admits exactly the app's own decrypted documents and nothing else.
|
||||
- A document uploaded while automatic AI reading is switched off no longer claims a summary is being generated forever. The upload stopped queueing summary work it is not allowed to do, a stale claim heals itself on the next pass, and documents already stuck in that state are repaired by the update. The manual summary action is offered instead, as it should have been.
|
||||
- When reading a document for search fails, the document now says why, in plain words: a scan with no readable text, a PDF whose pages would not render (scanned fax compression formats are a known cause), a provider error, or a provider that cannot take PDF files. Before, a failed read looked identical to one that never happened. An AI answer with no text in it also no longer counts as a successful read.
|
||||
- The "index all documents" confirmation shows the real number of documents queued instead of the word "true".
|
||||
|
||||
## [1.37.14] — 2026-08-13
|
||||
|
||||
### Fixed
|
||||
|
||||
+2
-2
@@ -1,11 +1,11 @@
|
||||
{
|
||||
"$comment": "Client-bundle budgets, enforced by scripts/check-bundle-budget.mjs (--check) after `next build` in CI. Values are KB gzip and include every emitted lazy route and locale chunk, while route budgets include the shared rootMainFiles baseline. Re-measured at v1.37.2: / 455, /insights 443, /measurements 441, /insights/mood 453, total 3328, baseline 130, lazy catalogs 849. Every per-route ceiling is unchanged. The /insights overview crept a kilobyte over its 445 ceiling when the ECG teaser began importing its overview cap from the classification module, dragging that module's label map and helpers into the eager page; the cap now lives in its own value-only module, and the Health Score's single provenance entry is split out of the full metric-provenance map, so the overview imports the bare number and citation it renders rather than every metric's label map behind them. Both are back under 445 with room to spare. The 42 KB aggregate rise from 3286 (v1.35.0) is all locale-catalog strings: v1.36 and v1.37 added the encounter, mood-context, mood-prognosis, practitioner, workout-batch and account-sharing surfaces, and their copy lands across the six locale catalogs the whole-build sum counts independently (804 to 849 KB). The non-catalog client chunks are flat against v1.35.0, so nothing a person downloads on a given route grew beyond its own ceiling. The aggregate cap therefore moves from 3310 to 3352 (24 KB headroom, matching the previous margin); per-route ceilings, the static-catalog guard and the one-Recharts-chunk guard remain unchanged. v1.37.4's honest-error-state charge then added recoverable read-failure handling to thirteen surfaces (the shared QueryErrorCard and QueryErrorRow primitives plus an isError branch per surface) and rewrote the MCP scope note and the managed-record error copy across the six locales; the measured total rose to 3404 while every per-route ceiling stayed met (/page 457, /insights 445, /measurements 443, /insights/mood 455). The aggregate cap moves from 3352 to 3428, keeping the same margin; nothing a person downloads on a given route grew beyond its ceiling.",
|
||||
"$comment": "Client-bundle budgets, enforced by scripts/check-bundle-budget.mjs (--check) after `next build` in CI. Values are KB gzip and include every emitted lazy route and locale chunk, while route budgets include the shared rootMainFiles baseline. Re-measured at v1.37.2: / 455, /insights 443, /measurements 441, /insights/mood 453, total 3328, baseline 130, lazy catalogs 849. Every per-route ceiling is unchanged. The /insights overview crept a kilobyte over its 445 ceiling when the ECG teaser began importing its overview cap from the classification module, dragging that module's label map and helpers into the eager page; the cap now lives in its own value-only module, and the Health Score's single provenance entry is split out of the full metric-provenance map, so the overview imports the bare number and citation it renders rather than every metric's label map behind them. Both are back under 445 with room to spare. The 42 KB aggregate rise from 3286 (v1.35.0) is all locale-catalog strings: v1.36 and v1.37 added the encounter, mood-context, mood-prognosis, practitioner, workout-batch and account-sharing surfaces, and their copy lands across the six locale catalogs the whole-build sum counts independently (804 to 849 KB). The non-catalog client chunks are flat against v1.35.0, so nothing a person downloads on a given route grew beyond its own ceiling. The aggregate cap therefore moves from 3310 to 3352 (24 KB headroom, matching the previous margin); per-route ceilings, the static-catalog guard and the one-Recharts-chunk guard remain unchanged. v1.37.4's honest-error-state charge then added recoverable read-failure handling to thirteen surfaces (the shared QueryErrorCard and QueryErrorRow primitives plus an isError branch per surface) and rewrote the MCP scope note and the managed-record error copy across the six locales; the measured total rose to 3404 while every per-route ceiling stayed met (/page 457, /insights 445, /measurements 443, /insights/mood 455). The aggregate cap moves from 3352 to 3428, keeping the same margin; nothing a person downloads on a given route grew beyond its ceiling. v1.37.15's async-truth charge then added the refresh settling state with its two-locale-key copy, the index-outcome plain-words line (seven keys across the six catalogs) and the reindex-count wiring; the measured total rose to 3429 while every per-route ceiling stayed met (/page 456, /insights 445, /measurements 442, /insights/mood 455). The aggregate cap moves from 3428 to 3452, keeping the same margin; nothing a person downloads on a given route grew beyond its ceiling.",
|
||||
"routesKbGz": {
|
||||
"/page": 460,
|
||||
"/insights/page": 445,
|
||||
"/measurements/page": 445,
|
||||
"/insights/mood/page": 460
|
||||
},
|
||||
"totalClientKbGz": 3428,
|
||||
"totalClientKbGz": 3452,
|
||||
"maxRechartsChunks": 1
|
||||
}
|
||||
+45
-4
@@ -1,7 +1,7 @@
|
||||
openapi: 3.1.0
|
||||
info:
|
||||
title: HealthLog API
|
||||
version: 1.37.14
|
||||
version: 1.37.15
|
||||
description: >-
|
||||
Self-hosted personal-health-tracking PWA — public API surface for the iOS native client and external ingest.
|
||||
|
||||
@@ -12792,7 +12792,8 @@ paths:
|
||||
immediately; the work runs off-request on the queue.
|
||||
responses:
|
||||
"200":
|
||||
description: Whether a backfill job was enqueued.
|
||||
description: How many live documents the enqueued backfill still has to index (0 when everything is indexed or no job
|
||||
was created).
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
@@ -37856,6 +37857,22 @@ components:
|
||||
- local-pdf
|
||||
- local-ocr
|
||||
- type: "null"
|
||||
lastIndexAttemptAt:
|
||||
anyOf:
|
||||
- type: string
|
||||
- type: "null"
|
||||
lastIndexOutcome:
|
||||
anyOf:
|
||||
- type: string
|
||||
enum:
|
||||
- local-empty
|
||||
- local-unsupported
|
||||
- decrypt-error
|
||||
- raster-failed
|
||||
- provider-error
|
||||
- pdf-needs-anthropic
|
||||
- empty-transcription
|
||||
- type: "null"
|
||||
hasThumbnail:
|
||||
type: boolean
|
||||
createdAt:
|
||||
@@ -37881,6 +37898,8 @@ components:
|
||||
- servingClass
|
||||
- hasContentIndex
|
||||
- contentIndexSource
|
||||
- lastIndexAttemptAt
|
||||
- lastIndexOutcome
|
||||
- hasThumbnail
|
||||
- createdAt
|
||||
- updatedAt
|
||||
@@ -37892,7 +37911,9 @@ components:
|
||||
download-only (`attachment`); render/download by it, never by MIME guess. `hasContentIndex` is true when the
|
||||
document has a content-search index (auto-indexed on upload); `contentIndexSource` says how that index was
|
||||
produced — `vision` means an AI provider read the original, the other values are provider-free local
|
||||
extractions, and it is null when `hasContentIndex` is false. `hasThumbnail` is true when a preview thumbnail has
|
||||
extractions, and it is null when `hasContentIndex` is false. `lastIndexAttemptAt` is when the most recent index
|
||||
attempt finished (successful or not; null when none ran); `lastIndexOutcome` says why that attempt produced no
|
||||
index, and is null when the attempt succeeded or none ran. `hasThumbnail` is true when a preview thumbnail has
|
||||
been rendered in the background (fetch it from `/api/documents/inbound/{id}/thumbnail`); false means no preview
|
||||
yet or an unsupported type — show a placeholder. Treat an unknown `kind` value as OTHER when decoding.
|
||||
DocumentConditionLink:
|
||||
@@ -38204,6 +38225,22 @@ components:
|
||||
- local-pdf
|
||||
- local-ocr
|
||||
- type: "null"
|
||||
lastIndexAttemptAt:
|
||||
anyOf:
|
||||
- type: string
|
||||
- type: "null"
|
||||
lastIndexOutcome:
|
||||
anyOf:
|
||||
- type: string
|
||||
enum:
|
||||
- local-empty
|
||||
- local-unsupported
|
||||
- decrypt-error
|
||||
- raster-failed
|
||||
- provider-error
|
||||
- pdf-needs-anthropic
|
||||
- empty-transcription
|
||||
- type: "null"
|
||||
hasThumbnail:
|
||||
type: boolean
|
||||
createdAt:
|
||||
@@ -38249,6 +38286,8 @@ components:
|
||||
- servingClass
|
||||
- hasContentIndex
|
||||
- contentIndexSource
|
||||
- lastIndexAttemptAt
|
||||
- lastIndexOutcome
|
||||
- hasThumbnail
|
||||
- createdAt
|
||||
- updatedAt
|
||||
@@ -38680,7 +38719,9 @@ components:
|
||||
type: object
|
||||
properties:
|
||||
enqueued:
|
||||
type: boolean
|
||||
type: integer
|
||||
minimum: 0
|
||||
maximum: 9007199254740991
|
||||
required:
|
||||
- enqueued
|
||||
additionalProperties: false
|
||||
|
||||
@@ -4172,6 +4172,8 @@
|
||||
"regenerateRateLimited": "Aktualisierungslimit erreicht — bitte später erneut versuchen.",
|
||||
"regenerateUnavailable": "Keine neue Analyse erhalten — bitte später erneut versuchen.",
|
||||
"regenerateError": "Aktualisierung läuft noch — bitte gleich erneut versuchen.",
|
||||
"regenerateSettling": "Die Einschätzung wird noch erstellt",
|
||||
"regenerateSettleFailed": "Die Aktualisierung konnte nicht abgeschlossen werden. Bitte später erneut versuchen.",
|
||||
"deviceScore": {
|
||||
"learning": "Deine Basislinie wird noch gelernt – ein paar weitere Nächte vervollständigen den Verlauf.",
|
||||
"average": "30-Tage-Durchschnitt {value}",
|
||||
@@ -9504,6 +9506,17 @@
|
||||
"generate": "Zusammenfassung erstellen",
|
||||
"generating": "Wird erstellt…"
|
||||
},
|
||||
"index": {
|
||||
"outcome": {
|
||||
"local-empty": "Es wurde kein lesbarer Text gefunden. Das sieht nach einem Scan aus; ein KI-Vision-Anbieter kann ihn lesen.",
|
||||
"local-unsupported": "Dieser Dateityp kann für die Suche nicht gelesen werden.",
|
||||
"decrypt-error": "Die gespeicherte Datei konnte zum Lesen nicht geöffnet werden.",
|
||||
"raster-failed": "Die PDF-Seiten konnten zum Lesen nicht gerendert werden. Gescannte Fax-Kompressionsformate sind eine bekannte Ursache.",
|
||||
"provider-error": "Der KI-Anbieter konnte dieses Dokument beim letzten Versuch nicht lesen.",
|
||||
"pdf-needs-anthropic": "Der eingerichtete KI-Anbieter kann keine PDF-Dateien lesen. Ein Claude-Anbieter oder lokale OCR kann es.",
|
||||
"empty-transcription": "Der KI-Anbieter hat für dieses Dokument keinen Text geliefert."
|
||||
}
|
||||
},
|
||||
"loadError": "Dieses Dokument konnte nicht geladen werden.",
|
||||
"saveError": "Die Änderung konnte nicht gespeichert werden.",
|
||||
"deleteError": "Das Dokument konnte nicht gelöscht werden.",
|
||||
|
||||
@@ -4172,6 +4172,8 @@
|
||||
"regenerateRateLimited": "Refresh limit reached — please try again later.",
|
||||
"regenerateUnavailable": "No fresh analysis arrived — please try again later.",
|
||||
"regenerateError": "Still refreshing — please try again in a moment.",
|
||||
"regenerateSettling": "Your assessment is still being prepared",
|
||||
"regenerateSettleFailed": "The refresh could not be completed. Please try again later.",
|
||||
"deviceScore": {
|
||||
"learning": "Still learning your baseline — a few more nights will fill in the trend.",
|
||||
"average": "30-day average {value}",
|
||||
@@ -9504,6 +9506,17 @@
|
||||
"generate": "Generate summary",
|
||||
"generating": "Generating…"
|
||||
},
|
||||
"index": {
|
||||
"outcome": {
|
||||
"local-empty": "No readable text was found. This looks like a scan; an AI vision provider can read it.",
|
||||
"local-unsupported": "This file type can't be read for search.",
|
||||
"decrypt-error": "The stored file couldn't be opened for reading.",
|
||||
"raster-failed": "The PDF pages couldn't be rendered for reading. Scanned fax compression formats are a known cause.",
|
||||
"provider-error": "The AI provider couldn't read this document on the last try.",
|
||||
"pdf-needs-anthropic": "The configured AI provider can't read PDF files. A Claude provider or local OCR can.",
|
||||
"empty-transcription": "The AI provider returned no text for this document."
|
||||
}
|
||||
},
|
||||
"loadError": "Couldn't load this document.",
|
||||
"saveError": "Couldn't save the change.",
|
||||
"deleteError": "Couldn't delete the document.",
|
||||
|
||||
@@ -4172,6 +4172,8 @@
|
||||
"regenerateRateLimited": "Límite de actualizaciones alcanzado: inténtalo de nuevo más tarde.",
|
||||
"regenerateUnavailable": "No llegó un análisis nuevo: inténtalo de nuevo más tarde.",
|
||||
"regenerateError": "Aún se está actualizando: inténtalo de nuevo en un momento.",
|
||||
"regenerateSettling": "La valoración todavía se está preparando",
|
||||
"regenerateSettleFailed": "No se pudo completar la actualización. Inténtalo de nuevo más tarde.",
|
||||
"deviceScore": {
|
||||
"learning": "Aún estamos aprendiendo tu línea base: unas noches más completarán la tendencia.",
|
||||
"average": "Media de 30 días {value}",
|
||||
@@ -9504,6 +9506,17 @@
|
||||
"generate": "Generar resumen",
|
||||
"generating": "Generando…"
|
||||
},
|
||||
"index": {
|
||||
"outcome": {
|
||||
"local-empty": "No se encontró texto legible. Parece un documento escaneado; un proveedor de visión de IA puede leerlo.",
|
||||
"local-unsupported": "Este tipo de archivo no se puede leer para la búsqueda.",
|
||||
"decrypt-error": "El archivo guardado no se pudo abrir para su lectura.",
|
||||
"raster-failed": "Las páginas del PDF no se pudieron procesar para su lectura. Los formatos de compresión de fax escaneados son una causa conocida.",
|
||||
"provider-error": "El proveedor de IA no pudo leer este documento en el último intento.",
|
||||
"pdf-needs-anthropic": "El proveedor de IA configurado no puede leer archivos PDF. Un proveedor Claude o el OCR local sí pueden.",
|
||||
"empty-transcription": "El proveedor de IA no devolvió texto para este documento."
|
||||
}
|
||||
},
|
||||
"loadError": "No se pudo cargar este documento.",
|
||||
"saveError": "No se pudo guardar el cambio.",
|
||||
"deleteError": "No se pudo eliminar el documento.",
|
||||
|
||||
@@ -4172,6 +4172,8 @@
|
||||
"regenerateRateLimited": "Limite d'actualisation atteinte — réessayez plus tard.",
|
||||
"regenerateUnavailable": "Aucune nouvelle analyse reçue — réessayez plus tard.",
|
||||
"regenerateError": "Actualisation en cours — réessayez dans un instant.",
|
||||
"regenerateSettling": "L’évaluation est encore en cours de préparation",
|
||||
"regenerateSettleFailed": "L’actualisation n’a pas pu aboutir. Réessayez plus tard.",
|
||||
"deviceScore": {
|
||||
"learning": "Nous apprenons encore votre référence : quelques nuits de plus compléteront la tendance.",
|
||||
"average": "Moyenne sur 30 jours {value}",
|
||||
@@ -9504,6 +9506,17 @@
|
||||
"generate": "Générer le résumé",
|
||||
"generating": "Génération…"
|
||||
},
|
||||
"index": {
|
||||
"outcome": {
|
||||
"local-empty": "Aucun texte lisible n’a été trouvé. Cela ressemble à un scan ; un fournisseur de vision IA peut le lire.",
|
||||
"local-unsupported": "Ce type de fichier ne peut pas être lu pour la recherche.",
|
||||
"decrypt-error": "Le fichier stocké n’a pas pu être ouvert pour la lecture.",
|
||||
"raster-failed": "Les pages du PDF n’ont pas pu être rendues pour la lecture. Les formats de compression de fax numérisés en sont une cause connue.",
|
||||
"provider-error": "Le fournisseur d’IA n’a pas pu lire ce document lors de la dernière tentative.",
|
||||
"pdf-needs-anthropic": "Le fournisseur d’IA configuré ne peut pas lire les fichiers PDF. Un fournisseur Claude ou l’OCR local le peut.",
|
||||
"empty-transcription": "Le fournisseur d’IA n’a renvoyé aucun texte pour ce document."
|
||||
}
|
||||
},
|
||||
"loadError": "Impossible de charger ce document.",
|
||||
"saveError": "Impossible d'enregistrer la modification.",
|
||||
"deleteError": "Impossible de supprimer le document.",
|
||||
|
||||
@@ -4172,6 +4172,8 @@
|
||||
"regenerateRateLimited": "Limite di aggiornamento raggiunto — riprova più tardi.",
|
||||
"regenerateUnavailable": "Nessuna nuova analisi ricevuta — riprova più tardi.",
|
||||
"regenerateError": "Aggiornamento ancora in corso — riprova tra un momento.",
|
||||
"regenerateSettling": "La valutazione è ancora in preparazione",
|
||||
"regenerateSettleFailed": "Non è stato possibile completare l’aggiornamento. Riprova più tardi.",
|
||||
"deviceScore": {
|
||||
"learning": "Stiamo ancora imparando la tua linea di base: qualche notte in più completerà l'andamento.",
|
||||
"average": "Media a 30 giorni {value}",
|
||||
@@ -9504,6 +9506,17 @@
|
||||
"generate": "Genera riepilogo",
|
||||
"generating": "Generazione…"
|
||||
},
|
||||
"index": {
|
||||
"outcome": {
|
||||
"local-empty": "Non è stato trovato testo leggibile. Sembra una scansione; un provider di visione IA può leggerla.",
|
||||
"local-unsupported": "Questo tipo di file non può essere letto per la ricerca.",
|
||||
"decrypt-error": "Il file archiviato non può essere aperto per la lettura.",
|
||||
"raster-failed": "Le pagine del PDF non possono essere elaborate per la lettura. I formati di compressione fax digitalizzati sono una causa nota.",
|
||||
"provider-error": "Il provider IA non è riuscito a leggere questo documento all’ultimo tentativo.",
|
||||
"pdf-needs-anthropic": "Il provider IA configurato non può leggere i file PDF. Un provider Claude o l’OCR locale sì.",
|
||||
"empty-transcription": "Il provider IA non ha restituito testo per questo documento."
|
||||
}
|
||||
},
|
||||
"loadError": "Impossibile caricare questo documento.",
|
||||
"saveError": "Impossibile salvare la modifica.",
|
||||
"deleteError": "Impossibile eliminare il documento.",
|
||||
|
||||
@@ -4172,6 +4172,8 @@
|
||||
"regenerateRateLimited": "Osiągnięto limit odświeżeń — spróbuj ponownie później.",
|
||||
"regenerateUnavailable": "Nie otrzymano nowej analizy — spróbuj ponownie później.",
|
||||
"regenerateError": "Trwa odświeżanie — spróbuj ponownie za chwilę.",
|
||||
"regenerateSettling": "Ocena jest wciąż przygotowywana",
|
||||
"regenerateSettleFailed": "Nie udało się ukończyć odświeżania. Spróbuj ponownie później.",
|
||||
"deviceScore": {
|
||||
"learning": "Wciąż poznajemy Twoją linię bazową – kilka kolejnych nocy uzupełni trend.",
|
||||
"average": "Średnia z 30 dni {value}",
|
||||
@@ -9504,6 +9506,17 @@
|
||||
"generate": "Wygeneruj podsumowanie",
|
||||
"generating": "Generowanie…"
|
||||
},
|
||||
"index": {
|
||||
"outcome": {
|
||||
"local-empty": "Nie znaleziono czytelnego tekstu. To wygląda na skan; dostawca wizji AI może go odczytać.",
|
||||
"local-unsupported": "Tego typu pliku nie można odczytać na potrzeby wyszukiwania.",
|
||||
"decrypt-error": "Nie udało się otworzyć zapisanego pliku do odczytu.",
|
||||
"raster-failed": "Nie udało się przetworzyć stron PDF do odczytu. Znaną przyczyną są skany w faksowych formatach kompresji.",
|
||||
"provider-error": "Dostawca AI nie zdołał odczytać tego dokumentu przy ostatniej próbie.",
|
||||
"pdf-needs-anthropic": "Skonfigurowany dostawca AI nie odczytuje plików PDF. Poradzi sobie z tym dostawca Claude lub lokalny OCR.",
|
||||
"empty-transcription": "Dostawca AI nie zwrócił tekstu dla tego dokumentu."
|
||||
}
|
||||
},
|
||||
"loadError": "Nie udało się wczytać tego dokumentu.",
|
||||
"saveError": "Nie udało się zapisać zmiany.",
|
||||
"deleteError": "Nie udało się usunąć dokumentu.",
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "healthlog",
|
||||
"version": "1.37.14",
|
||||
"version": "1.37.15",
|
||||
"description": "Self-hosted personal-health-tracking PWA with Withings integration, AI insights, and doctor-report PDF export.",
|
||||
"license": "PolyForm-Noncommercial-1.0.0",
|
||||
"homepage": "https://healthlog.dev",
|
||||
|
||||
@@ -0,0 +1,22 @@
|
||||
-- Refs #788 — heal phantom summary PENDING rows.
|
||||
-- `enqueueDocumentSummary` used to claim PENDING even when the owner's
|
||||
-- auto-read opt-in was OFF; the job's opt-out branch then left the state
|
||||
-- alone, so the detail view said "being generated" forever. Both writers now
|
||||
-- agree (the enqueue end no longer claims what the job cannot resolve, the
|
||||
-- job end heals PENDING→NONE on opt-out); this one-time data fix drains the
|
||||
-- rows already stranded. Safe by construction: READY / WITHHELD / UNAVAILABLE
|
||||
-- are never touched, and a job that is genuinely mid-flight rewrites its
|
||||
-- terminal state after this UPDATE anyway.
|
||||
UPDATE "inbound_documents"
|
||||
SET "summary_state" = 'NONE'
|
||||
WHERE "summary_state" = 'PENDING';
|
||||
|
||||
-- Refs #776 — record what became of a document's content-index attempt.
|
||||
-- Additive, nullable, no backfill: existing rows read NULL ("no attempt
|
||||
-- recorded"), which is exactly true for them — outcomes only exist from the
|
||||
-- release that writes them. `last_index_outcome` is a closed string enum
|
||||
-- enforced in code (see `DocumentIndexOutcomeValue`); NULL after a
|
||||
-- successful attempt, since the `document_content_index` row itself is the
|
||||
-- success signal.
|
||||
ALTER TABLE "inbound_documents" ADD COLUMN "last_index_attempt_at" TIMESTAMP(3);
|
||||
ALTER TABLE "inbound_documents" ADD COLUMN "last_index_outcome" TEXT;
|
||||
@@ -7536,6 +7536,22 @@ model InboundDocument {
|
||||
/// and one mid-flight all leave it null. NONE is the default, which keeps
|
||||
/// every pre-existing row correct — none of them were ever attempted.
|
||||
summaryState DocumentSummaryState @default(NONE) @map("summary_state")
|
||||
/// When the most recent content-index attempt finished (auto job, corpus
|
||||
/// backfill, or the manual index route — they share one recorder),
|
||||
/// successful or not. Null until any attempt ran. Diagnostic state the
|
||||
/// detail view pairs with `lastIndexOutcome`; carried verbatim in the
|
||||
/// disaster-recovery backup like its summary siblings.
|
||||
lastIndexAttemptAt DateTime? @map("last_index_attempt_at")
|
||||
/// Why the most recent index attempt produced NO index — a closed string
|
||||
/// enum (`DocumentIndexOutcomeValue` in
|
||||
/// `src/lib/validations/inbound-documents.ts`): local-empty |
|
||||
/// local-unsupported | decrypt-error | raster-failed | provider-error |
|
||||
/// pdf-needs-anthropic | empty-transcription. Null after a successful
|
||||
/// attempt (the `DocumentContentIndex` row itself is the success signal)
|
||||
/// and before the first attempt. Lets the detail view tell "never tried"
|
||||
/// from "tried and here is why it failed" instead of showing a bare
|
||||
/// not-indexed state for both.
|
||||
lastIndexOutcome String? @map("last_index_outcome")
|
||||
|
||||
facts ExtractedFact[]
|
||||
conditionLinks DocumentConditionLink[]
|
||||
|
||||
+1
-1
@@ -36,7 +36,7 @@ try {
|
||||
// v1.4.38.4 → v1.4.42. Do not hand-edit; bump `package.json` and rebuild.
|
||||
const CACHE_VERSION =
|
||||
(typeof self !== "undefined" && self.__APP_VERSION__) ||
|
||||
/* @sw-version-fallback */ "v1.37.14";
|
||||
/* @sw-version-fallback */ "v1.37.15";
|
||||
const STATIC_CACHE = `healthlog-static-${CACHE_VERSION}`;
|
||||
const PAGE_CACHE = `healthlog-pages-${CACHE_VERSION}`;
|
||||
// v1.18.6 — read-only data cache for a curated allowlist of safe GET `/api/*`
|
||||
|
||||
@@ -0,0 +1,100 @@
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { NextRequest } from "next/server";
|
||||
|
||||
/**
|
||||
* Refs #787 — the record fence made every vault preview load through
|
||||
* `useFencedObjectUrl`, i.e. `blob:` URLs. The app CSP had no `frame-src`, so
|
||||
* `default-src 'self'` blocked the PDF preview iframe loudly, and
|
||||
* `img-src 'self' data:` degraded every blob thumbnail to an icon silently.
|
||||
*
|
||||
* These tests pin the fix exactly:
|
||||
* - `frame-src` exists and admits precisely 'self' and blob: — nothing else.
|
||||
* - `img-src` admits blob: alongside 'self' and data:.
|
||||
* - The boundaries that make blob: safe stay untouched: `object-src 'none'`,
|
||||
* `frame-ancestors 'none'`, `default-src 'self'`, nonce-bound script-src.
|
||||
* - The vault SERVE route keeps its own stricter document CSP unchanged.
|
||||
*
|
||||
* Security rationale (mirrors the comment in proxy.ts): a `blob:` URL can only
|
||||
* be minted by same-origin script, and script execution stays nonce-bound, so
|
||||
* admitting blob: admits exactly the app's own decrypted documents.
|
||||
*/
|
||||
|
||||
vi.mock("@/lib/process-type", () => ({
|
||||
shouldRunWeb: () => true,
|
||||
}));
|
||||
|
||||
import { proxy } from "../proxy";
|
||||
|
||||
function setNodeEnv(value: "development" | "production") {
|
||||
vi.stubEnv("NODE_ENV", value);
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
setNodeEnv("production");
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
vi.unstubAllEnvs();
|
||||
});
|
||||
|
||||
function makeRequest(pathname: string): NextRequest {
|
||||
return new NextRequest(`http://localhost${pathname}`, {
|
||||
headers: { cookie: "healthlog_session=sess-1" },
|
||||
});
|
||||
}
|
||||
|
||||
/** Parse one directive's source list out of a CSP header string. */
|
||||
function directive(csp: string, name: string): string[] | null {
|
||||
const match = csp
|
||||
.split(";")
|
||||
.map((part) => part.trim())
|
||||
.find((part) => part.startsWith(`${name} `) || part === name);
|
||||
if (!match) return null;
|
||||
return match.split(/\s+/).slice(1);
|
||||
}
|
||||
|
||||
describe("proxy.ts blob preview CSP (#787)", () => {
|
||||
it("production frame-src admits exactly 'self' and blob:", () => {
|
||||
const res = proxy(makeRequest("/documents"));
|
||||
const csp = res.headers.get("content-security-policy") ?? "";
|
||||
expect(directive(csp, "frame-src")).toEqual(["'self'", "blob:"]);
|
||||
});
|
||||
|
||||
it("production img-src admits blob: alongside 'self' and data:", () => {
|
||||
const res = proxy(makeRequest("/documents"));
|
||||
const csp = res.headers.get("content-security-policy") ?? "";
|
||||
const sources = directive(csp, "img-src");
|
||||
expect(sources).toContain("blob:");
|
||||
expect(sources).toContain("'self'");
|
||||
expect(sources).toContain("data:");
|
||||
});
|
||||
|
||||
it("production keeps the boundaries that make blob: safe", () => {
|
||||
const res = proxy(makeRequest("/documents"));
|
||||
const csp = res.headers.get("content-security-policy") ?? "";
|
||||
expect(directive(csp, "object-src")).toEqual(["'none'"]);
|
||||
expect(directive(csp, "frame-ancestors")).toEqual(["'none'"]);
|
||||
expect(directive(csp, "default-src")).toEqual(["'self'"]);
|
||||
expect(csp).toMatch(/script-src 'self' 'nonce-/);
|
||||
});
|
||||
|
||||
it("development frame-src admits exactly 'self' and blob:", () => {
|
||||
setNodeEnv("development");
|
||||
const res = proxy(makeRequest("/documents"));
|
||||
const csp = res.headers.get("content-security-policy") ?? "";
|
||||
expect(directive(csp, "frame-src")).toEqual(["'self'", "blob:"]);
|
||||
});
|
||||
|
||||
it("development img-src admits blob:", () => {
|
||||
setNodeEnv("development");
|
||||
const res = proxy(makeRequest("/documents"));
|
||||
const csp = res.headers.get("content-security-policy") ?? "";
|
||||
expect(directive(csp, "img-src")).toContain("blob:");
|
||||
});
|
||||
|
||||
it("leaves the vault serve route's document CSP untouched (no frame-src, no blob:)", () => {
|
||||
const res = proxy(makeRequest("/api/documents/inbound/doc-1/original"));
|
||||
const csp = res.headers.get("content-security-policy") ?? "";
|
||||
expect(csp).toBe("default-src 'none'; frame-ancestors 'self';");
|
||||
});
|
||||
});
|
||||
@@ -1319,6 +1319,10 @@ const handler = apiHandler(
|
||||
? new Date(document.summaryGeneratedAt)
|
||||
: null,
|
||||
summaryState: document.summaryState as never,
|
||||
lastIndexAttemptAt: document.lastIndexAttemptAt
|
||||
? new Date(document.lastIndexAttemptAt)
|
||||
: null,
|
||||
lastIndexOutcome: document.lastIndexOutcome ?? null,
|
||||
createdAt: new Date(document.createdAt!),
|
||||
updatedAt: new Date(document.updatedAt!),
|
||||
})),
|
||||
|
||||
@@ -49,6 +49,7 @@ import {
|
||||
} from "@/lib/documents/ai-route-support";
|
||||
import { maybeAutoStageLabFacts } from "@/lib/documents/auto-stage-labs";
|
||||
import { upsertContentIndex } from "@/lib/documents/content-index";
|
||||
import { recordIndexAttempt } from "@/lib/documents/index-document";
|
||||
import {
|
||||
DocumentDescribeError,
|
||||
transcribeDocument,
|
||||
@@ -93,6 +94,15 @@ async function finishIndex(
|
||||
source: "vision" | "text-ocr",
|
||||
tokenCount: number,
|
||||
): Promise<Response> {
|
||||
// Refs #776 — the manual route is the second writer of the attempt record
|
||||
// (the auto job + backfill share `indexLoadedDocument`): a success clears
|
||||
// any stored failure reason so the detail view stops explaining a problem
|
||||
// that no longer exists.
|
||||
await recordIndexAttempt(userId, documentId, {
|
||||
indexed: true,
|
||||
source,
|
||||
tokenCount,
|
||||
});
|
||||
await auditLog("documents.inbound.index", {
|
||||
userId,
|
||||
ipAddress: getClientIp(request),
|
||||
@@ -183,21 +193,44 @@ async function handleVisionIndex(
|
||||
const vision = await prepareVisionInput(document, pick.pdfSupported);
|
||||
if (!vision.ok) {
|
||||
// Preparation failed before any provider dispatch — the slot goes back.
|
||||
// Refs #776 — each failure is also recorded on the row so the detail view
|
||||
// can explain the missing index after the toast is gone.
|
||||
await refundDocumentAiSlot(userId);
|
||||
if (vision.reason === "pdfNeedsAnthropic") {
|
||||
await recordIndexAttempt(userId, document.id, {
|
||||
indexed: false,
|
||||
reason: "pdf-needs-anthropic",
|
||||
});
|
||||
return apiError(
|
||||
"PDF scanning needs a Claude vision provider; use local OCR instead.",
|
||||
422,
|
||||
{ errorCode: "documents.inbound.pdfNeedsAnthropic" },
|
||||
);
|
||||
}
|
||||
if (vision.reason === "rasterFailed") {
|
||||
await recordIndexAttempt(userId, document.id, {
|
||||
indexed: false,
|
||||
reason: "raster-failed",
|
||||
});
|
||||
return apiError("The PDF pages couldn't be rendered for scanning.", 422, {
|
||||
errorCode: "documents.inbound.extractFailed",
|
||||
});
|
||||
}
|
||||
if (vision.reason === "fileType") {
|
||||
await recordIndexAttempt(userId, document.id, {
|
||||
indexed: false,
|
||||
reason: "local-unsupported",
|
||||
});
|
||||
return apiError(
|
||||
"This document can't be scanned. Use local OCR (text mode).",
|
||||
422,
|
||||
{ errorCode: "documents.inbound.fileType" },
|
||||
);
|
||||
}
|
||||
await recordIndexAttempt(userId, document.id, {
|
||||
indexed: false,
|
||||
reason: "decrypt-error",
|
||||
});
|
||||
return apiError("Couldn't read the stored document.", 422, {
|
||||
errorCode: "documents.inbound.extractFailed",
|
||||
});
|
||||
@@ -230,6 +263,22 @@ async function handleVisionIndex(
|
||||
reservation.reserved,
|
||||
dateKey,
|
||||
);
|
||||
// Refs #776 — the empty-transcription guard, same contract as the auto
|
||||
// path (`tryProviderIndex`): a provider answer with no text must never
|
||||
// become a "successful" empty index. The spend stays charged (the
|
||||
// provider was called); the honest answer is an error plus the recorded
|
||||
// reason.
|
||||
if (text.trim().length === 0) {
|
||||
await recordIndexAttempt(userId, document.id, {
|
||||
indexed: false,
|
||||
reason: "empty-transcription",
|
||||
});
|
||||
return apiError(
|
||||
"The provider returned no text for this document. Try a clearer copy.",
|
||||
422,
|
||||
{ errorCode: "documents.inbound.extractFailed" },
|
||||
);
|
||||
}
|
||||
const { tokenCount } = await upsertContentIndex({
|
||||
userId,
|
||||
documentId: document.id,
|
||||
@@ -240,6 +289,10 @@ async function handleVisionIndex(
|
||||
return finishIndex(request, userId, document.id, "vision", tokenCount);
|
||||
} catch (err) {
|
||||
await reconcileSpend(userId, reservation.reserved, 0, dateKey);
|
||||
await recordIndexAttempt(userId, document.id, {
|
||||
indexed: false,
|
||||
reason: "provider-error",
|
||||
});
|
||||
if (err instanceof DocumentDescribeError) {
|
||||
return apiError("Couldn't read the document. Try a clearer copy.", 422, {
|
||||
errorCode: "documents.inbound.extractFailed",
|
||||
|
||||
@@ -0,0 +1,116 @@
|
||||
import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { NextRequest } from "next/server";
|
||||
|
||||
/**
|
||||
* Refs #776 — the reindex trigger's wire honesty. The client types
|
||||
* `{ enqueued: number }` and interpolates it into "Indexing {count}
|
||||
* document(s)"; the route used to return `Boolean(jobId)`, so the toast said
|
||||
* "Indexing true document(s)". Pins: `enqueued` is a NUMBER — the count of
|
||||
* live documents still lacking a content index (the same figure the usage
|
||||
* gauge derives) when a job was created, and an honest 0 when none was.
|
||||
*/
|
||||
|
||||
vi.mock("@/lib/db", () => ({
|
||||
prisma: {
|
||||
inboundDocument: { count: vi.fn() },
|
||||
documentContentIndex: { count: vi.fn() },
|
||||
},
|
||||
}));
|
||||
vi.mock("@/lib/jobs/document-content-index-backfill", () => ({
|
||||
enqueueContentIndexBackfill: vi.fn(),
|
||||
}));
|
||||
vi.mock("@/lib/documents/provider-order", () => ({
|
||||
resolveDocumentVisionProvider: vi.fn(),
|
||||
}));
|
||||
vi.mock("@/lib/ai/consent-guard", () => ({
|
||||
assertDocumentEgressConsent: vi.fn().mockResolvedValue(undefined),
|
||||
ConsentRequiredError: class ConsentRequiredError extends Error {},
|
||||
}));
|
||||
vi.mock("@/lib/modules/gate", () => ({
|
||||
requireModuleEnabled: vi.fn().mockResolvedValue({ enabled: true }),
|
||||
}));
|
||||
vi.mock("@/lib/rate-limit", () => ({
|
||||
checkRateLimit: vi.fn().mockResolvedValue({ allowed: true }),
|
||||
rateLimitHeaders: () => ({}),
|
||||
}));
|
||||
vi.mock("@/lib/auth/session", () => ({ getSession: vi.fn() }));
|
||||
vi.mock("@/lib/auth/audit", () => ({
|
||||
auditLog: vi.fn().mockResolvedValue(undefined),
|
||||
}));
|
||||
vi.mock("@/lib/logging/transports", () => ({ emitIfSampled: vi.fn() }));
|
||||
vi.mock("@/lib/db-compat", () => ({
|
||||
ensureDbCompatibility: vi.fn().mockResolvedValue(undefined),
|
||||
}));
|
||||
vi.mock("next/headers", () => ({
|
||||
headers: vi.fn(async () => ({ get: () => null })),
|
||||
cookies: vi.fn(async () => ({
|
||||
get: () => undefined,
|
||||
set: () => {},
|
||||
delete: () => {},
|
||||
})),
|
||||
}));
|
||||
|
||||
import { POST } from "../route";
|
||||
import { prisma } from "@/lib/db";
|
||||
import { getSession } from "@/lib/auth/session";
|
||||
import { enqueueContentIndexBackfill } from "@/lib/jobs/document-content-index-backfill";
|
||||
import { resolveDocumentVisionProvider } from "@/lib/documents/provider-order";
|
||||
|
||||
const SESSION_OK = {
|
||||
session: { id: "s1", expiresAt: new Date(Date.now() + 3_600_000) },
|
||||
user: { id: "user-1", username: "tester", role: "USER" as const },
|
||||
};
|
||||
|
||||
const PICK = {
|
||||
chain: [{ providerType: "anthropic", instance: {} }],
|
||||
pick: {
|
||||
entry: { providerType: "anthropic", instance: {} },
|
||||
providerType: "anthropic",
|
||||
pdfSupported: true,
|
||||
},
|
||||
};
|
||||
|
||||
const req = () =>
|
||||
new NextRequest(new URL("http://localhost/api/documents/inbound/reindex"), {
|
||||
method: "POST",
|
||||
});
|
||||
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks();
|
||||
vi.mocked(getSession).mockResolvedValue(SESSION_OK as never);
|
||||
vi.mocked(resolveDocumentVisionProvider).mockResolvedValue(PICK as never);
|
||||
vi.mocked(enqueueContentIndexBackfill).mockResolvedValue({ enqueued: true });
|
||||
// 7 live documents, 4 already indexed → 3 remain.
|
||||
vi.mocked(prisma.inboundDocument.count).mockResolvedValue(7 as never);
|
||||
vi.mocked(prisma.documentContentIndex.count).mockResolvedValue(4 as never);
|
||||
});
|
||||
|
||||
describe("POST /api/documents/inbound/reindex", () => {
|
||||
it("returns the remaining un-indexed count as a NUMBER, never a boolean", async () => {
|
||||
const res = await POST(req() as never);
|
||||
expect(res.status).toBe(200);
|
||||
const body = await res.json();
|
||||
expect(typeof body.data.enqueued).toBe("number");
|
||||
expect(body.data.enqueued).toBe(3);
|
||||
});
|
||||
|
||||
it("returns an honest 0 when no job was created", async () => {
|
||||
vi.mocked(enqueueContentIndexBackfill).mockResolvedValue({
|
||||
enqueued: false,
|
||||
});
|
||||
const res = await POST(req() as never);
|
||||
expect(res.status).toBe(200);
|
||||
const body = await res.json();
|
||||
expect(body.data.enqueued).toBe(0);
|
||||
});
|
||||
|
||||
it("scopes both counts to the caller's LIVE documents (the gauge's own figures)", async () => {
|
||||
await POST(req() as never);
|
||||
expect(prisma.inboundDocument.count).toHaveBeenCalledWith({
|
||||
where: { userId: "user-1", deletedAt: null },
|
||||
});
|
||||
expect(prisma.documentContentIndex.count).toHaveBeenCalledWith({
|
||||
where: { userId: "user-1", document: { deletedAt: null } },
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -11,6 +11,7 @@ import { apiHandler, requireAuth } from "@/lib/api-handler";
|
||||
import { apiError, apiSuccess, getClientIp } from "@/lib/api-response";
|
||||
import { assertDocumentEgressConsent } from "@/lib/ai/consent-guard";
|
||||
import { auditLog } from "@/lib/auth/audit";
|
||||
import { prisma } from "@/lib/db";
|
||||
import { enqueueContentIndexBackfill } from "@/lib/jobs/document-content-index-backfill";
|
||||
import { resolveDocumentVisionProvider } from "@/lib/documents/provider-order";
|
||||
import { annotate } from "@/lib/logging/context";
|
||||
@@ -58,16 +59,37 @@ export const POST = apiHandler(async (request) => {
|
||||
surface: "insights",
|
||||
});
|
||||
|
||||
const { enqueued } = await enqueueContentIndexBackfill(user.id);
|
||||
const { enqueued: jobCreated } = await enqueueContentIndexBackfill(user.id);
|
||||
|
||||
// Wire honesty (#776): the client types `enqueued` as a NUMBER and
|
||||
// interpolates it into "Indexing {count} document(s)" — a boolean here once
|
||||
// rendered as "Indexing true document(s)". Report the count of live
|
||||
// documents still lacking a content index (the exact figures the usage
|
||||
// gauge already derives, same live-scoping: a soft-deleted document keeps
|
||||
// its index row until hard purge, so both counts filter `deletedAt: null`).
|
||||
// An honest 0 when no job was created — the client then says "everything is
|
||||
// already indexed" instead of promising work nobody queued.
|
||||
let enqueued = 0;
|
||||
if (jobCreated) {
|
||||
const [indexedCount, totalCount] = await Promise.all([
|
||||
prisma.documentContentIndex.count({
|
||||
where: { userId: user.id, document: { deletedAt: null } },
|
||||
}),
|
||||
prisma.inboundDocument.count({
|
||||
where: { userId: user.id, deletedAt: null },
|
||||
}),
|
||||
]);
|
||||
enqueued = Math.max(0, totalCount - indexedCount);
|
||||
}
|
||||
|
||||
await auditLog("documents.inbound.reindex", {
|
||||
userId: user.id,
|
||||
ipAddress: getClientIp(request),
|
||||
details: { enqueued },
|
||||
details: { enqueued, jobCreated },
|
||||
});
|
||||
annotate({
|
||||
action: { name: "documents.contentIndex.backfillEnqueue" },
|
||||
meta: { enqueued },
|
||||
meta: { enqueued, jobCreated },
|
||||
});
|
||||
|
||||
return apiSuccess({ enqueued });
|
||||
|
||||
@@ -1,3 +1,16 @@
|
||||
/**
|
||||
* Refs #786 — async-truth contract, server end (the client end lives in
|
||||
* `src/components/insights/use-insights-advisor.ts`, `AdvisorSettleState`):
|
||||
* the force POST generates INLINE and, on success, writes the fresh briefing
|
||||
* to the user cache (`insightsCachedAt` advances) — even when the client has
|
||||
* long since aborted its 45 s request. A failed inline generation writes a
|
||||
* failure marker the GET reports as `generationFailed`. The client relies on
|
||||
* exactly those two signals to settle a timed-out regenerate: it polls the
|
||||
* GET until `cachedAt` advances past its baseline (success) or a NEWER
|
||||
* failure marker appears (honest error). Renaming or short-circuiting either
|
||||
* write breaks that convergence — the settling seam tests pin the client
|
||||
* half, this comment is the server half of the contract.
|
||||
*/
|
||||
import { prisma } from "@/lib/db";
|
||||
import { auditLog } from "@/lib/auth/audit";
|
||||
import {
|
||||
|
||||
@@ -196,7 +196,10 @@ describe("POST /api/labs/ocr/extract — vision PDF rasterization", () => {
|
||||
});
|
||||
|
||||
it("falls back to pdfNeedsAnthropic when rasterization fails", async () => {
|
||||
vi.mocked(rasterizePdf).mockResolvedValue({ ok: false });
|
||||
vi.mocked(rasterizePdf).mockResolvedValue({
|
||||
ok: false,
|
||||
reason: "render-failed",
|
||||
});
|
||||
|
||||
const res = await POST(pdfReq());
|
||||
expect(res.status).toBe(422);
|
||||
|
||||
@@ -213,3 +213,44 @@ describe("<DocumentAiSection>", () => {
|
||||
expect(html).toContain("Summarise");
|
||||
});
|
||||
});
|
||||
|
||||
describe("<DocumentAiSection> index outcome (#776)", () => {
|
||||
it("explains a failed index attempt in plain words, muted", () => {
|
||||
const html = render(
|
||||
base({ hasContentIndex: false, lastIndexOutcome: "local-empty" }),
|
||||
);
|
||||
expect(html).toContain('data-slot="document-index-outcome"');
|
||||
expect(html).toContain("No readable text was found");
|
||||
expect(html).toContain("text-muted-foreground");
|
||||
});
|
||||
|
||||
it("also explains it on the auto-read unread branch", () => {
|
||||
const html = render(
|
||||
base({
|
||||
autoReadEnabled: true,
|
||||
hasContentIndex: false,
|
||||
lastIndexOutcome: "raster-failed",
|
||||
}),
|
||||
);
|
||||
expect(html).toContain('data-slot="document-index-outcome"');
|
||||
expect(html).toContain("couldn't be rendered");
|
||||
});
|
||||
|
||||
it("says nothing once the document IS indexed (a past failure is history)", () => {
|
||||
const html = render(
|
||||
base({
|
||||
hasContentIndex: true,
|
||||
contentIndexSource: "vision",
|
||||
lastIndexOutcome: "provider-error",
|
||||
}),
|
||||
);
|
||||
expect(html).not.toContain('data-slot="document-index-outcome"');
|
||||
});
|
||||
|
||||
it("says nothing when no attempt ever ran (never tried is not a failure)", () => {
|
||||
const html = render(
|
||||
base({ hasContentIndex: false, lastIndexOutcome: null }),
|
||||
);
|
||||
expect(html).not.toContain('data-slot="document-index-outcome"');
|
||||
});
|
||||
});
|
||||
@@ -34,6 +34,8 @@ function doc(overrides: Partial<InboundDocumentDto> = {}): InboundDocumentDto {
|
||||
servingClass: "inline",
|
||||
hasContentIndex: false,
|
||||
contentIndexSource: null,
|
||||
lastIndexAttemptAt: null,
|
||||
lastIndexOutcome: null,
|
||||
hasThumbnail: false,
|
||||
createdAt: "2025-10-05T08:00:00.000Z",
|
||||
updatedAt: "2025-10-05T08:00:00.000Z",
|
||||
|
||||
@@ -85,6 +85,8 @@ function doc(
|
||||
servingClass: "inline",
|
||||
hasContentIndex: true,
|
||||
contentIndexSource: "vision",
|
||||
lastIndexAttemptAt: null,
|
||||
lastIndexOutcome: null,
|
||||
hasThumbnail: false,
|
||||
createdAt: "2026-08-01T10:00:00.000Z",
|
||||
updatedAt: "2026-08-01T10:00:00.000Z",
|
||||
|
||||
@@ -0,0 +1,131 @@
|
||||
import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { renderToStaticMarkup } from "react-dom/server";
|
||||
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
|
||||
|
||||
/**
|
||||
* Refs #776 — the "index all" toast's count honesty.
|
||||
*
|
||||
* The reindex route reports `enqueued` as a NUMBER (documents still to
|
||||
* index); the toast interpolates it into "Indexing {count} document(s)".
|
||||
* Before the route fix the wire carried `Boolean(jobId)` and the toast said
|
||||
* "Indexing true document(s)". This pins the component end of that contract:
|
||||
* fed the real wire shape, the success toast renders the count and NEVER the
|
||||
* word "true"; a 0 says "everything is already indexed" instead of promising
|
||||
* queued work.
|
||||
*
|
||||
* House convention: `renderToStaticMarkup` + handler exercised via a mocked
|
||||
* child reference (the filter bar captures `onIndexAll`), no DOM events.
|
||||
*/
|
||||
|
||||
let mockSearch = "";
|
||||
vi.mock("next/navigation", () => ({
|
||||
useRouter: () => ({ push: vi.fn(), replace: vi.fn() }),
|
||||
usePathname: () => "/documents",
|
||||
useSearchParams: () => new URLSearchParams(mockSearch),
|
||||
}));
|
||||
|
||||
vi.mock("@/hooks/use-auth", () => ({
|
||||
useAuth: () => ({
|
||||
user: { id: "u1", modules: { inboundDocuments: true } },
|
||||
isLoading: false,
|
||||
isAuthenticated: true,
|
||||
}),
|
||||
}));
|
||||
|
||||
vi.mock("@/lib/api/api-fetch", async () => {
|
||||
const actual = await vi.importActual<typeof import("@/lib/api/api-fetch")>(
|
||||
"@/lib/api/api-fetch",
|
||||
);
|
||||
return {
|
||||
...actual,
|
||||
apiGet: () => new Promise(() => {}),
|
||||
apiPatch: vi.fn(),
|
||||
apiPost: vi.fn(),
|
||||
apiDelete: vi.fn(),
|
||||
};
|
||||
});
|
||||
|
||||
const toastSuccess = vi.fn();
|
||||
const toastError = vi.fn();
|
||||
vi.mock("sonner", () => ({
|
||||
toast: {
|
||||
success: (...args: unknown[]) => toastSuccess(...args),
|
||||
error: (...args: unknown[]) => toastError(...args),
|
||||
},
|
||||
}));
|
||||
|
||||
// Capture the view's index-all handler through the filter bar's prop — the
|
||||
// static render cannot click, so the mocked child is the seam.
|
||||
let capturedOnIndexAll: (() => void) | undefined;
|
||||
vi.mock("../document-filter-bar", () => ({
|
||||
DocumentFilterBar: (props: { onIndexAll?: () => void }) => {
|
||||
capturedOnIndexAll = props.onIndexAll;
|
||||
return null;
|
||||
},
|
||||
}));
|
||||
|
||||
// The reindex mutation resolves synchronously with a configurable wire
|
||||
// payload — exactly what `apiPost` would unwrap from the route envelope.
|
||||
let wirePayload: { enqueued: number };
|
||||
vi.mock("../use-content-index", () => ({
|
||||
useIndexDocument: () => ({ mutate: vi.fn(), isPending: false }),
|
||||
useReindexAll: () => ({
|
||||
mutate: (
|
||||
_vars: void,
|
||||
opts?: { onSuccess?: (result: { enqueued: number }) => void },
|
||||
) => {
|
||||
opts?.onSuccess?.(wirePayload);
|
||||
},
|
||||
isPending: false,
|
||||
}),
|
||||
}));
|
||||
|
||||
import { I18nProvider } from "@/lib/i18n/context";
|
||||
import { DocumentsView } from "../documents-view";
|
||||
|
||||
function render() {
|
||||
mockSearch = "";
|
||||
capturedOnIndexAll = undefined;
|
||||
const queryClient = new QueryClient({
|
||||
defaultOptions: { queries: { retry: false } },
|
||||
});
|
||||
renderToStaticMarkup(
|
||||
<QueryClientProvider client={queryClient}>
|
||||
<I18nProvider initialLocale="en">
|
||||
<DocumentsView />
|
||||
</I18nProvider>
|
||||
</QueryClientProvider>,
|
||||
);
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks();
|
||||
});
|
||||
|
||||
describe("<DocumentsView> index-all toast (#776)", () => {
|
||||
it("renders the remaining count — never the word 'true'", () => {
|
||||
wirePayload = { enqueued: 3 };
|
||||
render();
|
||||
expect(capturedOnIndexAll).toBeDefined();
|
||||
|
||||
capturedOnIndexAll!();
|
||||
|
||||
expect(toastSuccess).toHaveBeenCalledTimes(1);
|
||||
const message = String(toastSuccess.mock.calls[0]![0]);
|
||||
expect(message).toContain("3");
|
||||
expect(message).not.toContain("true");
|
||||
expect(toastError).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("says everything is indexed on an honest 0, not 'indexing 0 documents'", () => {
|
||||
wirePayload = { enqueued: 0 };
|
||||
render();
|
||||
|
||||
capturedOnIndexAll!();
|
||||
|
||||
expect(toastSuccess).toHaveBeenCalledTimes(1);
|
||||
const message = String(toastSuccess.mock.calls[0]![0]);
|
||||
expect(message).toContain("already indexed");
|
||||
expect(message).not.toContain("true");
|
||||
});
|
||||
});
|
||||
@@ -54,6 +54,8 @@ function doc(id: string, title: string): InboundDocumentDto {
|
||||
servingClass: "inline",
|
||||
hasContentIndex: false,
|
||||
contentIndexSource: null,
|
||||
lastIndexAttemptAt: null,
|
||||
lastIndexOutcome: null,
|
||||
hasThumbnail: false,
|
||||
createdAt: "2025-10-05T08:00:00.000Z",
|
||||
updatedAt: "2025-10-05T08:00:00.000Z",
|
||||
|
||||
@@ -42,6 +42,8 @@ function doc(overrides: Partial<InboundDocumentDto> = {}): InboundDocumentDto {
|
||||
servingClass: "inline",
|
||||
hasContentIndex: false,
|
||||
contentIndexSource: null,
|
||||
lastIndexAttemptAt: null,
|
||||
lastIndexOutcome: null,
|
||||
hasThumbnail: false,
|
||||
createdAt: "2026-03-11T08:00:00.000Z",
|
||||
updatedAt: "2026-03-11T08:00:00.000Z",
|
||||
|
||||
@@ -38,6 +38,7 @@ import { cn } from "@/lib/utils";
|
||||
import {
|
||||
isAiReadSource,
|
||||
type DocumentContentIndexSourceValue,
|
||||
type DocumentIndexOutcomeValue,
|
||||
type DocumentSuggestionDto,
|
||||
type DocumentSummaryMode,
|
||||
} from "@/lib/validations/inbound-documents";
|
||||
@@ -74,6 +75,7 @@ export function DocumentAiSection({
|
||||
onCloseSummary,
|
||||
hasContentIndex,
|
||||
contentIndexSource,
|
||||
lastIndexOutcome = null,
|
||||
indexPending,
|
||||
onIndex,
|
||||
}: {
|
||||
@@ -111,11 +113,24 @@ export function DocumentAiSection({
|
||||
onCloseSummary: () => void;
|
||||
hasContentIndex: boolean;
|
||||
contentIndexSource: DocumentContentIndexSourceValue | null;
|
||||
/**
|
||||
* Refs #776 — why the last index attempt produced no index (null: never
|
||||
* tried, or the document IS indexed). Rendered as a muted plain-words line
|
||||
* beside the read action so a failed read is explainable, not a mystery.
|
||||
*/
|
||||
lastIndexOutcome?: DocumentIndexOutcomeValue | null;
|
||||
indexPending: boolean;
|
||||
onIndex: () => void;
|
||||
}) {
|
||||
const { t } = useTranslations();
|
||||
const aiRead = hasContentIndex && isAiReadSource(contentIndexSource);
|
||||
// Refs #776 — only an un-indexed document explains itself; once an index
|
||||
// exists the last failure is history. Muted per the text-colour rule: this
|
||||
// is meta about the document, not content.
|
||||
const indexOutcomeText =
|
||||
!hasContentIndex && lastIndexOutcome
|
||||
? t(`documents.detail.index.outcome.${lastIndexOutcome}`)
|
||||
: null;
|
||||
const readLabel = indexPending
|
||||
? t("documents.ai.reading")
|
||||
: aiRead
|
||||
@@ -233,6 +248,15 @@ export function DocumentAiSection({
|
||||
</div>
|
||||
) : null}
|
||||
|
||||
{indexOutcomeText ? (
|
||||
<p
|
||||
data-slot="document-index-outcome"
|
||||
className="text-muted-foreground text-xs"
|
||||
>
|
||||
{indexOutcomeText}
|
||||
</p>
|
||||
) : null}
|
||||
|
||||
{suggestErrorKey ? (
|
||||
<p role="alert" className="text-destructive text-sm">
|
||||
{suggestErrorKey}
|
||||
|
||||
@@ -875,6 +875,7 @@ export function DocumentDetailSheet({
|
||||
}}
|
||||
hasContentIndex={doc.hasContentIndex}
|
||||
contentIndexSource={doc.contentIndexSource}
|
||||
lastIndexOutcome={doc.lastIndexOutcome}
|
||||
indexPending={indexDoc.isPending}
|
||||
onIndex={runIndex}
|
||||
/>
|
||||
|
||||
@@ -41,7 +41,16 @@ export function useIndexDocument() {
|
||||
});
|
||||
}
|
||||
|
||||
/** Fire the corpus backfill; resolves with how many docs were enqueued. */
|
||||
/**
|
||||
* Fire the corpus backfill; resolves with how many docs were enqueued.
|
||||
*
|
||||
* Wire contract (#776): `enqueued` is the NUMBER of live documents the
|
||||
* backfill still has to index — the reindex route computes it from the same
|
||||
* live-scoped counts the usage gauge reads, and returns an honest 0 when no
|
||||
* job was created. The toast interpolates it as a count, so this type is
|
||||
* load-bearing: it must match the route's shape exactly (a boolean here once
|
||||
* rendered as "Indexing true document(s)").
|
||||
*/
|
||||
export function useReindexAll() {
|
||||
const queryClient = useQueryClient();
|
||||
return useMutation<{ enqueued: number }, Error, void>({
|
||||
|
||||
@@ -0,0 +1,89 @@
|
||||
import { describe, it, expect, vi } from "vitest";
|
||||
import { renderToStaticMarkup } from "react-dom/server";
|
||||
|
||||
/**
|
||||
* Refs #786 — the regenerate button's settling state.
|
||||
*
|
||||
* While a timed-out force regenerate is settling (the server is still
|
||||
* generating; the hook polls for the outcome) the button may NOT offer a
|
||||
* retry or read as failed: it stays disabled, keeps the spinner, and its
|
||||
* label says the assessment is still being prepared. Static renders pin the
|
||||
* three visible facts; the toast edges live in the hook/strip effect and are
|
||||
* pinned by the outcome mapping (`regenerateToastKind`).
|
||||
*/
|
||||
|
||||
vi.mock("next/navigation", () => ({
|
||||
usePathname: () => "/insights",
|
||||
}));
|
||||
vi.mock("sonner", () => ({ toast: { success: vi.fn(), error: vi.fn() } }));
|
||||
|
||||
import { InsightsTabStrip, regenerateToastKind } from "../insights-tab-strip";
|
||||
import { I18nProvider } from "@/lib/i18n/context";
|
||||
|
||||
function render(node: React.ReactNode) {
|
||||
return renderToStaticMarkup(
|
||||
<I18nProvider initialLocale="en">{node}</I18nProvider>,
|
||||
);
|
||||
}
|
||||
|
||||
describe("<InsightsTabStrip> regenerate settling state (#786)", () => {
|
||||
it("settling: button disabled, spinner on, label says still being prepared", () => {
|
||||
const html = render(
|
||||
<InsightsTabStrip
|
||||
onRegenerate={() => {}}
|
||||
regenerating={false}
|
||||
regenerateSettling
|
||||
/>,
|
||||
);
|
||||
expect(html).toContain('data-slot="insights-tab-strip-regenerate"');
|
||||
expect(html).toContain("Your assessment is still being prepared");
|
||||
expect(html).toContain("disabled");
|
||||
expect(html).toContain("animate-spin");
|
||||
});
|
||||
|
||||
it("idle: button enabled with the regenerate label", () => {
|
||||
const html = render(
|
||||
<InsightsTabStrip onRegenerate={() => {}} regenerating={false} />,
|
||||
);
|
||||
expect(html).toContain("Re-run assessment");
|
||||
expect(html).not.toContain("Your assessment is still being prepared");
|
||||
});
|
||||
});
|
||||
|
||||
describe("regenerateToastKind — falling-edge outcome mapping (#786)", () => {
|
||||
it("maps settle-failed to the honest settle error, never success", () => {
|
||||
expect(regenerateToastKind("settle-failed")).toEqual({
|
||||
kind: "error",
|
||||
messageKey: "insights.regenerateSettleFailed",
|
||||
});
|
||||
});
|
||||
|
||||
it("keeps the existing mappings", () => {
|
||||
expect(regenerateToastKind("fresh")).toEqual({
|
||||
kind: "success",
|
||||
messageKey: "insights.regenerateSuccess",
|
||||
});
|
||||
expect(regenerateToastKind("timeout")).toEqual({
|
||||
kind: "error",
|
||||
messageKey: "insights.regenerateError",
|
||||
});
|
||||
expect(regenerateToastKind("rate-limited")).toEqual({
|
||||
kind: "error",
|
||||
messageKey: "insights.regenerateRateLimited",
|
||||
});
|
||||
expect(regenerateToastKind("empty")).toEqual({
|
||||
kind: "error",
|
||||
messageKey: "insights.regenerateUnavailable",
|
||||
});
|
||||
expect(regenerateToastKind("no-provider")).toBe(null);
|
||||
// Legacy mounts without an outcome keep the unconditional success.
|
||||
expect(regenerateToastKind(null)).toEqual({
|
||||
kind: "success",
|
||||
messageKey: "insights.regenerateSuccess",
|
||||
});
|
||||
expect(regenerateToastKind(undefined)).toEqual({
|
||||
kind: "success",
|
||||
messageKey: "insights.regenerateSuccess",
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,143 @@
|
||||
import { describe, it, expect } from "vitest";
|
||||
|
||||
/**
|
||||
* Refs #786 — the async-truth contract for the force regenerate.
|
||||
*
|
||||
* A regenerate the client gives up on at 45 s is NOT a failure: the server's
|
||||
* inline generation keeps running and usually writes the cache moments later.
|
||||
* The old behaviour fired an error toast and then never converged (staleTime
|
||||
* 1 h + focus refetch off), so the page kept showing the old assessment while
|
||||
* claiming the refresh failed. The fix enters a "settling" state: remember
|
||||
* the baseline `cachedAt`, keep polling on the existing bounded cadence, and
|
||||
* resolve to a REAL outcome — fresh (cache advanced), settle-failed (a newer
|
||||
* failure marker, or the attempt cap).
|
||||
*
|
||||
* Two pure seams pin the state machine:
|
||||
* - `nextAdvisorPollInterval` gains settling as the second poll reason,
|
||||
* with attempts counted from the settle start.
|
||||
* - `resolveAdvisorSettle` decides fresh / settle-failed / keep-waiting.
|
||||
*/
|
||||
|
||||
import {
|
||||
ADVISOR_REVALIDATE_POLL_MAX_ATTEMPTS,
|
||||
ADVISOR_REVALIDATE_POLL_MS,
|
||||
nextAdvisorPollInterval,
|
||||
resolveAdvisorSettle,
|
||||
type AdvisorSettleState,
|
||||
} from "../use-insights-advisor";
|
||||
|
||||
function settle(
|
||||
overrides: Partial<AdvisorSettleState> = {},
|
||||
): AdvisorSettleState {
|
||||
return {
|
||||
baselineCachedAt: "2026-08-13T06:00:00.000Z",
|
||||
baselineFailed: false,
|
||||
startedAtUpdateCount: 4,
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
describe("nextAdvisorPollInterval — settling is the second poll reason (#786)", () => {
|
||||
it("polls while settling even when nothing reports revalidating", () => {
|
||||
expect(nextAdvisorPollInterval(false, 5, settle())).toBe(
|
||||
ADVISOR_REVALIDATE_POLL_MS,
|
||||
);
|
||||
expect(nextAdvisorPollInterval(undefined, 5, settle())).toBe(
|
||||
ADVISOR_REVALIDATE_POLL_MS,
|
||||
);
|
||||
});
|
||||
|
||||
it("counts settling attempts from the settle start, not from mount", () => {
|
||||
// 12 data updates total but only 8 since settling began → keep polling.
|
||||
expect(
|
||||
nextAdvisorPollInterval(false, 12, settle({ startedAtUpdateCount: 4 })),
|
||||
).toBe(ADVISOR_REVALIDATE_POLL_MS);
|
||||
});
|
||||
|
||||
it("stops at the settling attempt ceiling", () => {
|
||||
expect(
|
||||
nextAdvisorPollInterval(
|
||||
false,
|
||||
4 + ADVISOR_REVALIDATE_POLL_MAX_ATTEMPTS,
|
||||
settle({ startedAtUpdateCount: 4 }),
|
||||
),
|
||||
).toBe(false);
|
||||
});
|
||||
|
||||
it("keeps the pre-existing revalidating behaviour when not settling", () => {
|
||||
expect(nextAdvisorPollInterval(true, 1, null)).toBe(
|
||||
ADVISOR_REVALIDATE_POLL_MS,
|
||||
);
|
||||
expect(nextAdvisorPollInterval(false, 1, null)).toBe(false);
|
||||
expect(nextAdvisorPollInterval(true, 1)).toBe(ADVISOR_REVALIDATE_POLL_MS);
|
||||
});
|
||||
});
|
||||
|
||||
describe("resolveAdvisorSettle — the settle state machine (#786)", () => {
|
||||
it("resolves fresh once cachedAt advances past the baseline", () => {
|
||||
expect(
|
||||
resolveAdvisorSettle(
|
||||
settle(),
|
||||
{ cachedAt: "2026-08-13T06:02:31.000Z", generationFailed: false },
|
||||
6,
|
||||
),
|
||||
).toBe("fresh");
|
||||
});
|
||||
|
||||
it("resolves fresh on ANY cachedAt when there was no baseline (first generation)", () => {
|
||||
expect(
|
||||
resolveAdvisorSettle(
|
||||
settle({ baselineCachedAt: null }),
|
||||
{ cachedAt: "2026-08-13T06:02:31.000Z", generationFailed: false },
|
||||
6,
|
||||
),
|
||||
).toBe("fresh");
|
||||
});
|
||||
|
||||
it("keeps waiting while the cache is unchanged and attempts remain", () => {
|
||||
expect(
|
||||
resolveAdvisorSettle(
|
||||
settle(),
|
||||
{ cachedAt: "2026-08-13T06:00:00.000Z", generationFailed: false },
|
||||
6,
|
||||
),
|
||||
).toBe(null);
|
||||
});
|
||||
|
||||
it("resolves settle-failed on a NEWER failure marker (failed flipped during settling)", () => {
|
||||
expect(
|
||||
resolveAdvisorSettle(
|
||||
settle({ baselineFailed: false }),
|
||||
{ cachedAt: "2026-08-13T06:00:00.000Z", generationFailed: true },
|
||||
6,
|
||||
),
|
||||
).toBe("settle-failed");
|
||||
});
|
||||
|
||||
it("does NOT treat a failure marker that predates the settle as fresh evidence", () => {
|
||||
// The baseline already carried the failed flag — an unchanged flag says
|
||||
// nothing about THIS generation; keep waiting for cachedAt or the cap.
|
||||
expect(
|
||||
resolveAdvisorSettle(
|
||||
settle({ baselineFailed: true }),
|
||||
{ cachedAt: "2026-08-13T06:00:00.000Z", generationFailed: true },
|
||||
6,
|
||||
),
|
||||
).toBe(null);
|
||||
});
|
||||
|
||||
it("resolves settle-failed at the attempt cap — never waits forever", () => {
|
||||
expect(
|
||||
resolveAdvisorSettle(
|
||||
settle({ startedAtUpdateCount: 4 }),
|
||||
{ cachedAt: "2026-08-13T06:00:00.000Z", generationFailed: false },
|
||||
4 + ADVISOR_REVALIDATE_POLL_MAX_ATTEMPTS,
|
||||
),
|
||||
).toBe("settle-failed");
|
||||
});
|
||||
|
||||
it("a missing payload keeps waiting (no evidence either way)", () => {
|
||||
expect(resolveAdvisorSettle(settle(), null, 6)).toBe(null);
|
||||
expect(resolveAdvisorSettle(settle(), undefined, 6)).toBe(null);
|
||||
});
|
||||
});
|
||||
@@ -200,6 +200,7 @@ export function InsightsLayoutShell({ children }: { children: ReactNode }) {
|
||||
onRegenerate={advisorEnabled ? advisor.regenerate : undefined}
|
||||
regenerating={advisor.isRegenerating}
|
||||
regenerateOutcome={advisor.regenerateOutcome}
|
||||
regenerateSettling={advisor.regenerateSettling}
|
||||
availability={availability}
|
||||
visibleTileIds={visibleTileIds}
|
||||
tileOrder={tileOrder}
|
||||
|
||||
@@ -88,7 +88,20 @@ export interface InsightsTabStripProps {
|
||||
* mounts / no regenerate) falls back to the prior unconditional success.
|
||||
*/
|
||||
regenerateOutcome?:
|
||||
"fresh" | "empty" | "rate-limited" | "timeout" | "no-provider" | null;
|
||||
| "fresh"
|
||||
| "empty"
|
||||
| "rate-limited"
|
||||
| "timeout"
|
||||
| "no-provider"
|
||||
| "settle-failed"
|
||||
| null;
|
||||
/**
|
||||
* Refs #786 — true while a timed-out regenerate is settling (the server is
|
||||
* still generating; the hook polls for the proven outcome). The button
|
||||
* stays busy with a "still being prepared" label, and the falling-edge
|
||||
* toast waits for the COMBINED busy state so no error fires mid-settle.
|
||||
*/
|
||||
regenerateSettling?: boolean;
|
||||
/**
|
||||
* v1.4.27 F19 — analytics + event-driven availability inputs the
|
||||
* gating helper reads. When omitted the strip falls back to its
|
||||
@@ -637,10 +650,38 @@ function buildTabs(
|
||||
return entries;
|
||||
}
|
||||
|
||||
/**
|
||||
* Refs #786 — the falling-edge toast decision as a pure seam. Maps the last
|
||||
* settled outcome to the toast that fires when the busy state falls:
|
||||
* `settle-failed` (a settling poll proved the generation failed, or the
|
||||
* attempt cap ran out) gets its own honest error copy; `no-provider` stays
|
||||
* silent (the surface already shows the connect-AI empty state); a missing
|
||||
* outcome (legacy mounts) keeps the historical unconditional success.
|
||||
*/
|
||||
export function regenerateToastKind(
|
||||
outcome: InsightsTabStripProps["regenerateOutcome"] | undefined,
|
||||
): { kind: "success" | "error"; messageKey: string } | null {
|
||||
switch (outcome) {
|
||||
case "timeout":
|
||||
return { kind: "error", messageKey: "insights.regenerateError" };
|
||||
case "rate-limited":
|
||||
return { kind: "error", messageKey: "insights.regenerateRateLimited" };
|
||||
case "empty":
|
||||
return { kind: "error", messageKey: "insights.regenerateUnavailable" };
|
||||
case "settle-failed":
|
||||
return { kind: "error", messageKey: "insights.regenerateSettleFailed" };
|
||||
case "no-provider":
|
||||
return null;
|
||||
default:
|
||||
return { kind: "success", messageKey: "insights.regenerateSuccess" };
|
||||
}
|
||||
}
|
||||
|
||||
function InsightsTabStripImpl({
|
||||
onRegenerate,
|
||||
regenerating = false,
|
||||
regenerateOutcome,
|
||||
regenerateSettling = false,
|
||||
availability,
|
||||
visibleTileIds,
|
||||
tileOrder,
|
||||
@@ -686,24 +727,33 @@ function InsightsTabStripImpl({
|
||||
// regenerate claimed "refreshed" while serving the old text. The latest
|
||||
// outcome is read through a ref-free dep so the effect fires only on the
|
||||
// spinner edge.
|
||||
const lastRegeneratingRef = useRef<boolean>(regenerating);
|
||||
//
|
||||
// Refs #786 — the edge watches the COMBINED busy state (`regenerating ||
|
||||
// regenerateSettling`): a timed-out regenerate flows straight into
|
||||
// settling, so the spinner never falls at 45 s and NO error toast fires
|
||||
// while the server may still deliver. The edge fires once settling
|
||||
// resolves, with the proven outcome — `"fresh"` (success) or
|
||||
// `"settle-failed"` (its own honest error copy). The mapping is the pure
|
||||
// `regenerateToastKind` seam above.
|
||||
const busy = regenerating || regenerateSettling;
|
||||
const lastBusyRef = useRef<boolean>(busy);
|
||||
useEffect(() => {
|
||||
if (lastRegeneratingRef.current && !regenerating) {
|
||||
if (regenerateOutcome === "timeout") {
|
||||
toast.error(t("insights.regenerateError"));
|
||||
} else if (regenerateOutcome === "rate-limited") {
|
||||
toast.error(t("insights.regenerateRateLimited"));
|
||||
} else if (regenerateOutcome === "empty") {
|
||||
toast.error(t("insights.regenerateUnavailable"));
|
||||
} else if (regenerateOutcome !== "no-provider") {
|
||||
// `"fresh"` or legacy `undefined`/null → success.
|
||||
toast.success(t("insights.regenerateSuccess"));
|
||||
if (lastBusyRef.current && !busy) {
|
||||
const decision = regenerateToastKind(regenerateOutcome);
|
||||
if (decision) {
|
||||
if (decision.kind === "error") {
|
||||
toast.error(t(decision.messageKey));
|
||||
} else {
|
||||
toast.success(t(decision.messageKey));
|
||||
}
|
||||
}
|
||||
}
|
||||
lastRegeneratingRef.current = regenerating;
|
||||
}, [regenerating, regenerateOutcome, t]);
|
||||
lastBusyRef.current = busy;
|
||||
}, [busy, regenerateOutcome, t]);
|
||||
|
||||
const regenerateLabel = t("insights.regenerateAnalysis");
|
||||
const regenerateLabel = regenerateSettling
|
||||
? t("insights.regenerateSettling")
|
||||
: t("insights.regenerateAnalysis");
|
||||
const customizeLabel = t("insights.customize");
|
||||
|
||||
// Overflow affordance for the pill row. The row hides its scrollbar
|
||||
@@ -1000,7 +1050,9 @@ function InsightsTabStripImpl({
|
||||
<button
|
||||
type="button"
|
||||
onClick={onRegenerate}
|
||||
disabled={regenerating}
|
||||
// Refs #786 — settling keeps the button busy: the work is still
|
||||
// running server-side, so a retry now would only pile on.
|
||||
disabled={busy}
|
||||
aria-label={regenerateLabel}
|
||||
title={regenerateLabel}
|
||||
data-slot="insights-tab-strip-regenerate"
|
||||
@@ -1016,7 +1068,7 @@ function InsightsTabStripImpl({
|
||||
"disabled:cursor-not-allowed disabled:opacity-50",
|
||||
)}
|
||||
>
|
||||
{regenerating ? (
|
||||
{busy ? (
|
||||
<Loader2
|
||||
className="h-4 w-4 animate-spin motion-reduce:animate-none"
|
||||
aria-hidden="true"
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
"use client";
|
||||
|
||||
import { useCallback } from "react";
|
||||
import { useCallback, useState } from "react";
|
||||
import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
|
||||
import type { InsightResult } from "@/lib/ai/types";
|
||||
// Type-only — the runtime schemas load lazily inside `fetchAdvisor` so
|
||||
@@ -139,16 +139,97 @@ const FORCE_ADVISOR_TIMEOUT_MS = 45_000;
|
||||
export const ADVISOR_REVALIDATE_POLL_MS = 25_000;
|
||||
export const ADVISOR_REVALIDATE_POLL_MAX_ATTEMPTS = 10;
|
||||
|
||||
/**
|
||||
* Refs #786 — the settling state a timed-out force regenerate enters.
|
||||
*
|
||||
* Async-truth contract, client end (the server end is documented at the head
|
||||
* of `src/app/api/insights/generate/route.ts`): the force POST generates
|
||||
* INLINE and keeps running after the client aborts at 45 s — the server
|
||||
* usually writes the fresh cache moments later. A page that knows work is
|
||||
* running must keep saying so until it can prove an outcome, so the timeout
|
||||
* is NOT surfaced as an error. Instead the hook remembers the baseline it was
|
||||
* showing and polls (bounded, same cadence as the revalidating poll) until it
|
||||
* can prove one of three outcomes: the cache advanced past the baseline
|
||||
* (fresh), a failure marker newer than the baseline appeared (settle-failed),
|
||||
* or the attempt cap ran out (settle-failed — never wait forever).
|
||||
*/
|
||||
export interface AdvisorSettleState {
|
||||
/** `cachedAt` of the payload shown when the client gave up (null = none). */
|
||||
baselineCachedAt: string | null;
|
||||
/** Whether a failure marker was ALREADY reported at that time — an
|
||||
* unchanged flag proves nothing about THIS generation. */
|
||||
baselineFailed: boolean;
|
||||
/** The query's `dataUpdateCount` when settling began; poll attempts count
|
||||
* from here, not from mount. */
|
||||
startedAtUpdateCount: number;
|
||||
}
|
||||
|
||||
/** The outcome the falling-edge toast reports for a settled regenerate. */
|
||||
export type AdvisorRegenerateOutcome = AdvisorFetchOutcome | "settle-failed";
|
||||
|
||||
/**
|
||||
* Decide what a settling poll result proves. Pure test seam — the effect in
|
||||
* `useInsightsAdvisorQuery` applies the returned resolution verbatim.
|
||||
* `null` = no evidence yet, keep polling.
|
||||
*/
|
||||
export function resolveAdvisorSettle(
|
||||
settling: AdvisorSettleState,
|
||||
payload:
|
||||
| Pick<InsightAdvisorPayload, "cachedAt" | "generationFailed">
|
||||
| null
|
||||
| undefined,
|
||||
dataUpdateCount: number,
|
||||
): "fresh" | "settle-failed" | null {
|
||||
const cachedAt = payload?.cachedAt ?? null;
|
||||
if (
|
||||
cachedAt != null &&
|
||||
(settling.baselineCachedAt == null ||
|
||||
new Date(cachedAt).getTime() >
|
||||
new Date(settling.baselineCachedAt).getTime())
|
||||
) {
|
||||
return "fresh";
|
||||
}
|
||||
// A failure marker that FLIPPED during settling is the server saying the
|
||||
// inline generation died after we stopped watching — an honest error. One
|
||||
// that was already set at the baseline predates this attempt and proves
|
||||
// nothing; keep waiting for the cache or the cap.
|
||||
if ((payload?.generationFailed ?? false) && !settling.baselineFailed) {
|
||||
return "settle-failed";
|
||||
}
|
||||
if (
|
||||
dataUpdateCount - settling.startedAtUpdateCount >=
|
||||
ADVISOR_REVALIDATE_POLL_MAX_ATTEMPTS
|
||||
) {
|
||||
return "settle-failed";
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Decide whether the advisor query schedules its next poll. Pure so the
|
||||
* ceiling + stop conditions are unit-testable: returns the interval
|
||||
* while the last payload carries `revalidating: true`, `false` once a
|
||||
* response comes back with the flag falsy OR the attempt cap is hit.
|
||||
*
|
||||
* Refs #786 — settling is the SECOND poll reason: while a timed-out force
|
||||
* regenerate awaits its real outcome the query polls regardless of the
|
||||
* `revalidating` flag, with attempts counted from the settle start so a
|
||||
* long-mounted page gets the full budget.
|
||||
*/
|
||||
export function nextAdvisorPollInterval(
|
||||
revalidating: boolean | undefined,
|
||||
dataUpdateCount: number,
|
||||
settling?: Pick<AdvisorSettleState, "startedAtUpdateCount"> | null,
|
||||
): number | false {
|
||||
if (settling) {
|
||||
if (
|
||||
dataUpdateCount - settling.startedAtUpdateCount >=
|
||||
ADVISOR_REVALIDATE_POLL_MAX_ATTEMPTS
|
||||
) {
|
||||
return false;
|
||||
}
|
||||
return ADVISOR_REVALIDATE_POLL_MS;
|
||||
}
|
||||
if (!revalidating) return false;
|
||||
if (dataUpdateCount >= ADVISOR_REVALIDATE_POLL_MAX_ATTEMPTS) return false;
|
||||
return ADVISOR_REVALIDATE_POLL_MS;
|
||||
@@ -290,8 +371,19 @@ export interface UseInsightsAdvisorResult {
|
||||
* fires the "refreshed" toast only when this is `"fresh"`, so a slow gen
|
||||
* the client gave up on (`"timeout"`) or a missing provider
|
||||
* (`"no-provider"`) never reads as "done".
|
||||
*
|
||||
* Refs #786 — a timed-out regenerate no longer surfaces `"timeout"` at the
|
||||
* spinner edge: the hook keeps `regenerateSettling` true and polls until it
|
||||
* can PROVE an outcome, then reports `"fresh"` (cache advanced) or
|
||||
* `"settle-failed"` (newer failure marker / attempt cap) here.
|
||||
*/
|
||||
regenerateOutcome: AdvisorFetchOutcome | null;
|
||||
regenerateOutcome: AdvisorRegenerateOutcome | null;
|
||||
/**
|
||||
* Refs #786 — true while a timed-out force regenerate awaits its real
|
||||
* outcome (bounded poll). The strip keeps the button busy and says the
|
||||
* assessment is still being prepared; NO toast fires until this falls.
|
||||
*/
|
||||
regenerateSettling: boolean;
|
||||
/**
|
||||
* v1.15.20 — the outcome of the last settled READ. Lets surfaces
|
||||
* distinguish "no briefing yet, a generate could help" (`empty` /
|
||||
@@ -340,6 +432,13 @@ export function useInsightsAdvisorQuery(
|
||||
enabled: boolean,
|
||||
): UseInsightsAdvisorResult {
|
||||
const queryClient = useQueryClient();
|
||||
// Refs #786 — settling state for a timed-out force regenerate (see the
|
||||
// contract comment above `AdvisorSettleState`). `settleOutcome` carries the
|
||||
// PROVEN outcome once settling resolves; both reset on the next regenerate.
|
||||
const [settling, setSettling] = useState<AdvisorSettleState | null>(null);
|
||||
const [settleOutcome, setSettleOutcome] = useState<
|
||||
"fresh" | "settle-failed" | null
|
||||
>(null);
|
||||
const query = useQuery({
|
||||
queryKey: queryKeys.insightsAdvisor(),
|
||||
// v1.15.20 — the cache stores the full tagged result (payload +
|
||||
@@ -353,13 +452,49 @@ export function useInsightsAdvisorQuery(
|
||||
// v1.16.7 — converge a stale-served briefing in-session: while the
|
||||
// last GET reported `revalidating: true` (warm enqueued), poll on a
|
||||
// bounded interval until a response comes back with the flag falsy.
|
||||
// Refs #786 — settling (timed-out regenerate awaiting its outcome) is
|
||||
// the second poll reason, attempts counted from the settle start.
|
||||
refetchInterval: (query) =>
|
||||
nextAdvisorPollInterval(
|
||||
query.state.data?.payload?.revalidating,
|
||||
query.state.dataUpdateCount,
|
||||
settling,
|
||||
),
|
||||
});
|
||||
|
||||
// `dataUpdateCount` lives on the query STATE (the refetchInterval callback
|
||||
// reads it there); the observer result only exposes `dataUpdatedAt`. Read
|
||||
// it imperatively from the cache where the settle machinery needs it.
|
||||
const readDataUpdateCount = useCallback(
|
||||
() =>
|
||||
queryClient
|
||||
.getQueryCache()
|
||||
.find({ queryKey: queryKeys.insightsAdvisor() })?.state
|
||||
.dataUpdateCount ?? 0,
|
||||
[queryClient],
|
||||
);
|
||||
|
||||
// Apply the settle resolution the pure seam proves. Each poll result lands
|
||||
// in `query.data` (the poll's own cache write IS the fresh payload — no
|
||||
// second setQueryData needed) and re-renders this hook; the seam decides
|
||||
// fresh / settle-failed / keep-waiting. Render-phase state adjustment (the
|
||||
// React-documented alternative to a setState-in-effect): guarded so it
|
||||
// fires at most once per settle — resolution non-null clears `settling`,
|
||||
// and the next render skips the whole block. Both flips land in the SAME
|
||||
// committed render, so the strip's falling-edge toast sees the proven
|
||||
// outcome exactly when the busy state drops.
|
||||
if (settling) {
|
||||
const resolution = resolveAdvisorSettle(
|
||||
settling,
|
||||
query.data?.payload,
|
||||
readDataUpdateCount(),
|
||||
);
|
||||
if (resolution) {
|
||||
setSettling(null);
|
||||
setSettleOutcome(resolution);
|
||||
}
|
||||
}
|
||||
|
||||
const mutation = useMutation({
|
||||
mutationFn: () => fetchAdvisor({ force: true }),
|
||||
onSuccess: (result) => {
|
||||
@@ -373,6 +508,25 @@ export function useInsightsAdvisorQuery(
|
||||
// client gave up at 45 s, so re-read the GET to converge — but the
|
||||
// honest toast is gated on `regenerateOutcome` below, not on this
|
||||
// invalidate.
|
||||
//
|
||||
// Refs #786 — on a TIMEOUT specifically, one early invalidate is not
|
||||
// convergence: the generation routinely outlives it, the re-read
|
||||
// serves the OLD cache, and staleTime 1 h + focus-refetch-off remove
|
||||
// every later trigger. Enter the settling state instead: remember
|
||||
// the baseline this page is showing and poll (bounded) until the
|
||||
// cache provably advances or fails — the strip stays busy and no
|
||||
// error toast fires until then.
|
||||
if (result.outcome === "timeout") {
|
||||
const current = queryClient.getQueryData<AdvisorFetchResult>(
|
||||
queryKeys.insightsAdvisor(),
|
||||
);
|
||||
setSettleOutcome(null);
|
||||
setSettling({
|
||||
baselineCachedAt: current?.payload?.cachedAt ?? null,
|
||||
baselineFailed: current?.payload?.generationFailed ?? false,
|
||||
startedAtUpdateCount: readDataUpdateCount(),
|
||||
});
|
||||
}
|
||||
queryClient.invalidateQueries({
|
||||
queryKey: queryKeys.insightsAdvisor(),
|
||||
});
|
||||
@@ -403,7 +557,13 @@ export function useInsightsAdvisorQuery(
|
||||
// status-query flip on a sub-page that re-renders the shell would
|
||||
// otherwise re-reconcile the whole strip mid-gesture and eat taps.
|
||||
const { mutate } = mutation;
|
||||
const regenerate = useCallback(() => mutate(), [mutate]);
|
||||
const regenerate = useCallback(() => {
|
||||
// Refs #786 — a new attempt clears any previous settle verdict so a
|
||||
// stale "fresh"/"settle-failed" cannot leak into this cycle's toast.
|
||||
setSettling(null);
|
||||
setSettleOutcome(null);
|
||||
mutate();
|
||||
}, [mutate]);
|
||||
|
||||
return {
|
||||
payload: query.data?.payload ?? null,
|
||||
@@ -413,7 +573,10 @@ export function useInsightsAdvisorQuery(
|
||||
regenerate,
|
||||
isRegenerating: mutation.isPending,
|
||||
regenerateError: (mutation.error as Error | null) ?? null,
|
||||
regenerateOutcome: mutation.data?.outcome ?? null,
|
||||
// Refs #786 — a proven settle verdict wins over the mutation's raw
|
||||
// outcome ("timeout" while settling is an interim state, not a result).
|
||||
regenerateOutcome: settleOutcome ?? mutation.data?.outcome ?? null,
|
||||
regenerateSettling: settling !== null,
|
||||
readOutcome: query.data?.outcome ?? null,
|
||||
// Absent field (pre-v1.18.9 cached payload, or query still settling) →
|
||||
// assume a provider is present so a transient unknown never flashes a
|
||||
|
||||
@@ -0,0 +1,278 @@
|
||||
import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||
|
||||
/**
|
||||
* Refs #776 — an index attempt that ends without an index must record why.
|
||||
*
|
||||
* Before this, a failed attempt (raster fail, empty scan, provider error)
|
||||
* left its reason only in wide-events: `hasContentIndex=false` was
|
||||
* indistinguishable from "never tried". Pins:
|
||||
*
|
||||
* - `indexLoadedDocument` (the choke point the job AND the backfill share)
|
||||
* records every attempt on the row: `lastIndexAttemptAt` always,
|
||||
* `lastIndexOutcome` = the failure reason, or null on success.
|
||||
* - The provider path's specific failure (raster-failed /
|
||||
* pdf-needs-anthropic / provider-error / empty-transcription) survives
|
||||
* the local fallback: when local can only say "no text", the recorded
|
||||
* reason is the provider path's more actionable one.
|
||||
* - An empty provider transcription NEVER becomes a "successful" empty
|
||||
* index — it falls through to local instead.
|
||||
*/
|
||||
|
||||
vi.mock("@/lib/db", () => ({
|
||||
prisma: {
|
||||
inboundDocument: { updateMany: vi.fn() },
|
||||
},
|
||||
}));
|
||||
vi.mock("@/lib/documents/ai-route-support", () => ({
|
||||
loadOwnedDocument: vi.fn(),
|
||||
prepareVisionInput: vi.fn(),
|
||||
}));
|
||||
vi.mock("@/lib/documents/content-index", () => ({
|
||||
upsertContentIndex: vi.fn().mockResolvedValue({ tokenCount: 5 }),
|
||||
}));
|
||||
vi.mock("@/lib/documents/describe", () => ({
|
||||
transcribeDocument: vi.fn(),
|
||||
}));
|
||||
vi.mock("@/lib/documents/local-extract", () => ({
|
||||
localExtractText: vi.fn(),
|
||||
}));
|
||||
vi.mock("@/lib/documents/store", () => ({
|
||||
decryptDocumentContent: vi.fn(() => Buffer.from([1, 2, 3])),
|
||||
}));
|
||||
vi.mock("@/lib/labs/ocr-upload", () => ({
|
||||
detectOcrMimeType: vi.fn(() => "application/pdf"),
|
||||
}));
|
||||
vi.mock("@/lib/documents/provider-order", () => ({
|
||||
resolveDocumentVisionProvider: vi.fn(),
|
||||
}));
|
||||
vi.mock("@/lib/documents/document-settings", () => ({
|
||||
documentAutoReadEnabled: vi.fn(),
|
||||
}));
|
||||
vi.mock("@/lib/ai/consent-guard", () => ({
|
||||
isExternalDocumentEgress: (providerType: string) => providerType !== "local",
|
||||
}));
|
||||
vi.mock("@/lib/ai/coach/budget", () => ({
|
||||
buildDateKey: vi.fn(() => "2026-08-13"),
|
||||
reserveBudget: vi.fn(),
|
||||
reconcileSpend: vi.fn().mockResolvedValue(undefined),
|
||||
resolveDailyCap: vi.fn(() => 1000),
|
||||
}));
|
||||
vi.mock("@/lib/ai/ai-budgets", () => ({
|
||||
AI_BUDGETS: { documentTranscribe: { temperature: 0, maxTokens: 4000 } },
|
||||
}));
|
||||
|
||||
import { indexDocumentContent, recordIndexAttempt } from "../index-document";
|
||||
import { prisma } from "@/lib/db";
|
||||
import {
|
||||
loadOwnedDocument,
|
||||
prepareVisionInput,
|
||||
} from "@/lib/documents/ai-route-support";
|
||||
import { upsertContentIndex } from "@/lib/documents/content-index";
|
||||
import { transcribeDocument } from "@/lib/documents/describe";
|
||||
import { documentAutoReadEnabled } from "@/lib/documents/document-settings";
|
||||
import { localExtractText } from "@/lib/documents/local-extract";
|
||||
import { resolveDocumentVisionProvider } from "@/lib/documents/provider-order";
|
||||
import { reserveBudget } from "@/lib/ai/coach/budget";
|
||||
|
||||
const DOC = {
|
||||
id: "doc-1",
|
||||
kind: "OTHER",
|
||||
contentEncrypted: new Uint8Array([1, 2, 3]),
|
||||
contentCodec: "binary2",
|
||||
mimeType: "application/pdf",
|
||||
status: "STORED",
|
||||
};
|
||||
|
||||
const PICK = {
|
||||
chain: [{ providerType: "anthropic", instance: {} }],
|
||||
pick: {
|
||||
entry: { providerType: "anthropic", instance: {} },
|
||||
providerType: "anthropic",
|
||||
pdfSupported: true,
|
||||
},
|
||||
};
|
||||
|
||||
function visionOk() {
|
||||
vi.mocked(prepareVisionInput).mockResolvedValue({
|
||||
ok: true,
|
||||
images: [],
|
||||
documents: [{ mediaType: "application/pdf", dataBase64: "AA==" }],
|
||||
} as never);
|
||||
}
|
||||
|
||||
function localEmpty() {
|
||||
vi.mocked(localExtractText).mockResolvedValue({
|
||||
ok: false,
|
||||
reason: "empty",
|
||||
} as never);
|
||||
}
|
||||
|
||||
function lastRecordedOutcome(): unknown {
|
||||
const calls = vi.mocked(prisma.inboundDocument.updateMany).mock.calls;
|
||||
const withOutcome = calls.filter(
|
||||
([arg]) => arg.data && "lastIndexOutcome" in (arg.data as object),
|
||||
);
|
||||
const last = withOutcome[withOutcome.length - 1];
|
||||
return last ? (last[0].data as { lastIndexOutcome: unknown }) : undefined;
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks();
|
||||
vi.mocked(loadOwnedDocument).mockResolvedValue(DOC as never);
|
||||
vi.mocked(documentAutoReadEnabled).mockResolvedValue(true);
|
||||
vi.mocked(resolveDocumentVisionProvider).mockResolvedValue(PICK as never);
|
||||
vi.mocked(reserveBudget).mockResolvedValue({
|
||||
allowed: true,
|
||||
reserved: 1,
|
||||
} as never);
|
||||
vi.mocked(localExtractText).mockResolvedValue({
|
||||
ok: true,
|
||||
text: "glucose fasting creatinine values report",
|
||||
source: "local-pdf",
|
||||
} as never);
|
||||
vi.mocked(prisma.inboundDocument.updateMany).mockResolvedValue({
|
||||
count: 1,
|
||||
} as never);
|
||||
});
|
||||
|
||||
describe("recordIndexAttempt — the row-level writer", () => {
|
||||
it("records a failure reason with the attempt instant, owner-scoped", async () => {
|
||||
await recordIndexAttempt("user-1", "doc-1", {
|
||||
indexed: false,
|
||||
reason: "local-empty",
|
||||
});
|
||||
expect(prisma.inboundDocument.updateMany).toHaveBeenCalledWith({
|
||||
where: { id: "doc-1", userId: "user-1", deletedAt: null },
|
||||
data: {
|
||||
lastIndexAttemptAt: expect.any(Date),
|
||||
lastIndexOutcome: "local-empty",
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it("clears the outcome on a successful attempt (the index row is the success signal)", async () => {
|
||||
await recordIndexAttempt("user-1", "doc-1", {
|
||||
indexed: true,
|
||||
source: "vision",
|
||||
tokenCount: 5,
|
||||
});
|
||||
expect(prisma.inboundDocument.updateMany).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
data: expect.objectContaining({ lastIndexOutcome: null }),
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it("writes nothing for not-found (no row to explain) and never throws", async () => {
|
||||
await recordIndexAttempt("user-1", "missing", {
|
||||
indexed: false,
|
||||
reason: "not-found",
|
||||
});
|
||||
expect(prisma.inboundDocument.updateMany).not.toHaveBeenCalled();
|
||||
|
||||
vi.mocked(prisma.inboundDocument.updateMany).mockRejectedValueOnce(
|
||||
new Error("db down"),
|
||||
);
|
||||
await expect(
|
||||
recordIndexAttempt("user-1", "doc-1", {
|
||||
indexed: false,
|
||||
reason: "local-empty",
|
||||
}),
|
||||
).resolves.toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
describe("indexDocumentContent — the attempt record at the choke point", () => {
|
||||
it("records the outcome on failure (job + backfill share this path)", async () => {
|
||||
vi.mocked(resolveDocumentVisionProvider).mockResolvedValue({
|
||||
chain: [],
|
||||
pick: null,
|
||||
} as never);
|
||||
localEmpty();
|
||||
|
||||
await indexDocumentContent("user-1", "doc-1");
|
||||
expect(lastRecordedOutcome()).toMatchObject({
|
||||
lastIndexOutcome: "local-empty",
|
||||
});
|
||||
});
|
||||
|
||||
it("records a cleared outcome on success", async () => {
|
||||
visionOk();
|
||||
vi.mocked(transcribeDocument).mockResolvedValue({
|
||||
text: "haemoglobin 14.2 cholesterol 190",
|
||||
} as never);
|
||||
|
||||
await indexDocumentContent("user-1", "doc-1");
|
||||
expect(lastRecordedOutcome()).toMatchObject({ lastIndexOutcome: null });
|
||||
});
|
||||
});
|
||||
|
||||
describe("indexDocumentContent — the provider path's reason survives the local fallback", () => {
|
||||
it("provider-error: the provider threw and local had no text", async () => {
|
||||
visionOk();
|
||||
vi.mocked(transcribeDocument).mockRejectedValueOnce(new Error("boom"));
|
||||
localEmpty();
|
||||
|
||||
const outcome = await indexDocumentContent("user-1", "doc-1");
|
||||
expect(outcome).toEqual({ indexed: false, reason: "provider-error" });
|
||||
});
|
||||
|
||||
it("raster-failed: the PDF could not be rendered and local had no text", async () => {
|
||||
vi.mocked(prepareVisionInput).mockResolvedValue({
|
||||
ok: false,
|
||||
reason: "rasterFailed",
|
||||
} as never);
|
||||
localEmpty();
|
||||
|
||||
const outcome = await indexDocumentContent("user-1", "doc-1");
|
||||
expect(outcome).toEqual({ indexed: false, reason: "raster-failed" });
|
||||
});
|
||||
|
||||
it("pdf-needs-anthropic: the provider cannot read PDFs and local had no text", async () => {
|
||||
vi.mocked(prepareVisionInput).mockResolvedValue({
|
||||
ok: false,
|
||||
reason: "pdfNeedsAnthropic",
|
||||
} as never);
|
||||
localEmpty();
|
||||
|
||||
const outcome = await indexDocumentContent("user-1", "doc-1");
|
||||
expect(outcome).toEqual({ indexed: false, reason: "pdf-needs-anthropic" });
|
||||
});
|
||||
|
||||
it("a plain local failure keeps its own reason when the provider path had none", async () => {
|
||||
vi.mocked(resolveDocumentVisionProvider).mockResolvedValue({
|
||||
chain: [],
|
||||
pick: null,
|
||||
} as never);
|
||||
localEmpty();
|
||||
|
||||
const outcome = await indexDocumentContent("user-1", "doc-1");
|
||||
expect(outcome).toEqual({ indexed: false, reason: "local-empty" });
|
||||
});
|
||||
});
|
||||
|
||||
describe("indexDocumentContent — the empty-transcription guard", () => {
|
||||
it("an empty provider answer NEVER becomes a successful empty index", async () => {
|
||||
visionOk();
|
||||
vi.mocked(transcribeDocument).mockResolvedValue({ text: " " } as never);
|
||||
localEmpty();
|
||||
|
||||
const outcome = await indexDocumentContent("user-1", "doc-1");
|
||||
expect(outcome).toEqual({
|
||||
indexed: false,
|
||||
reason: "empty-transcription",
|
||||
});
|
||||
expect(upsertContentIndex).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("an empty provider answer still lands the LOCAL text when there is one", async () => {
|
||||
visionOk();
|
||||
vi.mocked(transcribeDocument).mockResolvedValue({ text: "" } as never);
|
||||
|
||||
const outcome = await indexDocumentContent("user-1", "doc-1");
|
||||
expect(outcome).toMatchObject({ indexed: true, source: "local-pdf" });
|
||||
expect(upsertContentIndex).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ source: "local-pdf" }),
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,71 @@
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import { Buffer } from "node:buffer";
|
||||
|
||||
/**
|
||||
* Refs #776 — the raster-failed chain, end to end through the REAL
|
||||
* rasterizer: a PDF that pdfjs cannot open must surface as
|
||||
* `{ ok: false, reason: "rasterFailed" }` from `prepareVisionInput` (for an
|
||||
* image-only provider), which the index tree records as
|
||||
* `lastIndexOutcome: "raster-failed"` — never silently, never mislabelled as
|
||||
* "needs Anthropic".
|
||||
*
|
||||
* Fixture note (deliberate abstention, verified 2026-08-13): the reporter's
|
||||
* residual scan is a CCITT Group 4 (`/CCITTFaxDecode`, `/K -1`) single-image
|
||||
* PDF — a codec class pdfjs rasterization is known to fail on. A minimal
|
||||
* programmatically generated CCITT-G4 PDF was probed against THIS repo's
|
||||
* pdfjs: it renders the page as blank (`ok: true`, 1 page) rather than
|
||||
* throwing, because pdfjs skips the undecodable image object — so a tiny
|
||||
* in-repo CCITT fixture cannot deterministically go red on this version and
|
||||
* would be flaky across pdfjs upgrades. The deterministic stand-in below (a
|
||||
* PDF magic header over an unparseable body) exercises the same failure arm:
|
||||
* pdfjs throws at open, `rasterizePdf` reports `render-failed` (with the
|
||||
* pdfjs error text annotated), and the mapping to `rasterFailed` is what the
|
||||
* outcome column persists.
|
||||
*/
|
||||
|
||||
vi.mock("@/lib/documents/store", () => ({
|
||||
decryptDocumentContent: vi.fn(),
|
||||
}));
|
||||
|
||||
import { prepareVisionInput } from "../ai-route-support";
|
||||
import { rasterizePdf } from "../rasterize-pdf";
|
||||
import { decryptDocumentContent } from "@/lib/documents/store";
|
||||
|
||||
/** PDF magic bytes over a body pdfjs cannot parse. */
|
||||
const UNRENDERABLE_PDF = Buffer.concat([
|
||||
Buffer.from("%PDF-1.4\n"),
|
||||
Buffer.from(Array.from({ length: 64 }, (_, i) => (i * 37) % 256)),
|
||||
]);
|
||||
|
||||
const DOC = {
|
||||
id: "doc-1",
|
||||
kind: "OTHER",
|
||||
contentEncrypted: new Uint8Array([1]),
|
||||
contentCodec: "binary2",
|
||||
mimeType: "application/pdf",
|
||||
status: "STORED",
|
||||
};
|
||||
|
||||
describe("raster-failed end to end (#776)", () => {
|
||||
it("rasterizePdf reports render-failed for a PDF pdfjs cannot open", async () => {
|
||||
const result = await rasterizePdf(UNRENDERABLE_PDF);
|
||||
expect(result).toEqual({ ok: false, reason: "render-failed" });
|
||||
});
|
||||
|
||||
it("prepareVisionInput maps it to rasterFailed for an image-only provider", async () => {
|
||||
vi.mocked(decryptDocumentContent).mockReturnValue(
|
||||
UNRENDERABLE_PDF as never,
|
||||
);
|
||||
// pdfSupported=false → the raster path runs for real.
|
||||
const vision = await prepareVisionInput(DOC as never, false);
|
||||
expect(vision).toEqual({ ok: false, reason: "rasterFailed" });
|
||||
});
|
||||
|
||||
it("a PDF-capable provider never rasterizes, so the same bytes stay readable natively", async () => {
|
||||
vi.mocked(decryptDocumentContent).mockReturnValue(
|
||||
UNRENDERABLE_PDF as never,
|
||||
);
|
||||
const vision = await prepareVisionInput(DOC as never, true);
|
||||
expect(vision).toMatchObject({ ok: true });
|
||||
});
|
||||
});
|
||||
@@ -102,10 +102,13 @@ describe("rasterizePdf", () => {
|
||||
it("returns { ok: false } and never throws on malformed input", async () => {
|
||||
await expect(
|
||||
rasterizePdf(Buffer.from("this is not a pdf")),
|
||||
).resolves.toEqual({ ok: false });
|
||||
).resolves.toEqual({ ok: false, reason: "render-failed" });
|
||||
});
|
||||
|
||||
it("returns { ok: false } on an empty buffer", async () => {
|
||||
await expect(rasterizePdf(Buffer.alloc(0))).resolves.toEqual({ ok: false });
|
||||
await expect(rasterizePdf(Buffer.alloc(0))).resolves.toEqual({
|
||||
ok: false,
|
||||
reason: "render-failed",
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -143,7 +143,15 @@ export type VisionInput =
|
||||
}[];
|
||||
documents: { mediaType: "application/pdf"; dataBase64: string }[];
|
||||
}
|
||||
| { ok: false; reason: "decryptFailed" | "fileType" | "pdfNeedsAnthropic" };
|
||||
| {
|
||||
ok: false;
|
||||
// Refs #776 — `rasterFailed` (this PDF would not render on a raster-
|
||||
// capable build) is distinct from `pdfNeedsAnthropic` (this build /
|
||||
// provider cannot take a PDF at all): they need different fixes, so
|
||||
// they must not share a label.
|
||||
reason:
|
||||
"decryptFailed" | "fileType" | "pdfNeedsAnthropic" | "rasterFailed";
|
||||
};
|
||||
|
||||
/**
|
||||
* Decrypt the stored original, re-derive its MIME from the bytes (never trust
|
||||
@@ -197,7 +205,14 @@ export async function prepareVisionInput(
|
||||
if (raster.ok) {
|
||||
return { ok: true, images: raster.images, documents: [] };
|
||||
}
|
||||
return { ok: false, reason: "pdfNeedsAnthropic" };
|
||||
// Refs #776 — a build that cannot rasterize at all is a capability gap a
|
||||
// PDF-native (Anthropic) provider would cover; a render failure on a
|
||||
// capable build is a property of THIS document. Report them apart.
|
||||
return {
|
||||
ok: false,
|
||||
reason:
|
||||
raster.reason === "unsupported" ? "pdfNeedsAnthropic" : "rasterFailed",
|
||||
};
|
||||
}
|
||||
|
||||
return {
|
||||
|
||||
@@ -52,22 +52,71 @@ import { documentAutoReadEnabled } from "@/lib/documents/document-settings";
|
||||
import { localExtractText } from "@/lib/documents/local-extract";
|
||||
import { resolveDocumentVisionProvider } from "@/lib/documents/provider-order";
|
||||
import { decryptDocumentContent } from "@/lib/documents/store";
|
||||
import { prisma } from "@/lib/db";
|
||||
import { annotate } from "@/lib/logging/context";
|
||||
import { detectOcrMimeType } from "@/lib/labs/ocr-upload";
|
||||
import type { ProviderChainResolved } from "@/lib/ai/provider-runner";
|
||||
|
||||
/**
|
||||
* Why one document's index attempt ended without an index. Refs #776 — the
|
||||
* provider path's specific failures are terminal reasons now: when the local
|
||||
* fallback can only say "no text", the recorded reason is the provider path's
|
||||
* more actionable one (a raster failure, a provider error, an empty
|
||||
* transcription, or a PDF the picked provider cannot take), so the detail
|
||||
* view can say WHY instead of a bare not-indexed.
|
||||
*/
|
||||
export type IndexFailureReason =
|
||||
| "not-found"
|
||||
| "local-empty"
|
||||
| "local-unsupported"
|
||||
| "decrypt-error"
|
||||
| "raster-failed"
|
||||
| "provider-error"
|
||||
| "pdf-needs-anthropic"
|
||||
| "empty-transcription";
|
||||
|
||||
/**
|
||||
* Outcome of one document's index attempt. A provider that is not usable (none
|
||||
* configured, no consent, budget exhausted, cannot read the file, or errored)
|
||||
* is never itself a terminal outcome — the tree always falls through to the
|
||||
* free local path, so the terminal reasons are only the local ones.
|
||||
* free local path — but its failure NOTE survives that fall-through (see
|
||||
* `IndexFailureReason`).
|
||||
*/
|
||||
export type IndexOutcome =
|
||||
| { indexed: true; source: ContentIndexSource; tokenCount: number }
|
||||
| {
|
||||
indexed: false;
|
||||
reason:
|
||||
"not-found" | "local-empty" | "local-unsupported" | "decrypt-error";
|
||||
};
|
||||
| { indexed: false; reason: IndexFailureReason };
|
||||
|
||||
/**
|
||||
* Refs #776 — record what became of this attempt on the document row, so a
|
||||
* missing index is explainable. `lastIndexAttemptAt` always advances;
|
||||
* `lastIndexOutcome` carries the failure reason, or null on success (the
|
||||
* `DocumentContentIndex` row itself is the success signal). Called from
|
||||
* `indexLoadedDocument` (the choke point the auto job and the corpus backfill
|
||||
* share) and from the manual index route. Owner-scoped; never throws — a
|
||||
* diagnostic write must not fail the index work it describes.
|
||||
*/
|
||||
export async function recordIndexAttempt(
|
||||
userId: string,
|
||||
documentId: string,
|
||||
outcome: IndexOutcome,
|
||||
): Promise<void> {
|
||||
// No row to explain — the document is gone or not the caller's.
|
||||
if (!outcome.indexed && outcome.reason === "not-found") return;
|
||||
try {
|
||||
await prisma.inboundDocument.updateMany({
|
||||
where: { id: documentId, userId, deletedAt: null },
|
||||
data: {
|
||||
lastIndexAttemptAt: new Date(),
|
||||
lastIndexOutcome: outcome.indexed ? null : outcome.reason,
|
||||
},
|
||||
});
|
||||
} catch (err) {
|
||||
annotate({
|
||||
action: { name: "documents.contentIndex.recordFailed" },
|
||||
meta: { documentId, reason: err instanceof Error ? err.name : "unknown" },
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* A resolved, consent-checked provider context. Resolve ONCE per user and reuse
|
||||
@@ -114,29 +163,56 @@ export async function resolveIndexProvider(
|
||||
return { chain, pick, consentOk, dailyCap };
|
||||
}
|
||||
|
||||
/**
|
||||
* The provider path's failure note when it falls through to local without a
|
||||
* terminal outcome. Refs #776 — the note survives the fall-through: when the
|
||||
* local path can only report "no text", the note is the recorded reason.
|
||||
*/
|
||||
type ProviderPathNote =
|
||||
| "raster-failed"
|
||||
| "pdf-needs-anthropic"
|
||||
| "provider-error"
|
||||
| "empty-transcription";
|
||||
|
||||
interface ProviderPathResult {
|
||||
outcome: IndexOutcome | null;
|
||||
note: ProviderPathNote | null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Try the PROVIDER (vision) path for one already-loaded document. Returns a
|
||||
* terminal outcome on success/budget/provider-error, or `null` to signal
|
||||
* "fall through to local" (no usable provider, or the provider cannot read this
|
||||
* particular file — e.g. a PDF on a non-Anthropic account).
|
||||
* terminal outcome on success / decrypt-error, or `outcome: null` to signal
|
||||
* "fall through to local" (no usable provider, budget out, or the provider
|
||||
* cannot read this particular file) — with a `note` naming the specific
|
||||
* provider-side failure when there was one.
|
||||
*/
|
||||
async function tryProviderIndex(
|
||||
userId: string,
|
||||
document: LoadedDocument,
|
||||
provider: ResolvedIndexProvider,
|
||||
): Promise<IndexOutcome | null> {
|
||||
): Promise<ProviderPathResult> {
|
||||
const { pick } = provider;
|
||||
if (!pick || !provider.consentOk) return null;
|
||||
if (!pick || !provider.consentOk) return { outcome: null, note: null };
|
||||
|
||||
const vision = await prepareVisionInput(document, pick.pdfSupported);
|
||||
if (!vision.ok) {
|
||||
// A decrypt failure is terminal (local would fail the same way); anything
|
||||
// else (fileType / pdfNeedsAnthropic) falls through to the local path — a
|
||||
// text-layer PDF on a non-Anthropic account is read locally for free.
|
||||
// else falls through to the local path — a text-layer PDF on a
|
||||
// non-Anthropic account is read locally for free. The PDF-specific
|
||||
// failures keep their name as the note.
|
||||
if (vision.reason === "decryptFailed") {
|
||||
return { indexed: false, reason: "decrypt-error" };
|
||||
return {
|
||||
outcome: { indexed: false, reason: "decrypt-error" },
|
||||
note: null,
|
||||
};
|
||||
}
|
||||
return null;
|
||||
if (vision.reason === "rasterFailed") {
|
||||
return { outcome: null, note: "raster-failed" };
|
||||
}
|
||||
if (vision.reason === "pdfNeedsAnthropic") {
|
||||
return { outcome: null, note: "pdf-needs-anthropic" };
|
||||
}
|
||||
return { outcome: null, note: null };
|
||||
}
|
||||
|
||||
const dateKey = buildDateKey();
|
||||
@@ -148,7 +224,7 @@ async function tryProviderIndex(
|
||||
);
|
||||
// Budget exhausted → fall through to the free local path rather than stall;
|
||||
// a text-layer PDF stays searchable even once the AI allowance is spent.
|
||||
if (!reservation.allowed) return null;
|
||||
if (!reservation.allowed) return { outcome: null, note: null };
|
||||
|
||||
try {
|
||||
const { text } = await transcribeDocument({
|
||||
@@ -163,6 +239,14 @@ async function tryProviderIndex(
|
||||
reservation.reserved,
|
||||
dateKey,
|
||||
);
|
||||
// Refs #776 — the empty-transcription guard: a provider answer with no
|
||||
// text must NEVER become a "successful" empty index (it would mark the
|
||||
// document indexed while making it unfindable). Fall through to local —
|
||||
// a text-layer PDF still lands its text — and keep the note. The spend
|
||||
// stays reconciled at the full amount: the provider was called.
|
||||
if (text.trim().length === 0) {
|
||||
return { outcome: null, note: "empty-transcription" };
|
||||
}
|
||||
const { tokenCount } = await upsertContentIndex({
|
||||
userId,
|
||||
documentId: document.id,
|
||||
@@ -170,12 +254,15 @@ async function tryProviderIndex(
|
||||
source: "vision",
|
||||
providerType: pick.providerType,
|
||||
});
|
||||
return { indexed: true, source: "vision", tokenCount };
|
||||
return {
|
||||
outcome: { indexed: true, source: "vision", tokenCount },
|
||||
note: null,
|
||||
};
|
||||
} catch {
|
||||
// Refund the reservation and let the caller fall through to local — a
|
||||
// transient provider miss must never leave a text-layer PDF unsearchable.
|
||||
await reconcileSpend(userId, reservation.reserved, 0, dateKey);
|
||||
return null;
|
||||
return { outcome: null, note: "provider-error" };
|
||||
}
|
||||
}
|
||||
|
||||
@@ -220,15 +307,38 @@ async function tryLocalIndex(
|
||||
/**
|
||||
* Index one already-loaded document: provider-first, local-fallback. Reuses a
|
||||
* pre-resolved provider context so a batch resolves the chain once.
|
||||
*
|
||||
* Refs #776 — this is the choke point the auto job and the corpus backfill
|
||||
* share, so the attempt record is written HERE: every terminal outcome lands
|
||||
* on the row via `recordIndexAttempt`, and a provider-path failure note
|
||||
* survives a mute local fallback ("no text" is less actionable than "the
|
||||
* provider errored" / "the PDF would not render").
|
||||
*/
|
||||
export async function indexLoadedDocument(
|
||||
userId: string,
|
||||
document: LoadedDocument,
|
||||
provider: ResolvedIndexProvider,
|
||||
): Promise<IndexOutcome> {
|
||||
const providerOutcome = await tryProviderIndex(userId, document, provider);
|
||||
if (providerOutcome) return providerOutcome;
|
||||
return tryLocalIndex(userId, document);
|
||||
const providerResult = await tryProviderIndex(userId, document, provider);
|
||||
let outcome: IndexOutcome;
|
||||
if (providerResult.outcome) {
|
||||
outcome = providerResult.outcome;
|
||||
} else {
|
||||
const local = await tryLocalIndex(userId, document);
|
||||
if (
|
||||
!local.indexed &&
|
||||
providerResult.note &&
|
||||
(local.reason === "local-empty" || local.reason === "local-unsupported")
|
||||
) {
|
||||
// The local path can only say "no text" — the provider path knows why
|
||||
// the richer read never happened. Record the actionable reason.
|
||||
outcome = { indexed: false, reason: providerResult.note };
|
||||
} else {
|
||||
outcome = local;
|
||||
}
|
||||
}
|
||||
await recordIndexAttempt(userId, document.id, outcome);
|
||||
return outcome;
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -66,8 +66,16 @@ export interface RasterImage {
|
||||
dataBase64: string;
|
||||
}
|
||||
|
||||
/** Best-effort rasterization outcome — never an exception. */
|
||||
export type RasterResult = { ok: true; images: RasterImage[] } | { ok: false };
|
||||
/**
|
||||
* Best-effort rasterization outcome — never an exception. Refs #776: a
|
||||
* failure names its class so the caller can tell "this build cannot raster
|
||||
* at all" (`unsupported` — the native canvas binary is unusable, a capability
|
||||
* gap Anthropic's native PDF path would cover) from "this particular PDF
|
||||
* would not render" (`render-failed` — malformed / encrypted / zero pages).
|
||||
*/
|
||||
export type RasterResult =
|
||||
| { ok: true; images: RasterImage[] }
|
||||
| { ok: false; reason: "unsupported" | "render-failed" };
|
||||
|
||||
// Minimal structural types for the slice of the pdfjs API this module uses. A
|
||||
// type-only shape (erased at build) so we never eagerly evaluate the pdfjs
|
||||
@@ -138,7 +146,7 @@ export async function rasterizePdf(
|
||||
action: { name: "documents.rasterize.failed" },
|
||||
meta: { reason: "native_canvas_unsupported" },
|
||||
});
|
||||
return { ok: false };
|
||||
return { ok: false, reason: "unsupported" };
|
||||
}
|
||||
try {
|
||||
// Lazy import: keeps pdfjs (DOMMatrix) + @napi-rs/canvas out of the eager
|
||||
@@ -190,7 +198,16 @@ export async function rasterizePdf(
|
||||
}
|
||||
}
|
||||
|
||||
if (images.length === 0) return { ok: false };
|
||||
if (images.length === 0) {
|
||||
// Refs #776 — the zero-pages shape must be as loud as a throw: a
|
||||
// document that "rendered" nothing is a failed rasterization, and the
|
||||
// wide event is what lets an operator see WHY a scan stayed unread.
|
||||
annotate({
|
||||
action: { name: "documents.rasterize.failed" },
|
||||
meta: { reason: "zero_pages", pageCount: doc.numPages },
|
||||
});
|
||||
return { ok: false, reason: "render-failed" };
|
||||
}
|
||||
annotate({
|
||||
action: { name: "documents.rasterize.ok" },
|
||||
meta: { pages: images.length, cappedFrom: doc.numPages },
|
||||
@@ -207,7 +224,7 @@ export async function rasterizePdf(
|
||||
message: err instanceof Error ? err.message.slice(0, 300) : String(err),
|
||||
},
|
||||
});
|
||||
return { ok: false };
|
||||
return { ok: false, reason: "render-failed" };
|
||||
} finally {
|
||||
if (task) {
|
||||
try {
|
||||
|
||||
@@ -17,6 +17,7 @@ import type {
|
||||
InboundDocumentStatus,
|
||||
} from "@/generated/prisma/client";
|
||||
import { servingClassFor } from "@/lib/documents/upload-policy";
|
||||
import { toIndexOutcome } from "@/lib/validations/inbound-documents";
|
||||
import type {
|
||||
DocumentConditionLinkDto,
|
||||
DocumentEncounterLinkDto,
|
||||
@@ -225,6 +226,15 @@ export function serialiseDocument(
|
||||
servingClass: servingClassFor(doc.mimeType),
|
||||
hasContentIndex,
|
||||
contentIndexSource: hasContentIndex ? contentIndexSource : null,
|
||||
// Refs #776 — the attempt record travels on every DTO so the detail view
|
||||
// can explain a missing index. Outcome only surfaces while there IS no
|
||||
// index: once indexed, the last failure is history, not status.
|
||||
lastIndexAttemptAt: doc.lastIndexAttemptAt
|
||||
? doc.lastIndexAttemptAt.toISOString()
|
||||
: null,
|
||||
lastIndexOutcome: hasContentIndex
|
||||
? null
|
||||
: toIndexOutcome(doc.lastIndexOutcome),
|
||||
hasThumbnail,
|
||||
createdAt: doc.createdAt.toISOString(),
|
||||
updatedAt: doc.updatedAt.toISOString(),
|
||||
|
||||
@@ -133,6 +133,9 @@ export interface DocumentBackupEntry {
|
||||
summaryEncrypted?: string | null;
|
||||
summaryGeneratedAt?: string | null;
|
||||
summaryState?: string;
|
||||
/** Refs #776 — the index-attempt record, carried verbatim (canonical DR). */
|
||||
lastIndexAttemptAt?: string | null;
|
||||
lastIndexOutcome?: string | null;
|
||||
}
|
||||
|
||||
export interface RecordsBackupOptions {
|
||||
@@ -148,6 +151,8 @@ interface DisasterRecoveryDocumentRow {
|
||||
summaryEncrypted: Uint8Array | null;
|
||||
summaryGeneratedAt: Date | null;
|
||||
summaryState: string;
|
||||
lastIndexAttemptAt: Date | null;
|
||||
lastIndexOutcome: string | null;
|
||||
updatedAt: Date;
|
||||
}
|
||||
|
||||
@@ -254,6 +259,8 @@ export async function buildRecordsBackupSection(
|
||||
summaryEncrypted: true,
|
||||
summaryGeneratedAt: true,
|
||||
summaryState: true,
|
||||
lastIndexAttemptAt: true,
|
||||
lastIndexOutcome: true,
|
||||
createdAt: true,
|
||||
updatedAt: true,
|
||||
},
|
||||
@@ -488,6 +495,8 @@ export async function buildRecordsBackupSection(
|
||||
: null,
|
||||
summaryGeneratedAt: dr.summaryGeneratedAt?.toISOString() ?? null,
|
||||
summaryState: dr.summaryState,
|
||||
lastIndexAttemptAt: dr.lastIndexAttemptAt?.toISOString() ?? null,
|
||||
lastIndexOutcome: dr.lastIndexOutcome,
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,415 @@
|
||||
import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||
|
||||
/**
|
||||
* The async-truth contract, pinned per state machine (v1.37.15).
|
||||
*
|
||||
* > A state that claims work is running must have a writer for every terminal
|
||||
* > branch; an enqueue must never claim a pending state its job cannot
|
||||
* > terminally resolve.
|
||||
*
|
||||
* Two machines are pinned — the document summary job and the document index
|
||||
* tree. Honest scope: these enumerate the branches of the two KNOWN machines;
|
||||
* a generic phantom-pending detector for arbitrary future jobs is not
|
||||
* provable and is not claimed. When a branch is added to either job, add it
|
||||
* here — the enumerations below are the contract.
|
||||
*
|
||||
* (a) Every branch of `runDocumentSummaryJob` ends in a terminal
|
||||
* `summaryState` write — the opt-out branch included (it heals a phantom
|
||||
* PENDING back to NONE). The only silent exits are "row gone" and
|
||||
* "summary already stored", whose states are terminal by precondition.
|
||||
* (b) Every branch of the index tree behind `runDocumentIndex` ends in an
|
||||
* attempt record (`lastIndexAttemptAt` + `lastIndexOutcome`) — success
|
||||
* included (it clears the outcome). The only silent exit is not-found.
|
||||
* (c) `enqueueDocumentSummary` never claims PENDING when the job's only
|
||||
* branch would be the opt-out no-op, and never before a job id exists.
|
||||
*/
|
||||
|
||||
vi.mock("@/lib/logging/context", () => ({ annotate: vi.fn() }));
|
||||
vi.mock("@/lib/db", () => ({
|
||||
prisma: {
|
||||
inboundDocument: { findFirst: vi.fn(), updateMany: vi.fn() },
|
||||
user: { findUnique: vi.fn() },
|
||||
},
|
||||
}));
|
||||
vi.mock("@/lib/jobs/boss-instance", () => ({ getGlobalBoss: vi.fn() }));
|
||||
vi.mock("@/lib/documents/document-settings", () => ({
|
||||
documentAutoReadEnabled: vi.fn(),
|
||||
}));
|
||||
vi.mock("@/lib/documents/provider-order", () => ({
|
||||
resolveDocumentVisionProvider: vi.fn(),
|
||||
}));
|
||||
vi.mock("@/lib/documents/ai-route-support", () => ({
|
||||
loadOwnedDocument: vi.fn(),
|
||||
prepareVisionInput: vi.fn(),
|
||||
}));
|
||||
vi.mock("@/lib/documents/describe", () => ({
|
||||
runDocumentSummary: vi.fn(),
|
||||
transcribeDocument: vi.fn(),
|
||||
}));
|
||||
vi.mock("@/lib/documents/store", () => ({
|
||||
encryptDocumentSummary: vi.fn(() => new Uint8Array([9])),
|
||||
decryptDocumentContent: vi.fn(() => Buffer.from([1, 2, 3])),
|
||||
}));
|
||||
vi.mock("@/lib/documents/content-index", () => ({
|
||||
upsertContentIndex: vi.fn().mockResolvedValue({ tokenCount: 5 }),
|
||||
}));
|
||||
vi.mock("@/lib/documents/local-extract", () => ({
|
||||
localExtractText: vi.fn(),
|
||||
}));
|
||||
vi.mock("@/lib/documents/auto-stage-labs", () => ({
|
||||
maybeAutoStageLabFacts: vi.fn().mockResolvedValue({ staged: false }),
|
||||
}));
|
||||
vi.mock("@/lib/labs/ocr-upload", () => ({
|
||||
detectOcrMimeType: vi.fn(() => "application/pdf"),
|
||||
}));
|
||||
vi.mock("@/lib/ai/consent-guard", () => {
|
||||
class ConsentRequiredError extends Error {}
|
||||
return {
|
||||
ConsentRequiredError,
|
||||
assertDocumentEgressConsent: vi.fn(),
|
||||
isExternalDocumentEgress: (providerType: string) =>
|
||||
providerType !== "local",
|
||||
};
|
||||
});
|
||||
vi.mock("@/lib/ai/coach/budget", () => ({
|
||||
buildDateKey: vi.fn(() => "2026-08-13"),
|
||||
reserveBudget: vi.fn(),
|
||||
reconcileSpend: vi.fn().mockResolvedValue(undefined),
|
||||
resolveDailyCap: vi.fn(() => 1000),
|
||||
}));
|
||||
vi.mock("@/lib/ai/ai-budgets", () => ({
|
||||
AI_BUDGETS: {
|
||||
documentSummary: { temperature: 0.3, maxTokens: 600 },
|
||||
documentTranscribe: { temperature: 0, maxTokens: 4000 },
|
||||
},
|
||||
}));
|
||||
|
||||
import {
|
||||
enqueueDocumentSummary,
|
||||
runDocumentSummaryJob,
|
||||
} from "../document-summary";
|
||||
import { runDocumentIndex } from "../document-index";
|
||||
import { prisma } from "@/lib/db";
|
||||
import { getGlobalBoss } from "@/lib/jobs/boss-instance";
|
||||
import { documentAutoReadEnabled } from "@/lib/documents/document-settings";
|
||||
import { resolveDocumentVisionProvider } from "@/lib/documents/provider-order";
|
||||
import {
|
||||
loadOwnedDocument,
|
||||
prepareVisionInput,
|
||||
} from "@/lib/documents/ai-route-support";
|
||||
import {
|
||||
runDocumentSummary,
|
||||
transcribeDocument,
|
||||
} from "@/lib/documents/describe";
|
||||
import { localExtractText } from "@/lib/documents/local-extract";
|
||||
import {
|
||||
assertDocumentEgressConsent,
|
||||
ConsentRequiredError,
|
||||
} from "@/lib/ai/consent-guard";
|
||||
import { reserveBudget } from "@/lib/ai/coach/budget";
|
||||
|
||||
const DOC = {
|
||||
id: "doc-1",
|
||||
kind: "OTHER",
|
||||
contentEncrypted: new Uint8Array([1, 2, 3]),
|
||||
contentCodec: "binary2",
|
||||
mimeType: "application/pdf",
|
||||
status: "STORED",
|
||||
};
|
||||
|
||||
const PICK = {
|
||||
chain: [{ providerType: "anthropic", instance: {} }],
|
||||
pick: {
|
||||
entry: { providerType: "anthropic", instance: {} },
|
||||
providerType: "anthropic",
|
||||
pdfSupported: true,
|
||||
},
|
||||
};
|
||||
|
||||
/** All updateMany data objects that carried a summaryState. */
|
||||
function summaryStateWrites(): unknown[] {
|
||||
return vi
|
||||
.mocked(prisma.inboundDocument.updateMany)
|
||||
.mock.calls.filter(
|
||||
([arg]) => arg.data && "summaryState" in (arg.data as object),
|
||||
)
|
||||
.map(([arg]) => (arg.data as { summaryState: unknown }).summaryState);
|
||||
}
|
||||
|
||||
/** All updateMany data objects that carried the index-attempt record. */
|
||||
function indexAttemptWrites(): unknown[] {
|
||||
return vi
|
||||
.mocked(prisma.inboundDocument.updateMany)
|
||||
.mock.calls.filter(
|
||||
([arg]) => arg.data && "lastIndexAttemptAt" in (arg.data as object),
|
||||
)
|
||||
.map(
|
||||
([arg]) => (arg.data as { lastIndexOutcome: unknown }).lastIndexOutcome,
|
||||
);
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks();
|
||||
vi.mocked(prisma.inboundDocument.findFirst).mockResolvedValue({
|
||||
id: "doc-1",
|
||||
summaryEncrypted: null,
|
||||
} as never);
|
||||
vi.mocked(prisma.inboundDocument.updateMany).mockResolvedValue({
|
||||
count: 1,
|
||||
} as never);
|
||||
vi.mocked(prisma.user.findUnique).mockResolvedValue({
|
||||
locale: "en",
|
||||
} as never);
|
||||
vi.mocked(documentAutoReadEnabled).mockResolvedValue(true);
|
||||
vi.mocked(resolveDocumentVisionProvider).mockResolvedValue(PICK as never);
|
||||
vi.mocked(assertDocumentEgressConsent).mockResolvedValue(undefined);
|
||||
vi.mocked(loadOwnedDocument).mockResolvedValue(DOC as never);
|
||||
vi.mocked(prepareVisionInput).mockResolvedValue({
|
||||
ok: true,
|
||||
images: [],
|
||||
documents: [{ mediaType: "application/pdf", dataBase64: "AA==" }],
|
||||
} as never);
|
||||
vi.mocked(runDocumentSummary).mockResolvedValue({
|
||||
summary: "A short descriptive summary.",
|
||||
blocked: null,
|
||||
} as never);
|
||||
vi.mocked(transcribeDocument).mockResolvedValue({
|
||||
text: "haemoglobin 14.2",
|
||||
} as never);
|
||||
vi.mocked(reserveBudget).mockResolvedValue({
|
||||
allowed: true,
|
||||
reserved: 1,
|
||||
} as never);
|
||||
vi.mocked(localExtractText).mockResolvedValue({
|
||||
ok: true,
|
||||
text: "glucose fasting values",
|
||||
source: "local-pdf",
|
||||
} as never);
|
||||
});
|
||||
|
||||
const PAYLOAD = { userId: "user-1", documentId: "doc-1" };
|
||||
|
||||
describe("contract (a): every runDocumentSummaryJob branch writes a terminal state", () => {
|
||||
const branches: Array<{
|
||||
name: string;
|
||||
setup: () => void;
|
||||
state: string;
|
||||
}> = [
|
||||
{
|
||||
name: "opt-out heals PENDING to NONE",
|
||||
setup: () => {
|
||||
vi.mocked(documentAutoReadEnabled).mockResolvedValue(false);
|
||||
},
|
||||
state: "NONE",
|
||||
},
|
||||
{
|
||||
name: "no provider configured",
|
||||
setup: () => {
|
||||
vi.mocked(resolveDocumentVisionProvider).mockResolvedValue({
|
||||
chain: [],
|
||||
pick: null,
|
||||
} as never);
|
||||
},
|
||||
state: "UNAVAILABLE",
|
||||
},
|
||||
{
|
||||
name: "consent refused",
|
||||
setup: () => {
|
||||
vi.mocked(assertDocumentEgressConsent).mockRejectedValue(
|
||||
new ConsentRequiredError("insights"),
|
||||
);
|
||||
},
|
||||
state: "UNAVAILABLE",
|
||||
},
|
||||
{
|
||||
name: "document unreadable by the picked provider",
|
||||
setup: () => {
|
||||
vi.mocked(prepareVisionInput).mockResolvedValue({
|
||||
ok: false,
|
||||
reason: "fileType",
|
||||
} as never);
|
||||
},
|
||||
state: "UNAVAILABLE",
|
||||
},
|
||||
{
|
||||
name: "budget exhausted",
|
||||
setup: () => {
|
||||
vi.mocked(reserveBudget).mockResolvedValue({
|
||||
allowed: false,
|
||||
reserved: 0,
|
||||
} as never);
|
||||
},
|
||||
state: "UNAVAILABLE",
|
||||
},
|
||||
{
|
||||
name: "safety screen blocked the summary",
|
||||
setup: () => {
|
||||
vi.mocked(runDocumentSummary).mockResolvedValue({
|
||||
summary: "",
|
||||
blocked: "dose_directive",
|
||||
} as never);
|
||||
},
|
||||
state: "WITHHELD",
|
||||
},
|
||||
{
|
||||
name: "provider error",
|
||||
setup: () => {
|
||||
vi.mocked(runDocumentSummary).mockRejectedValue(new Error("boom"));
|
||||
},
|
||||
state: "UNAVAILABLE",
|
||||
},
|
||||
{
|
||||
name: "success",
|
||||
setup: () => {},
|
||||
state: "READY",
|
||||
},
|
||||
];
|
||||
|
||||
for (const branch of branches) {
|
||||
it(`${branch.name} → writes ${branch.state}`, async () => {
|
||||
branch.setup();
|
||||
await runDocumentSummaryJob(PAYLOAD);
|
||||
expect(summaryStateWrites()).toContain(branch.state);
|
||||
});
|
||||
}
|
||||
|
||||
it("the only silent exits are row-gone and already-summarised (terminal by precondition)", async () => {
|
||||
vi.mocked(prisma.inboundDocument.findFirst).mockResolvedValue(
|
||||
null as never,
|
||||
);
|
||||
await runDocumentSummaryJob(PAYLOAD);
|
||||
expect(summaryStateWrites()).toEqual([]);
|
||||
|
||||
vi.clearAllMocks();
|
||||
vi.mocked(prisma.inboundDocument.findFirst).mockResolvedValue({
|
||||
id: "doc-1",
|
||||
summaryEncrypted: new Uint8Array([1]),
|
||||
} as never);
|
||||
await runDocumentSummaryJob(PAYLOAD);
|
||||
expect(summaryStateWrites()).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("contract (a): every runDocumentIndex branch records the attempt", () => {
|
||||
const branches: Array<{
|
||||
name: string;
|
||||
setup: () => void;
|
||||
outcome: string | null;
|
||||
}> = [
|
||||
{
|
||||
name: "provider success",
|
||||
setup: () => {},
|
||||
outcome: null,
|
||||
},
|
||||
{
|
||||
name: "local success (no provider)",
|
||||
setup: () => {
|
||||
vi.mocked(resolveDocumentVisionProvider).mockResolvedValue({
|
||||
chain: [],
|
||||
pick: null,
|
||||
} as never);
|
||||
},
|
||||
outcome: null,
|
||||
},
|
||||
{
|
||||
name: "scan with no text and no provider",
|
||||
setup: () => {
|
||||
vi.mocked(resolveDocumentVisionProvider).mockResolvedValue({
|
||||
chain: [],
|
||||
pick: null,
|
||||
} as never);
|
||||
vi.mocked(localExtractText).mockResolvedValue({
|
||||
ok: false,
|
||||
reason: "empty",
|
||||
} as never);
|
||||
},
|
||||
outcome: "local-empty",
|
||||
},
|
||||
{
|
||||
name: "provider error with no local text",
|
||||
setup: () => {
|
||||
vi.mocked(transcribeDocument).mockRejectedValue(new Error("boom"));
|
||||
vi.mocked(localExtractText).mockResolvedValue({
|
||||
ok: false,
|
||||
reason: "empty",
|
||||
} as never);
|
||||
},
|
||||
outcome: "provider-error",
|
||||
},
|
||||
{
|
||||
name: "empty transcription with no local text",
|
||||
setup: () => {
|
||||
vi.mocked(transcribeDocument).mockResolvedValue({
|
||||
text: " ",
|
||||
} as never);
|
||||
vi.mocked(localExtractText).mockResolvedValue({
|
||||
ok: false,
|
||||
reason: "empty",
|
||||
} as never);
|
||||
},
|
||||
outcome: "empty-transcription",
|
||||
},
|
||||
{
|
||||
name: "raster failure with no local text",
|
||||
setup: () => {
|
||||
vi.mocked(prepareVisionInput).mockResolvedValue({
|
||||
ok: false,
|
||||
reason: "rasterFailed",
|
||||
} as never);
|
||||
vi.mocked(localExtractText).mockResolvedValue({
|
||||
ok: false,
|
||||
reason: "empty",
|
||||
} as never);
|
||||
},
|
||||
outcome: "raster-failed",
|
||||
},
|
||||
{
|
||||
name: "decrypt failure",
|
||||
setup: () => {
|
||||
vi.mocked(prepareVisionInput).mockResolvedValue({
|
||||
ok: false,
|
||||
reason: "decryptFailed",
|
||||
} as never);
|
||||
},
|
||||
outcome: "decrypt-error",
|
||||
},
|
||||
];
|
||||
|
||||
for (const branch of branches) {
|
||||
it(`${branch.name} → records outcome ${JSON.stringify(branch.outcome)}`, async () => {
|
||||
branch.setup();
|
||||
await runDocumentIndex(PAYLOAD);
|
||||
expect(indexAttemptWrites()).toContain(branch.outcome);
|
||||
});
|
||||
}
|
||||
|
||||
it("the only silent exit is not-found (no row to explain)", async () => {
|
||||
vi.mocked(loadOwnedDocument).mockResolvedValue(null as never);
|
||||
await runDocumentIndex(PAYLOAD);
|
||||
expect(indexAttemptWrites()).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("contract (b): enqueue never claims a pending state its job cannot resolve", () => {
|
||||
it("enqueueDocumentSummary with the opt-in OFF claims nothing and mints no job", async () => {
|
||||
vi.mocked(documentAutoReadEnabled).mockResolvedValue(false);
|
||||
const send = vi.fn().mockResolvedValue("job-1");
|
||||
vi.mocked(getGlobalBoss).mockReturnValue({ send } as never);
|
||||
|
||||
const result = await enqueueDocumentSummary("user-1", "doc-1");
|
||||
expect(result).toEqual({ enqueued: false });
|
||||
expect(send).not.toHaveBeenCalled();
|
||||
expect(summaryStateWrites()).toEqual([]);
|
||||
});
|
||||
|
||||
it("enqueueDocumentSummary claims PENDING only once a job id exists", async () => {
|
||||
const send = vi.fn().mockResolvedValue(null);
|
||||
vi.mocked(getGlobalBoss).mockReturnValue({ send } as never);
|
||||
await enqueueDocumentSummary("user-1", "doc-1");
|
||||
expect(summaryStateWrites()).toEqual([]);
|
||||
|
||||
send.mockResolvedValue("job-1");
|
||||
await enqueueDocumentSummary("user-1", "doc-1");
|
||||
expect(summaryStateWrites()).toContain("PENDING");
|
||||
});
|
||||
});
|
||||
@@ -55,8 +55,12 @@ vi.mock("@/lib/ai/ai-budgets", () => ({
|
||||
AI_BUDGETS: { documentSummary: { temperature: 0.3, maxTokens: 600 } },
|
||||
}));
|
||||
|
||||
import { runDocumentSummaryJob } from "../document-summary";
|
||||
import {
|
||||
enqueueDocumentSummary,
|
||||
runDocumentSummaryJob,
|
||||
} from "../document-summary";
|
||||
import { prisma } from "@/lib/db";
|
||||
import { getGlobalBoss } from "@/lib/jobs/boss-instance";
|
||||
import { documentAutoReadEnabled } from "@/lib/documents/document-settings";
|
||||
import { resolveDocumentVisionProvider } from "@/lib/documents/provider-order";
|
||||
import {
|
||||
@@ -124,14 +128,19 @@ beforeEach(() => {
|
||||
});
|
||||
|
||||
describe("runDocumentSummaryJob — gating", () => {
|
||||
it("does NOTHING when the opt-in is OFF (no provider call, no write)", async () => {
|
||||
it("does no provider work when the opt-in is OFF (only the PENDING heal may write)", async () => {
|
||||
vi.mocked(documentAutoReadEnabled).mockResolvedValue(false);
|
||||
|
||||
await runDocumentSummaryJob({ userId: "user-1", documentId: "doc-1" });
|
||||
|
||||
expect(resolveDocumentVisionProvider).not.toHaveBeenCalled();
|
||||
expect(runDocumentSummary).not.toHaveBeenCalled();
|
||||
expect(prisma.inboundDocument.updateMany).not.toHaveBeenCalled();
|
||||
// The single permitted write is the PENDING→NONE heal; nothing may claim
|
||||
// an attempt (UNAVAILABLE/WITHHELD) or a summary (READY) here.
|
||||
const calls = vi.mocked(prisma.inboundDocument.updateMany).mock.calls;
|
||||
for (const [arg] of calls) {
|
||||
expect(arg.data).toEqual({ summaryState: "NONE" });
|
||||
}
|
||||
});
|
||||
|
||||
it("generates and PERSISTS the summary encrypted when opt-in is ON + a provider is configured", async () => {
|
||||
@@ -248,4 +257,64 @@ describe("runDocumentSummaryJob — gating", () => {
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it("heals a phantom PENDING back to NONE when the opt-in is OFF", async () => {
|
||||
vi.mocked(documentAutoReadEnabled).mockResolvedValue(false);
|
||||
|
||||
await runDocumentSummaryJob({ userId: "user-1", documentId: "doc-1" });
|
||||
|
||||
// The opt-out branch may not leave a claimed PENDING standing — the view
|
||||
// reads PENDING as "being generated" and this run is the only writer that
|
||||
// can resolve it. The heal is scoped to PENDING: UNAVAILABLE / WITHHELD
|
||||
// record real attempts and stay, READY is never touched.
|
||||
expect(prisma.inboundDocument.updateMany).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
where: expect.objectContaining({ summaryState: "PENDING" }),
|
||||
data: { summaryState: "NONE" },
|
||||
}),
|
||||
);
|
||||
// Still strictly no provider work.
|
||||
expect(resolveDocumentVisionProvider).not.toHaveBeenCalled();
|
||||
expect(runDocumentSummary).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
describe("enqueueDocumentSummary — the pending contract", () => {
|
||||
it("does not enqueue and does not claim PENDING when the opt-in is OFF", async () => {
|
||||
vi.mocked(documentAutoReadEnabled).mockResolvedValue(false);
|
||||
const send = vi.fn().mockResolvedValue("job-1");
|
||||
vi.mocked(getGlobalBoss).mockReturnValue({ send } as never);
|
||||
|
||||
const result = await enqueueDocumentSummary("user-1", "doc-1");
|
||||
|
||||
// With auto-read OFF the job's only move is the opt-out no-op, so a
|
||||
// PENDING claimed here could never be resolved to a real outcome. No job,
|
||||
// no state write — the view keeps offering the manual action.
|
||||
expect(result).toEqual({ enqueued: false });
|
||||
expect(send).not.toHaveBeenCalled();
|
||||
expect(prisma.inboundDocument.updateMany).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("enqueues and claims PENDING when the opt-in is ON and a job was created", async () => {
|
||||
const send = vi.fn().mockResolvedValue("job-1");
|
||||
vi.mocked(getGlobalBoss).mockReturnValue({ send } as never);
|
||||
|
||||
const result = await enqueueDocumentSummary("user-1", "doc-1");
|
||||
|
||||
expect(result).toEqual({ enqueued: true });
|
||||
expect(send).toHaveBeenCalledTimes(1);
|
||||
expect(prisma.inboundDocument.updateMany).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ data: { summaryState: "PENDING" } }),
|
||||
);
|
||||
});
|
||||
|
||||
it("claims nothing when the boss dropped the send (no job id)", async () => {
|
||||
const send = vi.fn().mockResolvedValue(null);
|
||||
vi.mocked(getGlobalBoss).mockReturnValue({ send } as never);
|
||||
|
||||
const result = await enqueueDocumentSummary("user-1", "doc-1");
|
||||
|
||||
expect(result).toEqual({ enqueued: false });
|
||||
expect(prisma.inboundDocument.updateMany).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
@@ -109,8 +109,23 @@ export async function runDocumentSummaryJob(
|
||||
// on-demand route stays the only way to summarise). Fail-closed on a missing
|
||||
// row.
|
||||
if (!(await documentAutoReadEnabled(userId))) {
|
||||
// Opting out is not a failed attempt — leave the state alone so the view
|
||||
// keeps offering the manual action rather than claiming we tried.
|
||||
// Opting out is not a failed attempt — but a claimed PENDING may not
|
||||
// outlive this run either. PENDING promises the reader "a job will resolve
|
||||
// this", and this job IS that resolution: with the opt-in OFF the honest
|
||||
// terminal state is NONE (never attempted), which is what the enqueue end
|
||||
// of the contract (`enqueueDocumentSummary`) also concludes when the pref
|
||||
// is off. The heal is scoped to PENDING only — NONE needs no write,
|
||||
// UNAVAILABLE / WITHHELD record real attempts, READY is a stored summary.
|
||||
// This also drains any phantom PENDING left by a pref flip mid-flight.
|
||||
await prisma.inboundDocument.updateMany({
|
||||
where: {
|
||||
id: documentId,
|
||||
userId,
|
||||
deletedAt: null,
|
||||
summaryState: "PENDING",
|
||||
},
|
||||
data: { summaryState: "NONE" },
|
||||
});
|
||||
annotate({
|
||||
action: { name: "documents.summary.autoSkipped" },
|
||||
meta: { documentId, reason: "opt-out" },
|
||||
@@ -282,14 +297,29 @@ export async function enqueueDocumentSummary(
|
||||
userId: string,
|
||||
documentId: string,
|
||||
): Promise<{ enqueued: boolean }> {
|
||||
const boss = getGlobalBoss();
|
||||
if (!boss) return { enqueued: false };
|
||||
const payload: DocumentSummaryPayload = {
|
||||
userId,
|
||||
documentId,
|
||||
enqueuedAt: new Date().toISOString(),
|
||||
};
|
||||
try {
|
||||
// Contract, enqueue end: PENDING may only be claimed for work the job can
|
||||
// terminally resolve. With the auto-read opt-in OFF the job's only branch
|
||||
// is the opt-out no-op, so enqueueing would mint a job whose sole purpose
|
||||
// is to undo this function's own PENDING — don't. No job, no state write;
|
||||
// the document stays NONE and the view offers the manual action. The job
|
||||
// keeps its own pref check (belt-and-braces for a pref flipped between
|
||||
// enqueue and run) and heals PENDING→NONE there — the other end of this
|
||||
// contract lives in `runDocumentSummaryJob`'s opt-out branch.
|
||||
if (!(await documentAutoReadEnabled(userId))) {
|
||||
annotate({
|
||||
action: { name: "documents.summary.autoSkipped" },
|
||||
meta: { documentId, reason: "opt-out" },
|
||||
});
|
||||
return { enqueued: false };
|
||||
}
|
||||
const boss = getGlobalBoss();
|
||||
if (!boss) return { enqueued: false };
|
||||
const payload: DocumentSummaryPayload = {
|
||||
userId,
|
||||
documentId,
|
||||
enqueuedAt: new Date().toISOString(),
|
||||
};
|
||||
const jobId = await boss.send(DOCUMENT_SUMMARY_QUEUE, payload, {
|
||||
retryLimit: 2,
|
||||
retryDelay: 30,
|
||||
|
||||
@@ -171,6 +171,18 @@ const inboundDocument = z
|
||||
contentIndexSource: z
|
||||
.enum(["vision", "text-ocr", "local-pdf", "local-ocr"])
|
||||
.nullable(),
|
||||
lastIndexAttemptAt: z.string().nullable(),
|
||||
lastIndexOutcome: z
|
||||
.enum([
|
||||
"local-empty",
|
||||
"local-unsupported",
|
||||
"decrypt-error",
|
||||
"raster-failed",
|
||||
"provider-error",
|
||||
"pdf-needs-anthropic",
|
||||
"empty-transcription",
|
||||
])
|
||||
.nullable(),
|
||||
hasThumbnail: z.boolean(),
|
||||
createdAt: z.string(),
|
||||
updatedAt: z.string(),
|
||||
@@ -178,7 +190,7 @@ const inboundDocument = z
|
||||
.meta({
|
||||
id: "InboundDocument",
|
||||
description:
|
||||
"A stored document. The raw bytes are stored encrypted at rest and never returned; this is the metadata + staging summary. `status` is STORED for a freshly uploaded file (no extraction run). `title` is the user label (plaintext); `documentDate` is the user filing date; `reportDate` is the model-transcribed date (null until extraction runs). `servingClass` says how `/original` delivers the file — render inline (`inline`) or download-only (`attachment`); render/download by it, never by MIME guess. `hasContentIndex` is true when the document has a content-search index (auto-indexed on upload); `contentIndexSource` says how that index was produced — `vision` means an AI provider read the original, the other values are provider-free local extractions, and it is null when `hasContentIndex` is false. `hasThumbnail` is true when a preview thumbnail has been rendered in the background (fetch it from `/api/documents/inbound/{id}/thumbnail`); false means no preview yet or an unsupported type — show a placeholder. Treat an unknown `kind` value as OTHER when decoding.",
|
||||
"A stored document. The raw bytes are stored encrypted at rest and never returned; this is the metadata + staging summary. `status` is STORED for a freshly uploaded file (no extraction run). `title` is the user label (plaintext); `documentDate` is the user filing date; `reportDate` is the model-transcribed date (null until extraction runs). `servingClass` says how `/original` delivers the file — render inline (`inline`) or download-only (`attachment`); render/download by it, never by MIME guess. `hasContentIndex` is true when the document has a content-search index (auto-indexed on upload); `contentIndexSource` says how that index was produced — `vision` means an AI provider read the original, the other values are provider-free local extractions, and it is null when `hasContentIndex` is false. `lastIndexAttemptAt` is when the most recent index attempt finished (successful or not; null when none ran); `lastIndexOutcome` says why that attempt produced no index, and is null when the attempt succeeded or none ran. `hasThumbnail` is true when a preview thumbnail has been rendered in the background (fetch it from `/api/documents/inbound/{id}/thumbnail`); false means no preview yet or an unsupported type — show a placeholder. Treat an unknown `kind` value as OTHER when decoding.",
|
||||
});
|
||||
|
||||
const inboundDocumentDetail = inboundDocument
|
||||
@@ -1028,12 +1040,13 @@ export const inboundDocumentPaths: NonNullable<ZodOpenApiObject["paths"]> = {
|
||||
"Enqueues a background job that content-indexes the caller's not-yet-indexed documents (one provider transcription each, bounded + resumable). Gated on the module, a configured vision provider (422 `documents.inbound.providerUnsupported` otherwise), and the existing AI consent (403 otherwise). Returns immediately; the work runs off-request on the queue.",
|
||||
responses: {
|
||||
"200": {
|
||||
description: "Whether a backfill job was enqueued.",
|
||||
description:
|
||||
"How many live documents the enqueued backfill still has to index (0 when everything is indexed or no job was created).",
|
||||
content: {
|
||||
"application/json": {
|
||||
schema: dataEnvelope(
|
||||
z
|
||||
.object({ enqueued: z.boolean() })
|
||||
.object({ enqueued: z.number().int().nonnegative() })
|
||||
.meta({ id: "DocumentReindexResponse" }),
|
||||
"DocumentReindexEnvelope",
|
||||
),
|
||||
|
||||
@@ -1022,6 +1022,10 @@ const documentBackupSchema = z
|
||||
summaryEncrypted: base64BytesSchema.nullable().optional(),
|
||||
summaryGeneratedAt: isoDateTime.nullable().optional(),
|
||||
summaryState: z.enum(DocumentSummaryState).optional(),
|
||||
// Refs #776 — the index-attempt record (canonical DR payloads only).
|
||||
// Free string on purpose: a future outcome value must not fail a restore.
|
||||
lastIndexAttemptAt: isoDateTime.nullable().optional(),
|
||||
lastIndexOutcome: z.string().nullable().optional(),
|
||||
summary: z.string().nullable().optional(),
|
||||
createdAt: isoDateTime.optional(),
|
||||
updatedAt: isoDateTime.optional(),
|
||||
|
||||
@@ -114,6 +114,34 @@ export const DOCUMENT_SUMMARY_STATES = [
|
||||
export type DocumentSummaryStateValue =
|
||||
(typeof DOCUMENT_SUMMARY_STATES)[number];
|
||||
|
||||
/**
|
||||
* Refs #776 — why a document's most recent content-index attempt produced no
|
||||
* index. Mirrors the free-String `lastIndexOutcome` column (closed here, not
|
||||
* in the schema, so a new reason needs no migration); the detail view renders
|
||||
* a plain-words line per value instead of a bare "not indexed" that cannot
|
||||
* tell "never tried" from "tried and failed".
|
||||
*/
|
||||
export const DOCUMENT_INDEX_OUTCOMES = [
|
||||
"local-empty",
|
||||
"local-unsupported",
|
||||
"decrypt-error",
|
||||
"raster-failed",
|
||||
"provider-error",
|
||||
"pdf-needs-anthropic",
|
||||
"empty-transcription",
|
||||
] as const;
|
||||
export type DocumentIndexOutcomeValue =
|
||||
(typeof DOCUMENT_INDEX_OUTCOMES)[number];
|
||||
|
||||
/** Narrow the free-String DB column to the DTO union (null when unknown). */
|
||||
export function toIndexOutcome(
|
||||
outcome: string | null | undefined,
|
||||
): DocumentIndexOutcomeValue | null {
|
||||
return DOCUMENT_INDEX_OUTCOMES.includes(outcome as DocumentIndexOutcomeValue)
|
||||
? (outcome as DocumentIndexOutcomeValue)
|
||||
: null;
|
||||
}
|
||||
|
||||
/** The document lifecycle states. STORED is the library default. */
|
||||
export const INBOUND_DOCUMENT_STATUSES = [
|
||||
"STORED",
|
||||
@@ -347,6 +375,18 @@ export interface InboundDocumentDto {
|
||||
* a locally-indexed one and offer a richer "Read with AI" pass on the latter.
|
||||
*/
|
||||
contentIndexSource: DocumentContentIndexSourceValue | null;
|
||||
/**
|
||||
* Refs #776 — when the most recent index attempt finished (ISO 8601),
|
||||
* successful or not. Null until any attempt ran (pre-existing rows, or a
|
||||
* document whose auto-index never fired).
|
||||
*/
|
||||
lastIndexAttemptAt: string | null;
|
||||
/**
|
||||
* Refs #776 — why the most recent index attempt produced no index, or null
|
||||
* (attempt succeeded, or never ran — `lastIndexAttemptAt` tells the two
|
||||
* apart). The detail view renders the reason in plain words.
|
||||
*/
|
||||
lastIndexOutcome: DocumentIndexOutcomeValue | null;
|
||||
/**
|
||||
* Whether the document has an encrypted preview thumbnail (rendered in the
|
||||
* background after upload). Gates the card's `<img>`: when false the card
|
||||
|
||||
+15
-3
@@ -528,9 +528,21 @@ export function proxy(request: NextRequest) {
|
||||
// HTML/SVG denied entirely, and attachment-only posture for every
|
||||
// non-passive format.
|
||||
`default-src 'none'; frame-ancestors 'self';`
|
||||
: isDev
|
||||
? `default-src 'self'; script-src 'self' 'unsafe-eval' 'unsafe-inline'; style-src 'self' 'unsafe-inline'; img-src 'self' data:; connect-src 'self'; font-src 'self';`
|
||||
: `default-src 'self'; script-src 'self' 'nonce-${nonce}'; style-src 'self' 'unsafe-inline'; img-src 'self' data:; connect-src 'self'${aiConnectSrc}${withingsConnectSrc}${whoopConnectSrc}${stravaConnectSrc}; font-src 'self'; object-src 'none'; base-uri 'self'; form-action 'self'; frame-ancestors 'none'; worker-src 'self'; report-uri ${cspReportEndpoint}; report-to csp-endpoint;`;
|
||||
: // v1.37.15 — `frame-src 'self' blob:` + `blob:` in `img-src` (both dev
|
||||
// and prod). Since the record fence (v1.37.0) every vault preview loads
|
||||
// through `useFencedObjectUrl`, i.e. a `blob:` object URL: without a
|
||||
// `frame-src` the PDF preview iframe fell through to `default-src
|
||||
// 'self'` and was blocked loudly, and `img-src 'self' data:` degraded
|
||||
// every blob thumbnail to its fallback icon silently. Admitting `blob:`
|
||||
// is same-origin-tight: a blob: URL can only be minted by script running
|
||||
// on this origin (`URL.createObjectURL`), script execution stays
|
||||
// nonce-bound (prod), and `object-src 'none'` / `frame-ancestors
|
||||
// 'none'` / `default-src 'self'` are unchanged — no foreign origin
|
||||
// gains anything. Reverting the previews to direct `/api` src instead
|
||||
// is barred by the record-fence guard.
|
||||
isDev
|
||||
? `default-src 'self'; script-src 'self' 'unsafe-eval' 'unsafe-inline'; style-src 'self' 'unsafe-inline'; img-src 'self' data: blob:; frame-src 'self' blob:; connect-src 'self'; font-src 'self';`
|
||||
: `default-src 'self'; script-src 'self' 'nonce-${nonce}'; style-src 'self' 'unsafe-inline'; img-src 'self' data: blob:; frame-src 'self' blob:; connect-src 'self'${aiConnectSrc}${withingsConnectSrc}${whoopConnectSrc}${stravaConnectSrc}; font-src 'self'; object-src 'none'; base-uri 'self'; form-action 'self'; frame-ancestors 'none'; worker-src 'self'; report-uri ${cspReportEndpoint}; report-to csp-endpoint;`;
|
||||
response.headers.set("Content-Security-Policy", csp);
|
||||
|
||||
// Production-only headers. HSTS carries `preload` so the domain stays
|
||||
|
||||
Reference in new issue
Block a user