Commit Graph
9 Commits
Author SHA1 Message Date
Marc-André Bombeck 4b73099e75 fix(documents): render PDFs with pdfjs font data and refuse blank pages
Refs #1124. The rasterizer opened pdfjs without its standard font, CMap
and wasm directories and told it to use system fonts. The runtime image
has none, so any text pdfjs could not draw from an embedded program (a
non-embedded standard font, or an embedded one it cannot parse) came out
as nothing, and an image-only provider was sent the page background. A
workstation hides this because its own fonts stand in.

pdfjs now gets the directories from its own package, resolved at runtime
from the working directory, and Liberation Sans from that package is
registered as the canvas' sans-serif, serif and monospace faces for the
text pdfjs draws with a substitute.

A document whose every rendered page is blank now reports render-failed,
so the read records raster-failed instead of indexing a transcript of an
empty page as a success.
2026-10-04 10:37:24 +02:00
Marc-André Bombeck 5f548d331f fix(labs): say when a scanned PDF was read from its first pages only
A PDF longer than the pages a scan renders was read from its first ten
pages, and the cut was recorded only in the wide event. The person checking
the extracted rows had no way to know that values on later pages were never
read.

The rasterizer now returns the PDF's page count, the extract response
carries pageCoverage { read, total } when the cap applied, and the review
screen says how many of the pages were read.
2026-10-01 10:05:35 +02:00
Marc-André Bombeck 589b35589b Prove the raster-failed chain and name the fax-codec cause
The residual scan from the report is a CCITT Group 4 single-image PDF,
the codec class pdfjs rasterization is known to fail on. Chain checks:
a pdfjs open failure or throw already lands as render-failed with the
error text annotated; the zero-pages shape now annotates too instead
of returning silently. An end-to-end test drives the REAL rasterizer
through prepareVisionInput and pins the rasterFailed mapping; the test
header documents why a tiny in-repo CCITT fixture is deliberately not
checked in (probed against this pdfjs version, it renders the page as
blank instead of throwing, so the fixture cannot deterministically
fail and would be flaky across upgrades). The detail view's
raster-failed line now names scanned fax compression formats as a
known cause, in all six locales.

Refs #776
2026-08-13 13:04:41 +02:00
Marc-André Bombeck 893db887e9 Record why a document's index attempt failed and say so
A content-index attempt that failed (raster failure, a scan with no
text layer, a provider error) recorded its reason only in the server's
observability stream. In the vault, hasContentIndex false looked
identical for a document nobody ever tried and one that failed three
times, and the detail sheet had nothing to say about either.

Apply the summaryState pattern to the index. Two additive columns on
InboundDocument (migration 0333) record when the last attempt finished
and why it produced no index, as a closed reason set: local-empty,
local-unsupported, decrypt-error, raster-failed, provider-error,
pdf-needs-anthropic, empty-transcription. The writer sits at the choke
point the auto job and the corpus backfill share, and in the manual
index route; a success clears the reason. The provider path's specific
failure now survives the local fallback, since "no text" is less
actionable than "the PDF would not render". rasterizePdf names its
failure class so a build that cannot rasterize at all is distinguished
from a PDF that would not render. New guard in both index paths: an
empty provider transcription never becomes a successful empty index.

The detail sheet renders the reason in plain words (muted, six
locales). The DTO and OpenAPI carry the two fields; the disaster
recovery backup and restore carry them verbatim like their summary
siblings.

Refs #776
2026-08-13 12:40:50 +02:00
Marc-André Bombeck 74a386fcfd chore(deps): update July dependency bundle 2026-07-20 14:18:48 +02:00
Marc-André Bombeck 45da6c3c7d chore(release): v1.28.27 — runs on CPUs without AVX2
The bundled Skia x64 build (document thumbnails, scanned-PDF rasterization)
uses AVX2 instructions; on an x86-64 host without them — Celeron/Atom-class
NAS CPUs, pre-2013 Xeons, VMs masking CPU flags — the first canvas call is an
uncatchable SIGILL that kills the process and crash-loops the container (live
incident: a Synology DS1520+, exit 132 ~18 s after every boot, reported with
kernel traps and the correct root cause in hand).

The renderer is now gated on a one-time CPU-feature check before the module
ever loads: linux/x64 requires the avx2 cpuinfo flag; arm64 and non-Linux are
unaffected; unreadable cpuinfo fails open. Behind a closed gate, thumbnail
generation and PDF rasterization take their existing fail-soft paths (vault
shows type icons; PDFs read as text) and the boot-time thumbnail backfill
skips enqueueing. NATIVE_CANVAS=off|on overrides detection, documented in the
env example and whitelisted in compose.
2026-07-11 10:02:29 +02:00
Marc-André Bombeck ee76672798 feat(documents): preview thumbnails in the vault overview
Render a small JPEG preview for each stored document and show it at the
card's leading edge, falling back to the kind icon when none exists.

- Generate previews in the background on upload (new document-thumbnail
  pg-boss queue) plus a boot-time corpus backfill for existing documents;
  upload never blocks on or fails because of it. Images are downscaled via
  @napi-rs/canvas, a PDF's first page is rasterised then downscaled, every
  other type is skipped. Generation never throws.
- Store the preview encrypted at rest in a 1:1 DocumentThumbnail side table
  (migration 0236) so the list/detail queries that omit the original blob
  never drag the preview either. Same AES-256-GCM string codec + fail-closed
  decrypt as the content index; wired into the key-rotation walk.
- Serve it through an owner-scoped GET /api/documents/inbound/[id]/thumbnail
  (auth + module gate + userId-scoped query + fail-closed decrypt), always
  image/jpeg + nosniff + private, no-store; 404 on miss. The card renders an
  authed lazy <img>, onError falls back to the icon.
- Add hasThumbnail to the document DTO so the card only fetches a preview
  that exists.

Also drop the "search looks inside your documents" caption from the overview
(the index-all prompt stays) and the per-card searchable marker; both were
noise. Remove the now-unused i18n keys across all six locales.
2026-07-10 01:15:32 +02:00
Marc-André Bombeck a89e511359 fix(documents): run pdfjs + canvas un-bundled so scanned PDFs actually rasterize
The bundled (Turbopack) copy of pdfjs-dist failed its render path in the
standalone image with a bare Error, so a scanned PDF on a non-Anthropic
provider fell back to 'PDF scanning needs a Claude vision provider' — while the
identical un-bundled module rasterizes the same document fine (verified in the
container end to end on the real document). Externalize pdfjs-dist and
@napi-rs/canvas so the server loads the real modules, and hoist pdfjs-dist to
the top level in the image so the runtime import resolves. Also log the render
error message (not just its name) and drop the redundant 'not saved' note from
the document summary panel.
2026-07-09 16:22:00 +02:00
Marc-André Bombeck d2451021bc feat(documents): rasterize PDFs so image-only-wire providers can read them
Only Anthropic accepts a native PDF block; codex and other image-only-wire
providers dropped it. Add a bounded, best-effort server-side rasterizer that
renders a PDF's pages to JPEG images those providers read as ordinary image
input, so a scanned/image-only PDF is legible on the auto-read path.

- @napi-rs/canvas promoted to a direct dependency; its musl prebuilts + the
  pdfjs loader pinned into the standalone trace (next.config.ts)
- rasterize-pdf.ts: lazy pdfjs import (keeps DOMMatrix out of the eager chunk),
  first 10 pages, 1600px long edge, JPEG q80, never throws
- prepareVisionInput rasterizes a PDF for a non-Anthropic vision provider and
  keeps the native PDF block for Anthropic; the document capability reports
  PDFs readable via raster so the UI offers the read
2026-07-09 10:49:17 +02:00