From c809a811e6379091213ed78437cebdf1f96eab79 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Marc-Andr=C3=A9=20Bombeck?= Date: Sat, 3 Oct 2026 23:32:04 +0200 Subject: [PATCH] docs(self-hosting): say which reads run on the worker and why a split worker must match the web version --- docs/self-hosting/scaling.md | 37 ++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) diff --git a/docs/self-hosting/scaling.md b/docs/self-hosting/scaling.md index 940abdc4d..01b1c950f 100644 --- a/docs/self-hosting/scaling.md +++ b/docs/self-hosting/scaling.md @@ -86,6 +86,43 @@ boot and every 15 minutes, flipping orphaned rows to `failed` (`interrupted_by_restart`) so the UI offers a retry instead of an endless spinner. +## Document and lab-scan reads run on the worker (v1.40) + +Since v1.40, every AI read of a document or a lab report runs as a +background job on the worker, not inside the HTTP request: + +- reading a document for search (the content index), +- a document summary, stored or shown once, +- filing suggestions (title, type, date), +- extracting facts from a stored document for review, +- a lab scan (photo or PDF of a lab report). + +The request only queues the read and answers `202`; the client then +polls the run until it ends. That is what keeps a +slow model from being cut off by a proxy timeout in front of the +server. When the enqueue itself fails (no pg-boss connection at all), +the request answers `503` straight away, and nothing is charged. + +**"Waiting for the background worker"** is the line the app shows when +a read has sat in the queue for more than 30 seconds without a worker +picking it up. On a single `all` container that is rare and short +(the worker is busy with another job). If it does not go away, no +worker process is consuming the queue: the worker container is +stopped, crashing on boot, or pointed at another database. A run that +no worker picks up within 15 minutes fails as +`aiRuns.workerUnavailable` and its AI budget reservation is handed +back. That sweep is itself a worker job, so with no worker running at +all the run simply stays queued until one starts. + +**In a web/worker split, the worker must run the same version as the +web containers.** The web container queues the read; only a worker +that knows the `document-ai-run` queue can execute it. A worker left +on a pre-v1.40 image never takes these jobs, so every document read +and lab scan waits and then fails as above, while everything else +looks healthy. Deploy both from the same image tag +(`HEALTHLOG_IMAGE_REF` is shared by `app` and the `app-worker` block) +and restart the worker together with the web containers. + ## Healthchecks - `app` (web) healthcheck: `wget /api/health` every 30s.