mirror of
https://github.com/MBombeck/HealthLog.git
synced 2026-11-09 13:01:00 +00:00
fix(jobs): load every job handler once at worker boot and report failures
Since handlers load on their queue's first job, a module the image cannot load no longer fails the worker's boot. The worker comes up healthy, the queue burns its retries when it first has work, and a one-shot job (a restore, an import tail) ends failed with nobody told at deploy time. After binding every queue the worker now runs `preloadHandlers` in the background, without awaiting it, so readiness is unchanged. It loads each handler once through the same memoised loader its queue is bound to, so a module that loads stays loaded for the first job. Each failure is reported on the `job.run.failed` line at error, with the queue's name and `reason: "handler_load_failed"`, plus the worker error report (stderr and GlitchTip). A summary line on stderr names every queue that failed. The sweep never throws, and a failing report does not stop it. docs/ops/deploy.md says what to look for after a deploy.
This commit is contained in:
1 parent
f7b89f96e1
commit
c6bd27f06f
6 files changed
+305
-15
No files matched your search
@@ -237,6 +237,21 @@ Pass `--only=prod` / `--only=demo` to check a single target, or
|
||||
`--attempts=N --interval=ms` to widen the retry window while a slow
|
||||
pull settles.
|
||||
|
||||
The worker loads each job's handler module on that job's first run, so a
|
||||
module the image cannot load does not stop the worker from starting. Once
|
||||
it has bound its queues, the worker loads every handler a single time in
|
||||
the background and reports each one that fails, normally within a minute
|
||||
of start. Each failure is a `job.run.failed` event at `error` with
|
||||
`reason: "handler_load_failed"` and the queue's name, and stderr also gets
|
||||
`[pg-boss] [registry] preloaded N/M job handlers; failed to load: <queues>`.
|
||||
If either shows up after a deploy, the image is broken for those queues,
|
||||
and their jobs will fail when they run. Roll back (see Rollback above)
|
||||
instead of waiting for the retries to run out:
|
||||
|
||||
```sh
|
||||
docker compose logs --since 5m app 2>&1 | grep -E 'handler_load_failed|failed to load:'
|
||||
```
|
||||
|
||||
## Walk the synthetic journey
|
||||
|
||||
The version check proves the instance serves the right build. It does not
|
||||
|
||||
Reference in new issue
Block a user