Forty-two guard files carried their own comment stripper, block comments
first in one non-greedy pass, so a /* inside a line comment opened a
block that hid the code after it, and the stricter copies hid code
behind a /* inside a string, which every e2e route glob has. They all
read through the shared stripComments now, and it scans the way the
parser does: strings, template literals and regular expressions keep
every character, a comment inside a template expression goes, and line
numbers are kept. A parity test strips every source file both ways,
with the scanner and with the TypeScript parser, and fails on the first
that differs.
No guard reports anything new. Each one fails when the stripper returns
nothing; three of them passed on an empty file and assert that the code
they ban is read at all now.
The five hand-written registrars (about 5,500 lines, one 997-line function)
are replaced by `jobs/registry/`: one `defineJob` entry per queue with its
cron, schedule send options, policy, work options, lock, lane and a handler
loader. `registerJobGroup` creates, schedules, retires and binds each group
in the order the registrars used; the worker boot walks the groups and runs
their boot discovery.
Handlers load on their queue's first job. The facts a declaration needs at
boot (names, crons, concurrency, send options) moved out of the heavy job
modules into light `*-queue.ts` siblings, and their importers point there,
so loading the registry evaluates 288 modules instead of the 1,164 the
registrars pulled in. The formerly inline handlers moved verbatim into
handler modules under `jobs/reminder/`.
Fitbit's queues live in `fitbit/jobs/definitions.ts` and enter the registry
through one spread, so removing the integration later is that folder and
that line.
`tests/integration/queue-registration-parity.test.ts` proves pg-boss holds
exactly what the registrars left (queues, schedules, bindings, byte for
byte) and loads every handler once. The queue tests and the runtime guard
assert on the registry objects instead of reading registrar source.
The fold was declared short on the strength of its 60-second
statements, but four aggregates and four persist transactions of up
to 120 s come to about thirteen minutes at worst under the defaults,
and nothing bounds them once an operator lifts the statement timeout.
Past pg-boss's fifteen-minute default the retry folded the account a
second time. The send now carries an hour's expiry, the fold stops
between granularities on the job's budget, and the binding holds a
per-account lockedPass outside the lane. The runtime guard now requires
every heavy queue to be declared long.
pg-boss counts a job's expiry from the fetch and ends the handler at it, so
a job that waited for the heavy lane spent its expiry in line: a long import
let the weekly backup expire through every retry, and an upload that waited
past its own expiry was failed, stopped counting as in use and lost its
staged file to the sweep. A job that finds the lane taken now returns to
created with start_after a minute ahead, at the same attempt, in one
statement, and runs when the lane is free.
The full-history integration drain joins the lane, and the guard checks that
every heavy queue's policy lets a job go back to created.
Web and worker share one process and one heap limit, and the jobs that
hold a large working set (the Apple Health and Health Connect imports,
restore, the weekly and off-host backups, the restore drill, the rollup
backfill, the nightly status batch, the pre-generation walk and the
narrative walk) ran side by side whenever their schedules or an upload
met, so the peak was their sum. `heavyLane` queues them in the process,
first come first served. A waiter leaves the line when pg-boss expires it
and its retry takes the place; a one-shot whose input lives outside
pg-boss keeps its place. The runtime guard binds every heavy queue in the
lane, keeps `lockedPass` outside it, and allows the one-shot rule only
where the send options forbid a retry.
pg-boss polls each idle worker slot every two seconds, so the hundred-odd
bindings asked the database for work about fifty times a second while
most of them wait for one tick a day. `createAndWork` now gives a queue
that a registrar schedules and nothing else sends to a 30-second poll.
Scheduled queues that also take sends keep the default, because with one
job per fetch a slow poll would drain a burst at one job per interval;
the runtime guard holds that list to the send sites in the tree,
including sends through a variable.
Every queue bound with createAndWork is now declared in
src/lib/jobs/queue-runtime.ts as short (cannot come near the default
fifteen minutes, with the reason), budgeted (stops itself at three
quarters of the default and resumes) or long (an explicit expiry, a
stop on the job's budget, and a lock or claim per identity; or a
declared one-shot with no retries).
The guard reads every binding, every boss.send and every schedule tuple
in the tree: each names a queue in the table, every entry is bound, a
long queue's sends and schedules all carry its expiry, its handler
stops on jobBudget / jobDeadline where declared and holds lockedPass or
the named marker, and a short or budgeted queue sets no expiry of its
own. Floors pin the counts (112 bindings, 64 sends, 74 schedules), and
planted long queues missing each protection in turn are caught. Run
against the tree before the previous commit it reports 59 findings;
dropping lockedPass and the cron expiry from pr-detection reports both.
Marc-André Bombeck