The Debian base carries HIGH advisories the distribution has not fixed
(util-linux, ncurses, systemd, acl, sqlite, perl-base), which no change
here can remove. The blocking image scan now runs with --ignore-unfixed,
so a finding with a published fix still fails the job, and a separate
step before it lists every HIGH and CRITICAL finding without failing.
The scheduled scan's critical gate follows the same rule; its SARIF
report keeps everything.
The dependency and build stages stay on node:22-trixie-slim, where pnpm
picks the glibc native packages. The runner moves to wolfi-base, pinned
by its index digest: glibc, about fifty packages, and every one of them
known to the image scan, Node and the PostgreSQL client included.
nodejs-22, tzdata and postgresql-18-client come from apk, so the pgclient
stage and its pin guard go. The builder writes its Node module ABI and
glibc version out and the runner refuses to build on another ABI or an
older glibc. Wolfi's npm is major 12 and runs no install scripts unless
package.json allows them, so the Prisma CLI install names the three
packages that have one, and the build checks that the schema engine file
is there before asking the CLI about it. npm is installed for that step
and removed in the same layer; the check that nothing is left asks a
fresh shell, whose command cache cannot remember the deleted binary.
Every stage now starts from node:22-trixie-slim, pinned by its multi-arch
index digest, so the native modules load their glibc builds. The arm64
musl Skia build needs ARMv8.2 (LSE atomics, RCpc loads) and dies with
SIGILL on an ARMv8.0 core such as the Raspberry Pi 4's Cortex-A72; the
glibc build runs there (#1311).
The runner keeps what it had: tzdata, TZ, the build-time OS upgrade, the
nextjs user (uid 1001, now in group nodejs gid 1001), the backup and
GeoLite2 directories and every build-time assertion. pg_dump and
pg_restore of major 18 come from the PGDG apt repository, whose signing
key is fetched by node and checked against a pinned SHA-256 before apt
trusts it; PATH names the major 18 directory so the binary does not
depend on Debian's version-picking wrapper. The canvas hoist takes the
-gnu binary package by the pinned version and still fails the build on a
mismatch. bookworm-slim was the other candidate; on the day of the
switch it carried a CRITICAL zlib finding that trixie-slim does not.
The drill now runs an S3-compatible bucket beside the stack (Versity S3
Gateway v1.8.0, pinned by digest; the MinIO images are no longer published
on Docker Hub or quay.io) on a network and a volume of its own, so it
outlives the stack as a real bucket does. The app takes the dump through the
scheduled job and uploads it there; the drill waits for a run that both
kept and uploaded it, destroys every volume of the stack, the local backups
included, and restores on a new stack with only the bucket and the keys:
`docker compose run --rm db-restore latest --from-bucket`. The independent
row count reads the manifest from the bucket with the restore's own reader.
DRILL_SOURCE=local keeps the earlier path (dump files copied off the host,
then `db-restore latest`), and CI runs both. A new mutation,
wrong-backup-key, restores with a different BACKUP_ENCRYPTION_KEY and must
be refused. Ports are free ones picked at start.
scripts/compose-dr-drill.sh plays docs/ops/disaster-recovery.md against
real containers: it builds the image from the checkout, starts the bundled
compose stack with fixed throwaway keys, seeds the demo account and a
Guardian with a managed profile (through the app's own code), writes a
reading with an encrypted note through the API, and waits for the
scheduled database-backup job to take the dump. It then keeps only the
dump files, destroys every volume, starts a new stack, restores with
`docker compose run --rm db-restore latest`, counts every table against
the manifest apart from the restore, and boots the app: health, the demo
login, the note read back, a document opened, the Guardian still seeing
the profile.
DRILL_MUTATION breaks one step on purpose (wrong-key, skip-table,
non-empty-target), and the drill fails on each. The workflow runs it on
pull requests that touch the compose file, the image, the entrypoint or
the dump and restore code, nightly, and by hand.
The compose file runs Postgres 18. An install from v1.45 or earlier keeps
its Postgres 16 data in `pgdata`, which 18 cannot open, so a one-shot
`db-upgrade` service moves it on the first start: it starts Postgres 16 on
the old volume inside its own container, streams a dump into 18 in one
transaction, compares every table's row count, gathers planner statistics
and marks the old volume as moved. The app waits for it to succeed.
The old volume is never changed apart from the marker, so the way back is
the previous image and volume lines. A move cut short leaves a "moving"
marker and runs again from scratch; an emptied new volume after a finished
move is refused rather than refilled with the old state.
scripts/compose-db-upgrade-check.sh plays all of it against real
containers from the v1.45.0 compose file, and runs in CI on every change
to the compose file. Pull requests run the integration suite and e2e on
Postgres 18; the nightly integration run keeps 16 covered for installs
that bring their own database.
tablet-shell.spec.ts runs on WebKit with an iPad profile (webkit-tablet,
in the WebKit job) and on Chromium with an Android tablet profile
(chromium-tablet, in the shards), across iPad mini to iPad Pro 13 in
both orientations, Safari's bars, Split View and Slide Over. It checks
that the shell fills the window and leaves the document nothing to
scroll, that every rail destination scrolls into view on top and takes a
tap, that the scroll cue follows the overflow, that main reaches its
last line clear of the bottom bar, that the page scrolls again after a
sheet and a dialog close, and that a tab strip leaves the vertical pan
to the page.
ui-standards records the shell and rail rules.
The floors step piped scripts/mutation-floors.mjs into tee without
pipefail, so it took tee's exit status and a module under its floor could
never fail the run. It now runs under bash with pipefail, as do the other
steps that pipe into tee, and a guard reads every workflow and fails on a
tee pipe without it. The other piped steps were checked: each either
validates what the pipe produced or pipes a command that cannot fail.
The nightly runs set FC_NUM_RUNS, an absolute count that replaced every
property's own: the database backup round trip, which states twelve cases
for its cost, ran a hundred times against a timeout sized for twelve.
FC_RUNS_FACTOR now multiplies each property's count (five for the unit
suite, four for integration), propertyTimeout grows a test's timeout by
the same factor, and the backup round trip uses it.
next stays on main's 16.4.2 (the RCE and Server Actions advisories) with its two companions; release's newer dev tools and its own additions are kept. The lockfile is release's, re-resolved. pt-BR, which main does not have, takes the one key main added.
Installing Chromium's system packages from apt stalled a shard for 29
minutes, the same way WebKit's did. The shards now use the image the
WebKit job already runs in, whose tag follows the lockfile's
playwright-core, so no browser or system package is installed at run
time.
Installing WebKit's system packages in every shard stalled one shard for
29 minutes on apt and left another without time for its tests. The
bottom-sheet spec now runs in its own job inside Microsoft's Playwright
image, whose tag follows the playwright-core version the lockfile
resolves; the shards install Chromium only and skip the WebKit project.
The e2e gate answers for the shards and the WebKit job.
A webkit-iphone project runs one spec: drag, spring back, flick, scroll before drag, a slider sideways, the three closes, focus return, the page's scroll position and pull-to-refresh. CI installs WebKit beside Chromium.
Stryker runs against encryption at rest, medication compliance, the rollups,
source ranking, unit conversion and backup validation, with the unit tests
that live next to them (vitest.mutation.config.mts). It is too slow for a
pull request and runs nightly and on demand (.github/workflows/mutation.yml).
scripts/mutation-floors.mjs reads the JSON report and fails when a module
scores below its floor in stryker.floors.json, so one well-tested module
cannot hide a weak one. The floors sit two points below the first measured
scores: crypto 46.9 %, compliance 66.8 %, rollups 41.7 %, source ranking
57.0 %, units 87.4 %, backup validation 25.5 %.
@stryker-mutator/vitest-runner 10.0.0 predates Vitest 5, which names a test
"describe > test" where the runner still built "describe test". Every test
inside a describe block was filtered out of the mutant runs, so nearly every
mutant survived (5.5 % across the set). A pnpm patch builds the name the way
Vitest 5 does; the Docker dependency stage copies the patches directory,
which the frozen install needs.
Once a night both suites run with files and tests in a random order, seeded
and printed, so order dependence surfaces on purpose rather than on somebody's
pull request; a failure replays with --sequence.seed. The same run gives every
fast-check property a fresh seed and five times the cases. The integration
reports are merged into a regenerated timing file and uploaded for a
maintainer to commit when the shards drift apart. Not a gate.
Vitest's --shard cut the file list into slices of equal length, which gave
the integration shards 954 s to 1 449 s. A sequencer now deals files out by
their measured duration, plus a fixed per-file cost for import and setup,
longest first to the shard with the least work, and starts the longest files
first inside a shard. Timings live in tests/durations/*.json and are
regenerated from Vitest JSON reports with scripts/test-durations.mjs; every
CI shard uploads its report.
Integration shards run three workers each, one cloned database per worker.
The unit suite runs as three projects decided per file from its source:
guards and unit share a module graph inside a worker, isolated keeps a fresh
one for every file that mocks, spies, stubs or resets. In CI it is three
duration-balanced shards next to a job for lint, format, types and the
OpenAPI drift check.
The required check names stay as they were. "Lint, Typecheck & Test" is now
the job that answers for the static checks and every unit shard, the way
"integration" already answers for its shards. The browser-install guard
recognises a direct, flags-only vitest run as a run of the unit suite.
The gate summed every emitted chunk once and watched five pages. The sum
counts a module once per chunk group it lands in, so it rose whenever a
page got lighter by moving code into on-demand chunks, and it needed a
raise in most releases while no reader downloaded more; and a heavy import
into a page outside the five went unseen.
scripts/check-bundle-budget.mjs now measures:
- the first load of every page: rootMainFiles plus the page's
client-reference chunks, each file counted once by content hash;
- p50, p95 and max of those first loads across pages;
- the distinct client code: every chunk evaluated in a node:vm sandbox
with a stub TURBOPACK array, each module factory counted once by its
module id (not its text: each chunk is minified on its own, so copies of
one module differ in mangled names), catalogs excluded, gzipped once.
The summed aggregate is still printed but no longer gated. The catalog,
per-catalog and one-Recharts checks are unchanged. `--write-baselines`
rewrites the per-page baselines, `--dir` measures another build directory,
`--json` writes the report.
bundle-budget.json is re-based from the v1.44.0 build (local arm64): 110
pages, first load p50 336, p95 437, max 451 KB gz; distinct client code
1904 KB gz in 1224 modules; summed aggregate 3875 KB for comparison. Every
page has a baseline with the same 20 KB allowance; caps are p95 447, max
461 and distinct code 1930 KB. A page without a baseline fails until one
is written in the same change.
Proof: importing zod into the weight insights page measured
/insights/weight at 427.8 KB gz, 88.8 KB over its 339 KB baseline, and the
gate failed; the old gate passed the same build, because the page was not
watched and the summed aggregate did not move (3875 KB). The import was
removed again.
The multi-year Google Health backfill run walked three and four years of
heart rate on every pull request, some 3.7 million readings in over twenty
minutes, which filled most of its shard's time budget. What a run costs is
its pages, so a shorter history only helps with shorter slices.
HEALTHLOG_BACKFILL_TEST_DAYS now sets the size. Unset, the runs walk 120
and 160 days in 50-page slices with the faults at the same places within
them and the light collections already walked, about a tenth of the
pages. The heap growth bound applies to the full run only. The full three
and four years run nightly and on demand in integration-long.yml.
Refs #1220
The live evaluation now captures the tool calls a generation makes, with
their arguments, and hands them to the judge. runScenarioLive puts a
scenario to the model in the chat turn's own context (system prompt,
tool-mode and dialog addenda, DATA INVENTORY, EARLIER TABLES, transcript)
over the real tool catalogue, answers each call with a stand-in result,
and reads the reply through the turn's clarification and chip parsers.
runScenarioJudge grades each scenario a few times with the same
evaluateScenario the deterministic suite uses and reports a pass rate per
scenario; the server-only scenarios and checks are left out.
Like the existing judge it runs only with COACH_EVAL_API_KEY, from the
nightly workflow's non-blocking step, and never throws.
The delegated-writes spec checked for the invitation form's level control
right after page load, before the form had painted, found nothing and
skipped every test on every run, CI included.
- Give the journey its own account pair, seeded and reset in global setup,
so it no longer invites the same pair as the account-sharing journey
(a second live grant between two accounts is refused with 409).
- Wait for the form through its data-slot and fail when the WRITE level
control is missing instead of skipping.
- Revoke the journey's grant in afterAll.
- Keep it out of the mobile project in the config rather than through a
runtime skip.
Add two checks so a skip cannot hide a spec again:
- e2e-conditional-skip-guard.test.ts requires every conditional
test.skip/test.fixme in e2e/ to carry a reason and refuses a bare
test.skip().
- scripts/check-e2e-skipped-specs.mjs reads the Playwright JSON reports of
both shards in the e2e gate job and fails when a spec file skipped all of
its tests across every project, apart from an allowlist with reasons.
A commit that reaches main without a pull request, such as an asset
uploaded through the web UI, was never analysed until the weekly run.
Run on push to main as well. Pull-request runs still cancel their
superseded heads; push runs are grouped per commit and never cancelled,
so back-to-back merges each keep their own analysis.
The Trivy install step piped a script fetched from a tag straight into a
shell. A tag can be moved to other content, so the job ran whatever the
ref pointed at, with the runner's privileges, and the Scorecard check
flagged it as an unpinned download-then-run.
The script now comes from the commit the tag points at, is checked
against its SHA-256 before it runs, and the binary version stays what it
was.
S1. The journey signs in with a real account and POSTs that account's
password to whatever BASE_URL names, and the workflow took BASE_URL as
free text. Anyone with repo write, or a leaked token, could dispatch the
run at a collector of their own and keep a working credential; secret
masking guards the log, never the egress.
The host now has to be on a short const allowlist in the script, checked
before anything is sent, with exit 2 and a named refusal otherwise. The
workflow input is a choice between the two hosted instances, so the form
cannot offer a third; the script's own check still stands behind it for a
dispatch that goes round the form.
--self-test covers the refusal, the prefix lookalike a startsWith check
would have waved through included.
Post-publish verify proves the published image boots. Nothing proved that
on a running instance a person can sign in, write a reading and read it
back — which is exactly the shape of the defects this line keeps
rediscovering: a value written and never read.
`scripts/synthetic-journey.mjs` walks that journey in plain Node 22 with no
dependencies, so it runs from a workstation, a runner or the deploy host.
It warms `/`, `/api/health`, `/api/version` and the measurements list first
and discards those timings (the first hit on each route after a deploy is a
cold start), then asserts five legs in order: the served version equals the
expected one, the password flow signs in through a cookie jar, a weight
reading is written with an Idempotency-Key, the same row reads back with
its value, unit and source, and the reversible delete leaves the account
clean. One line per leg with the elapsed time; the first failure ends the
run non-zero with the status and a body excerpt that has every token shape
redacted out of it.
The workflow is `workflow_dispatch` only, with `base_url` and
`expected_version` as inputs and the credentials read from repository
secrets. It is deliberately not wired into the deploy path: the target
instance and the secrets are the maintainer's call.
Mutation proof, against a production build served from
`.next/standalone/server.js` on a throwaway Postgres.
Green:
warm 35 ms (discarded)
ok version 2 ms serves 1.38.13
ok sign in 78 ms session cookie held
ok write 64 ms wrote cmtup9lwu0007zhovn0mxajdz
ok read back 8 ms 77.7 kg (MANUAL)
ok delete 27 ms account left clean
synthetic journey green
Read leg broken deliberately — the same server behind a throwaway forwarder
that empties the `GET /api/measurements` list and changes nothing else:
ok version 4 ms serves 1.38.13
ok sign in 38 ms session cookie held
ok write 40 ms wrote cmtupa4ck000gzhovwnij64fz
FAILED read back 8 ms read back: the list came back without the reading
status 200
body {"data":{"measurements":[],"meta":{"total":0}},"error":null}
synthetic journey RED on "read back"
Version leg broken the same way, by asking the instance for the previous
tag:
FAILED version 3 ms version: instance serves "1.38.13", expected "v1.38.12"
synthetic journey RED on "version"
`--self-test` carries that proof in the repository: it runs the journey
against an in-process instance nine times, once with nothing broken and
once per broken leg, and fails if a broken leg still comes back green. The
workflow runs it before it touches the real instance.
Gates: typecheck, `node --check`, lint (the three known warnings, no
errors), prettier, the workflow and CI-determinism guards.
The audit had moved off the pull_request trigger entirely, which left a window in which a tag could be cut from a merge the audit had not seen. It runs on pull requests again as an advisory job (continue-on-error), stays blocking on the push run for main and on the daily schedule, and a tag is only ever cut from a green push run.
The hero-geometry check stopped skipping under CI in v1.38 and started failing
instead, which was the point: it had caught its own launch failure and passed
for months, measuring nothing. The install step went into the quality job of
`security.yml` and nowhere else, and a second job runs the same suite — the
auto-merge job for dependency bumps. Every Dependabot pull request has failed
since on `browserType.launch: Executable doesn't exist`, four of them on
2026-09-06 alone (run 34043840314), on an error that says nothing about the
bump. One workflow was fixed; the class was not.
That job installs the same headless shell now, off the same cache key. The
error the check throws names both jobs rather than sending a reader to the
workflow that was already correct, and a guard sweeps the workflow directory so
a third job cannot run the suite with no browser anywhere in it. Its limit is
written down: the sweep is file-scoped, so a workflow with two jobs where only
one installs still satisfies it.
Gates:
CI=1, browser path hidden exit=1, "This check is the gate in CI and it
has no browser to measure with"
CI=1, browser present exit=0, 4 tests passed (it measures)
install step deleted from the guard fails: ["dependabot-auto-merge.yml"]
auto-merge workflow
`pnpm audit --prod` answers a question about the published advisory database,
not about the diff. The same lockfile passes on Monday and fails on Tuesday
because someone else published, and the branch that turns red changed nothing:
seventeen of the forty-one Security & Quality failures in the last thirty days
were advisory publications on branches that touched no dependency
(GHSA-jqff-g426-hqxp, GHSA-fph4-wmhf-6fwf, GHSA-f65p-4m7j-42xc and
GHSA-5jgf-p345-68v8 in one run alone). A gate that fails for a reason the author
cannot act on is a gate people re-run instead of read.
It keeps running on every push to `main` and once a day on a new schedule
trigger, which is the cadence the question actually has. The other three jobs
sit out the scheduled run. Nothing about what ships is loosened: the container
scan still builds and scans the image on every push to `main`, and the daily
Trivy workflow still scans the lockfile with dev dependencies included and the
published image.
Gate — the trigger set, read back from the parsed workflow:
triggers: push, pull_request, schedule
quality | if: github.event_name != 'schedule'
dependency-audit | if: github.event_name != 'pull_request'
secret-scan | if: github.event_name != 'schedule'
docker-security | if: github.event_name == 'push' && github.ref == 'refs/heads/main'
dependabot[bot]