Installing Chromium's system packages from apt stalled a shard for 29
minutes, the same way WebKit's did. The shards now use the image the
WebKit job already runs in, whose tag follows the lockfile's
playwright-core, so no browser or system package is installed at run
time.
Installing WebKit's system packages in every shard stalled one shard for
29 minutes on apt and left another without time for its tests. The
bottom-sheet spec now runs in its own job inside Microsoft's Playwright
image, whose tag follows the playwright-core version the lockfile
resolves; the shards install Chromium only and skip the WebKit project.
The e2e gate answers for the shards and the WebKit job.
A webkit-iphone project runs one spec: drag, spring back, flick, scroll before drag, a slider sideways, the three closes, focus return, the page's scroll position and pull-to-refresh. CI installs WebKit beside Chromium.
Next 16.4.2 carries the fixes for GHSA-673q-mjm6-r5xr and
GHSA-2464-fhr4-w9xw (unauthenticated remote code execution) and
GHSA-p3m5-59mp-jv5r (denial of service through Server Actions).
eslint-config-next and @next/bundle-analyzer move with it, both pinned
exactly. nodemailer 10.1.7 fixes the address-parser advisories.
The 16.4.0 ignore in the Dependabot config is gone: the history journeys
it was guarding pass on 16.4.2 with the day-step fix, and the note says
what was measured.
The multi-year Google Health backfill run walked three and four years of
heart rate on every pull request, some 3.7 million readings in over twenty
minutes, which filled most of its shard's time budget. What a run costs is
its pages, so a shorter history only helps with shorter slices.
HEALTHLOG_BACKFILL_TEST_DAYS now sets the size. Unset, the runs walk 120
and 160 days in 50-page slices with the faults at the same places within
them and the light collections already walked, about a tenth of the
pages. The heap growth bound applies to the full run only. The full three
and four years run nightly and on demand in integration-long.yml.
Refs #1220
A red Next bump inside the production group held the twelve harmless
updates next to it red as well. The three Next packages open as single
pull requests, so each is judged on its own checks.
Adds messages/nl.json (AI-initial, informal je/jij register, full key
parity with en.json) and registers nl everywhere the locale list lives:
config, loaders, Intl/date-fns mapping, zod enums, OpenAPI spec, scripts,
the managed-profile picker and the notification locale guards.
Dutch follows the Korean pattern for the AI surfaces (English body plus a
Dutch reply-language footer, no safety-contract matrix) and gets its own
banks in the outbound safety screen (dose, risk, causal, continuation),
the prose-grounding and citation-coverage tables, the reference-range
prose and the no-key fallback copy, each with Dutch test fixtures.
PHQ-9 / GAD-7 / WHO-5 are not marked validated for nl yet: the Dutch
wording has not been read against the published forms, so Dutch readers
get the English items plus the existing note until it has.
With 16.4.0 a document sheet opened through ?doc= stays open after Back,
and the [ / ] day keys sometimes leave ?day= where it was. Three end-to-end
journeys fail 3 of 40 repeated runs on 16.4.0 and 0 of 80 on 16.3.8 with
every other update in place, and CI hit the same journeys on both
viewports. 16.3.8 already carries every published security fix, so
Dependabot skips 16.4.0 for next and the two packages that move with it.
Next 16.4.0, pg-boss 12.37.1, the AWS SDK 3.1146, @napi-rs/canvas 1.0.10,
@node-rs/argon2 2.2.2, @simplewebauthn/server 14.0.3, TanStack Query and
Virtual, lucide-react, mmdb-lib, nodemailer, otpauth, pg, radix-ui,
react-day-picker, react-hook-form, tailwind-merge and yaml.
pg-boss moves its schema from 42 to 45 (three additive migrations, nullable
columns and one partial index, no table rewrite); the worker applies them on
start as before. The retention test pins the installed version, so it moves
to 12.37.1 after re-checking every source line it asserts.
zod stays on 4.4.3. 4.5 requires seconds in `z.iso.datetime()` and 4.6
rewrites nullable unions in the generated JSON Schema into `type` arrays,
which reshapes the OpenAPI document and the MCP tool schemas. Dependabot
skips zod 4.5 and later until that move is made on purpose.
The live evaluation now captures the tool calls a generation makes, with
their arguments, and hands them to the judge. runScenarioLive puts a
scenario to the model in the chat turn's own context (system prompt,
tool-mode and dialog addenda, DATA INVENTORY, EARLIER TABLES, transcript)
over the real tool catalogue, answers each call with a stand-in result,
and reads the reply through the turn's clarification and chip parsers.
runScenarioJudge grades each scenario a few times with the same
evaluateScenario the deterministic suite uses and reports a pass rate per
scenario; the server-only scenarios and checks are left out.
Like the existing judge it runs only with COACH_EVAL_API_KEY, from the
nightly workflow's non-blocking step, and never throws.
The delegated-writes spec checked for the invitation form's level control
right after page load, before the form had painted, found nothing and
skipped every test on every run, CI included.
- Give the journey its own account pair, seeded and reset in global setup,
so it no longer invites the same pair as the account-sharing journey
(a second live grant between two accounts is refused with 409).
- Wait for the form through its data-slot and fail when the WRITE level
control is missing instead of skipping.
- Revoke the journey's grant in afterAll.
- Keep it out of the mobile project in the config rather than through a
runtime skip.
Add two checks so a skip cannot hide a spec again:
- e2e-conditional-skip-guard.test.ts requires every conditional
test.skip/test.fixme in e2e/ to carry a reason and refuses a bare
test.skip().
- scripts/check-e2e-skipped-specs.mjs reads the Playwright JSON reports of
both shards in the e2e gate job and fails when a spec file skipped all of
its tests across every project, apart from an allowlist with reasons.
A commit that reaches main without a pull request, such as an asset
uploaded through the web UI, was never analysed until the weekly run.
Run on push to main as well. Pull-request runs still cancel their
superseded heads; push runs are grouped per commit and never cancelled,
so back-to-back merges each keep their own analysis.
The Trivy install step piped a script fetched from a tag straight into a
shell. A tag can be moved to other content, so the job ran whatever the
ref pointed at, with the runner's privileges, and the Scorecard check
flagged it as an unpinned download-then-run.
The script now comes from the commit the tag points at, is checked
against its SHA-256 before it runs, and the binary version stays what it
was.
The compose ecosystem added last change did its job on the first run and
immediately proposed 16.15 to 18.6. A PostgreSQL major changes the
on-disk format, so merging that would hand self-hosters a container that
cannot read their data directory without pg_upgrade or a dump and
restore. Minor releases keep arriving as reviewable PRs; a major is a
project decision with a migration path behind it, not a weekly bump.
The guard grows a case for the ignore rule, so dropping it fails rather
than quietly re-opening the door.
The bundled PostgreSQL was pinned to 16.14 while 16.15 had been out for
five weeks. The pin itself is deliberate, a database change should be
reviewable rather than whatever the registry serves on pull day, but
nothing watched it: Dependabot was configured with the docker ecosystem,
which reads the Dockerfile and never opens a compose file, so no update
PR was ever raised. CI and the integration suite meanwhile run against
the floating 16-alpine, so the shipped database was older than the
tested one.
Adds the docker-compose ecosystem so each PostgreSQL minor release
arrives as a reviewable PR, moves the pin to 16.15, and pins both halves
with a test that fails if the exact minor or the watch is dropped.
Refs #987
S1. The journey signs in with a real account and POSTs that account's
password to whatever BASE_URL names, and the workflow took BASE_URL as
free text. Anyone with repo write, or a leaked token, could dispatch the
run at a collector of their own and keep a working credential; secret
masking guards the log, never the egress.
The host now has to be on a short const allowlist in the script, checked
before anything is sent, with exit 2 and a named refusal otherwise. The
workflow input is a choice between the two hosted instances, so the form
cannot offer a third; the script's own check still stands behind it for a
dispatch that goes round the form.
--self-test covers the refusal, the prefix lookalike a startsWith check
would have waved through included.
Post-publish verify proves the published image boots. Nothing proved that
on a running instance a person can sign in, write a reading and read it
back — which is exactly the shape of the defects this line keeps
rediscovering: a value written and never read.
`scripts/synthetic-journey.mjs` walks that journey in plain Node 22 with no
dependencies, so it runs from a workstation, a runner or the deploy host.
It warms `/`, `/api/health`, `/api/version` and the measurements list first
and discards those timings (the first hit on each route after a deploy is a
cold start), then asserts five legs in order: the served version equals the
expected one, the password flow signs in through a cookie jar, a weight
reading is written with an Idempotency-Key, the same row reads back with
its value, unit and source, and the reversible delete leaves the account
clean. One line per leg with the elapsed time; the first failure ends the
run non-zero with the status and a body excerpt that has every token shape
redacted out of it.
The workflow is `workflow_dispatch` only, with `base_url` and
`expected_version` as inputs and the credentials read from repository
secrets. It is deliberately not wired into the deploy path: the target
instance and the secrets are the maintainer's call.
Mutation proof, against a production build served from
`.next/standalone/server.js` on a throwaway Postgres.
Green:
warm 35 ms (discarded)
ok version 2 ms serves 1.38.13
ok sign in 78 ms session cookie held
ok write 64 ms wrote cmtup9lwu0007zhovn0mxajdz
ok read back 8 ms 77.7 kg (MANUAL)
ok delete 27 ms account left clean
synthetic journey green
Read leg broken deliberately — the same server behind a throwaway forwarder
that empties the `GET /api/measurements` list and changes nothing else:
ok version 4 ms serves 1.38.13
ok sign in 38 ms session cookie held
ok write 40 ms wrote cmtupa4ck000gzhovwnij64fz
FAILED read back 8 ms read back: the list came back without the reading
status 200
body {"data":{"measurements":[],"meta":{"total":0}},"error":null}
synthetic journey RED on "read back"
Version leg broken the same way, by asking the instance for the previous
tag:
FAILED version 3 ms version: instance serves "1.38.13", expected "v1.38.12"
synthetic journey RED on "version"
`--self-test` carries that proof in the repository: it runs the journey
against an in-process instance nine times, once with nothing broken and
once per broken leg, and fails if a broken leg still comes back green. The
workflow runs it before it touches the real instance.
Gates: typecheck, `node --check`, lint (the three known warnings, no
errors), prettier, the workflow and CI-determinism guards.
The audit had moved off the pull_request trigger entirely, which left a window in which a tag could be cut from a merge the audit had not seen. It runs on pull requests again as an advisory job (continue-on-error), stays blocking on the push run for main and on the daily schedule, and a tag is only ever cut from a green push run.
The hero-geometry check stopped skipping under CI in v1.38 and started failing
instead, which was the point: it had caught its own launch failure and passed
for months, measuring nothing. The install step went into the quality job of
`security.yml` and nowhere else, and a second job runs the same suite — the
auto-merge job for dependency bumps. Every Dependabot pull request has failed
since on `browserType.launch: Executable doesn't exist`, four of them on
2026-09-06 alone (run 34043840314), on an error that says nothing about the
bump. One workflow was fixed; the class was not.
That job installs the same headless shell now, off the same cache key. The
error the check throws names both jobs rather than sending a reader to the
workflow that was already correct, and a guard sweeps the workflow directory so
a third job cannot run the suite with no browser anywhere in it. Its limit is
written down: the sweep is file-scoped, so a workflow with two jobs where only
one installs still satisfies it.
Gates:
CI=1, browser path hidden exit=1, "This check is the gate in CI and it
has no browser to measure with"
CI=1, browser present exit=0, 4 tests passed (it measures)
install step deleted from the guard fails: ["dependabot-auto-merge.yml"]
auto-merge workflow
`pnpm audit --prod` answers a question about the published advisory database,
not about the diff. The same lockfile passes on Monday and fails on Tuesday
because someone else published, and the branch that turns red changed nothing:
seventeen of the forty-one Security & Quality failures in the last thirty days
were advisory publications on branches that touched no dependency
(GHSA-jqff-g426-hqxp, GHSA-fph4-wmhf-6fwf, GHSA-f65p-4m7j-42xc and
GHSA-5jgf-p345-68v8 in one run alone). A gate that fails for a reason the author
cannot act on is a gate people re-run instead of read.
It keeps running on every push to `main` and once a day on a new schedule
trigger, which is the cadence the question actually has. The other three jobs
sit out the scheduled run. Nothing about what ships is loosened: the container
scan still builds and scans the image on every push to `main`, and the daily
Trivy workflow still scans the lockfile with dev dependencies included and the
published image.
Gate — the trigger set, read back from the parsed workflow:
triggers: push, pull_request, schedule
quality | if: github.event_name != 'schedule'
dependency-audit | if: github.event_name != 'pull_request'
secret-scan | if: github.event_name != 'schedule'
docker-security | if: github.event_name == 'push' && github.ref == 'refs/heads/main'
Marc-André Bombeck