ci: give every job that runs the unit suite a browser

The hero-geometry check stopped skipping under CI in v1.38 and started failing
instead, which was the point: it had caught its own launch failure and passed
for months, measuring nothing. The install step went into the quality job of
`security.yml` and nowhere else, and a second job runs the same suite — the
auto-merge job for dependency bumps. Every Dependabot pull request has failed
since on `browserType.launch: Executable doesn't exist`, four of them on
2026-09-06 alone (run 34043840314), on an error that says nothing about the
bump. One workflow was fixed; the class was not.

That job installs the same headless shell now, off the same cache key. The
error the check throws names both jobs rather than sending a reader to the
workflow that was already correct, and a guard sweeps the workflow directory so
a third job cannot run the suite with no browser anywhere in it. Its limit is
written down: the sweep is file-scoped, so a workflow with two jobs where only
one installs still satisfies it.

Gates:

  CI=1, browser path hidden      exit=1, "This check is the gate in CI and it
                                 has no browser to measure with"
  CI=1, browser present          exit=0, 4 tests passed (it measures)
  install step deleted from the  guard fails: ["dependabot-auto-merge.yml"]
  auto-merge workflow
This commit is contained in:
Marc-André Bombeck committed 2026-09-09 22:47:31 +02:00
1 parent d48bbff01d
commit 626a95ff33
3 files changed
+90 -7

No files matched your search

@@ -31,6 +31,23 @@ jobs:
- run: pnpm db:generate
# The hero-geometry check in the unit suite renders the real panel in a
# real browser and measures the boxes, and it refuses to skip when CI is
# set — a check that skips measures nothing. This job runs the same suite
# as the quality job in `security.yml` and needs the same browser: without
# it every Dependabot pull request went red on a launch failure that said
# nothing about the bump (four of them on 2026-09-06 alone). Headless
# shell only, matching that job.
- name: Cache Playwright headless shell
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: ~/.cache/ms-playwright
key: playwright-shell-${{ runner.os }}-${{ hashFiles('pnpm-lock.yaml') }}
restore-keys: |
playwright-shell-${{ runner.os }}-
- run: pnpm exec playwright install --only-shell chromium
- run: pnpm test
# Auto-merge is gated on the Dependabot-declared update type:
@@ -0,0 +1,59 @@
/**
* Every job that runs the unit suite installs the browser the suite needs.
*
* `health-score-card-geometry.test.tsx` renders the real hero band in a real
* Chromium and measures the boxes. It used to catch its own launch failure and
* pass, so for months it measured nothing in CI; since v1.38 it throws when
* `CI` is set instead, which is right — and which turns a job that runs
* `pnpm test` without a browser from silently useless into loudly red.
*
* That is exactly what happened. The install step was added to the quality job
* in `security.yml` and nowhere else, and `dependabot-auto-merge.yml` runs the
* same suite: every dependency bump failed on a launch error that said nothing
* about the bump. One workflow was fixed, one class was not.
*
* The sweep is textual and file-scoped, which is its limit: a workflow with two
* jobs, only one of which installs the browser, satisfies it. It catches the
* thing that actually went wrong — a whole workflow running the suite with no
* browser anywhere in it.
*/
import { describe, it, expect } from "vitest";
import { readFileSync, readdirSync } from "node:fs";
import { join } from "node:path";
const WORKFLOWS = join(process.cwd(), ".github", "workflows");
/** `run: pnpm test` — the whole unit suite, geometry check included. */
const RUNS_UNIT_SUITE = /^\s*(?:-\s*)?run:\s*pnpm test\s*$/m;
const INSTALLS_BROWSER = /playwright install --only-shell chromium/;
describe("the browser the unit suite measures with", () => {
it("is installed by every workflow that runs the suite", () => {
const files = readdirSync(WORKFLOWS).filter((f) => f.endsWith(".yml"));
expect(
files.length,
"no workflow files found — the walk is broken, not the tree clean",
).toBeGreaterThanOrEqual(5);
const runners = files.filter((f) =>
RUNS_UNIT_SUITE.test(readFileSync(join(WORKFLOWS, f), "utf8")),
);
// An empty match set would agree with an empty violation list. It is two
// workflows today; pinned at one so removing a job is not a guard failure,
// but a matcher that stops matching is.
expect(
runners.length,
"no workflow runs `pnpm test` — the matcher stopped matching",
).toBeGreaterThanOrEqual(1);
const missing = runners.filter(
(f) => !INSTALLS_BROWSER.test(readFileSync(join(WORKFLOWS, f), "utf8")),
);
expect(
missing,
"a workflow runs the unit suite without installing the headless " +
"Chromium the hero-geometry check measures with — that check refuses " +
"to skip under CI, so the job fails on a launch error instead",
).toEqual([]);
});
});
@@ -223,16 +223,23 @@ beforeAll(async () => {
browser = await chromium.launch();
} catch (error) {
const reason = `no Chromium build available (${String(error).slice(0, 200)})`;
// In CI the browser is installed by the quality job, so a launch failure
// is the job being wrong, not the machine being bare. Throwing from the
// hook fails every test in this suite: a geometry check that quietly
// skips is the same thing as no geometry check, and this repo shipped
// exactly that for months because the skip only ever reached a console
// line nobody reads.
// In CI the browser is installed by the job that runs the suite, so a
// launch failure is the job being wrong, not the machine being bare.
// Throwing from the hook fails every test in this suite: a geometry check
// that quietly skips is the same thing as no geometry check, and this repo
// shipped exactly that for months because the skip only ever reached a
// console line nobody reads.
//
// TWO jobs run `pnpm test`, and naming only one of them here sent a reader
// to a workflow that was already correct: the quality job in
// `security.yml` and the auto-merge job in `dependabot-auto-merge.yml`.
// The second had no install step for three days after this check started
// failing instead of skipping, and turned every dependency bump red.
if (process.env.CI) {
throw new Error(
`[hero geometry] ${reason}. This check is the gate in CI and it has no browser to measure with. ` +
`Restore the "pnpm exec playwright install --only-shell chromium" step in the quality job of .github/workflows/security.yml.`,
`Add the "pnpm exec playwright install --only-shell chromium" step to whichever job ran this suite — ` +
`the quality job of .github/workflows/security.yml, or the auto-merge job of .github/workflows/dependabot-auto-merge.yml.`,
);
}
skipReason = reason;