ci: fail the mutation floors step, and scale nightly property runs by a factor

The floors step piped scripts/mutation-floors.mjs into tee without
pipefail, so it took tee's exit status and a module under its floor could
never fail the run. It now runs under bash with pipefail, as do the other
steps that pipe into tee, and a guard reads every workflow and fails on a
tee pipe without it. The other piped steps were checked: each either
validates what the pipe produced or pipes a command that cannot fail.

The nightly runs set FC_NUM_RUNS, an absolute count that replaced every
property's own: the database backup round trip, which states twelve cases
for its cost, ran a hundred times against a timeout sized for twelve.
FC_RUNS_FACTOR now multiplies each property's count (five for the unit
suite, four for integration), propertyTimeout grows a test's timeout by
the same factor, and the backup round trip uses it.
This commit is contained in:
Marc-André Bombeck committed 2026-10-22 19:04:20 +02:00
1 parent 422adf2986
commit 298515d156
7 files changed
+242 -61

No files matched your search

+6 -1
View File
@@ -35,8 +35,13 @@ jobs:
- name: Run Stryker
run: pnpm exec stryker run --concurrency 4
# Without pipefail the step takes the exit status of `tee`, which is
# always 0, so a module under its floor could never fail the run.
- name: Check the per-module floors
run: node scripts/mutation-floors.mjs | tee -a "$GITHUB_STEP_SUMMARY"
shell: bash
run: |
set -o pipefail
node scripts/mutation-floors.mjs | tee -a "$GITHUB_STEP_SUMMARY"
- name: Upload the report
if: always()
+11 -2
View File
@@ -58,9 +58,12 @@ jobs:
echo "FC_SEED=$(( $(date +%s%N) % 2147483647 ))" >> "$GITHUB_ENV"
- name: Unit tests, shuffled
shell: bash
env:
FC_NUM_RUNS: "500"
# Five times each property's own case count (tests/property/fast-check.ts).
FC_RUNS_FACTOR: "5"
run: |
set -o pipefail
echo "Order seed ${TEST_SEED}, property seed ${FC_SEED}" | tee -a "$GITHUB_STEP_SUMMARY"
pnpm exec vitest run --sequence.shuffle --sequence.seed="${TEST_SEED}" \
--reporter=default --reporter=json --outputFile.json=unit-report.json
@@ -104,14 +107,18 @@ jobs:
echo "FC_SEED=$(( ${{ github.run_id }} % 2147483647 ))" >> "$GITHUB_ENV"
- name: Integration tests, shuffled
shell: bash
env:
HEALTHLOG_INTEGRATION_WORKERS: "3"
FC_NUM_RUNS: "100"
# Four times each property's own case count; a property's timeout
# grows by the same factor (`propertyTimeout`).
FC_RUNS_FACTOR: "4"
# Quoted for the reason given in integration.yml.
API_TOKEN_HMAC_KEY: "dddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd"
ENCRYPTION_KEY: "0000000000000000000000000000000000000000000000000000000000000000"
SESSION_SECRET: "integration-tests"
run: |
set -o pipefail
echo "Order seed ${TEST_SEED}, property seed ${FC_SEED}" | tee -a "$GITHUB_STEP_SUMMARY"
pnpm exec vitest run --config vitest.integration.config.mts \
--shard=${{ matrix.shard }}/4 \
@@ -150,7 +157,9 @@ jobs:
path: reports
- name: Regenerate
shell: bash
run: |
set -o pipefail
shopt -s nullglob
integration=(reports/nightly-integration-report-*/*.json)
unit=(reports/nightly-unit-report/*.json)