ci: restore the drill from the bucket after the host is lost

The drill now runs an S3-compatible bucket beside the stack (Versity S3
Gateway v1.8.0, pinned by digest; the MinIO images are no longer published
on Docker Hub or quay.io) on a network and a volume of its own, so it
outlives the stack as a real bucket does. The app takes the dump through the
scheduled job and uploads it there; the drill waits for a run that both
kept and uploaded it, destroys every volume of the stack, the local backups
included, and restores on a new stack with only the bucket and the keys:
`docker compose run --rm db-restore latest --from-bucket`. The independent
row count reads the manifest from the bucket with the restore's own reader.

DRILL_SOURCE=local keeps the earlier path (dump files copied off the host,
then `db-restore latest`), and CI runs both. A new mutation,
wrong-backup-key, restores with a different BACKUP_ENCRYPTION_KEY and must
be refused. Ports are free ones picked at start.
This commit is contained in:
Marc-André Bombeck committed 2026-10-25 13:43:23 +01:00
1 parent 30a2ba2102
commit 4ffa3352bd
2 files changed
+128 -46

No files matched your search

+15 -6
View File
@@ -2,11 +2,14 @@ name: Disaster recovery drill
# Rebuilds a lost server from the nightly database dump and the keys alone,
# against real containers: the app image built from this checkout, the
# bundled compose file, the demo seed plus a Guardian with a managed profile,
# the dump taken by the scheduled job, every volume destroyed, a new stack,
# `docker compose run --rm db-restore latest`, and the app booted on the
# result (health, login, an encrypted note, a document, the Guardian's
# profile, every table's row count against the manifest).
# bundled compose file with an S3-compatible bucket outside it, the demo seed
# plus a Guardian with a managed profile, the dump taken and uploaded by the
# scheduled job, every volume of the stack destroyed, a new stack, the
# restore, and the app booted on the result (health, login, an encrypted
# note, a document, the Guardian's profile, every table's row count against
# the manifest). Two sources: from the bucket (`db-restore latest
# --from-bucket`, the disaster path) and from dump files copied off the host
# (`db-restore latest`).
# `scripts/compose-dr-drill.sh` is the same drill, runnable locally; its
# header lists what it checks and the mutations that must turn it red.
@@ -40,9 +43,13 @@ concurrency:
jobs:
drill:
name: Restore a lost server from its dump
name: Restore a lost server from its dump (${{ matrix.source }})
runs-on: ubuntu-24.04
timeout-minutes: 60
strategy:
fail-fast: false
matrix:
source: [bucket, local]
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
@@ -58,4 +65,6 @@ jobs:
- run: pnpm db:generate
- name: Drill
env:
DRILL_SOURCE: ${{ matrix.source }}
run: bash scripts/compose-dr-drill.sh