fix(docker): a boot storm cannot flip the healthcheck to unhealthy

/api/health gives the database two seconds and answers 503 past that.
With three retries, a pool saturated for a minute after the start period
could mark the container unhealthy and take it out of the proxy's
routing. The compose healthcheck and the image's HEALTHCHECK now allow
five misses (two and a half minutes) and end wget after four seconds,
inside Docker's five, so a stalled request is one clean miss. A test
holds both to these bounds and to each other.
This commit is contained in:
Marc-André Bombeck committed 2026-10-21 22:03:48 +02:00
1 parent 59c3c38379
commit d8fcf814b2
4 files changed
+129 -5

No files matched your search

+10 -2
View File
@@ -512,11 +512,19 @@ services:
test:
[
"CMD-SHELL",
"wget --no-verbose --tries=1 --spider http://127.0.0.1:3000/api/health || exit 1",
"wget --no-verbose --tries=1 --timeout=4 --spider http://127.0.0.1:3000/api/health || exit 1",
]
interval: 30s
timeout: 5s
retries: 3
# Five misses in a row, two and a half minutes, before "unhealthy". The
# probe gives the database two seconds to answer SELECT 1 and says 503
# past that, so a boot storm that saturates the pool for a minute after
# the start period (backfills, a restore, another tenant's import) costs
# a probe or two, not the container's health and the proxy's routing.
# wget's own 4 s timeout ends the request inside Docker's 5 s, so a
# stalled connection reads as one clean miss rather than a killed probe.
# A database that stays away for 2.5 minutes is a real outage and shows.
retries: 5
# The entrypoint applies migrations before the server listens. A
# migration that waits out a held lock retries for up to about eight
# minutes (docs/ops/deploy.md), and failed probes in that time must not