fix(ci): make the GeoLite2 build step tolerate missing keys and MaxMind-side download failures

Build on the previous fetch-script softening (674c6b5b) by hardening
the download path itself, so a transient MaxMind-side issue never
takes the release pipeline down. The runtime gate stays the only
place that surfaces `offline geo unavailable` to the maintainer.

`fetch_edition` now uses `curl --fail-with-body` and captures the
exit code instead of letting `set -e` abort the script. Any non-zero
result — 401 EULA-pending, 403 throttled, 5xx server hiccup, 404
edition-removed, network timeout — flips into the same fallback shape
as the missing-key path: write the `.empty` marker, log a clear
reason on stderr, and return 0 so the second edition still gets its
chance. Tarball-extract and MMDB-locate failures route through the
same fallback. After both fetches, the marker state is summarised:
marker present means at least one edition fell back and the runtime
resolver will use ipwho.is; marker absent means both MMDBs landed
cleanly. The pre-fetch `rm -f "$OUT_DIR/.empty"` from the previous
commit still seeds the clean state; the per-edition fallback re-adds
the marker on demand.

`set -euo pipefail` stays, but every recoverable path either captures
the exit code (`|| curl_exit=$?`) or guards with `if !`, so the
strict-mode abort never reaches a fallback branch.

This unblocks PR #169, where the MAXMIND_LICENSE_KEY secret is set
but MaxMind's account-side GeoLite2 EULA acceptance has not
propagated yet and the download endpoint returns 401. The build no
longer crashes on `curl: (22) The requested URL returned error: 401`.
This commit is contained in:
Marc-André Bombeck committed 2026-05-15 22:12:22 +02:00
1 parent b2361e9ddc
commit 76783431e3
1 file changed
+38 -6
+38 -6
View File
@@ -52,20 +52,44 @@ fetch_edition() {
tmp_tarball="$(mktemp -t "${edition_id}.tar.gz.XXXXXX")"
echo "fetch-geolite2: downloading $edition_id ..." >&2
curl --silent --show-error --fail --location \
# Use --fail-with-body so curl prints the response on failure; capture
# the exit code without aborting the script. Any non-zero exit (401
# EULA-pending, 403 throttled, 5xx, network timeout, etc.) flips the
# build into the same fallback shape as the missing-key path: write
# the `.empty` marker and let the runtime gate surface the issue to
# the maintainer.
local curl_exit=0
curl --silent --show-error --fail-with-body --location \
--output "$tmp_tarball" \
"https://download.maxmind.com/app/geoip_download?edition_id=${edition_id}&license_key=${LICENSE_KEY}&suffix=tar.gz"
"https://download.maxmind.com/app/geoip_download?edition_id=${edition_id}&license_key=${LICENSE_KEY}&suffix=tar.gz" || curl_exit=$?
if [[ "$curl_exit" -ne 0 ]]; then
echo "fetch-geolite2: $edition_id download failed (curl exit $curl_exit) — falling back to runtime ipwho.is." >&2
rm -f "$tmp_tarball"
# Reinstate the marker so partial state from a previous fetch does
# not look like a healthy populated directory.
touch "$OUT_DIR/.empty"
return 0
fi
# The tarball ships under a date-stamped top-level directory
# (`GeoLite2-City_YYYYMMDD/`). Extract the MMDB into a flat layout
# so the Dockerfile COPY uses a stable path.
tar -xzf "$tmp_tarball" -C "$(dirname "$tmp_tarball")"
if ! tar -xzf "$tmp_tarball" -C "$(dirname "$tmp_tarball")"; then
echo "fetch-geolite2: $edition_id tarball extraction failed — falling back to runtime ipwho.is." >&2
rm -f "$tmp_tarball"
touch "$OUT_DIR/.empty"
return 0
fi
local extracted
extracted="$(find "$(dirname "$tmp_tarball")" -maxdepth 2 -name "${mmdb_basename}" -print -quit)"
if [[ -z "$extracted" ]]; then
echo "fetch-geolite2: expected ${mmdb_basename} inside the ${edition_id} tarball" >&2
echo "fetch-geolite2: expected ${mmdb_basename} inside the ${edition_id} tarball — falling back to runtime ipwho.is." >&2
rm -f "$tmp_tarball"
exit 1
find "$(dirname "$tmp_tarball")" -maxdepth 1 -type d -name "${edition_id}_*" \
-exec rm -rf {} + 2>/dev/null || true
touch "$OUT_DIR/.empty"
return 0
fi
mv "$extracted" "$OUT_DIR/$mmdb_basename"
rm -f "$tmp_tarball"
@@ -78,7 +102,15 @@ fetch_edition() {
echo "fetch-geolite2: $mmdb_basename SHA256 $sha" >&2
}
# Track whether every edition landed cleanly. If any fetch fell back to
# the runtime path, leave the `.empty` marker so the runtime gate +
# admin notification fires; otherwise clear it so a healthy populated
# directory is not misread as the fallback state.
fetch_edition "GeoLite2-City" "GeoLite2-City.mmdb"
fetch_edition "GeoLite2-ASN" "GeoLite2-ASN.mmdb"
echo "fetch-geolite2: done. Files in $OUT_DIR." >&2
if [[ -f "$OUT_DIR/.empty" ]]; then
echo "fetch-geolite2: at least one edition fell back; runtime resolver will use ipwho.is." >&2
else
echo "fetch-geolite2: done. Files in $OUT_DIR." >&2
fi