feat(geo): bundle GeoLite2-City and add an offline-first lookup path

Rewrites src/lib/geo.ts so lookupIpLocation reads from the bundled
MaxMind GeoLite2-City MMDB first and falls back to the existing
ipwho.is path only for IPs the offline DB cannot resolve. Adds a
sibling lookupIpAsn that returns { asn, carrier } from the
GeoLite2-ASN MMDB so the admin login-overview can grow a carrier
chip alongside the auth-provider chip.

The MMDB files are too large to vendor in git. scripts/fetch-geolite2.sh
downloads them at build time using a MaxMind license key; the
Dockerfile copies assets/geolite2/ into /opt/geolite2/. When the
license key is absent or the files are missing the image still
builds and the resolver falls back to ipwho.is (v1.4.26 behaviour).

GeoLite2 is CC BY-SA 4.0 — attribution lives on the new public
/about page and in docs/audit/v1427-summary.md.
This commit is contained in:
Marc-André Bombeck committed 2026-05-15 16:36:55 +02:00
1 parent 2c9d17c333
commit 50150e35ef
11 files changed
+834 -18

No files matched your search

+6
View File
@@ -46,6 +46,12 @@ next-env.d.ts
# prisma generated
/src/generated/prisma
# v1.4.27 B3 — MaxMind GeoLite2 databases are fetched at build time
# via `scripts/fetch-geolite2.sh`. Too large for git; license key
# required. The folder + a `.gitkeep` are tracked so the Dockerfile
# COPY target always exists.
/assets/geolite2/*.mmdb
# AI session artefacts (kept locally, not committed)
/docs/superpowers/
.claude/
+15
View File
@@ -63,6 +63,21 @@ RUN mkdir -p /opt/prisma-cli && \
npm init -y && \
npm install --omit=dev prisma@7.8 @prisma/engines@7.8
# v1.4.27 B3 — offline GeoLite2 databases for IP→location and IP→ASN
# lookups. The MMDB files live in `/opt/geolite2/` and are read by
# `src/lib/geo.ts` via `mmdb-lib`. They are downloaded outside the
# Docker build by `scripts/fetch-geolite2.sh` (operator runs it before
# `docker build` with a MaxMind license key) and staged in
# `assets/geolite2/`. The README + .gitkeep are always present so the
# COPY target exists; if the maintainer skipped the fetch step the
# image builds without the DBs and the resolver falls back to the
# online `ipwho.is` provider — matches v1.4.26 behaviour.
#
# Attribution (CC BY-SA 4.0): see `docs/audit/v1427-summary.md` and
# `/about` in the running app.
RUN mkdir -p /opt/geolite2
COPY assets/geolite2/ /opt/geolite2/
# Entrypoint script (runs migrations, then starts app)
COPY docker-entrypoint.sh ./
RUN chmod +x docker-entrypoint.sh
+20
View File
@@ -0,0 +1,20 @@
# GeoLite2 databases
`GeoLite2-City.mmdb` and `GeoLite2-ASN.mmdb` are downloaded at build
time by `scripts/fetch-geolite2.sh` and copied into the Docker image
at `/opt/geolite2/`. The runtime resolver in `src/lib/geo.ts` reads
them via `mmdb-lib`.
Run before `docker build`:
```
MAXMIND_LICENSE_KEY=xxxx ./scripts/fetch-geolite2.sh
```
Without the key the script exits cleanly and the image builds without
the databases — the resolver falls back to the `ipwho.is` online
provider (the v1.4.26 behaviour).
This product includes GeoLite2 data created by MaxMind, available from
<https://www.maxmind.com>. The databases are licensed under
[CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/).
+72
View File
@@ -0,0 +1,72 @@
# v1.4.27 release summary
## Overview
v1.4.27 polishes the settings shell rhythm and lands the offline-first
geolocation pipeline that drives the `/admin/login-overview` table.
Two MaxMind GeoLite2 databases (`GeoLite2-City` + `GeoLite2-ASN`) are
now bundled into the runtime image; the resolver in
`src/lib/geo.ts` reads them with `mmdb-lib` and falls back to the
existing `ipwho.is` HTTPS path only when the local DB cannot resolve
the address.
## GeoLite2 attribution
This product includes GeoLite2 data created by MaxMind, available from
<https://www.maxmind.com>. The GeoLite2 databases (`GeoLite2-City`
and `GeoLite2-ASN`) are distributed under the
[Creative Commons Attribution-ShareAlike 4.0 International License][cc-by-sa-4].
The same attribution renders on the `/about` page so an end user can
discover it without inspecting the repository.
[cc-by-sa-4]: https://creativecommons.org/licenses/by-sa/4.0/
## Operating the offline databases
The MMDB files are too large to vendor in git. They are downloaded
outside the Docker build by `scripts/fetch-geolite2.sh`. The operator
runs the script before `docker build` with a free MaxMind license
key:
```
MAXMIND_LICENSE_KEY=xxxx ./scripts/fetch-geolite2.sh
```
The script writes `GeoLite2-City.mmdb` and `GeoLite2-ASN.mmdb` to
`assets/geolite2/`. The Dockerfile then copies the folder into
`/opt/geolite2/` so the runtime resolver finds the databases on disk.
If the script is skipped the image builds cleanly, and the resolver
falls back to the online `ipwho.is` provider — the v1.4.26 behaviour.
MaxMind reissues the databases on the first Tuesday of each month;
re-run the script before each release to pick up new ASN allocations
and city renames.
## Schema delta
`AuditLog` grows two nullable columns:
- `asn Int?` — the autonomous-system number resolved from the
`GeoLite2-ASN` DB at audit-creation time.
- `carrier String?` — the matching `autonomous_system_organization`
string (e.g. `"Deutsche Telekom AG"`).
Both columns are additive and `IF NOT EXISTS`-guarded so the
migration is idempotent on the demo server which may already carry
hand-edited columns from a prior partial run.
## Backfill helper
`src/lib/jobs/geo-backfill.ts` walks rows where `location IS NULL`
and `createdAt > now() - 30 days`, capped at 5 000 rows per pass, and
re-resolves them through the offline DB. Useful after a fresh
GeoLite2 DB roll so the historical rows that landed during an
`ipwho.is` outage finally pick up a city + carrier label.
## Image-size budget
The two MMDB files add ~80 MB to the production image. The
`tzdata`-bearing Alpine runner stage already sits at ~330 MB
compressed; the v1.4.27 image lands well under the 90 MB image-size
delta budget set out in the fix plan.
+1
View File
@@ -60,6 +60,7 @@
"jspdf": "^4.2.1",
"jspdf-autotable": "^5.0.7",
"lucide-react": "^1.14.0",
"mmdb-lib": "^3.0.2",
"next": "16.2.6",
"next-themes": "^0.4.6",
"pg-boss": "^12.18.2",
+9
View File
@@ -53,6 +53,9 @@ importers:
lucide-react:
specifier: ^1.14.0
version: 1.14.0(react@19.2.5)
mmdb-lib:
specifier: ^3.0.2
version: 3.0.2
next:
specifier: 16.2.6
version: 16.2.6(@babel/core@7.29.0)(@playwright/test@1.59.1)(react-dom@19.2.5(react@19.2.5))(react@19.2.5)
@@ -4777,6 +4780,10 @@ packages:
engines: {node: '>=10'}
hasBin: true
mmdb-lib@3.0.2:
resolution: {integrity: sha512-7e87vk0DdWT647wjcfEtWeMtjm+zVGqNohN/aeIymbUfjHQ2T4Sx5kM+1irVDBSloNC3CkGKxswdMoo8yhqTDg==}
engines: {node: '>=10', npm: '>=6'}
ms@2.1.3:
resolution: {integrity: sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==}
@@ -11320,6 +11327,8 @@ snapshots:
mkdirp@3.0.1: {}
mmdb-lib@3.0.2: {}
ms@2.1.3: {}
msw@2.14.5(@types/node@25.6.2)(typescript@6.0.3):
+76
View File
@@ -0,0 +1,76 @@
#!/usr/bin/env bash
# v1.4.27 B3 — GeoLite2-City + GeoLite2-ASN download helper.
#
# The MMDB files are too large (~80 MB combined) to vendor in git.
# Instead, an operator runs this script before `docker build` so the
# files land at `assets/geolite2/` where the Dockerfile picks them up
# and copies them into `/opt/geolite2/` inside the image.
#
# The MaxMind GeoLite2 databases require a free MaxMind account and
# a licence key. The key is taken from the `MAXMIND_LICENSE_KEY`
# environment variable. Without the key, the script exits 0 and the
# Docker image builds without the offline databases — the runtime
# resolver in `src/lib/geo.ts` falls back to the online `ipwho.is`
# provider, which is the v1.4.26 behaviour.
#
# Licence: the databases are distributed under
# Creative Commons Attribution-ShareAlike 4.0
# (https://creativecommons.org/licenses/by-sa/4.0/). The attribution
# lives in `docs/audit/v1427-summary.md` and on the `/about` page.
#
# Usage:
# MAXMIND_LICENSE_KEY=xxxx ./scripts/fetch-geolite2.sh
#
# Refresh schedule: re-run before each release. The MMDB layout is
# stable; MaxMind reissues the databases on the first Tuesday of each
# month.
set -euo pipefail
OUT_DIR="${GEOLITE2_OUT_DIR:-assets/geolite2}"
LICENSE_KEY="${MAXMIND_LICENSE_KEY:-}"
if [[ -z "$LICENSE_KEY" ]]; then
echo "fetch-geolite2: MAXMIND_LICENSE_KEY is not set — skipping download." >&2
echo "fetch-geolite2: the runtime resolver will fall back to ipwho.is." >&2
exit 0
fi
mkdir -p "$OUT_DIR"
fetch_edition() {
local edition_id="$1"
local mmdb_basename="$2"
local tmp_tarball
tmp_tarball="$(mktemp -t "${edition_id}.tar.gz.XXXXXX")"
echo "fetch-geolite2: downloading $edition_id ..." >&2
curl --silent --show-error --fail --location \
--output "$tmp_tarball" \
"https://download.maxmind.com/app/geoip_download?edition_id=${edition_id}&license_key=${LICENSE_KEY}&suffix=tar.gz"
# The tarball ships under a date-stamped top-level directory
# (`GeoLite2-City_YYYYMMDD/`). Extract the MMDB into a flat layout
# so the Dockerfile COPY uses a stable path.
tar -xzf "$tmp_tarball" -C "$(dirname "$tmp_tarball")"
local extracted
extracted="$(find "$(dirname "$tmp_tarball")" -maxdepth 2 -name "${mmdb_basename}" -print -quit)"
if [[ -z "$extracted" ]]; then
echo "fetch-geolite2: expected ${mmdb_basename} inside the ${edition_id} tarball" >&2
rm -f "$tmp_tarball"
exit 1
fi
mv "$extracted" "$OUT_DIR/$mmdb_basename"
rm -f "$tmp_tarball"
# Clean up the date-stamped extraction directory.
find "$(dirname "$tmp_tarball")" -maxdepth 1 -type d -name "${edition_id}_*" \
-exec rm -rf {} +
local sha
sha="$(shasum -a 256 "$OUT_DIR/$mmdb_basename" | awk '{print $1}')"
echo "fetch-geolite2: $mmdb_basename SHA256 $sha" >&2
}
fetch_edition "GeoLite2-City" "GeoLite2-City.mmdb"
fetch_edition "GeoLite2-ASN" "GeoLite2-ASN.mmdb"
echo "fetch-geolite2: done. Files in $OUT_DIR." >&2
+181
View File
@@ -0,0 +1,181 @@
import type { Metadata } from "next";
import Link from "next/link";
/**
* v1.4.27 B3 — Public about / credits page.
*
* Two reasons this page exists:
*
* 1. The runtime image bundles MaxMind GeoLite2-City and
* GeoLite2-ASN databases for offline IP-to-location and
* IP-to-ASN lookups. The CC BY-SA 4.0 licence requires the
* attribution to be reachable from the running application,
* not just the source repository.
* 2. A general home for open-source credits as the project
* accumulates more third-party data sources (already true for
* the upcoming ICD-10 reference table in the iOS Health import
* flow).
*
* The page mirrors `/privacy` in layout and is reachable without a
* session — see `src/proxy.ts` PUBLIC_PATHS.
*/
export const dynamic = "force-static";
export const revalidate = false;
const LAST_UPDATED = "2026-05-15";
export const metadata: Metadata = {
title: "About — HealthLog",
description:
"Open-source credits and third-party data attributions for the HealthLog project.",
robots: { index: true, follow: true },
};
function Section({
id,
title,
children,
}: {
id: string;
title: string;
children: React.ReactNode;
}) {
return (
<section id={id} className="space-y-3 scroll-mt-20">
<h2 className="text-xl font-semibold tracking-tight md:text-2xl">
{title}
</h2>
<div className="text-muted-foreground space-y-3 text-sm leading-relaxed md:text-base">
{children}
</div>
</section>
);
}
export default function AboutPage() {
return (
<div className="bg-background text-foreground min-h-dvh">
<header className="border-border/60 bg-background/80 sticky top-0 z-10 border-b backdrop-blur">
<div className="mx-auto flex max-w-3xl items-center justify-between px-4 py-3 md:px-6">
<Link
href="/"
className="text-foreground hover:text-primary text-sm font-semibold tracking-tight"
>
HealthLog
</Link>
<Link
href="/auth/login"
className="text-muted-foreground hover:text-foreground text-sm"
>
Sign in
</Link>
</div>
</header>
<main
id="main-content"
className="mx-auto max-w-3xl space-y-10 px-4 py-8 md:px-6 md:py-12"
>
<div className="space-y-3">
<p className="text-muted-foreground text-xs tracking-wider uppercase">
About
</p>
<h1 className="text-3xl font-bold tracking-tight md:text-4xl">
About HealthLog
</h1>
<p className="text-muted-foreground text-sm">
Last updated: {LAST_UPDATED}
</p>
</div>
<Section id="project" title="Project">
<p>
HealthLog is an open-source, self-hostable personal-health-tracking
application. The source code lives at{" "}
<a
href="https://github.com/MBombeck/HealthLog"
target="_blank"
rel="noopener noreferrer"
className="text-primary hover:underline"
>
github.com/MBombeck/HealthLog
</a>{" "}
and is licensed under the GNU Affero General Public License v3.0.
</p>
</Section>
<Section id="credits" title="Credits">
<p>
HealthLog stands on a number of open-source libraries and public
data sources. The list below covers the third-party assets that
ship with the runtime image and whose licences require an explicit
attribution.
</p>
<h3 className="text-foreground text-base font-semibold md:text-lg">
MaxMind GeoLite2
</h3>
<p>
This product includes GeoLite2 data created by MaxMind, available
from{" "}
<a
href="https://www.maxmind.com"
target="_blank"
rel="noopener noreferrer"
className="text-primary hover:underline"
>
www.maxmind.com
</a>
. The bundled databases (
<code className="bg-muted rounded px-1 py-0.5 text-xs">
GeoLite2-City
</code>{" "}
and{" "}
<code className="bg-muted rounded px-1 py-0.5 text-xs">
GeoLite2-ASN
</code>
) power the offline IP-to-location and IP-to-carrier lookups that
decorate the admin login-overview audit table. They are
distributed under the{" "}
<a
href="https://creativecommons.org/licenses/by-sa/4.0/"
target="_blank"
rel="noopener noreferrer"
className="text-primary hover:underline"
>
Creative Commons Attribution-ShareAlike 4.0 International
License
</a>
.
</p>
</Section>
<footer
className="border-border/60 mt-12 border-t pt-6 text-xs text-muted-foreground"
data-slot="about-footer"
>
<p>
HealthLog — open source under{" "}
<a
href="https://github.com/MBombeck/HealthLog/blob/main/LICENSE"
target="_blank"
rel="noopener noreferrer"
className="hover:text-foreground hover:underline"
>
AGPL-3.0
</a>
. See{" "}
<Link
href="/privacy"
className="hover:text-foreground hover:underline"
>
privacy policy
</Link>
.
</p>
</footer>
</main>
</div>
);
}
+267
View File
@@ -0,0 +1,267 @@
import { describe, it, expect, beforeEach, afterEach, vi } from "vitest";
/**
* v1.4.27 B3 — ASN + carrier lookup tests.
*
* The MaxMind GeoLite2-ASN MMDB is too large for the test fixtures
* directory (~10 MB) and shipping it would bloat git history. These
* tests mock `mmdb-lib` directly so the assertion focuses on the
* resolver contract:
*
* - Private / loopback IPs short-circuit to `null` without touching
* the reader.
* - A missing MMDB file (no offline DB on the host) yields `null`.
* - A row from the DB with both `autonomous_system_number` and
* `autonomous_system_organization` is folded into `{ asn, carrier }`.
* - Reader throws are swallowed (the helper is called from a
* fire-and-forget audit-log path; a thrown error would surface as
* an unhandled rejection).
* - The offline-first `lookupIpLocation` path picks the German city
* name first, English second, with the country ISO code from
* `country` or `registered_country`.
*
* The MMDB Reader is mocked once at the top of the file; per-test
* shape control happens through the `setReaderRow` + `setReaderThrows`
* helpers below. The `fs` stub is also installed module-wide so the
* lazy loader believes the DB exists.
*/
interface AsnRow {
autonomous_system_number?: number;
autonomous_system_organization?: string;
}
interface CityRow {
city?: { names?: Record<string, string> };
country?: { iso_code?: string };
registered_country?: { iso_code?: string };
}
// Test-time spy state — captured by the mocked Reader. The Reader
// constructor inspects the buffer header to decide whether it's
// serving City or ASN rows.
const readerState = {
city: null as ((ip: string) => CityRow | null) | null,
asn: null as ((ip: string) => AsnRow | null) | null,
cityThrows: false,
asnThrows: false,
cityExists: false,
asnExists: false,
};
vi.mock("mmdb-lib", () => ({
Reader: class {
private kind: "city" | "asn";
constructor(buf: Buffer) {
const head = buf.toString("utf-8", 0, 4);
this.kind = head === "city" ? "city" : "asn";
}
get(ip: string): AsnRow | CityRow | null {
if (this.kind === "asn") {
if (readerState.asnThrows) throw new Error("mmdb-asn boom");
return readerState.asn ? readerState.asn(ip) : null;
}
if (readerState.cityThrows) throw new Error("mmdb-city boom");
return readerState.city ? readerState.city(ip) : null;
}
},
}));
vi.mock("node:fs", async () => {
const actual = await vi.importActual<typeof import("node:fs")>("node:fs");
const existsSync = (p: string) => {
if (p.endsWith("GeoLite2-City.mmdb")) return readerState.cityExists;
if (p.endsWith("GeoLite2-ASN.mmdb")) return readerState.asnExists;
return false;
};
const readFileSync = (p: string) => {
if (p.endsWith("GeoLite2-City.mmdb"))
return Buffer.from("city-mmdb-stub");
if (p.endsWith("GeoLite2-ASN.mmdb"))
return Buffer.from("asn-mmdb-stub");
throw new Error(`unexpected read ${p}`);
};
return {
...actual,
existsSync,
readFileSync,
default: { ...actual, existsSync, readFileSync },
};
});
const ORIGINAL_ENV = { ...process.env };
beforeEach(async () => {
process.env = { ...ORIGINAL_ENV };
delete process.env.IP_GEO_LOOKUP_DISABLED;
delete process.env.GEOLITE2_DIR;
readerState.city = null;
readerState.asn = null;
readerState.cityThrows = false;
readerState.asnThrows = false;
readerState.cityExists = false;
readerState.asnExists = false;
vi.resetModules();
// Reset the lazy cache so each test gets a fresh Reader load.
const mod = await import("../geo");
mod.__resetGeoLite2CacheForTests();
});
afterEach(() => {
process.env = ORIGINAL_ENV;
vi.restoreAllMocks();
vi.unstubAllGlobals();
});
describe("lookupIpAsn — offline ASN resolver (v1.4.27 B3)", () => {
it("returns null for private + loopback addresses without touching the reader", async () => {
const get = vi.fn();
readerState.asn = get;
readerState.asnExists = true;
const { lookupIpAsn } = await import("../geo");
expect(lookupIpAsn("10.0.0.5")).toBeNull();
expect(lookupIpAsn("127.0.0.1")).toBeNull();
expect(lookupIpAsn("::1")).toBeNull();
expect(lookupIpAsn(null)).toBeNull();
expect(get).not.toHaveBeenCalled();
});
it("returns null when the GeoLite2-ASN MMDB is missing", async () => {
readerState.asnExists = false;
const { lookupIpAsn } = await import("../geo");
expect(lookupIpAsn("8.8.8.8")).toBeNull();
});
it("returns { asn, carrier } when the reader resolves a public IP", async () => {
readerState.asnExists = true;
readerState.asn = () => ({
autonomous_system_number: 3320,
autonomous_system_organization: "Deutsche Telekom AG",
});
const { lookupIpAsn } = await import("../geo");
expect(lookupIpAsn("84.131.0.1")).toEqual({
asn: 3320,
carrier: "Deutsche Telekom AG",
});
});
it("returns null when the reader has no row for the IP", async () => {
readerState.asnExists = true;
readerState.asn = () => null;
const { lookupIpAsn } = await import("../geo");
expect(lookupIpAsn("8.8.8.8")).toBeNull();
});
it("returns the asn with carrier=null when the org field is missing", async () => {
readerState.asnExists = true;
readerState.asn = () => ({ autonomous_system_number: 64500 });
const { lookupIpAsn } = await import("../geo");
expect(lookupIpAsn("8.8.8.8")).toEqual({ asn: 64500, carrier: null });
});
it("swallows reader throws and returns null (caller is fire-and-forget)", async () => {
readerState.asnExists = true;
readerState.asnThrows = true;
const { lookupIpAsn } = await import("../geo");
expect(lookupIpAsn("8.8.8.8")).toBeNull();
});
});
describe("lookupIpLocation — offline-first city resolver (v1.4.27 B3)", () => {
it("prefers the German exonym from the MMDB Names record", async () => {
readerState.cityExists = true;
readerState.city = () => ({
city: { names: { de: "München", en: "Munich" } },
country: { iso_code: "DE" },
});
const fetchSpy = vi.fn();
vi.stubGlobal("fetch", fetchSpy);
const { lookupIpLocation } = await import("../geo");
expect(await lookupIpLocation("85.214.0.1")).toBe("München, DE");
// Offline hit — no online fallback was needed.
expect(fetchSpy).not.toHaveBeenCalled();
});
it("falls back to the English city name when the German exonym is absent", async () => {
readerState.cityExists = true;
readerState.city = () => ({
city: { names: { en: "Birmingham" } },
country: { iso_code: "GB" },
});
const fetchSpy = vi.fn();
vi.stubGlobal("fetch", fetchSpy);
const { lookupIpLocation } = await import("../geo");
expect(await lookupIpLocation("212.58.224.1")).toBe("Birmingham, GB");
});
it("accepts the registered_country ISO when country is absent", async () => {
readerState.cityExists = true;
readerState.city = () => ({
city: { names: { en: "Geneva" } },
registered_country: { iso_code: "CH" },
});
const { lookupIpLocation } = await import("../geo");
expect(await lookupIpLocation("194.158.0.1")).toBe("Geneva, CH");
});
it("falls back to the online provider when the offline DB has no city/country", async () => {
readerState.cityExists = true;
readerState.city = () => ({ city: { names: {} } });
const fetchSpy = vi.fn().mockResolvedValue(
new Response(
new TextEncoder().encode(
JSON.stringify({ success: true, city: "Hamburg", country_code: "DE" }),
).buffer as ArrayBuffer,
{
status: 200,
headers: { "content-type": "application/json; charset=utf-8" },
},
),
);
vi.stubGlobal("fetch", fetchSpy);
const { lookupIpLocation } = await import("../geo");
expect(await lookupIpLocation("8.8.8.8")).toBe("Hamburg, DE");
expect(fetchSpy).toHaveBeenCalledTimes(1);
});
it("falls back to the online provider when the offline DB is missing entirely", async () => {
readerState.cityExists = false;
const fetchSpy = vi.fn().mockResolvedValue(
new Response(
new TextEncoder().encode(
JSON.stringify({ success: true, city: "Berlin", country_code: "DE" }),
).buffer as ArrayBuffer,
{
status: 200,
headers: { "content-type": "application/json; charset=utf-8" },
},
),
);
vi.stubGlobal("fetch", fetchSpy);
const { lookupIpLocation } = await import("../geo");
expect(await lookupIpLocation("8.8.8.8")).toBe("Berlin, DE");
expect(fetchSpy).toHaveBeenCalledTimes(1);
});
it("skips the online fallback when IP_GEO_LOOKUP_DISABLED=1 and offline misses", async () => {
process.env.IP_GEO_LOOKUP_DISABLED = "1";
readerState.cityExists = false;
const fetchSpy = vi.fn();
vi.stubGlobal("fetch", fetchSpy);
const { lookupIpLocation } = await import("../geo");
expect(await lookupIpLocation("8.8.8.8")).toBeNull();
expect(fetchSpy).not.toHaveBeenCalled();
});
});
+183 -18
View File
@@ -1,23 +1,61 @@
/**
* IP geolocation lookup for audit-log enrichment.
*
* Default provider is ipwho.is (HTTPS, free, no key). Both the response
* shape from ipwho.is and the fallback ip-api.com pro endpoint are
* accepted, so swapping providers via IP_GEO_LOOKUP_URL only requires
* matching one of those response shapes.
* v1.4.27 B3: offline-first resolver. The hot path is now a microsecond-
* scale local read against the bundled MaxMind GeoLite2-City MMDB at
* `/opt/geolite2/GeoLite2-City.mmdb` (and the matching GeoLite2-ASN
* database for carrier lookups). The previous `ipwho.is` HTTPS path is
* kept as the second-tier fallback for IPs the offline DB cannot
* resolve (freshly-allocated ranges that lag the monthly GeoLite2
* release roll).
*
* Setting IP_GEO_LOOKUP_DISABLED=1 disables lookup entirely — used by
* deployments that do not want any IP egress to a third-party service
* (V3 audit: GDPR Art. 32 + Art. 44, plaintext HTTP IP egress).
* 1. Private / loopback / opt-out → null (no lookup).
* 2. Offline MMDB hit → return immediately.
* 3. Offline miss → fall back to the existing `ipwho.is` path.
*
* v1.4.16 A8a: the body is decoded as UTF-8 explicitly via
* `TextDecoder('utf-8')` instead of `Response.json()`, and an
* The legacy contract still holds: `lookupIpLocation` returns a
* `"City, CC"` string or `null`, never throws. `lookupIpAsn` is the
* new sibling that resolves the autonomous-system number + the carrier
* organisation string (so the admin login overview can render a
* carrier chip next to the auth provider).
*
* Both helpers are safe to call from any request context. The MMDB
* Reader is loaded lazily on first call and held in a module-level
* cache; the file load is synchronous and not cheap, but it only
* happens once per worker process.
*
* The base path is overridable via `GEOLITE2_DIR` (default
* `/opt/geolite2`). When the DB files are missing the helpers silently
* skip the offline tier — local dev without the MMDBs still works and
* falls straight back to the online provider.
*
* Default provider for the online fallback is ipwho.is (HTTPS, free,
* no key). Both the response shape from ipwho.is and the fallback
* ip-api.com pro endpoint are accepted, so swapping providers via
* `IP_GEO_LOOKUP_URL` only requires matching one of those response
* shapes.
*
* Setting `IP_GEO_LOOKUP_DISABLED=1` disables the online fallback
* entirely — used by deployments that do not want any IP egress to a
* third-party service (V3 audit: GDPR Art. 32 + Art. 44, plaintext
* HTTP IP egress). The offline tier still runs because no egress is
* involved.
*
* v1.4.16 A8a: the online-fallback body is decoded as UTF-8 explicitly
* via `TextDecoder('utf-8')` instead of `Response.json()`, and an
* `Accept-Language: de, en;q=0.5` hint is sent so providers return
* native city names ("Nürnberg") rather than ASCII folds
* ("Nuremberg"). The previous path lost umlauts in production for at
* least one maintainer-flagged login row that rendered as "Nrnberg" in
* /admin/login-overview — see `docs/audit/v1416-summary.md` once cut.
* native city names ("Nürnberg") rather than ASCII folds ("Nuremberg").
* The previous path lost umlauts in production for at least one
* maintainer-flagged login row that rendered as "Nrnberg" in
* /admin/login-overview — see `docs/audit/v1416-summary.md`.
*/
import fs from "node:fs";
import path from "node:path";
import { Reader as MmdbReader } from "mmdb-lib";
import type {
AsnResponse,
CityResponse,
} from "mmdb-lib/lib/reader/response";
interface IpwhoIsResponse {
success?: boolean;
@@ -38,6 +76,98 @@ const PRIVATE_IP =
const DEFAULT_GEO_URL = "https://ipwho.is";
function geoLiteDir(): string {
return process.env.GEOLITE2_DIR ?? "/opt/geolite2";
}
// ── Offline tier (MaxMind GeoLite2) ──────────────────────────────────
//
// Both readers are held in a process-local cache. We use `undefined`
// for "never tried", and `null` for "tried, file missing" so we don't
// pay the `existsSync + readFileSync + parse` cost on every lookup.
interface MmdbCache {
city?: MmdbReader<CityResponse> | null;
asn?: MmdbReader<AsnResponse> | null;
}
const cache: MmdbCache = {};
function loadMmdbReader<T extends import("mmdb-lib/lib/reader/response").Response>(
file: string,
): MmdbReader<T> | null {
try {
const full = path.join(geoLiteDir(), file);
if (!fs.existsSync(full)) return null;
const buf = fs.readFileSync(full);
return new MmdbReader<T>(buf);
} catch {
// Corrupt MMDB or unreadable file → fall back to online tier.
// Never throw from the geo lookup path; the auth-audit caller is
// fire-and-forget and a thrown error would propagate into an
// unhandled rejection.
return null;
}
}
function getCityReader(): MmdbReader<CityResponse> | null {
if (cache.city === undefined) {
cache.city = loadMmdbReader<CityResponse>("GeoLite2-City.mmdb");
}
return cache.city;
}
function getAsnReader(): MmdbReader<AsnResponse> | null {
if (cache.asn === undefined) {
cache.asn = loadMmdbReader<AsnResponse>("GeoLite2-ASN.mmdb");
}
return cache.asn;
}
/**
* Test-only — reset the lazy reader cache so a test can swap the
* `GEOLITE2_DIR` between cases without leaking the previous Reader.
* Not exported from the public surface; the tests import via the
* module-level alias.
*/
export function __resetGeoLite2CacheForTests(): void {
cache.city = undefined;
cache.asn = undefined;
}
/**
* Pick the best-available localised city name from a GeoLite2
* record. German first because the user-base is DACH-skewed; English
* as a graceful fallback for cities that don't have a German exonym.
* The Names map is incomplete in MaxMind for some small German
* towns, so we accept any of the locales the MMDB layout exposes.
*/
function pickCityName(city: CityResponse["city"]): string | null {
if (!city?.names) return null;
return (
city.names.de ??
city.names.en ??
city.names.fr ??
city.names.es ??
null
);
}
function lookupIpLocationOffline(ip: string): string | null {
const reader = getCityReader();
if (!reader) return null;
try {
const row = reader.get(ip);
if (!row) return null;
const city = pickCityName(row.city);
const cc = row.country?.iso_code ?? row.registered_country?.iso_code;
if (!city || !cc) return null;
return `${city}, ${cc}`;
} catch {
return null;
}
}
function buildLookupUrl(ip: string): string {
const base = (process.env.IP_GEO_LOOKUP_URL ?? DEFAULT_GEO_URL).replace(
/\/+$/,
@@ -70,12 +200,8 @@ async function readUtf8Json(res: Response): Promise<unknown> {
return JSON.parse(text);
}
export async function lookupIpLocation(
ip: string | null,
): Promise<string | null> {
if (!ip || PRIVATE_IP.test(ip)) return null;
async function lookupIpLocationOnline(ip: string): Promise<string | null> {
if (process.env.IP_GEO_LOOKUP_DISABLED === "1") return null;
try {
const res = await fetch(buildLookupUrl(ip), {
signal: AbortSignal.timeout(3000),
@@ -109,3 +235,42 @@ export async function lookupIpLocation(
return null;
}
}
export async function lookupIpLocation(
ip: string | null,
): Promise<string | null> {
if (!ip || PRIVATE_IP.test(ip)) return null;
const offline = lookupIpLocationOffline(ip);
if (offline) return offline;
return lookupIpLocationOnline(ip);
}
/**
* Resolve the autonomous-system number + carrier organisation for an
* IP. Offline-only against the bundled GeoLite2-ASN MMDB — the public
* `ipwho.is` endpoint does not expose an ASN field in its free tier,
* so a miss returns `null` rather than falling back to an online
* lookup.
*
* Private / loopback IPs return `null` without touching the reader.
* Same gate the location helper applies.
*/
export function lookupIpAsn(
ip: string | null,
): { asn: number; carrier: string | null } | null {
if (!ip || PRIVATE_IP.test(ip)) return null;
const reader = getAsnReader();
if (!reader) return null;
try {
const row = reader.get(ip);
if (!row) return null;
const asn = row.autonomous_system_number;
if (typeof asn !== "number") return null;
const carrier = row.autonomous_system_organization || null;
return { asn, carrier };
} catch {
return null;
}
}
+4
View File
@@ -29,6 +29,10 @@ const PUBLIC_PATHS = [
// requires a publicly reachable Privacy-Policy URL during submission;
// GDPR Art. 13 expects the same for any visitor before they sign up.
"/privacy",
// v1.4.27 B3 — `/about` carries the GeoLite2 CC BY-SA 4.0 attribution
// alongside the project credits. The CC licence requires the
// attribution to be reachable without a sign-in.
"/about",
// `/onboarding` itself + its subroutes are matched exactly via
// `isPublicPath()` so we don't admit `/onboarding-export` etc.
"/robots.txt",