mirror of
https://github.com/MBombeck/HealthLog.git
synced 2026-11-09 04:50:59 +00:00
Merge pull request #1121 from MBombeck/fix/key-check-hardening
Make the encryption key check unable to lock out a correct key
This commit is contained in:
34 files changed
+1745
-221
No files matched your search
@@ -67,6 +67,13 @@ NODE_ENV="development"
|
||||
# your encryption key" step shows first; unset means compose.
|
||||
# HEALTHLOG_PLATFORM="compose"
|
||||
|
||||
# Start-up encryption key check: enforce (default) refuses to serve when the
|
||||
# key does not open the stored data; warn only logs it and serves anyway. A
|
||||
# last resort for a refusal you are certain is false (a first start, with no
|
||||
# recorded key to reset). It disables a safety check: with a wrong key every
|
||||
# encrypted value fails and new rows are written under that key.
|
||||
# ENCRYPTION_KEY_CHECK="enforce"
|
||||
|
||||
|
||||
# -----------------------------------------------------------------------------
|
||||
# Required Secrets — generate with `openssl rand -hex 32`
|
||||
|
||||
@@ -266,6 +266,14 @@ API_TOKEN_HMAC_KEY="<openssl rand -hex 32>"
|
||||
# it for you.
|
||||
# HEALTHLOG_PLATFORM="compose"
|
||||
|
||||
# Start-up encryption key check: enforce (default) refuses to serve when the
|
||||
# key does not open the stored data; warn only logs it, reports it as a
|
||||
# warning on /api/health, and serves anyway. A last resort for a refusal you
|
||||
# are certain is false (a first start, with no recorded key to reset). It
|
||||
# disables a safety check: with a wrong key every encrypted value fails and
|
||||
# new rows are written under that key. Remove it once the key is confirmed.
|
||||
# ENCRYPTION_KEY_CHECK="enforce"
|
||||
|
||||
|
||||
# -----------------------------------------------------------------------------
|
||||
# Deploy webhook -- Coolify -> HealthLog auto-deploy feedback
|
||||
|
||||
@@ -135,6 +135,11 @@ x-healthlog-env: &healthlog_env
|
||||
# compose | truenas | coolify | unraid | portainer. Only chooses which
|
||||
# instructions the admin's "Back up your encryption key" step shows first.
|
||||
HEALTHLOG_PLATFORM: "${HEALTHLOG_PLATFORM:-}"
|
||||
# enforce (default) | warn. The start-up encryption key check refuses to
|
||||
# serve when the key does not open the stored data; warn only logs it and
|
||||
# serves anyway. A last resort for a refusal you are certain is false: it
|
||||
# disables a safety check. See docs/self-hosting/getting-started.md.
|
||||
ENCRYPTION_KEY_CHECK: "${ENCRYPTION_KEY_CHECK:-}"
|
||||
BACKUP_ENCRYPTION_KEY: "${BACKUP_ENCRYPTION_KEY:-}"
|
||||
BACKUP_ENCRYPTION_PREVIOUS_KEYS: "${BACKUP_ENCRYPTION_PREVIOUS_KEYS:-}"
|
||||
BACKUP_S3_ENDPOINT: "${BACKUP_S3_ENDPOINT:-}"
|
||||
|
||||
@@ -103,6 +103,13 @@ under `v1` (the synthetic id assigned to the existing `ENCRYPTION_KEY`).
|
||||
a full `Columns walked` line. Both read the same registry the rotation
|
||||
walks.
|
||||
|
||||
A run that walked every column without an error also removes the boot
|
||||
key check's record of each previous key id that no longer holds a single
|
||||
value (`encryption_key_canaries`); the script prints which ids it removed
|
||||
and, for an id that still holds values, the columns that do. With the record
|
||||
gone, the id can later carry a different key without the start-up check
|
||||
refusing it.
|
||||
|
||||
A zero only means "nothing left" when the run also says it walked every
|
||||
registered column. A zero from a run that skipped columns means "never
|
||||
looked", and dropping the old key on it makes those rows permanently
|
||||
|
||||
@@ -267,6 +267,43 @@ every API request answers `503` with `meta.errorCode`
|
||||
Restore the original key, point `DATABASE_URL` at the matching database, or
|
||||
start with an empty database; then restart.
|
||||
|
||||
The check remembers the key it first saw for each key id in the table
|
||||
`encryption_key_canaries`. Before it records a key, it reads the oldest
|
||||
encrypted values in the database and records the key only if they open, or
|
||||
if the database holds no encrypted value under that key id at all. When the
|
||||
stored data proves neither (a single unreadable value, the oldest values
|
||||
failing while newer ones open, or a probe that ran out of time), the app
|
||||
keeps serving, records nothing, logs a warning that starts with
|
||||
`[boot] Encryption key check inconclusive`, and checks again at the next
|
||||
start.
|
||||
|
||||
If you are certain the configured key is the one your data was written
|
||||
with, and the check still refuses because it recorded a different key
|
||||
earlier, reset the check for that key id and restart. The refusal in the log
|
||||
prints the exact statement; for the default key id it is:
|
||||
|
||||
```bash
|
||||
docker compose exec -T db psql -U healthlog healthlog \
|
||||
-c "DELETE FROM encryption_key_canaries WHERE key_id = 'v1';"
|
||||
docker compose restart app
|
||||
```
|
||||
|
||||
The next start probes the stored data again and records the key only if
|
||||
that data opens with it. Do this only when you are sure: if the key is in
|
||||
fact wrong, the app serves errors on every encrypted value instead of the
|
||||
one clear refusal. After a key rotation the rotation script removes the
|
||||
record of a retired key id itself, once it reports that no values remain
|
||||
under that id.
|
||||
|
||||
At a first start there is no record yet, so there is nothing to reset. If
|
||||
the check refuses there and you are certain the key is right, the last
|
||||
resort is `ENCRYPTION_KEY_CHECK=warn` (default `enforce`): the check still
|
||||
runs and logs its full result, `/api/health` names it as
|
||||
`warning: "encryption_key_mismatch"` without failing, requests are served,
|
||||
and no key is recorded while the check does not pass. This disables a
|
||||
safety check. With a wrong key every encrypted value fails and new rows are
|
||||
written under that key, so remove the setting once the key is confirmed.
|
||||
|
||||
## Backup and restore
|
||||
|
||||
Your database is the only stateful piece — back it up, and back up the
|
||||
|
||||
@@ -36,6 +36,12 @@
|
||||
"description": "Opens the admin's encryption key backup step on the Portainer instructions.",
|
||||
"default": "portainer"
|
||||
},
|
||||
{
|
||||
"name": "ENCRYPTION_KEY_CHECK",
|
||||
"label": "Encryption key check",
|
||||
"description": "enforce refuses to serve when the key does not open the stored data. warn only logs it: a last resort for a refusal you are certain is false; it disables a safety check.",
|
||||
"default": "enforce"
|
||||
},
|
||||
{
|
||||
"name": "SESSION_COOKIE_SECURE",
|
||||
"label": "Session cookie Secure flag",
|
||||
|
||||
@@ -80,6 +80,41 @@ Fix it one of three ways, then restart the app:
|
||||
3. Start fresh with an empty database. The old data and its backups need the
|
||||
old key.
|
||||
|
||||
Before it records a key as the right one, the check reads the oldest
|
||||
encrypted values in the database and records the key only if they open (or if
|
||||
the database holds no encrypted value under that key id yet). When the data
|
||||
proves neither, the app keeps serving, records nothing, writes a warning
|
||||
starting with `[boot] Encryption key check inconclusive` to the log, and
|
||||
checks again at the next start.
|
||||
|
||||
### Resetting the check
|
||||
|
||||
If you are certain the configured key is the one the data was written with,
|
||||
and the app still refuses because the check recorded a different key earlier,
|
||||
remove that record and restart. Open a shell on the database container (Apps,
|
||||
HealthLog, the database container's Shell) and run, with the key id the log
|
||||
names (`v1` unless you use `ENCRYPTION_KEYS`):
|
||||
|
||||
```bash
|
||||
psql -U healthlog healthlog \
|
||||
-c "DELETE FROM encryption_key_canaries WHERE key_id = 'v1';"
|
||||
```
|
||||
|
||||
Then restart the app. It probes the stored data again and records the key
|
||||
only if that data opens with it. If the key is in fact wrong, the app then
|
||||
fails on every encrypted value instead of refusing clearly, so only do this
|
||||
when you are sure. After a key rotation you never need this for the old key
|
||||
id: the rotation script removes its record once no values remain under it.
|
||||
|
||||
On a first start there is no record to remove. If the check refuses there and
|
||||
you are certain the key is right, add `ENCRYPTION_KEY_CHECK` = `warn` under
|
||||
Apps, HealthLog, Edit, Environment as a last resort. The check still runs and
|
||||
logs, `/api/health` reports `warning: "encryption_key_mismatch"` without
|
||||
failing, the app serves, and no key is recorded until the check passes. This
|
||||
switches off a safety check: with a wrong key every encrypted value fails and
|
||||
new data is written under that key. Remove the setting once the key is
|
||||
confirmed.
|
||||
|
||||
## Updates
|
||||
|
||||
Pick the new tag, take a snapshot of the database dataset first, and update.
|
||||
|
||||
@@ -54,6 +54,7 @@
|
||||
|
||||
<Config Name="SESSION_COOKIE_SECURE" Target="SESSION_COOKIE_SECURE" Default="false" Mode="" Description="Set to false for LAN / plain-HTTP installs (http://tower:3000) or login fails. Set to true only behind an HTTPS reverse proxy." Type="Variable" Display="always" Required="true" Mask="false">false</Config>
|
||||
<Config Name="HEALTHLOG_PLATFORM" Target="HEALTHLOG_PLATFORM" Default="unraid" Mode="" Description="Where this server is configured. Opens the admin's encryption key backup step on the Unraid instructions." Type="Variable" Display="advanced" Required="false" Mask="false">unraid</Config>
|
||||
<Config Name="ENCRYPTION_KEY_CHECK" Target="ENCRYPTION_KEY_CHECK" Default="enforce" Mode="" Description="Start-up encryption key check. enforce refuses to serve when the key does not open the stored data. warn only logs it: a last resort for a refusal you are certain is false, and it disables a safety check." Type="Variable" Display="advanced" Required="false" Mask="false">enforce</Config>
|
||||
|
||||
<Config Name="NEXT_PUBLIC_APP_URL" Target="NEXT_PUBLIC_APP_URL" Default="http://tower:3000" Mode="" Description="Public URL users reach HealthLog at. Use the LAN URL (e.g. http://tower:3000) or your HTTPS domain behind a reverse proxy." Type="Variable" Display="always" Required="true" Mask="false"/>
|
||||
|
||||
|
||||
@@ -7657,6 +7657,7 @@
|
||||
"fixDatabase": "Oder lass DATABASE_URL auf die Datenbank zeigen, die zu diesem Schlüssel gehört.",
|
||||
"fixFresh": "Oder starte neu mit einer leeren Datenbank. Die alten Daten und ihre Sicherungen lassen sich nur mit dem alten Schlüssel lesen.",
|
||||
"restartNote": "Starte HealthLog nach der Änderung neu.",
|
||||
"lastResort": "Letzter Ausweg, nur wenn du sicher bist, dass der Schlüssel stimmt, und keiner der drei Wege passt: setze {setting} und starte neu. HealthLog liefert dann trotzdem aus und schreibt den Befund nur ins Log. Damit ist eine Sicherheitsprüfung abgeschaltet: Mit einem falschen Schlüssel scheitert jeder verschlüsselte Wert, und neue Daten werden unter diesem Schlüssel geschrieben.",
|
||||
"codes": "Für den Betrieb: API-Anfragen antworten mit 503 und dem Fehlercode {code}, /api/health meldet den Grund {reason}."
|
||||
}
|
||||
},
|
||||
|
||||
@@ -7657,6 +7657,7 @@
|
||||
"fixDatabase": "Or point DATABASE_URL at the database that belongs to this key.",
|
||||
"fixFresh": "Or start fresh with an empty database. The old data and its backups can only be read with the old key.",
|
||||
"restartNote": "Restart HealthLog after the change.",
|
||||
"lastResort": "Last resort, only if you are certain the key is right and none of the three applies: set {setting} and restart. HealthLog then serves anyway and only logs the finding. This disables a safety check: with a wrong key every encrypted value fails and new data is written under that key.",
|
||||
"codes": "For the operator: API requests answer 503 with error code {code}, and /api/health reports reason {reason}."
|
||||
}
|
||||
},
|
||||
|
||||
@@ -7657,6 +7657,7 @@
|
||||
"fixDatabase": "O haz que DATABASE_URL apunte a la base de datos que corresponde a esta clave.",
|
||||
"fixFresh": "O empieza de cero con una base de datos vacía. Los datos antiguos y sus copias solo se pueden leer con la clave antigua.",
|
||||
"restartNote": "Reinicia HealthLog después del cambio.",
|
||||
"lastResort": "Último recurso, solo si estás seguro de que la clave es correcta y ninguna de las tres opciones sirve: define {setting} y reinicia. HealthLog sirve igualmente y solo registra el hallazgo. Esto desactiva una comprobación de seguridad: con una clave incorrecta falla cada valor cifrado y los datos nuevos se escriben con esa clave.",
|
||||
"codes": "Para el administrador: las peticiones a la API responden 503 con el código de error {code}, y /api/health indica el motivo {reason}."
|
||||
}
|
||||
},
|
||||
|
||||
@@ -7657,6 +7657,7 @@
|
||||
"fixDatabase": "Ou fais pointer DATABASE_URL vers la base de données qui correspond à cette clé.",
|
||||
"fixFresh": "Ou repars de zéro avec une base vide. Les anciennes données et leurs sauvegardes ne se lisent qu'avec l'ancienne clé.",
|
||||
"restartNote": "Redémarre HealthLog après la modification.",
|
||||
"lastResort": "Dernier recours, seulement si tu es sûr que la clé est la bonne et qu'aucune des trois solutions ne convient : définis {setting} et redémarre. HealthLog sert alors quand même et se contente de journaliser le constat. Cela désactive une vérification de sécurité : avec une mauvaise clé, chaque valeur chiffrée échoue et les nouvelles données sont écrites sous cette clé.",
|
||||
"codes": "Pour l'administrateur : les requêtes API répondent 503 avec le code d'erreur {code}, et /api/health indique la raison {reason}."
|
||||
}
|
||||
},
|
||||
|
||||
@@ -7657,6 +7657,7 @@
|
||||
"fixDatabase": "Oppure fai puntare DATABASE_URL al database che appartiene a questa chiave.",
|
||||
"fixFresh": "Oppure ricomincia con un database vuoto. I vecchi dati e i loro backup si leggono solo con la vecchia chiave.",
|
||||
"restartNote": "Riavvia HealthLog dopo la modifica.",
|
||||
"lastResort": "Ultima risorsa, solo se sei certo che la chiave sia giusta e nessuna delle tre soluzioni vale: imposta {setting} e riavvia. HealthLog serve comunque e si limita a registrare il problema nel log. Così si disattiva un controllo di sicurezza: con una chiave sbagliata ogni valore cifrato fallisce e i nuovi dati vengono scritti con quella chiave.",
|
||||
"codes": "Per chi gestisce il server: le richieste API rispondono 503 con il codice di errore {code}, e /api/health indica il motivo {reason}."
|
||||
}
|
||||
},
|
||||
|
||||
@@ -7657,6 +7657,7 @@
|
||||
"fixDatabase": "또는 DATABASE_URL이 이 키에 맞는 데이터베이스를 가리키게 하세요.",
|
||||
"fixFresh": "또는 빈 데이터베이스로 새로 시작하세요. 이전 데이터와 백업은 이전 키로만 읽을 수 있습니다.",
|
||||
"restartNote": "변경 후 HealthLog를 다시 시작하세요.",
|
||||
"lastResort": "최후의 수단: 키가 맞다고 확신하고 위 세 가지가 모두 해당하지 않을 때만 {setting}을 설정하고 다시 시작하세요. HealthLog는 그래도 서비스하며 결과를 로그에만 남깁니다. 이 설정은 안전 검사를 끕니다. 키가 틀리면 모든 암호화된 값이 실패하고 새 데이터가 그 키로 기록됩니다.",
|
||||
"codes": "운영자용: API 요청은 오류 코드 {code}와 함께 503으로 응답하고, /api/health는 원인 {reason}을 알려 줍니다."
|
||||
}
|
||||
},
|
||||
|
||||
@@ -7657,6 +7657,7 @@
|
||||
"fixDatabase": "Albo ustaw DATABASE_URL na bazę danych, która należy do tego klucza.",
|
||||
"fixFresh": "Albo zacznij od nowa z pustą bazą. Stare dane i ich kopie można odczytać tylko starym kluczem.",
|
||||
"restartNote": "Po zmianie uruchom HealthLog ponownie.",
|
||||
"lastResort": "Ostateczność, tylko jeśli masz pewność, że klucz jest właściwy, a żadna z trzech dróg nie pasuje: ustaw {setting} i uruchom ponownie. HealthLog będzie wtedy działać mimo to i tylko zapisze wynik w logu. To wyłącza kontrolę bezpieczeństwa: przy złym kluczu każda zaszyfrowana wartość zawiedzie, a nowe dane zostaną zapisane tym kluczem.",
|
||||
"codes": "Dla administratora: żądania API odpowiadają 503 z kodem błędu {code}, a /api/health podaje powód {reason}."
|
||||
}
|
||||
},
|
||||
|
||||
@@ -327,6 +327,11 @@
|
||||
"purpose": "Where this server's environment is configured (compose, truenas, coolify, unraid, portainer). Shown back to the admin: the \"Back up your encryption key\" step opens on that platform's instructions. Unset means compose.",
|
||||
"example": "compose"
|
||||
},
|
||||
{
|
||||
"name": "ENCRYPTION_KEY_CHECK",
|
||||
"purpose": "The start-up encryption key check. enforce (default) refuses to serve when the configured key does not open the stored data; warn only logs the finding, names it as a warning on /api/health, serves anyway and records no key until the check passes. A last resort for a refusal the operator is certain is false; it disables a safety check.",
|
||||
"example": "enforce"
|
||||
},
|
||||
{
|
||||
"name": "TRUST_PROXY_HOPS",
|
||||
"purpose": "Trusted reverse-proxy hop count; the client IP is read that many entries from the right of X-Forwarded-For.",
|
||||
|
||||
@@ -17,6 +17,10 @@
|
||||
* non-zero, because an operator reads a zero here as permission to drop the
|
||||
* previous key, and "nothing left" and "never looked" must not print alike.
|
||||
*
|
||||
* After a complete, error-free run it checks every other configured key id
|
||||
* for remaining values and removes the boot key check's record (canary) of
|
||||
* each id that holds none, naming the columns of each id that still does.
|
||||
*
|
||||
* Usage:
|
||||
* ENCRYPTION_KEYS='{"v1":"<old>","v2":"<new>"}' \
|
||||
* ENCRYPTION_ACTIVE_KEY_ID=v2 \
|
||||
@@ -27,6 +31,7 @@ import { Buffer } from "node:buffer";
|
||||
import { PrismaClient } from "@/generated/prisma/client";
|
||||
import { PrismaPg } from "@prisma/adapter-pg";
|
||||
import { decrypt, encrypt, extractKeyId, getActiveKeyId } from "@/lib/crypto";
|
||||
import { retireCanariesWithoutData } from "@/lib/crypto/canary";
|
||||
import {
|
||||
rotateColumn,
|
||||
type CorpusClient,
|
||||
@@ -1034,6 +1039,30 @@ async function main() {
|
||||
`drop the previous key:\n ${notWalked.join("\n ")}`,
|
||||
);
|
||||
}
|
||||
|
||||
// ───── Key-check records of retired key ids ─────
|
||||
// The boot key check keeps one canary per configured key id. Once no
|
||||
// registered column holds a single value under an id, that id's key can
|
||||
// leave ENCRYPTION_KEYS, and its canary goes with it: left behind, it would
|
||||
// refuse a later, different key configured under the same id. Only after a
|
||||
// complete, error-free run; the existence check walks every column itself
|
||||
// rather than trusting the counts above.
|
||||
if (notWalked.length === 0 && totalErrors === 0) {
|
||||
const { removed, remaining } = await retireCanariesWithoutData(prisma);
|
||||
for (const [keyId, columns] of Object.entries(remaining)) {
|
||||
console.log(
|
||||
`\nKey id '${keyId}': values remain in ${columns.length} column(s); ` +
|
||||
`keep its key:\n ${columns.join("\n ")}`,
|
||||
);
|
||||
}
|
||||
for (const keyId of removed) {
|
||||
console.log(
|
||||
`\nKey id '${keyId}': no values remain. Its key-check record was ` +
|
||||
`removed; its key can now leave ENCRYPTION_KEYS.`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
await prisma.$disconnect();
|
||||
if (notWalked.length > 0) process.exit(4);
|
||||
process.exit(totalErrors > 0 ? 3 : 0);
|
||||
|
||||
@@ -0,0 +1,127 @@
|
||||
/**
|
||||
* Every request surface outside `apiHandler` refuses while the boot key check
|
||||
* has found a mismatch.
|
||||
*
|
||||
* `apiHandler` answers 503 `encryption.key_mismatch` before auth and before
|
||||
* the handler, so a process holding the wrong key never writes through it.
|
||||
* A route that does not wrap in `apiHandler` gets none of that: the remote
|
||||
* MCP endpoint and its OAuth bridge went without it, and an MCP write tool
|
||||
* would have sealed new rows under the wrong key, rows the next boot's probe
|
||||
* could then read as evidence that the wrong key is right.
|
||||
*
|
||||
* So: a `route.ts` under `src/app` that does not wrap in `apiHandler` must
|
||||
* call `refuseOnKeyMismatch` / `refuseOAuthOnKeyMismatch`, or sit in the
|
||||
* allowlist below with the reason it cannot touch encrypted data. A server
|
||||
* action (`"use server"`) is held to the same rule; there are none today.
|
||||
*/
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { readFileSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
|
||||
import { walkSourceFiles } from "./helpers/source-files";
|
||||
|
||||
const SRC = join(process.cwd(), "src");
|
||||
|
||||
/**
|
||||
* Routes outside `apiHandler` that need no refusal, each with why. A file
|
||||
* belongs here only if it reads nothing from the database and writes nothing.
|
||||
*/
|
||||
const NO_DATA_ROUTES: Record<string, string> = {
|
||||
"app/.well-known/apple-app-site-association/route.ts":
|
||||
"static JSON built from environment variables; no database access",
|
||||
"app/.well-known/oauth-authorization-server/route.ts":
|
||||
"RFC 8414 metadata built from the configured origin; no database access",
|
||||
"app/.well-known/oauth-protected-resource/route.ts":
|
||||
"RFC 9728 metadata built from the configured origin; no database access",
|
||||
};
|
||||
|
||||
const REFUSAL_CALL = /\brefuse(?:OAuth)?OnKeyMismatch\s*\(/;
|
||||
|
||||
function sourceFiles(): string[] {
|
||||
return walkSourceFiles(SRC, { floor: 3000 })
|
||||
.filter((p) => !p.includes("__tests__"))
|
||||
.filter((p) => !p.endsWith(".test.ts") && !p.endsWith(".test.tsx"))
|
||||
.sort();
|
||||
}
|
||||
|
||||
/** The text of a top-level function starting at `from`, up to its closing brace. */
|
||||
function functionBody(src: string, from: number): string {
|
||||
const end = src.indexOf("\n}\n", from);
|
||||
return src.slice(from, end === -1 ? undefined : end);
|
||||
}
|
||||
|
||||
function read(rel: string): string {
|
||||
return readFileSync(join(SRC, rel), "utf8");
|
||||
}
|
||||
|
||||
describe("key mismatch refusal outside apiHandler", () => {
|
||||
const routes = sourceFiles().filter(
|
||||
(p) => p.startsWith("app/") && /\/route\.tsx?$/.test(p),
|
||||
);
|
||||
const outside = routes.filter((p) => !/\bapiHandler\s*\(/.test(read(p)));
|
||||
|
||||
it("finds the route tree and the routes outside apiHandler", () => {
|
||||
// A matcher that matches nothing would pass every check below.
|
||||
expect(routes.length).toBeGreaterThan(300);
|
||||
expect(outside).toEqual(
|
||||
expect.arrayContaining([
|
||||
"app/mcp/route.ts",
|
||||
"app/api/mcp/oauth/authorize/route.ts",
|
||||
"app/api/mcp/oauth/register/route.ts",
|
||||
"app/api/mcp/oauth/token/route.ts",
|
||||
]),
|
||||
);
|
||||
});
|
||||
|
||||
it("every route outside apiHandler refuses, or is allowlisted with a reason", () => {
|
||||
const missing = outside.filter(
|
||||
(p) => !(p in NO_DATA_ROUTES) && !REFUSAL_CALL.test(read(p)),
|
||||
);
|
||||
expect(missing).toEqual([]);
|
||||
});
|
||||
|
||||
it("an allowlisted route really stays away from the database", () => {
|
||||
for (const p of Object.keys(NO_DATA_ROUTES)) {
|
||||
expect(outside, `${p} is no longer outside apiHandler`).toContain(p);
|
||||
const src = read(p);
|
||||
expect(src, p).not.toMatch(/@\/lib\/db\b|prisma|\$queryRaw/);
|
||||
}
|
||||
});
|
||||
|
||||
it("every exported handler of a refusing route calls the refusal itself", () => {
|
||||
// `export async function POST(...) { ... }` must refuse in its own body;
|
||||
// `export const POST = handleX` must refuse in `handleX`. A refusal in one
|
||||
// handler of a file proves nothing about its siblings.
|
||||
let checked = 0;
|
||||
for (const p of outside.filter((f) => !(f in NO_DATA_ROUTES))) {
|
||||
const src = read(p);
|
||||
const bodies: Array<[string, string]> = [];
|
||||
for (const m of src.matchAll(
|
||||
/export\s+(?:async\s+)?function\s+(GET|POST|PUT|PATCH|DELETE)\b/g,
|
||||
)) {
|
||||
bodies.push([m[1], functionBody(src, m.index)]);
|
||||
}
|
||||
for (const m of src.matchAll(
|
||||
/export\s+const\s+(GET|POST|PUT|PATCH|DELETE)\s*=\s*(\w+)\s*;/g,
|
||||
)) {
|
||||
const def = new RegExp(`function\\s+${m[2]}\\s*\\(`).exec(src);
|
||||
expect(def, `${p}: ${m[1]} → ${m[2]} not found`).not.toBeNull();
|
||||
bodies.push([m[1], functionBody(src, def!.index)]);
|
||||
}
|
||||
expect(bodies.length, p).toBeGreaterThan(0);
|
||||
for (const [name, body] of bodies) {
|
||||
expect(REFUSAL_CALL.test(body), `${p} ${name}`).toBe(true);
|
||||
checked += 1;
|
||||
}
|
||||
}
|
||||
expect(checked).toBeGreaterThanOrEqual(6);
|
||||
});
|
||||
|
||||
it("server actions refuse too", () => {
|
||||
const actions = sourceFiles().filter((p) =>
|
||||
/^\s*["']use server["']/m.test(read(p)),
|
||||
);
|
||||
const missing = actions.filter((p) => !REFUSAL_CALL.test(read(p)));
|
||||
expect(missing).toEqual([]);
|
||||
});
|
||||
});
|
||||
@@ -76,6 +76,11 @@ const PUBLISHED_TO_USERS = [
|
||||
shownBy:
|
||||
"the admin's encryption key backup step, which opens on that platform",
|
||||
},
|
||||
{
|
||||
name: "ENCRYPTION_KEY_CHECK",
|
||||
shownBy:
|
||||
"the key mismatch page and the start-up log, which offer it as the last resort",
|
||||
},
|
||||
{
|
||||
name: "AUDIT_LOG_RETENTION_DAYS",
|
||||
shownBy: "the shared-access activity window and the revoke dialog",
|
||||
|
||||
@@ -140,3 +140,26 @@ describe("GET /api/health while the encryption key does not match", () => {
|
||||
expect(body).toEqual({ status: "ok" });
|
||||
});
|
||||
});
|
||||
|
||||
describe("GET /api/health under ENCRYPTION_KEY_CHECK=warn", () => {
|
||||
it("names the mismatch as a warning and stays ok", async () => {
|
||||
const { setKeyMismatchWarning } =
|
||||
await import("@/lib/boot/key-mismatch-state");
|
||||
vi.stubEnv("HEALTHLOG_PROCESS_TYPE", "web");
|
||||
getGlobalBoss.mockReturnValue({});
|
||||
setKeyMismatchWarning({
|
||||
keyIds: ["v1"],
|
||||
detectedAt: "2026-10-03T00:00:00Z",
|
||||
});
|
||||
try {
|
||||
const { response, body } = await health();
|
||||
expect(response.status).toBe(200);
|
||||
expect(body).toEqual({
|
||||
status: "ok",
|
||||
warning: "encryption_key_mismatch",
|
||||
});
|
||||
} finally {
|
||||
setKeyMismatchWarning(null);
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -7,6 +7,7 @@ import { NextResponse } from "next/server";
|
||||
import { apiHandler } from "@/lib/api-handler";
|
||||
import { annotate } from "@/lib/logging/context";
|
||||
import {
|
||||
getKeyMismatchWarning,
|
||||
isKeyMismatch,
|
||||
KEY_MISMATCH_HEALTH_REASON,
|
||||
} from "@/lib/boot/key-mismatch-state";
|
||||
@@ -31,6 +32,12 @@ export const GET = apiHandler(async () => {
|
||||
// it is the one place a catalog install's status page reads.
|
||||
const keyMismatch = isKeyMismatch();
|
||||
const reason = keyMismatch ? KEY_MISMATCH_HEALTH_REASON : undefined;
|
||||
// Under ENCRYPTION_KEY_CHECK=warn the same finding does not refuse; it is
|
||||
// named as a warning and leaves the status alone.
|
||||
const warning =
|
||||
!keyMismatch && getKeyMismatchWarning()
|
||||
? KEY_MISMATCH_HEALTH_REASON
|
||||
: undefined;
|
||||
const status =
|
||||
!keyMismatch &&
|
||||
dbOk &&
|
||||
@@ -51,6 +58,7 @@ export const GET = apiHandler(async () => {
|
||||
{
|
||||
status,
|
||||
...(reason ? { reason } : {}),
|
||||
...(warning ? { warning } : {}),
|
||||
timestamp: new Date().toISOString(),
|
||||
database: dbOk ? "connected" : "disconnected",
|
||||
worker: worker.running ? "running" : "stopped",
|
||||
@@ -63,7 +71,7 @@ export const GET = apiHandler(async () => {
|
||||
}
|
||||
|
||||
return NextResponse.json(
|
||||
{ status, ...(reason ? { reason } : {}) },
|
||||
{ status, ...(reason ? { reason } : {}), ...(warning ? { warning } : {}) },
|
||||
{ status: statusCode, headers: cacheHeaders },
|
||||
);
|
||||
});
|
||||
@@ -29,6 +29,7 @@ import { NextRequest } from "next/server";
|
||||
import { z } from "zod/v4";
|
||||
|
||||
import { annotate } from "@/lib/logging/context";
|
||||
import { refuseOAuthOnKeyMismatch } from "@/lib/boot/key-mismatch-refusal";
|
||||
import { withBackgroundEvent } from "@/lib/logging/background";
|
||||
import { auditLog } from "@/lib/auth/audit";
|
||||
import { getSession } from "@/lib/auth/session";
|
||||
@@ -344,10 +345,15 @@ function ensureAuthorizeCsp(response: Response): Response {
|
||||
}
|
||||
|
||||
export async function GET(request: NextRequest): Promise<Response> {
|
||||
// The boot key check refused this process: no grant is issued or stored.
|
||||
const refusal = refuseOAuthOnKeyMismatch();
|
||||
if (refusal) return ensureAuthorizeCsp(refusal);
|
||||
return ensureAuthorizeCsp(await authorizeGet(request));
|
||||
}
|
||||
|
||||
export async function POST(request: NextRequest): Promise<Response> {
|
||||
const refusal = refuseOAuthOnKeyMismatch();
|
||||
if (refusal) return ensureAuthorizeCsp(refusal);
|
||||
return ensureAuthorizeCsp(await authorizePost(request));
|
||||
}
|
||||
|
||||
|
||||
@@ -16,6 +16,7 @@ import { NextRequest } from "next/server";
|
||||
import { z } from "zod/v4";
|
||||
|
||||
import { annotate } from "@/lib/logging/context";
|
||||
import { refuseOAuthOnKeyMismatch } from "@/lib/boot/key-mismatch-refusal";
|
||||
import { withBackgroundEvent } from "@/lib/logging/background";
|
||||
import { checkAuthSurfaceRateLimit, rateLimitHeaders } from "@/lib/rate-limit";
|
||||
import { isApiGloballyEnabled } from "@/lib/app-settings";
|
||||
@@ -68,6 +69,10 @@ function oauthError(
|
||||
|
||||
export async function POST(request: NextRequest): Promise<Response> {
|
||||
return withBackgroundEvent("mcp.oauth.register", async () => {
|
||||
// The boot key check refused this process: mint nothing, write nothing.
|
||||
const refusal = refuseOAuthOnKeyMismatch();
|
||||
if (refusal) return refusal;
|
||||
|
||||
const rl = await checkAuthSurfaceRateLimit(
|
||||
request,
|
||||
"mcp:oauth:register",
|
||||
|
||||
@@ -32,6 +32,7 @@ import { NextRequest } from "next/server";
|
||||
import type { Prisma } from "@/generated/prisma/client";
|
||||
import { prisma } from "@/lib/db";
|
||||
import { annotate } from "@/lib/logging/context";
|
||||
import { refuseOAuthOnKeyMismatch } from "@/lib/boot/key-mismatch-refusal";
|
||||
import { withBackgroundEvent } from "@/lib/logging/background";
|
||||
import { auditLog } from "@/lib/auth/audit";
|
||||
import { issueApiToken } from "@/lib/auth/issue-token";
|
||||
@@ -201,6 +202,10 @@ async function finalizeTokenPair(
|
||||
|
||||
export async function POST(request: NextRequest): Promise<Response> {
|
||||
return withBackgroundEvent("mcp.oauth.token", async () => {
|
||||
// The boot key check refused this process: mint nothing, write nothing.
|
||||
const refusal = refuseOAuthOnKeyMismatch();
|
||||
if (refusal) return refusal;
|
||||
|
||||
// M1 — fail closed without a pinned origin; M4 — honour the operator's
|
||||
// global API kill-switch. Either off and the bridge mints no tokens.
|
||||
if (!isMcpOriginConfigured() || !(await isApiGloballyEnabled())) {
|
||||
|
||||
@@ -31,6 +31,11 @@ export function KeyMismatchPage({ locale }: { locale: Locale }) {
|
||||
<p className="text-muted-foreground text-sm">
|
||||
{t("admin.keyMismatch.restartNote")}
|
||||
</p>
|
||||
<p className="text-muted-foreground text-sm">
|
||||
{t("admin.keyMismatch.lastResort", {
|
||||
setting: "ENCRYPTION_KEY_CHECK=warn",
|
||||
})}
|
||||
</p>
|
||||
<p className="text-muted-foreground text-xs">
|
||||
{t("admin.keyMismatch.codes", {
|
||||
code: "encryption.key_mismatch",
|
||||
|
||||
@@ -50,6 +50,7 @@ import {
|
||||
resolveMcpAuthContext,
|
||||
SCOPE_HEALTH_READ,
|
||||
} from "@/lib/mcp";
|
||||
import { refuseOnKeyMismatch } from "@/lib/boot/key-mismatch-refusal";
|
||||
import { withBackgroundEvent } from "@/lib/logging/background";
|
||||
import { annotate } from "@/lib/logging/context";
|
||||
import { isModuleEnabled } from "@/lib/modules/gate";
|
||||
@@ -135,6 +136,12 @@ function readBearer(request: Request): string | null {
|
||||
|
||||
async function handleMcp(request: Request): Promise<Response> {
|
||||
return withBackgroundEvent("mcp.request", async () => {
|
||||
// 00. KEY CHECK — the boot key check found that the configured encryption
|
||||
// key does not open this database. The write tools would seal new
|
||||
// rows under it, so refuse before anything else, as apiHandler does.
|
||||
const refusal = refuseOnKeyMismatch();
|
||||
if (refusal) return refusal;
|
||||
|
||||
// 0. ORIGIN — refuse a cross-origin browser request before anything else.
|
||||
if (!originAllowed(request)) {
|
||||
annotate({ action: { name: "mcp.origin.rejected" } });
|
||||
|
||||
+4
-16
@@ -53,11 +53,7 @@ import {
|
||||
assertRecordSessionFence,
|
||||
attachRecordContextEcho,
|
||||
} from "./sharing/record-session-fence";
|
||||
import {
|
||||
isKeyMismatch,
|
||||
KEY_MISMATCH_ERROR_CODE,
|
||||
KEY_MISMATCH_EXEMPT_PATHS,
|
||||
} from "./boot/key-mismatch-state";
|
||||
import { refuseOnKeyMismatch } from "./boot/key-mismatch-refusal";
|
||||
import {
|
||||
capturedRateLimit,
|
||||
rateLimitResponseHeaders,
|
||||
@@ -266,17 +262,9 @@ export function apiHandler<T extends (...args: any[]) => Promise<Response>>(
|
||||
// The boot key check found that the configured encryption key
|
||||
// cannot open this database. Refuse before auth, before the
|
||||
// handler: nothing here may read or write encrypted data.
|
||||
if (isKeyMismatch() && !KEY_MISMATCH_EXEMPT_PATHS.has(url.pathname)) {
|
||||
annotate({ action: { name: "encryption.key_mismatch.refused" } });
|
||||
response = NextResponse.json(
|
||||
{
|
||||
data: null,
|
||||
error:
|
||||
"The server's encryption key does not match its database. The operator has to restore the original key or point the server at the matching database.",
|
||||
meta: { errorCode: KEY_MISMATCH_ERROR_CODE },
|
||||
},
|
||||
{ status: 503, headers: { "Retry-After": "300" } },
|
||||
);
|
||||
const refusal = refuseOnKeyMismatch(url.pathname);
|
||||
if (refusal) {
|
||||
response = refusal;
|
||||
} else {
|
||||
response = await handler(...args);
|
||||
}
|
||||
|
||||
@@ -13,17 +13,50 @@ import {
|
||||
checkEncryptionKeyCanaries,
|
||||
keyMismatchLogBlock,
|
||||
type CanaryClient,
|
||||
type InconclusiveReason,
|
||||
} from "@/lib/crypto/canary";
|
||||
import { setKeyMismatchState } from "./key-mismatch-state";
|
||||
import {
|
||||
getKeyCheckMode,
|
||||
setKeyMismatchState,
|
||||
setKeyMismatchWarning,
|
||||
} from "./key-mismatch-state";
|
||||
|
||||
const INCONCLUSIVE_EXPLANATION: Record<InconclusiveReason, string> = {
|
||||
"single-value": "the only stored value found under that id did not open.",
|
||||
mixed:
|
||||
"the oldest stored values under that id do not open, newer ones do. " +
|
||||
"That is what a wrong key that served for a while leaves behind; check " +
|
||||
"that this is the key the database was first written with.",
|
||||
unsampled:
|
||||
"the values stored under that id are too large to probe at start-up.",
|
||||
incomplete:
|
||||
"the probe of the stored data did not finish in time or a table could " +
|
||||
"not be read.",
|
||||
};
|
||||
|
||||
export async function runBootKeyCheck(client: CanaryClient): Promise<boolean> {
|
||||
const outcome = await checkEncryptionKeyCanaries(client);
|
||||
const mode = getKeyCheckMode();
|
||||
const outcome = await checkEncryptionKeyCanaries(client, { mode });
|
||||
setKeyMismatchWarning(null);
|
||||
if (outcome.state === "mismatch") {
|
||||
setKeyMismatchState({
|
||||
const state = {
|
||||
keyIds: outcome.keyIds,
|
||||
detectedAt: new Date().toISOString(),
|
||||
});
|
||||
};
|
||||
console.error(keyMismatchLogBlock(outcome.keyIds));
|
||||
if (mode === "warn") {
|
||||
// The operator turned the refusal off. Say so next to the block, keep
|
||||
// the finding for /api/health, and serve.
|
||||
setKeyMismatchState(null);
|
||||
setKeyMismatchWarning(state);
|
||||
console.error(
|
||||
"[boot] ENCRYPTION_KEY_CHECK=warn: serving anyway. No key is recorded " +
|
||||
"while the check does not pass. Remove the setting once the key is " +
|
||||
"confirmed.",
|
||||
);
|
||||
return false;
|
||||
}
|
||||
setKeyMismatchState(state);
|
||||
return true;
|
||||
}
|
||||
setKeyMismatchState(null);
|
||||
@@ -34,11 +67,17 @@ export async function runBootKeyCheck(client: CanaryClient): Promise<boolean> {
|
||||
`[boot] Encryption key check skipped: ${outcome.message.slice(0, 300)}`,
|
||||
);
|
||||
} else {
|
||||
if (outcome.inconclusive.length > 0) {
|
||||
for (const { keyId, reason } of outcome.inconclusive) {
|
||||
console.warn(
|
||||
`[boot] Encryption key check inconclusive for key id(s) ${outcome.inconclusive.join(", ")}: ` +
|
||||
"the only stored value found under that id did not open. Serving; " +
|
||||
"no canary written; the check runs again at the next start.",
|
||||
`[boot] Encryption key check inconclusive for key id ${keyId}: ` +
|
||||
`${INCONCLUSIVE_EXPLANATION[reason]} Serving; no key recorded; ` +
|
||||
"the check runs again at the next start.",
|
||||
);
|
||||
}
|
||||
if (mode === "warn" && outcome.inconclusive.length > 0) {
|
||||
console.warn(
|
||||
"[boot] ENCRYPTION_KEY_CHECK=warn: no key recorded on this start " +
|
||||
"because the check did not pass for every key id.",
|
||||
);
|
||||
}
|
||||
if (outcome.written.length > 0) {
|
||||
|
||||
@@ -0,0 +1,67 @@
|
||||
/**
|
||||
* The refusal a request surface answers while the boot key check has found
|
||||
* that the configured encryption key does not open this database.
|
||||
*
|
||||
* `apiHandler` applies it to every API route. A route outside `apiHandler`
|
||||
* (the remote MCP endpoint, its OAuth bridge) calls one of the two helpers
|
||||
* below as the first thing it does, before auth and before any read or
|
||||
* write: a process holding the wrong key must never seal new rows, because
|
||||
* those rows would open under that key at the next boot and read as evidence
|
||||
* that it is the right one. `key-mismatch-refusal-guard.test.ts` fails when a
|
||||
* route outside `apiHandler` does neither.
|
||||
*/
|
||||
import { annotate } from "@/lib/logging/context";
|
||||
import {
|
||||
isKeyMismatch,
|
||||
KEY_MISMATCH_ERROR_CODE,
|
||||
KEY_MISMATCH_EXEMPT_PATHS,
|
||||
} from "./key-mismatch-state";
|
||||
|
||||
export const KEY_MISMATCH_MESSAGE =
|
||||
"The server's encryption key does not match its database. The operator has to restore the original key or point the server at the matching database.";
|
||||
|
||||
const RETRY_AFTER = { "Retry-After": "300" };
|
||||
|
||||
/** The 503 in the standard `{ data, error, meta }` envelope. */
|
||||
export function keyMismatchResponse(): Response {
|
||||
annotate({ action: { name: "encryption.key_mismatch.refused" } });
|
||||
return Response.json(
|
||||
{
|
||||
data: null,
|
||||
error: KEY_MISMATCH_MESSAGE,
|
||||
meta: { errorCode: KEY_MISMATCH_ERROR_CODE },
|
||||
},
|
||||
{ status: 503, headers: RETRY_AFTER },
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* The envelope refusal when the process refuses and `pathname` is not one of
|
||||
* the exempt paths; otherwise null and the caller carries on.
|
||||
*/
|
||||
export function refuseOnKeyMismatch(pathname?: string): Response | null {
|
||||
if (!isKeyMismatch()) return null;
|
||||
if (pathname !== undefined && KEY_MISMATCH_EXEMPT_PATHS.has(pathname)) {
|
||||
return null;
|
||||
}
|
||||
return keyMismatchResponse();
|
||||
}
|
||||
|
||||
/**
|
||||
* The same refusal in the RFC 6749 error shape the OAuth endpoints speak:
|
||||
* `temporarily_unavailable` with a description, plus the HealthLog error
|
||||
* code as an extension member a client may ignore.
|
||||
*/
|
||||
export function refuseOAuthOnKeyMismatch(): Response | null {
|
||||
if (!isKeyMismatch()) return null;
|
||||
annotate({ action: { name: "encryption.key_mismatch.refused" } });
|
||||
return Response.json(
|
||||
{
|
||||
error: "temporarily_unavailable",
|
||||
error_description:
|
||||
"The server's encryption key does not match its database.",
|
||||
error_code: KEY_MISMATCH_ERROR_CODE,
|
||||
},
|
||||
{ status: 503, headers: { ...RETRY_AFTER, "Cache-Control": "no-store" } },
|
||||
);
|
||||
}
|
||||
@@ -22,11 +22,30 @@ export interface KeyMismatchState {
|
||||
}
|
||||
|
||||
const STATE_SYMBOL = Symbol.for("healthlog.encryptionKeyMismatch");
|
||||
const WARNING_SYMBOL = Symbol.for("healthlog.encryptionKeyMismatchWarning");
|
||||
|
||||
type GlobalWithState = typeof globalThis & {
|
||||
[STATE_SYMBOL]?: KeyMismatchState | null;
|
||||
[WARNING_SYMBOL]?: KeyMismatchState | null;
|
||||
};
|
||||
|
||||
/**
|
||||
* `ENCRYPTION_KEY_CHECK`: `enforce` (the default, and what any other value
|
||||
* means) refuses on a mismatch; `warn` runs the check, logs it, reports it as
|
||||
* a warning on `/api/health`, and serves anyway. `warn` is the operator's way
|
||||
* out of a refusal they are certain is false at a first boot, where there is
|
||||
* no recorded key yet whose row they could delete. It disables a safety
|
||||
* check: a process with the wrong key then serves errors on every encrypted
|
||||
* value and writes new rows under that key.
|
||||
*/
|
||||
export type KeyCheckMode = "enforce" | "warn";
|
||||
|
||||
export function getKeyCheckMode(): KeyCheckMode {
|
||||
return process.env.ENCRYPTION_KEY_CHECK?.trim().toLowerCase() === "warn"
|
||||
? "warn"
|
||||
: "enforce";
|
||||
}
|
||||
|
||||
/** The paths that keep answering while the process refuses everything else. */
|
||||
export const KEY_MISMATCH_EXEMPT_PATHS: ReadonlySet<string> = new Set([
|
||||
"/api/health",
|
||||
@@ -50,3 +69,15 @@ export function isKeyMismatch(): boolean {
|
||||
export function setKeyMismatchState(state: KeyMismatchState | null): void {
|
||||
(globalThis as GlobalWithState)[STATE_SYMBOL] = state;
|
||||
}
|
||||
|
||||
/**
|
||||
* A mismatch the check found while `ENCRYPTION_KEY_CHECK=warn`: recorded so
|
||||
* `/api/health` can name it as a warning, never read by the refusal.
|
||||
*/
|
||||
export function getKeyMismatchWarning(): KeyMismatchState | null {
|
||||
return (globalThis as GlobalWithState)[WARNING_SYMBOL] ?? null;
|
||||
}
|
||||
|
||||
export function setKeyMismatchWarning(state: KeyMismatchState | null): void {
|
||||
(globalThis as GlobalWithState)[WARNING_SYMBOL] = state;
|
||||
}
|
||||
@@ -1,17 +1,30 @@
|
||||
import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { PrismaClient } from "@/generated/prisma/client";
|
||||
import { PrismaPg } from "@prisma/adapter-pg";
|
||||
import {
|
||||
_resetCryptoCacheForTests,
|
||||
candidateMatchesKey,
|
||||
encrypt,
|
||||
encryptBytes,
|
||||
encryptUnderKeyId,
|
||||
extractKeyIdFromBytes,
|
||||
fingerprintKeyBytes,
|
||||
getKeyFingerprint,
|
||||
} from "@/lib/crypto";
|
||||
import { encryptToBytes } from "@/lib/ai/coach/bytes-codec";
|
||||
import {
|
||||
ENCRYPTED_COLUMNS,
|
||||
WORKOUT_ROUTE_GEOMETRY_AAD,
|
||||
encryptedColumnKey,
|
||||
} from "@/lib/crypto/encrypted-columns";
|
||||
import {
|
||||
canaryPlaintext,
|
||||
checkEncryptionKeyCanaries,
|
||||
keyMismatchLogBlock,
|
||||
probeValueOpens,
|
||||
resolveColumnLocation,
|
||||
type CanaryClient,
|
||||
type ColumnSampler,
|
||||
} from "../canary";
|
||||
|
||||
const KEY_A =
|
||||
@@ -19,23 +32,49 @@ const KEY_A =
|
||||
const KEY_B =
|
||||
"fedcba9876543210fedcba9876543210fedcba9876543210fedcba9876543210";
|
||||
|
||||
function useKey(hex: string) {
|
||||
function configureKeys(keys: Record<string, string>, active: string) {
|
||||
vi.stubEnv("ENCRYPTION_KEYS", JSON.stringify(keys));
|
||||
vi.stubEnv("ENCRYPTION_ACTIVE_KEY_ID", active);
|
||||
vi.stubEnv("ENCRYPTION_KEY", "");
|
||||
_resetCryptoCacheForTests();
|
||||
}
|
||||
|
||||
function configureKey(hex: string) {
|
||||
vi.stubEnv("ENCRYPTION_KEYS", "");
|
||||
vi.stubEnv("ENCRYPTION_ACTIVE_KEY_ID", "");
|
||||
vi.stubEnv("ENCRYPTION_KEY", hex);
|
||||
_resetCryptoCacheForTests();
|
||||
}
|
||||
|
||||
/** An in-memory stand-in for the two raw statements and the probe delegates. */
|
||||
type Seeded =
|
||||
| string
|
||||
| Uint8Array
|
||||
| { value: string | Uint8Array | null; ts?: string; codec?: string };
|
||||
|
||||
/**
|
||||
* An in-memory stand-in for the raw statements, plus a sampler over seeded
|
||||
* values keyed by `Model.field`. A seeded value's `ts` orders it; values
|
||||
* without one keep their seeded order after every dated value, as the SQL
|
||||
* sampler orders them.
|
||||
*/
|
||||
function fakeClient(opts: {
|
||||
canaries?: Record<string, string>;
|
||||
/** delegate → field → stored values (in row order). */
|
||||
probe?: Record<string, Record<string, string[]>>;
|
||||
probe?: Record<string, Seeded[]>;
|
||||
/** Make the sampler throw for these columns. */
|
||||
unreadable?: string[];
|
||||
/** Another process seals this id (with this ciphertext) just before us. */
|
||||
race?: { keyId: string; ciphertext: string };
|
||||
}) {
|
||||
const canaries = new Map(Object.entries(opts.canaries ?? {}));
|
||||
const inserts: string[] = [];
|
||||
let raced = false;
|
||||
const client = {
|
||||
async $queryRaw() {
|
||||
async $queryRaw(_q: TemplateStringsArray, ...values: unknown[]) {
|
||||
const [arg] = values;
|
||||
if (typeof arg === "string") {
|
||||
const c = canaries.get(arg);
|
||||
return c ? [{ ciphertext: c }] : [];
|
||||
}
|
||||
return [...canaries.entries()].map(([key_id, ciphertext]) => ({
|
||||
key_id,
|
||||
ciphertext,
|
||||
@@ -43,104 +82,118 @@ function fakeClient(opts: {
|
||||
},
|
||||
async $executeRaw(_q: TemplateStringsArray, ...values: unknown[]) {
|
||||
const [keyId, ciphertext] = values as [string, string];
|
||||
if (opts.race && !raced && opts.race.keyId === keyId) {
|
||||
raced = true;
|
||||
canaries.set(keyId, opts.race.ciphertext);
|
||||
}
|
||||
inserts.push(keyId);
|
||||
if (!canaries.has(keyId)) canaries.set(keyId, ciphertext);
|
||||
if (canaries.has(keyId)) return 0;
|
||||
canaries.set(keyId, ciphertext);
|
||||
return 1;
|
||||
},
|
||||
} as unknown as CanaryClient & Record<string, unknown>;
|
||||
for (const [delegate, fields] of Object.entries(opts.probe ?? {})) {
|
||||
client[delegate] = {
|
||||
async findMany(args: {
|
||||
select: Record<string, boolean>;
|
||||
take: number;
|
||||
skip?: number;
|
||||
}) {
|
||||
const field = Object.keys(args.select)[0];
|
||||
const skip = args.skip ?? 0;
|
||||
return (fields[field] ?? [])
|
||||
.slice(skip, skip + args.take)
|
||||
.map((value) => ({ [field]: value }));
|
||||
},
|
||||
};
|
||||
}
|
||||
return { client, canaries, inserts };
|
||||
} as unknown as CanaryClient;
|
||||
const sampler: ColumnSampler = async (column, keyId, limit) => {
|
||||
const key = `${column.model}.${column.field}`;
|
||||
if (opts.unreadable?.includes(key)) throw new Error("relation missing");
|
||||
const rows = (opts.probe?.[key] ?? []).map((v) =>
|
||||
typeof v === "string" || v instanceof Uint8Array
|
||||
? { value: v, ts: null as Date | null, codec: null }
|
||||
: {
|
||||
value: v.value,
|
||||
ts: v.ts ? new Date(v.ts) : null,
|
||||
codec: v.codec ?? null,
|
||||
},
|
||||
);
|
||||
const under = rows.filter((r) => {
|
||||
if (r.value === null) return true;
|
||||
if (typeof r.value === "string") return r.value.startsWith(`${keyId}.`);
|
||||
const b = Buffer.from(r.value);
|
||||
return (
|
||||
b.toString("latin1").startsWith(`${keyId}.`) ||
|
||||
extractKeyIdFromBytes(b) === keyId
|
||||
);
|
||||
});
|
||||
return under
|
||||
.map((r, i) => ({ r, i }))
|
||||
.sort((a, b) => {
|
||||
const ta = a.r.ts?.getTime() ?? Infinity;
|
||||
const tb = b.r.ts?.getTime() ?? Infinity;
|
||||
return ta === tb ? a.i - b.i : ta - tb;
|
||||
})
|
||||
.slice(0, limit)
|
||||
.map(({ r }) => r);
|
||||
};
|
||||
const check = (budget?: number) =>
|
||||
checkEncryptionKeyCanaries(client, { sampler, probeBudgetMs: budget });
|
||||
return { client, sampler, canaries, inserts, check };
|
||||
}
|
||||
|
||||
const ok = (fields: {
|
||||
written?: string[];
|
||||
verified?: string[];
|
||||
inconclusive?: Array<{ keyId: string; reason: string }>;
|
||||
}) => ({
|
||||
state: "ok",
|
||||
written: fields.written ?? [],
|
||||
verified: fields.verified ?? [],
|
||||
inconclusive: fields.inconclusive ?? [],
|
||||
});
|
||||
|
||||
describe("encryption key canary", () => {
|
||||
beforeEach(() => {
|
||||
vi.unstubAllEnvs();
|
||||
useKey(KEY_A);
|
||||
configureKey(KEY_A);
|
||||
});
|
||||
|
||||
it("writes a canary for a configured key with no row and no data", async () => {
|
||||
const { client, canaries, inserts } = fakeClient({});
|
||||
const outcome = await checkEncryptionKeyCanaries(client);
|
||||
expect(outcome).toEqual({
|
||||
state: "ok",
|
||||
written: ["v1"],
|
||||
verified: [],
|
||||
inconclusive: [],
|
||||
});
|
||||
it("writes a canary for a configured key when no column holds a value under its id", async () => {
|
||||
const { canaries, inserts, check } = fakeClient({});
|
||||
expect(await check()).toEqual(ok({ written: ["v1"] }));
|
||||
expect(inserts).toEqual(["v1"]);
|
||||
expect(canaries.get("v1")?.startsWith("v1.")).toBe(true);
|
||||
});
|
||||
|
||||
it("verifies a canary written under the same key", async () => {
|
||||
const { client, inserts } = fakeClient({
|
||||
const { inserts, check } = fakeClient({
|
||||
canaries: { v1: encryptUnderKeyId(canaryPlaintext("v1"), "v1") },
|
||||
});
|
||||
const outcome = await checkEncryptionKeyCanaries(client);
|
||||
expect(outcome).toEqual({
|
||||
state: "ok",
|
||||
written: [],
|
||||
verified: ["v1"],
|
||||
inconclusive: [],
|
||||
});
|
||||
expect(await check()).toEqual(ok({ verified: ["v1"] }));
|
||||
expect(inserts).toEqual([]);
|
||||
});
|
||||
|
||||
it("reports a mismatch when the key changed under the same id", async () => {
|
||||
const sealedUnderA = encryptUnderKeyId(canaryPlaintext("v1"), "v1");
|
||||
useKey(KEY_B);
|
||||
const { client, inserts } = fakeClient({ canaries: { v1: sealedUnderA } });
|
||||
const outcome = await checkEncryptionKeyCanaries(client);
|
||||
expect(outcome).toEqual({ state: "mismatch", keyIds: ["v1"] });
|
||||
configureKey(KEY_B);
|
||||
const { inserts, check } = fakeClient({ canaries: { v1: sealedUnderA } });
|
||||
expect(await check()).toEqual({ state: "mismatch", keyIds: ["v1"] });
|
||||
expect(inserts).toEqual([]);
|
||||
});
|
||||
|
||||
it("reports a mismatch for a canary that opens to the wrong value", async () => {
|
||||
// A row from another key id copied over: opens, but says the wrong thing.
|
||||
const { client } = fakeClient({
|
||||
const { check } = fakeClient({
|
||||
canaries: { v1: encryptUnderKeyId(canaryPlaintext("v2"), "v1") },
|
||||
});
|
||||
const outcome = await checkEncryptionKeyCanaries(client);
|
||||
expect(outcome.state).toBe("mismatch");
|
||||
expect((await check()).state).toBe("mismatch");
|
||||
});
|
||||
|
||||
it("refuses to seal a wrong key when existing data predates the canary", async () => {
|
||||
const existing = [encrypt("a stored token"), encrypt("another token")];
|
||||
useKey(KEY_B);
|
||||
const { client, inserts } = fakeClient({
|
||||
configureKey(KEY_B);
|
||||
const { inserts, check } = fakeClient({
|
||||
probe: {
|
||||
user: {
|
||||
codexAccessTokenEncrypted: [existing[0]],
|
||||
codexRefreshTokenEncrypted: [existing[1]],
|
||||
},
|
||||
"User.codexAccessTokenEncrypted": [existing[0]],
|
||||
"User.codexRefreshTokenEncrypted": [existing[1]],
|
||||
},
|
||||
});
|
||||
const outcome = await checkEncryptionKeyCanaries(client);
|
||||
expect(outcome).toEqual({ state: "mismatch", keyIds: ["v1"] });
|
||||
expect(await check()).toEqual({ state: "mismatch", keyIds: ["v1"] });
|
||||
expect(inserts).toEqual([]);
|
||||
});
|
||||
|
||||
it("writes the canary when existing data opens under the key", async () => {
|
||||
const { client, inserts } = fakeClient({
|
||||
probe: {
|
||||
user: { codexAccessTokenEncrypted: [encrypt("a stored token")] },
|
||||
},
|
||||
const { inserts, check } = fakeClient({
|
||||
probe: { "User.codexAccessTokenEncrypted": [encrypt("a stored token")] },
|
||||
});
|
||||
const outcome = await checkEncryptionKeyCanaries(client);
|
||||
expect(outcome.state).toBe("ok");
|
||||
expect((await check()).state).toBe("ok");
|
||||
expect(inserts).toEqual(["v1"]);
|
||||
});
|
||||
|
||||
@@ -148,46 +201,270 @@ describe("encryption key canary", () => {
|
||||
// A row that claims key id v1 but does not open (damaged, or sealed by
|
||||
// something else), found FIRST, then values that open.
|
||||
const damaged = "v1." + Buffer.alloc(40, 7).toString("base64");
|
||||
const { client, inserts } = fakeClient({
|
||||
const { inserts, check } = fakeClient({
|
||||
probe: {
|
||||
user: {
|
||||
codexAccessTokenEncrypted: [damaged],
|
||||
codexRefreshTokenEncrypted: [encrypt("a stored token")],
|
||||
},
|
||||
"User.codexAccessTokenEncrypted": [damaged],
|
||||
"User.codexRefreshTokenEncrypted": [encrypt("a stored token")],
|
||||
},
|
||||
});
|
||||
const outcome = await checkEncryptionKeyCanaries(client);
|
||||
expect(outcome.state).toBe("ok");
|
||||
expect((await check()).state).toBe("ok");
|
||||
expect(inserts).toEqual(["v1"]);
|
||||
});
|
||||
|
||||
it("several values in one column count, so a wrong key with one busy column still refuses", async () => {
|
||||
const values = [encrypt("one"), encrypt("two"), encrypt("three")];
|
||||
useKey(KEY_B);
|
||||
const { client, inserts } = fakeClient({
|
||||
probe: { user: { codexAccessTokenEncrypted: values } },
|
||||
configureKey(KEY_B);
|
||||
const { inserts, check } = fakeClient({
|
||||
probe: { "User.codexAccessTokenEncrypted": values },
|
||||
});
|
||||
const outcome = await checkEncryptionKeyCanaries(client);
|
||||
expect(outcome).toEqual({ state: "mismatch", keyIds: ["v1"] });
|
||||
expect(await check()).toEqual({ state: "mismatch", keyIds: ["v1"] });
|
||||
expect(inserts).toEqual([]);
|
||||
});
|
||||
|
||||
it("a single value that does not open is inconclusive: no mismatch, no canary", async () => {
|
||||
const existing = encrypt("the only stored token");
|
||||
useKey(KEY_B);
|
||||
const { client, inserts } = fakeClient({
|
||||
probe: { user: { codexAccessTokenEncrypted: [existing] } },
|
||||
});
|
||||
const outcome = await checkEncryptionKeyCanaries(client);
|
||||
expect(outcome).toEqual({
|
||||
state: "ok",
|
||||
written: [],
|
||||
verified: [],
|
||||
inconclusive: ["v1"],
|
||||
configureKey(KEY_B);
|
||||
const { inserts, check } = fakeClient({
|
||||
probe: { "User.codexAccessTokenEncrypted": [existing] },
|
||||
});
|
||||
expect(await check()).toEqual(
|
||||
ok({ inconclusive: [{ keyId: "v1", reason: "single-value" }] }),
|
||||
);
|
||||
expect(inserts).toEqual([]);
|
||||
});
|
||||
|
||||
describe("Bytes columns", () => {
|
||||
it("a wrong key over Bytes-only ciphertext refuses instead of sealing", async () => {
|
||||
const notes = [encryptToBytes("note one"), encryptToBytes("note two")];
|
||||
configureKey(KEY_B);
|
||||
const { inserts, check } = fakeClient({
|
||||
probe: {
|
||||
"MoodEntry.noteEncrypted": [notes[0]],
|
||||
"Measurement.notesEncrypted": [notes[1]],
|
||||
},
|
||||
});
|
||||
expect(await check()).toEqual({ state: "mismatch", keyIds: ["v1"] });
|
||||
expect(inserts).toEqual([]);
|
||||
});
|
||||
|
||||
it("opens the binary layout with its label, and the UTF-8 layout", async () => {
|
||||
const route = encryptBytes(Buffer.from("{}"), WORKOUT_ROUTE_GEOMETRY_AAD);
|
||||
const { inserts, check } = fakeClient({
|
||||
probe: {
|
||||
"WorkoutRoute.geometryEncrypted": [
|
||||
{ value: route, ts: "2024-01-01" },
|
||||
],
|
||||
"MoodEntry.noteEncrypted": [
|
||||
{ value: encryptToBytes("a note"), ts: "2024-01-02" },
|
||||
],
|
||||
},
|
||||
});
|
||||
expect(await check()).toEqual(ok({ written: ["v1"] }));
|
||||
expect(inserts).toEqual(["v1"]);
|
||||
});
|
||||
|
||||
it("a binary value under the wrong label does not count as opening", () => {
|
||||
const sealed = encryptBytes(Buffer.from("x"), "some/other/label");
|
||||
const column = ENCRYPTED_COLUMNS.find((c) => c.model === "WorkoutRoute")!;
|
||||
expect(probeValueOpens(column, { value: sealed, ts: null }, "v1")).toBe(
|
||||
false,
|
||||
);
|
||||
});
|
||||
|
||||
it("dispatches a codec column by the row's codec", () => {
|
||||
const column = ENCRYPTED_COLUMNS.find(
|
||||
(c) => c.model === "InboundDocument" && c.field === "contentEncrypted",
|
||||
)!;
|
||||
const binary = encryptBytes(Buffer.from("%PDF"));
|
||||
const text = encryptToBytes("JVBERg==");
|
||||
expect(
|
||||
probeValueOpens(
|
||||
column,
|
||||
{ value: binary, codec: "binary2", ts: null },
|
||||
"v1",
|
||||
),
|
||||
).toBe(true);
|
||||
expect(
|
||||
probeValueOpens(
|
||||
column,
|
||||
{ value: text, codec: "base64v1", ts: null },
|
||||
"v1",
|
||||
),
|
||||
).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe("oldest values decide", () => {
|
||||
function seedOldAThenNewB() {
|
||||
const old = [
|
||||
encrypt("old 1"),
|
||||
encrypt("old 2"),
|
||||
encryptToBytes("old note 1"),
|
||||
encryptToBytes("old note 2"),
|
||||
];
|
||||
configureKey(KEY_B);
|
||||
const recent = [
|
||||
encrypt("new 1"),
|
||||
encrypt("new 2"),
|
||||
encryptToBytes("new note 1"),
|
||||
encryptToBytes("new note 2"),
|
||||
];
|
||||
configureKey(KEY_A);
|
||||
return fakeClient({
|
||||
probe: {
|
||||
"User.codexAccessTokenEncrypted": [
|
||||
{ value: recent[0], ts: "2026-09-01" },
|
||||
{ value: old[0], ts: "2024-01-01" },
|
||||
],
|
||||
"User.codexRefreshTokenEncrypted": [
|
||||
{ value: old[1], ts: "2024-01-02" },
|
||||
{ value: recent[1], ts: "2026-09-02" },
|
||||
],
|
||||
"MoodEntry.noteEncrypted": [
|
||||
{ value: recent[2], ts: "2026-09-03" },
|
||||
{ value: old[2], ts: "2024-01-03" },
|
||||
],
|
||||
"Measurement.notesEncrypted": [
|
||||
{ value: old[3], ts: "2024-01-04" },
|
||||
{ value: recent[3], ts: "2026-09-04" },
|
||||
],
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
it("the key that wrote only the newer rows is never sealed", async () => {
|
||||
const fake = seedOldAThenNewB();
|
||||
configureKey(KEY_B);
|
||||
const outcome = await fake.check();
|
||||
expect(outcome.state).not.toBe("error");
|
||||
if (outcome.state === "ok") expect(outcome.written).toEqual([]);
|
||||
expect(fake.canaries.size).toBe(0);
|
||||
});
|
||||
|
||||
it("the key that wrote the oldest rows is sealed, newer unreadable rows notwithstanding", async () => {
|
||||
const fake = seedOldAThenNewB();
|
||||
expect(await fake.check()).toEqual(ok({ written: ["v1"] }));
|
||||
});
|
||||
|
||||
it("two oldest values that fail before one that opens are mixed: inconclusive", async () => {
|
||||
const opens = encrypt("opens");
|
||||
configureKey(KEY_B);
|
||||
const foreign = [encrypt("x"), encrypt("y")];
|
||||
configureKey(KEY_A);
|
||||
const { inserts, check } = fakeClient({
|
||||
probe: {
|
||||
"User.codexAccessTokenEncrypted": [
|
||||
{ value: foreign[0], ts: "2024-01-01" },
|
||||
],
|
||||
"User.codexRefreshTokenEncrypted": [
|
||||
{ value: foreign[1], ts: "2024-01-02" },
|
||||
],
|
||||
"MoodEntry.noteEncrypted": [{ value: opens, ts: "2025-01-01" }],
|
||||
},
|
||||
});
|
||||
expect(await check()).toEqual(
|
||||
ok({ inconclusive: [{ keyId: "v1", reason: "mixed" }] }),
|
||||
);
|
||||
expect(inserts).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("nothing unproven seals", () => {
|
||||
it("a probe out of time is inconclusive, even over data that opens", async () => {
|
||||
const { inserts, check } = fakeClient({
|
||||
probe: { "User.codexAccessTokenEncrypted": [encrypt("opens")] },
|
||||
});
|
||||
expect(await check(-1)).toEqual(
|
||||
ok({ inconclusive: [{ keyId: "v1", reason: "incomplete" }] }),
|
||||
);
|
||||
expect(inserts).toEqual([]);
|
||||
});
|
||||
|
||||
it("a column that cannot be read is inconclusive, never a fresh install", async () => {
|
||||
const { inserts, check } = fakeClient({
|
||||
unreadable: ["MoodEntry.noteEncrypted"],
|
||||
});
|
||||
expect(await check()).toEqual(
|
||||
ok({ inconclusive: [{ keyId: "v1", reason: "incomplete" }] }),
|
||||
);
|
||||
expect(inserts).toEqual([]);
|
||||
});
|
||||
|
||||
it("values too large to fetch are present: inconclusive, not fresh", async () => {
|
||||
const { inserts, check } = fakeClient({
|
||||
probe: { "InboundDocument.contentEncrypted": [{ value: null }] },
|
||||
});
|
||||
expect(await check()).toEqual(
|
||||
ok({ inconclusive: [{ keyId: "v1", reason: "unsampled" }] }),
|
||||
);
|
||||
expect(inserts).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("ENCRYPTION_KEY_CHECK=warn", () => {
|
||||
it("records nothing for any key id while one is inconclusive", async () => {
|
||||
configureKeys({ v1: KEY_A, v2: KEY_B }, "v2");
|
||||
const lone = encryptUnderKeyId("only value", "v1");
|
||||
configureKeys({ v1: KEY_B, v2: KEY_A }, "v2");
|
||||
const fake = fakeClient({
|
||||
probe: { "User.codexAccessTokenEncrypted": [lone] },
|
||||
});
|
||||
// Enforce records the proven id beside the inconclusive one.
|
||||
expect(await fake.check()).toEqual(
|
||||
ok({
|
||||
written: ["v2"],
|
||||
inconclusive: [{ keyId: "v1", reason: "single-value" }],
|
||||
}),
|
||||
);
|
||||
const warn = fakeClient({
|
||||
probe: { "User.codexAccessTokenEncrypted": [lone] },
|
||||
});
|
||||
expect(
|
||||
await checkEncryptionKeyCanaries(warn.client, {
|
||||
sampler: warn.sampler,
|
||||
mode: "warn",
|
||||
}),
|
||||
).toEqual(
|
||||
ok({ inconclusive: [{ keyId: "v1", reason: "single-value" }] }),
|
||||
);
|
||||
expect(warn.inserts).toEqual([]);
|
||||
});
|
||||
|
||||
it("records a key that passes for every id, as enforce does", async () => {
|
||||
const warn = fakeClient({
|
||||
probe: { "User.codexAccessTokenEncrypted": [encrypt("opens")] },
|
||||
});
|
||||
expect(
|
||||
await checkEncryptionKeyCanaries(warn.client, {
|
||||
sampler: warn.sampler,
|
||||
mode: "warn",
|
||||
}),
|
||||
).toEqual(ok({ written: ["v1"] }));
|
||||
});
|
||||
});
|
||||
|
||||
describe("the first-seal race", () => {
|
||||
it("losing it to a process with another key is a mismatch", async () => {
|
||||
configureKey(KEY_B);
|
||||
const otherCanary = encryptUnderKeyId(canaryPlaintext("v1"), "v1");
|
||||
configureKey(KEY_A);
|
||||
const { check } = fakeClient({
|
||||
race: { keyId: "v1", ciphertext: otherCanary },
|
||||
});
|
||||
expect(await check()).toEqual({ state: "mismatch", keyIds: ["v1"] });
|
||||
});
|
||||
|
||||
it("losing it to a process with the same key verifies", async () => {
|
||||
const { check } = fakeClient({
|
||||
race: {
|
||||
keyId: "v1",
|
||||
ciphertext: encryptUnderKeyId(canaryPlaintext("v1"), "v1"),
|
||||
},
|
||||
});
|
||||
expect(await check()).toEqual(ok({ verified: ["v1"] }));
|
||||
});
|
||||
});
|
||||
|
||||
it("reports an error, not a mismatch, when the table cannot be read", async () => {
|
||||
const client = {
|
||||
$queryRaw: async () => {
|
||||
@@ -202,22 +479,49 @@ describe("encryption key canary", () => {
|
||||
it("reports an error when no key is configured", async () => {
|
||||
vi.stubEnv("ENCRYPTION_KEY", "");
|
||||
_resetCryptoCacheForTests();
|
||||
const outcome = await checkEncryptionKeyCanaries(fakeClient({}).client);
|
||||
const outcome = await fakeClient({}).check();
|
||||
expect(outcome.state).toBe("error");
|
||||
});
|
||||
|
||||
it("names the key ids and never a key in the log block", () => {
|
||||
it("names the key ids, the reset statement, and never a key in the log block", () => {
|
||||
const block = keyMismatchLogBlock(["v1"]);
|
||||
expect(block).toContain("'v1'");
|
||||
expect(block).toContain("encryption.key_mismatch");
|
||||
expect(block).toContain(
|
||||
"DELETE FROM encryption_key_canaries WHERE key_id = 'v1';",
|
||||
);
|
||||
expect(block).not.toContain(KEY_A);
|
||||
});
|
||||
});
|
||||
|
||||
describe("the SQL sampler's column map", () => {
|
||||
it("resolves every registered column to a table, a column and an order", () => {
|
||||
// A throwaway client: never connects, only exposes the data model the
|
||||
// probe reads table and column names from.
|
||||
const client = new PrismaClient({
|
||||
adapter: new PrismaPg({
|
||||
connectionString: "postgres://x:y@127.0.0.1:1/x",
|
||||
}),
|
||||
});
|
||||
for (const column of ENCRYPTED_COLUMNS) {
|
||||
const loc = resolveColumnLocation(client, column);
|
||||
expect(loc.table, encryptedColumnKey(column)).toMatch(/^[a-z_0-9]+$/);
|
||||
expect(loc.column, encryptedColumnKey(column)).toMatch(/^[a-z_0-9]+$/);
|
||||
if (column.codecField) expect(loc.codec).not.toBeNull();
|
||||
}
|
||||
const backupPieces = ENCRYPTED_COLUMNS.find(
|
||||
(c) => c.model === "DataBackupChunk",
|
||||
)!;
|
||||
expect(resolveColumnLocation(client, backupPieces).ts).toBeNull();
|
||||
const user = ENCRYPTED_COLUMNS.find((c) => c.model === "User")!;
|
||||
expect(resolveColumnLocation(client, user).ts).toBe("updated_at");
|
||||
});
|
||||
});
|
||||
|
||||
describe("key fingerprint and copy check", () => {
|
||||
beforeEach(() => {
|
||||
vi.unstubAllEnvs();
|
||||
useKey(KEY_A);
|
||||
configureKey(KEY_A);
|
||||
});
|
||||
|
||||
it("is the first 12 hex of SHA-256 over the raw key bytes", () => {
|
||||
|
||||
+476
-109
@@ -12,44 +12,85 @@
|
||||
*
|
||||
* - row present: it must decrypt to the expected value, else the key is not
|
||||
* the one the row was written with;
|
||||
* - row absent: before writing one, sample existing ciphertext under that
|
||||
* key id from the registered string columns and try it. Without this
|
||||
* probe, the first boot after an upgrade (or a re-keyed boot over a
|
||||
* database that predates the canary) would seal the wrong key as the
|
||||
* right one.
|
||||
* - row absent: before writing one, probe the ciphertext already stored
|
||||
* under that key id. Without the probe, the first boot after an upgrade
|
||||
* (or a re-keyed boot over a database that predates the canary) would
|
||||
* seal the wrong key as the right one, and from then on refuse the
|
||||
* right key.
|
||||
*
|
||||
* The probe samples up to `PROBE_SAMPLE_LIMIT` values, one per column first
|
||||
* (different columns are different writers, so one bad writer cannot speak
|
||||
* for the whole database), then further values from columns that had any.
|
||||
* Its verdict, and what each one does:
|
||||
* ## The probe
|
||||
*
|
||||
* - `opens`: at least one value opened. A wrong key opens nothing, so the
|
||||
* key is right and any value that failed is a damaged or foreign row, not
|
||||
* a key problem. The canary is written.
|
||||
* Every registered column is asked, string and Bytes alike (`ENCRYPTED_COLUMNS`),
|
||||
* for its OLDEST values under the key id: the `encrypt()` string form
|
||||
* (`<keyId>.<base64>`, or a bare legacy base64 value for the legacy id `v1`),
|
||||
* the streamed form (`~hlgcm1.<keyId>.`), the same string stored as UTF-8 in a
|
||||
* Bytes column, and the binary layout (`0x02 <len> <keyId>`). The answer for a
|
||||
* column also tells whether it holds ANY value under the id, which is the
|
||||
* existence check below.
|
||||
*
|
||||
* Oldest means the row's `updatedAt`, else its `createdAt`, else (no timestamp
|
||||
* on the model, e.g. backup pieces) the primary key, ordered after every
|
||||
* timestamped row. `updatedAt` rather than `createdAt` because a value is
|
||||
* sealed when its row is written, and a row updated yesterday may carry a
|
||||
* value sealed yesterday on a row created years ago; `updatedAt` is a ceiling
|
||||
* on when the value was sealed. The point of the order: a process that served
|
||||
* under a wrong key without a canary (an inconclusive boot, a version that
|
||||
* predates this check) wrote rows that open under that wrong key. Those rows
|
||||
* are by construction newer than the data the database was built with, so
|
||||
* reading the oldest values first means they are never what decides.
|
||||
*
|
||||
* Up to `PROBE_ROWS_PER_COLUMN` values per column are fetched, the merged set
|
||||
* is ordered by that timestamp, and the oldest `PROBE_SAMPLE_LIMIT` are tried.
|
||||
* Values larger than `PROBE_VALUE_MAX_BYTES` (stored documents, large backups)
|
||||
* count as present but are not fetched. The verdict:
|
||||
*
|
||||
* - `opens`: the oldest value that opens is preceded by at most ONE older
|
||||
* value that does not. That one is a damaged or foreign row, not a key
|
||||
* problem: a wrong key opens nothing. Values newer than the first opening
|
||||
* one do not count against it; they are what a wrong-key process would
|
||||
* have left behind. The canary is written.
|
||||
* - `fails`: two or more values were tried and none opened. That is what a
|
||||
* wrong key looks like; the process refuses (mismatch).
|
||||
* - `inconclusive`: exactly one value was found and it did not open. One
|
||||
* row cannot tell a wrong key from one damaged row, and the two possible
|
||||
* mistakes are not equal: declaring a mismatch takes a correctly keyed
|
||||
* server down with 503 on every request, while serving keeps today's
|
||||
* behaviour. So the process serves, but writes NO canary either (that
|
||||
* would seal a key that may be wrong), logs a warning, and probes again
|
||||
* at the next boot, by which time more data may decide it.
|
||||
* - `none`: nothing found inside the time budget. The canary is written,
|
||||
* which is exactly the behaviour before this check existed.
|
||||
* - `none`: every column answered, and none holds a single value under the
|
||||
* key id. A genuinely fresh install (or a new id added for rotation); the
|
||||
* canary is written.
|
||||
* - `inconclusive`, with a reason: anything not proven. Only one value and
|
||||
* it did not open (`single-value`); two or more of the oldest values do
|
||||
* not open but a newer one does (`mixed`); values exist but none could be
|
||||
* fetched (`unsampled`); the time budget ran out or a column could not be
|
||||
* read (`incomplete`). The process serves, as before this check existed,
|
||||
* writes NO canary, logs a warning, and probes again at the next boot.
|
||||
* Serving rather than refusing because a refusal takes a correctly keyed
|
||||
* server down with 503 on every request; not sealing because that would
|
||||
* lock the right key out later.
|
||||
*
|
||||
* Writes are `ON CONFLICT DO NOTHING`, so two processes racing is fine.
|
||||
* The first seal can race another process. `ON CONFLICT DO NOTHING` keeps the
|
||||
* row the other process wrote; when this process's insert affected no row,
|
||||
* the stored canary is read back and must open under this process's key, or
|
||||
* the outcome is a mismatch.
|
||||
*
|
||||
* Never logs or returns key material. The canary is written by raw SQL and is
|
||||
* deliberately not in the rotation registry: it belongs to its key id.
|
||||
* deliberately not in the rotation registry: it belongs to its key id, and the
|
||||
* rotation script removes it once no ciphertext remains under that id
|
||||
* (`retireCanariesWithoutData`).
|
||||
*/
|
||||
import {
|
||||
decrypt,
|
||||
decryptBytes,
|
||||
decryptStream,
|
||||
encryptUnderKeyId,
|
||||
extractKeyId,
|
||||
extractKeyIdFromBytes,
|
||||
extractStreamKeyId,
|
||||
getActiveKeyId,
|
||||
getConfiguredKeyIds,
|
||||
isStreamCiphertext,
|
||||
} from "@/lib/crypto";
|
||||
import { ENCRYPTED_COLUMNS } from "@/lib/crypto/encrypted-columns";
|
||||
import {
|
||||
ENCRYPTED_COLUMNS,
|
||||
encryptedColumnKey,
|
||||
type EncryptedColumn,
|
||||
} from "@/lib/crypto/encrypted-columns";
|
||||
|
||||
export const CANARY_PREFIX = "healthlog-canary:";
|
||||
|
||||
@@ -57,12 +98,21 @@ export function canaryPlaintext(keyId: string): string {
|
||||
return `${CANARY_PREFIX}${keyId}`;
|
||||
}
|
||||
|
||||
/** How many existing values the probe tries at most per key id. */
|
||||
export const PROBE_SAMPLE_LIMIT = 5;
|
||||
/** How many of the oldest values the probe tries at most per key id. */
|
||||
export const PROBE_SAMPLE_LIMIT = 8;
|
||||
|
||||
/** How long the existing-data probe may take before the canary is written anyway. */
|
||||
/** How many of its oldest values each column contributes to the pool. */
|
||||
export const PROBE_ROWS_PER_COLUMN = 2;
|
||||
|
||||
/** Values larger than this count as present but are not fetched. */
|
||||
export const PROBE_VALUE_MAX_BYTES = 2 * 1024 * 1024;
|
||||
|
||||
/** How long the probe may take before it gives up as `inconclusive`. */
|
||||
export const PROBE_BUDGET_MS = 10_000;
|
||||
|
||||
/** The key id the legacy bare-base64 string format belongs to. */
|
||||
const LEGACY_KEY_ID = "v1";
|
||||
|
||||
/** The slice of a Prisma client this module needs. */
|
||||
export interface CanaryClient {
|
||||
$queryRaw<T = unknown>(
|
||||
@@ -73,31 +123,53 @@ export interface CanaryClient {
|
||||
query: TemplateStringsArray,
|
||||
...values: unknown[]
|
||||
): Promise<number>;
|
||||
$queryRawUnsafe?<T = unknown>(
|
||||
query: string,
|
||||
...values: unknown[]
|
||||
): Promise<T>;
|
||||
$executeRawUnsafe?(query: string, ...values: unknown[]): Promise<number>;
|
||||
}
|
||||
|
||||
type ProbeDelegate = {
|
||||
findMany: (args: {
|
||||
where: Record<string, unknown>;
|
||||
select: Record<string, boolean>;
|
||||
take: number;
|
||||
skip?: number;
|
||||
}) => Promise<Array<Record<string, unknown>>>;
|
||||
};
|
||||
/** One stored value under the key id, as the probe sees it. */
|
||||
export interface ProbeRow {
|
||||
/** The stored value; null when it exists but is too large to fetch. */
|
||||
value: string | Uint8Array | null;
|
||||
/** The row's codec column, for a codec-dispatched column. */
|
||||
codec?: string | null;
|
||||
/** The row's write time (see the module comment); null when it has none. */
|
||||
ts: Date | null;
|
||||
}
|
||||
|
||||
export type ProbeVerdict = "opens" | "fails" | "inconclusive" | "none";
|
||||
/**
|
||||
* Reads up to `limit` of the oldest values under `keyId` in one column, oldest
|
||||
* first. Throws when the column cannot be read; an empty array means the
|
||||
* column holds no value under that id.
|
||||
*/
|
||||
export type ColumnSampler = (
|
||||
column: EncryptedColumn,
|
||||
keyId: string,
|
||||
limit: number,
|
||||
) => Promise<ProbeRow[]>;
|
||||
|
||||
export type InconclusiveReason =
|
||||
"single-value" | "mixed" | "unsampled" | "incomplete";
|
||||
|
||||
export type ProbeResult =
|
||||
| { verdict: "opens" | "fails" | "none" }
|
||||
| { verdict: "inconclusive"; reason: InconclusiveReason };
|
||||
|
||||
export type KeyCheckOutcome =
|
||||
| {
|
||||
state: "ok";
|
||||
/** Key ids whose canary this run wrote. */
|
||||
written: string[];
|
||||
/** Key ids whose canary existed and opened. */
|
||||
/** Key ids whose canary existed (or another process wrote) and opened. */
|
||||
verified: string[];
|
||||
/**
|
||||
* Key ids with no canary whose only existing value did not open: not a
|
||||
* mismatch, no canary written, probed again at the next boot.
|
||||
* Key ids with no canary whose stored data proved nothing either way:
|
||||
* not a mismatch, no canary written, probed again at the next boot.
|
||||
*/
|
||||
inconclusive: string[];
|
||||
inconclusive: Array<{ keyId: string; reason: InconclusiveReason }>;
|
||||
}
|
||||
| { state: "mismatch"; keyIds: string[] }
|
||||
| { state: "error"; message: string };
|
||||
@@ -112,77 +184,304 @@ function opensAs(ciphertext: string, keyId: string, expected?: string) {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Sample existing values sealed under `keyId` and try them; see the module
|
||||
* comment for what each verdict means and why.
|
||||
*/
|
||||
export async function probeExistingData(
|
||||
client: unknown,
|
||||
function isBinaryLayout(column: EncryptedColumn, row: ProbeRow): boolean {
|
||||
if (column.codec === "binary2") return true;
|
||||
return column.codecField !== undefined && row.codec === "binary2";
|
||||
}
|
||||
|
||||
/** Does one stored value open under `keyId`? Never throws. */
|
||||
export function probeValueOpens(
|
||||
column: EncryptedColumn,
|
||||
row: ProbeRow,
|
||||
keyId: string,
|
||||
deadline: number,
|
||||
): Promise<ProbeVerdict> {
|
||||
const delegates = client as Record<string, ProbeDelegate | undefined>;
|
||||
const columns = ENCRYPTED_COLUMNS.filter(
|
||||
(c) => c.kind === "string" && !c.codec && !c.codecField,
|
||||
);
|
||||
let tried = 0;
|
||||
const columnsWithData: typeof columns = [];
|
||||
|
||||
const sample = async (
|
||||
column: (typeof columns)[number],
|
||||
skip: number,
|
||||
take: number,
|
||||
): Promise<string[]> => {
|
||||
const delegate =
|
||||
delegates[column.model.charAt(0).toLowerCase() + column.model.slice(1)];
|
||||
if (!delegate || typeof delegate.findMany !== "function") return [];
|
||||
try {
|
||||
const rows = await delegate.findMany({
|
||||
where: { [column.field]: { startsWith: `${keyId}.` } },
|
||||
select: { [column.field]: true },
|
||||
take,
|
||||
...(skip > 0 ? { skip } : {}),
|
||||
});
|
||||
return rows
|
||||
.map((row) => row[column.field])
|
||||
.filter((v): v is string => typeof v === "string");
|
||||
} catch {
|
||||
return [];
|
||||
): boolean {
|
||||
const value = row.value;
|
||||
if (value === null) return false;
|
||||
try {
|
||||
if (typeof value === "string") {
|
||||
if (isStreamCiphertext(value)) {
|
||||
if (extractStreamKeyId(value) !== keyId) return false;
|
||||
decryptStream(value);
|
||||
return true;
|
||||
}
|
||||
const id = extractKeyId(value);
|
||||
if (id === null) {
|
||||
// Legacy bare base64: only ever the legacy id's.
|
||||
if (keyId !== LEGACY_KEY_ID) return false;
|
||||
decrypt(value);
|
||||
return true;
|
||||
}
|
||||
return opensAs(value, keyId);
|
||||
}
|
||||
};
|
||||
|
||||
// Pass 1: one value per column, so different writers are heard first.
|
||||
for (const column of columns) {
|
||||
if (tried >= PROBE_SAMPLE_LIMIT || Date.now() > deadline) break;
|
||||
const values = await sample(column, 0, 1);
|
||||
if (values.length === 0) continue;
|
||||
columnsWithData.push(column);
|
||||
tried += 1;
|
||||
if (opensAs(values[0], keyId)) return "opens";
|
||||
}
|
||||
// Pass 2: more values from the columns that had any, up to the limit.
|
||||
for (const column of columnsWithData) {
|
||||
if (tried >= PROBE_SAMPLE_LIMIT || Date.now() > deadline) break;
|
||||
const values = await sample(column, 1, PROBE_SAMPLE_LIMIT - tried);
|
||||
for (const value of values) {
|
||||
tried += 1;
|
||||
if (opensAs(value, keyId)) return "opens";
|
||||
const buf = Buffer.from(value.buffer, value.byteOffset, value.byteLength);
|
||||
if (isBinaryLayout(column, row)) {
|
||||
if (extractKeyIdFromBytes(buf) !== keyId) return false;
|
||||
decryptBytes(buf, column.aad);
|
||||
return true;
|
||||
}
|
||||
return opensAs(buf.toString("utf8"), keyId);
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
|
||||
if (tried >= 2) return "fails";
|
||||
if (tried === 1) return "inconclusive";
|
||||
return "none";
|
||||
}
|
||||
|
||||
/**
|
||||
* Probe the stored values under `keyId`; see the module comment for what each
|
||||
* verdict means and why.
|
||||
*/
|
||||
export async function probeExistingData(
|
||||
sampler: ColumnSampler,
|
||||
keyId: string,
|
||||
deadline: number,
|
||||
columns: readonly EncryptedColumn[] = ENCRYPTED_COLUMNS,
|
||||
): Promise<ProbeResult> {
|
||||
const pool: Array<{ column: EncryptedColumn; row: ProbeRow; order: number }> =
|
||||
[];
|
||||
let present = 0;
|
||||
for (const column of columns) {
|
||||
if (Date.now() > deadline) {
|
||||
return { verdict: "inconclusive", reason: "incomplete" };
|
||||
}
|
||||
let rows: ProbeRow[];
|
||||
try {
|
||||
rows = await sampler(column, keyId, PROBE_ROWS_PER_COLUMN);
|
||||
} catch {
|
||||
return { verdict: "inconclusive", reason: "incomplete" };
|
||||
}
|
||||
present += rows.length;
|
||||
for (const row of rows) {
|
||||
if (row.value !== null) pool.push({ column, row, order: pool.length });
|
||||
}
|
||||
}
|
||||
if (Date.now() > deadline) {
|
||||
return { verdict: "inconclusive", reason: "incomplete" };
|
||||
}
|
||||
if (present === 0) return { verdict: "none" };
|
||||
if (pool.length === 0) {
|
||||
return { verdict: "inconclusive", reason: "unsampled" };
|
||||
}
|
||||
|
||||
// Oldest first; rows without a timestamp after every row with one.
|
||||
pool.sort((a, b) => {
|
||||
const ta = a.row.ts?.getTime() ?? Number.POSITIVE_INFINITY;
|
||||
const tb = b.row.ts?.getTime() ?? Number.POSITIVE_INFINITY;
|
||||
return ta === tb ? a.order - b.order : ta - tb;
|
||||
});
|
||||
const tried = pool.slice(0, PROBE_SAMPLE_LIMIT);
|
||||
|
||||
let failedBeforeFirstOpen = 0;
|
||||
for (const { column, row } of tried) {
|
||||
if (probeValueOpens(column, row, keyId)) {
|
||||
return failedBeforeFirstOpen <= 1
|
||||
? { verdict: "opens" }
|
||||
: { verdict: "inconclusive", reason: "mixed" };
|
||||
}
|
||||
failedBeforeFirstOpen += 1;
|
||||
}
|
||||
return tried.length >= 2
|
||||
? { verdict: "fails" }
|
||||
: { verdict: "inconclusive", reason: "single-value" };
|
||||
}
|
||||
|
||||
// ─── The SQL sampler ────────────────────────────────────────────────────────
|
||||
|
||||
interface ColumnLocation {
|
||||
table: string;
|
||||
column: string;
|
||||
pk: string;
|
||||
ts: string | null;
|
||||
codec: string | null;
|
||||
}
|
||||
|
||||
type RuntimeDataModel = {
|
||||
models: Record<
|
||||
string,
|
||||
{
|
||||
dbName: string | null;
|
||||
fields: Array<{ name: string; dbName?: string | null }>;
|
||||
}
|
||||
>;
|
||||
};
|
||||
|
||||
const IDENTIFIER = /^[A-Za-z_][A-Za-z0-9_]{0,62}$/;
|
||||
const KEY_ID = /^[A-Za-z0-9_-]{1,32}$/;
|
||||
|
||||
function quoteIdentifier(name: string): string {
|
||||
if (!IDENTIFIER.test(name)) {
|
||||
throw new Error(`Unexpected identifier in the column registry: ${name}`);
|
||||
}
|
||||
return `"${name}"`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Table and column names for a registry entry, read from the Prisma client's
|
||||
* runtime data model (the same `@@map` / `@map` names the client queries
|
||||
* with). `canary.test.ts` pins that every registry column resolves, so a
|
||||
* Prisma upgrade that moves this fails a test rather than the probe.
|
||||
*/
|
||||
export function resolveColumnLocation(
|
||||
client: unknown,
|
||||
column: EncryptedColumn,
|
||||
): ColumnLocation {
|
||||
const dm = (client as { _runtimeDataModel?: RuntimeDataModel })
|
||||
._runtimeDataModel;
|
||||
const model = dm?.models?.[column.model];
|
||||
if (!model) {
|
||||
throw new Error(`No data model for ${column.model}`);
|
||||
}
|
||||
const dbField = (name: string): string | null => {
|
||||
const f = model.fields.find((x) => x.name === name);
|
||||
return f ? (f.dbName ?? f.name) : null;
|
||||
};
|
||||
const columnName = dbField(column.field);
|
||||
const pk = dbField(column.pkField ?? "id");
|
||||
if (!columnName || !pk) {
|
||||
throw new Error(`No column for ${encryptedColumnKey(column)}`);
|
||||
}
|
||||
return {
|
||||
table: model.dbName ?? column.model,
|
||||
column: columnName,
|
||||
pk,
|
||||
ts: dbField("updatedAt") ?? dbField("createdAt"),
|
||||
codec: column.codecField ? dbField(column.codecField) : null,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* The SQL that finds values under `keyId` in one column. Identifiers come from
|
||||
* the Prisma data model and are checked against `IDENTIFIER` before they are
|
||||
* spliced; the key id is checked against the key-id grammar and travels as a
|
||||
* bound parameter. The prefix tests use `substr` / `substring`, which read
|
||||
* only the head of a large out-of-line value instead of the whole of it.
|
||||
*/
|
||||
function columnQuery(
|
||||
loc: ColumnLocation,
|
||||
column: EncryptedColumn,
|
||||
keyId: string,
|
||||
mode: "sample" | "exists",
|
||||
): { sql: string; params: unknown[] } {
|
||||
if (!KEY_ID.test(keyId)) throw new Error("Malformed key id");
|
||||
const c = quoteIdentifier(loc.column);
|
||||
const params: unknown[] = [];
|
||||
const bind = (v: unknown) => {
|
||||
params.push(v);
|
||||
return `$${params.length}`;
|
||||
};
|
||||
const tests: string[] = [];
|
||||
if (column.kind === "string") {
|
||||
const versioned = `${keyId}.`;
|
||||
const streamed = `~hlgcm1.${keyId}.`;
|
||||
tests.push(
|
||||
`substr(${c}, 1, ${versioned.length}) = ${bind(versioned)}`,
|
||||
`substr(${c}, 1, ${streamed.length}) = ${bind(streamed)}`,
|
||||
);
|
||||
if (keyId === LEGACY_KEY_ID) {
|
||||
// A legacy value is bare base64 of at least IV + tag + one block.
|
||||
tests.push(`substr(${c}, 1, 40) ~ '^[A-Za-z0-9+/]{40}$'`);
|
||||
}
|
||||
} else {
|
||||
const utf8 = Buffer.from(`${keyId}.`, "ascii");
|
||||
const id = Buffer.from(keyId, "ascii");
|
||||
const binary = Buffer.concat([Buffer.from([0x02, id.length]), id]);
|
||||
if (column.codec !== "binary2") {
|
||||
tests.push(
|
||||
`substring(${c} from 1 for ${utf8.length}) = ${bind(utf8)}::bytea`,
|
||||
);
|
||||
}
|
||||
if (column.codec === "binary2" || column.codecField) {
|
||||
tests.push(
|
||||
`substring(${c} from 1 for ${binary.length}) = ${bind(binary)}::bytea`,
|
||||
);
|
||||
}
|
||||
}
|
||||
const table = quoteIdentifier(loc.table);
|
||||
const where = `${c} IS NOT NULL AND (${tests.join(" OR ")})`;
|
||||
if (mode === "exists") {
|
||||
return {
|
||||
sql: `SELECT 1 AS one FROM ${table} WHERE ${where} LIMIT 1`,
|
||||
params,
|
||||
};
|
||||
}
|
||||
const ts = loc.ts ? quoteIdentifier(loc.ts) : null;
|
||||
const codec = loc.codec ? quoteIdentifier(loc.codec) : "NULL";
|
||||
const max = Math.floor(PROBE_VALUE_MAX_BYTES);
|
||||
const sql =
|
||||
`SELECT CASE WHEN octet_length(${c}) <= ${max} THEN ${c} END AS value, ` +
|
||||
`${codec} AS codec, ${ts ?? "NULL::timestamp"} AS ts ` +
|
||||
`FROM ${table} WHERE ${where} ` +
|
||||
`ORDER BY ${ts ? `${ts} ASC NULLS LAST, ` : ""}${quoteIdentifier(loc.pk)} ASC ` +
|
||||
`LIMIT ${bind(0)}`;
|
||||
return { sql, params };
|
||||
}
|
||||
|
||||
function requireUnsafe(client: CanaryClient) {
|
||||
const query = client.$queryRawUnsafe;
|
||||
if (typeof query !== "function") {
|
||||
throw new Error("The database client cannot run the probe queries");
|
||||
}
|
||||
return query.bind(client);
|
||||
}
|
||||
|
||||
/** The production sampler: one bounded query per column. */
|
||||
export function sqlColumnSampler(client: CanaryClient): ColumnSampler {
|
||||
const query = requireUnsafe(client);
|
||||
return async (column, keyId, limit) => {
|
||||
const loc = resolveColumnLocation(client, column);
|
||||
const { sql, params } = columnQuery(loc, column, keyId, "sample");
|
||||
params[params.length - 1] = limit;
|
||||
const rows = await query<
|
||||
Array<{
|
||||
value: string | Uint8Array | null;
|
||||
codec: string | null;
|
||||
ts: Date | null;
|
||||
}>
|
||||
>(sql, ...params);
|
||||
return rows.map((r) => ({
|
||||
value: r.value,
|
||||
codec: r.codec,
|
||||
ts: r.ts instanceof Date ? r.ts : r.ts ? new Date(r.ts) : null,
|
||||
}));
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Every registered column that still holds a value under `keyId`, by
|
||||
* `Model.field`. Exhaustive: one existence query per column, no budget.
|
||||
*/
|
||||
export async function columnsHoldingKeyId(
|
||||
client: CanaryClient,
|
||||
keyId: string,
|
||||
): Promise<string[]> {
|
||||
const query = requireUnsafe(client);
|
||||
const holding: string[] = [];
|
||||
for (const column of ENCRYPTED_COLUMNS) {
|
||||
const loc = resolveColumnLocation(client, column);
|
||||
const { sql, params } = columnQuery(loc, column, keyId, "exists");
|
||||
const rows = await query<unknown[]>(sql, ...params);
|
||||
if (rows.length > 0) holding.push(encryptedColumnKey(column));
|
||||
}
|
||||
return holding;
|
||||
}
|
||||
|
||||
// ─── The check ──────────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Run the check for every configured key id. Never throws: a failure to run
|
||||
* the check at all is reported as `error`, and only a value that provably does
|
||||
* not open is reported as `mismatch`.
|
||||
* the check at all is reported as `error`, and only values that provably do
|
||||
* not open are reported as `mismatch`.
|
||||
*/
|
||||
export async function checkEncryptionKeyCanaries(
|
||||
client: CanaryClient,
|
||||
options: { probeBudgetMs?: number } = {},
|
||||
options: {
|
||||
probeBudgetMs?: number;
|
||||
sampler?: ColumnSampler;
|
||||
/**
|
||||
* `warn` (`ENCRYPTION_KEY_CHECK=warn`): record nothing unless every key
|
||||
* id passed. The process serves whatever the outcome, so a key that is
|
||||
* not proven for every id must not be recorded beside it.
|
||||
*/
|
||||
mode?: "enforce" | "warn";
|
||||
} = {},
|
||||
): Promise<KeyCheckOutcome> {
|
||||
let keyIds: string[];
|
||||
try {
|
||||
@@ -217,12 +516,21 @@ export async function checkEncryptionKeyCanaries(
|
||||
|
||||
const deadline = Date.now() + (options.probeBudgetMs ?? PROBE_BUDGET_MS);
|
||||
const toWrite: string[] = [];
|
||||
const inconclusive: string[] = [];
|
||||
for (const keyId of missing) {
|
||||
const probe = await probeExistingData(client, keyId, deadline);
|
||||
if (probe === "fails") mismatched.push(keyId);
|
||||
else if (probe === "inconclusive") inconclusive.push(keyId);
|
||||
else toWrite.push(keyId);
|
||||
const inconclusive: Array<{ keyId: string; reason: InconclusiveReason }> = [];
|
||||
if (missing.length > 0) {
|
||||
let sampler: ColumnSampler;
|
||||
try {
|
||||
sampler = options.sampler ?? sqlColumnSampler(client);
|
||||
} catch (err) {
|
||||
return { state: "error", message: (err as Error).message };
|
||||
}
|
||||
for (const keyId of missing) {
|
||||
const probe = await probeExistingData(sampler, keyId, deadline);
|
||||
if (probe.verdict === "fails") mismatched.push(keyId);
|
||||
else if (probe.verdict === "inconclusive") {
|
||||
inconclusive.push({ keyId, reason: probe.reason });
|
||||
} else toWrite.push(keyId);
|
||||
}
|
||||
}
|
||||
|
||||
if (mismatched.length > 0) {
|
||||
@@ -231,16 +539,63 @@ export async function checkEncryptionKeyCanaries(
|
||||
return { state: "mismatch", keyIds: mismatched.sort() };
|
||||
}
|
||||
|
||||
const written: string[] = [];
|
||||
const sealable =
|
||||
options.mode === "warn" && inconclusive.length > 0 ? [] : toWrite;
|
||||
try {
|
||||
for (const keyId of toWrite) {
|
||||
for (const keyId of sealable) {
|
||||
const ciphertext = encryptUnderKeyId(canaryPlaintext(keyId), keyId);
|
||||
await client.$executeRaw`INSERT INTO encryption_key_canaries (key_id, ciphertext) VALUES (${keyId}, ${ciphertext}) ON CONFLICT (key_id) DO NOTHING`;
|
||||
const inserted =
|
||||
await client.$executeRaw`INSERT INTO encryption_key_canaries (key_id, ciphertext) VALUES (${keyId}, ${ciphertext}) ON CONFLICT (key_id) DO NOTHING`;
|
||||
if (inserted > 0) {
|
||||
written.push(keyId);
|
||||
continue;
|
||||
}
|
||||
// Another process sealed this id first. Its canary has to open under
|
||||
// this process's key, or one of the two holds the wrong key.
|
||||
const [winner] = await client.$queryRaw<Array<{ ciphertext: string }>>`
|
||||
SELECT ciphertext FROM encryption_key_canaries WHERE key_id = ${keyId}`;
|
||||
if (winner && opensAs(winner.ciphertext, keyId, canaryPlaintext(keyId))) {
|
||||
verified.push(keyId);
|
||||
} else {
|
||||
mismatched.push(keyId);
|
||||
}
|
||||
}
|
||||
} catch (err) {
|
||||
return { state: "error", message: (err as Error).message };
|
||||
}
|
||||
if (mismatched.length > 0) {
|
||||
return { state: "mismatch", keyIds: mismatched.sort() };
|
||||
}
|
||||
|
||||
return { state: "ok", written: toWrite, verified, inconclusive };
|
||||
return { state: "ok", written, verified, inconclusive };
|
||||
}
|
||||
|
||||
/**
|
||||
* After a rotation: remove the canary of every configured key id other than
|
||||
* the active one that no longer holds a single value in any registered
|
||||
* column. The id's key can then leave `ENCRYPTION_KEYS`; and if the id is
|
||||
* ever reused for a different key, no stale canary refuses it.
|
||||
*/
|
||||
export async function retireCanariesWithoutData(client: CanaryClient): Promise<{
|
||||
removed: string[];
|
||||
remaining: Record<string, string[]>;
|
||||
}> {
|
||||
const active = getActiveKeyId();
|
||||
const removed: string[] = [];
|
||||
const remaining: Record<string, string[]> = {};
|
||||
for (const keyId of getConfiguredKeyIds()) {
|
||||
if (keyId === active) continue;
|
||||
const holding = await columnsHoldingKeyId(client, keyId);
|
||||
if (holding.length > 0) {
|
||||
remaining[keyId] = holding;
|
||||
continue;
|
||||
}
|
||||
const deleted =
|
||||
await client.$executeRaw`DELETE FROM encryption_key_canaries WHERE key_id = ${keyId}`;
|
||||
if (deleted > 0) removed.push(keyId);
|
||||
}
|
||||
return { removed, remaining };
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -266,6 +621,18 @@ export function keyMismatchLogBlock(keyIds: string[]): string {
|
||||
" 3. Start fresh: an empty database with this key. Stored data and",
|
||||
" in-database backups of the old database cannot be read without",
|
||||
" the old key.",
|
||||
"",
|
||||
" Only if you are CERTAIN the configured key is the one this data was",
|
||||
" written with (the check recorded a different key earlier), reset the",
|
||||
" check and restart; it then probes the stored data again:",
|
||||
...keyIds.map(
|
||||
(id) => ` DELETE FROM encryption_key_canaries WHERE key_id = '${id}';`,
|
||||
),
|
||||
"",
|
||||
" Last resort, when no record exists yet to reset (a first start) and you",
|
||||
" are certain the key is right: ENCRYPTION_KEY_CHECK=warn serves anyway",
|
||||
" and only logs. It disables this safety check; with a wrong key every",
|
||||
" encrypted value then fails and new rows are written under that key.",
|
||||
"============================================================",
|
||||
].join("\n");
|
||||
}
|
||||
@@ -32,6 +32,10 @@ import {
|
||||
type CorpusClient,
|
||||
} from "@/lib/crypto/encryption-corpus";
|
||||
import { getWorkerPrisma } from "@/lib/jobs/reminder/shared";
|
||||
import {
|
||||
retireCanariesWithoutData,
|
||||
type CanaryClient,
|
||||
} from "@/lib/crypto/canary";
|
||||
|
||||
export const ENCRYPTION_KEY_ROTATE_QUEUE = "encryption-key-rotate";
|
||||
export const ENCRYPTION_KEY_ROTATE_CONCURRENCY = 1;
|
||||
@@ -62,9 +66,20 @@ export async function runEncryptionKeyRotation(
|
||||
totalErrors: number;
|
||||
totalDropped: number;
|
||||
stoppedEarly: boolean;
|
||||
/** Key ids whose boot key-check record this run removed. */
|
||||
retiredKeyIds: string[];
|
||||
}> {
|
||||
const prisma = getWorkerPrisma();
|
||||
const out = await rotateCorpus(prisma as unknown as CorpusClient, shouldStop);
|
||||
// A complete, error-free pass: remove the boot key check's record of every
|
||||
// other key id that no longer holds a value in any registered column, so
|
||||
// its key can leave ENCRYPTION_KEYS and the id can never refuse a later,
|
||||
// different key. The existence check walks every column itself.
|
||||
const retiredKeyIds =
|
||||
!out.stoppedEarly && out.totalErrors === 0
|
||||
? (await retireCanariesWithoutData(prisma as unknown as CanaryClient))
|
||||
.removed
|
||||
: [];
|
||||
return {
|
||||
activeKeyId: out.activeKeyId,
|
||||
totalScanned: out.totalScanned,
|
||||
@@ -72,6 +87,7 @@ export async function runEncryptionKeyRotation(
|
||||
totalErrors: out.totalErrors,
|
||||
totalDropped: out.totalDropped,
|
||||
stoppedEarly: out.stoppedEarly,
|
||||
retiredKeyIds,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -98,6 +114,7 @@ export async function handleEncryptionKeyRotate(
|
||||
rotated: result.totalRotated,
|
||||
errors: result.totalErrors,
|
||||
dropped: result.totalDropped,
|
||||
retiredKeyIds: result.retiredKeyIds,
|
||||
},
|
||||
});
|
||||
// Per-row errors are the fail-closed skip of a row under a key the
|
||||
|
||||
@@ -47,12 +47,22 @@ vi.mock("@/lib/db-compat", () => ({
|
||||
import {
|
||||
_resetCryptoCacheForTests,
|
||||
encrypt,
|
||||
encryptUnderKeyId,
|
||||
getKeyFingerprint,
|
||||
} from "@/lib/crypto";
|
||||
import { encryptToBytes } from "@/lib/ai/coach/bytes-codec";
|
||||
import { runBootKeyCheck } from "@/lib/boot/key-check";
|
||||
import {
|
||||
canaryPlaintext,
|
||||
checkEncryptionKeyCanaries,
|
||||
retireCanariesWithoutData,
|
||||
type CanaryClient,
|
||||
} from "@/lib/crypto/canary";
|
||||
import {
|
||||
getKeyMismatchWarning,
|
||||
isKeyMismatch,
|
||||
setKeyMismatchState,
|
||||
setKeyMismatchWarning,
|
||||
} from "@/lib/boot/key-mismatch-state";
|
||||
|
||||
const KEY_A =
|
||||
@@ -223,6 +233,369 @@ describe("boot key check probe over existing data (real Postgres)", () => {
|
||||
});
|
||||
});
|
||||
|
||||
/** Two users, one oldest, with ciphertext in string AND Bytes columns. */
|
||||
async function seedUser(
|
||||
name: string,
|
||||
writtenAt: Date,
|
||||
seal: () => { text: string; bytes: Uint8Array<ArrayBuffer> },
|
||||
) {
|
||||
const prisma = getPrismaClient();
|
||||
const a = seal();
|
||||
const b = seal();
|
||||
const user = await prisma.user.create({
|
||||
data: {
|
||||
username: name,
|
||||
email: `${name}@example.test`,
|
||||
codexAccessTokenEncrypted: a.text,
|
||||
codexRefreshTokenEncrypted: b.text,
|
||||
createdAt: writtenAt,
|
||||
updatedAt: writtenAt,
|
||||
},
|
||||
});
|
||||
await prisma.userHealthProfile.create({
|
||||
data: {
|
||||
userId: user.id,
|
||||
aboutMeEncrypted: a.bytes,
|
||||
conditionsEncrypted: b.bytes,
|
||||
createdAt: writtenAt,
|
||||
updatedAt: writtenAt,
|
||||
},
|
||||
});
|
||||
return user.id;
|
||||
}
|
||||
|
||||
const sealNow = () => ({
|
||||
text: encrypt("a stored value"),
|
||||
bytes: encryptToBytes("a stored note"),
|
||||
});
|
||||
|
||||
describe("boot key check: what decides, and what may seal (real Postgres)", () => {
|
||||
it("probes Bytes columns: an install whose ciphertext is only Bytes does not seal a wrong key", async () => {
|
||||
const prisma = getPrismaClient();
|
||||
const user = await prisma.user.create({
|
||||
data: { username: "bytes-only", email: "bytes-only@example.test" },
|
||||
});
|
||||
await prisma.userHealthProfile.create({
|
||||
data: {
|
||||
userId: user.id,
|
||||
aboutMeEncrypted: encryptToBytes("about me"),
|
||||
conditionsEncrypted: encryptToBytes("conditions"),
|
||||
allergiesEncrypted: encryptToBytes("allergies"),
|
||||
},
|
||||
});
|
||||
|
||||
useKey(KEY_B);
|
||||
const errors = vi.spyOn(console, "error").mockImplementation(() => {});
|
||||
expect(await runBootKeyCheck(prisma)).toBe(true);
|
||||
errors.mockRestore();
|
||||
expect(await prisma.encryptionKeyCanary.count()).toBe(0);
|
||||
|
||||
useKey(KEY_A);
|
||||
expect(await runBootKeyCheck(prisma)).toBe(false);
|
||||
expect(await prisma.encryptionKeyCanary.count()).toBe(1);
|
||||
});
|
||||
|
||||
it("a probe that runs out of time seals nothing and keeps serving", async () => {
|
||||
const prisma = getPrismaClient();
|
||||
await seedUser("slow", new Date("2024-01-01T00:00:00Z"), sealNow);
|
||||
const outcome = await checkEncryptionKeyCanaries(prisma, {
|
||||
probeBudgetMs: -1,
|
||||
});
|
||||
expect(outcome.state).toBe("ok");
|
||||
if (outcome.state !== "ok") return;
|
||||
expect(outcome.written).toEqual([]);
|
||||
expect(outcome.inconclusive.map((i) => i.keyId)).toEqual(["v1"]);
|
||||
expect(await prisma.encryptionKeyCanary.count()).toBe(0);
|
||||
});
|
||||
|
||||
it("a fresh database with no ciphertext under the key id seals it", async () => {
|
||||
const prisma = getPrismaClient();
|
||||
// Ciphertext exists, but under another key id: not evidence about v1.
|
||||
vi.stubEnv("ENCRYPTION_KEYS", JSON.stringify({ v1: KEY_A, v2: KEY_B }));
|
||||
vi.stubEnv("ENCRYPTION_ACTIVE_KEY_ID", "v2");
|
||||
vi.stubEnv("ENCRYPTION_KEY", "");
|
||||
_resetCryptoCacheForTests();
|
||||
await seedUser("v2-only", new Date("2024-01-01T00:00:00Z"), sealNow);
|
||||
expect(await runBootKeyCheck(prisma)).toBe(false);
|
||||
const ids = (await prisma.encryptionKeyCanary.findMany())
|
||||
.map((r) => r.keyId)
|
||||
.sort();
|
||||
expect(ids).toEqual(["v1", "v2"]);
|
||||
});
|
||||
|
||||
it("rows a wrong key wrote recently never decide: old rows under A, newer under B", async () => {
|
||||
const prisma = getPrismaClient();
|
||||
// The original data, under A.
|
||||
await seedUser("original", new Date("2024-01-01T00:00:00Z"), sealNow);
|
||||
await seedUser("original-2", new Date("2024-02-01T00:00:00Z"), sealNow);
|
||||
// A process holding B served for a while (no canary yet) and wrote more.
|
||||
useKey(KEY_B);
|
||||
for (let i = 0; i < 6; i++) {
|
||||
await seedUser(
|
||||
`recent-${i}`,
|
||||
new Date(Date.UTC(2026, 8, 1 + i)),
|
||||
sealNow,
|
||||
);
|
||||
}
|
||||
|
||||
const warn = vi.spyOn(console, "warn").mockImplementation(() => {});
|
||||
const errors = vi.spyOn(console, "error").mockImplementation(() => {});
|
||||
await runBootKeyCheck(prisma);
|
||||
warn.mockRestore();
|
||||
errors.mockRestore();
|
||||
// B is not sealed as the right key, whatever else the verdict is.
|
||||
expect(await prisma.encryptionKeyCanary.count()).toBe(0);
|
||||
const outcome = await checkEncryptionKeyCanaries(prisma);
|
||||
expect(
|
||||
outcome.state === "mismatch" ||
|
||||
(outcome.state === "ok" && outcome.written.length === 0),
|
||||
).toBe(true);
|
||||
|
||||
// The key the oldest data was written with is the right one.
|
||||
useKey(KEY_A);
|
||||
setKeyMismatchState(null);
|
||||
expect(await runBootKeyCheck(prisma)).toBe(false);
|
||||
expect(isKeyMismatch()).toBe(false);
|
||||
expect(await prisma.encryptionKeyCanary.count()).toBe(1);
|
||||
});
|
||||
|
||||
it("losing the first-seal race to another key is a mismatch, not ok", async () => {
|
||||
const prisma = getPrismaClient();
|
||||
// The other process, holding A, wins the insert between our probe and
|
||||
// our write.
|
||||
const sealedUnderA = encryptUnderKeyId(canaryPlaintext("v1"), "v1");
|
||||
useKey(KEY_B);
|
||||
let raced = false;
|
||||
const racing = new Proxy(prisma, {
|
||||
get(target, prop, receiver) {
|
||||
if (prop === "$executeRaw") {
|
||||
return async (q: TemplateStringsArray, ...values: unknown[]) => {
|
||||
if (!raced) {
|
||||
raced = true;
|
||||
await target.$executeRaw`INSERT INTO encryption_key_canaries (key_id, ciphertext) VALUES ('v1', ${sealedUnderA})`;
|
||||
}
|
||||
return target.$executeRaw(q, ...values);
|
||||
};
|
||||
}
|
||||
const value = Reflect.get(target, prop, receiver);
|
||||
return typeof value === "function" ? value.bind(target) : value;
|
||||
},
|
||||
}) as unknown as CanaryClient;
|
||||
const outcome = await checkEncryptionKeyCanaries(racing);
|
||||
expect(raced).toBe(true);
|
||||
expect(outcome).toEqual({ state: "mismatch", keyIds: ["v1"] });
|
||||
|
||||
// The same race lost to a process holding the SAME key is fine.
|
||||
await prisma.$executeRaw`DELETE FROM encryption_key_canaries`;
|
||||
raced = false;
|
||||
useKey(KEY_A);
|
||||
const same = await checkEncryptionKeyCanaries(racing);
|
||||
expect(same.state).toBe("ok");
|
||||
});
|
||||
});
|
||||
|
||||
describe("ENCRYPTION_KEY_CHECK (real Postgres)", () => {
|
||||
afterEach(() => setKeyMismatchWarning(null));
|
||||
|
||||
it("warn: a mismatch is logged and reported, never refused, never recorded", async () => {
|
||||
const prisma = getPrismaClient();
|
||||
await seedUser("warned", new Date("2024-01-01T00:00:00Z"), sealNow);
|
||||
useKey(KEY_B);
|
||||
vi.stubEnv("ENCRYPTION_KEY_CHECK", "warn");
|
||||
const errors = vi.spyOn(console, "error").mockImplementation(() => {});
|
||||
expect(await runBootKeyCheck(prisma)).toBe(false);
|
||||
const logged = errors.mock.calls.map((c) => String(c[0])).join("\n");
|
||||
errors.mockRestore();
|
||||
expect(logged).toContain("encryption.key_mismatch");
|
||||
expect(logged).toContain("ENCRYPTION_KEY_CHECK=warn");
|
||||
expect(isKeyMismatch()).toBe(false);
|
||||
expect(getKeyMismatchWarning()?.keyIds).toEqual(["v1"]);
|
||||
expect(await prisma.encryptionKeyCanary.count()).toBe(0);
|
||||
|
||||
// Requests are served, through apiHandler and outside it.
|
||||
await seedAdmin();
|
||||
const { GET: keyBackupGet } =
|
||||
await import("@/app/api/admin/encryption/key-backup/route");
|
||||
const served = await asRoute(keyBackupGet)(
|
||||
new NextRequest("http://localhost/api/admin/encryption/key-backup"),
|
||||
);
|
||||
expect(served.status).toBe(200);
|
||||
const { POST: mcp } = await import("@/app/mcp/route");
|
||||
const mcpRes = await mcp(
|
||||
new Request("http://localhost/mcp", { method: "POST", body: "{}" }),
|
||||
);
|
||||
expect(mcpRes.status).not.toBe(503);
|
||||
|
||||
const { GET: healthGet } = await import("@/app/api/health/route");
|
||||
const healthBody = await (
|
||||
await asRoute(healthGet)(new NextRequest("http://localhost/api/health"))
|
||||
).json();
|
||||
expect(healthBody.warning).toBe("encryption_key_mismatch");
|
||||
expect(healthBody.reason).toBeUndefined();
|
||||
});
|
||||
|
||||
it("warn: nothing is recorded while any key id is inconclusive", async () => {
|
||||
const prisma = getPrismaClient();
|
||||
vi.stubEnv("ENCRYPTION_KEYS", JSON.stringify({ v1: KEY_A, v2: KEY_B }));
|
||||
vi.stubEnv("ENCRYPTION_ACTIVE_KEY_ID", "v1");
|
||||
vi.stubEnv("ENCRYPTION_KEY", "");
|
||||
_resetCryptoCacheForTests();
|
||||
await prisma.user.create({
|
||||
data: {
|
||||
username: "lone",
|
||||
email: "lone@example.test",
|
||||
codexAccessTokenEncrypted: encrypt("the only value"),
|
||||
},
|
||||
});
|
||||
// v1 now holds a different key: its one value does not open. v2 has no
|
||||
// data, which on its own would be recorded.
|
||||
vi.stubEnv("ENCRYPTION_KEYS", JSON.stringify({ v1: KEY_B, v2: KEY_A }));
|
||||
_resetCryptoCacheForTests();
|
||||
vi.stubEnv("ENCRYPTION_KEY_CHECK", "warn");
|
||||
const warn = vi.spyOn(console, "warn").mockImplementation(() => {});
|
||||
expect(await runBootKeyCheck(prisma)).toBe(false);
|
||||
warn.mockRestore();
|
||||
expect(await prisma.encryptionKeyCanary.count()).toBe(0);
|
||||
|
||||
// Enforce records the proven id, as before.
|
||||
vi.stubEnv("ENCRYPTION_KEY_CHECK", "enforce");
|
||||
const warn2 = vi.spyOn(console, "warn").mockImplementation(() => {});
|
||||
expect(await runBootKeyCheck(prisma)).toBe(false);
|
||||
warn2.mockRestore();
|
||||
const ids = (await prisma.encryptionKeyCanary.findMany()).map(
|
||||
(r) => r.keyId,
|
||||
);
|
||||
expect(ids).toEqual(["v2"]);
|
||||
});
|
||||
|
||||
it("enforce, set explicitly or by any other value, refuses as before", async () => {
|
||||
const prisma = getPrismaClient();
|
||||
await seedUser("enforced", new Date("2024-01-01T00:00:00Z"), sealNow);
|
||||
useKey(KEY_B);
|
||||
for (const value of ["enforce", "", "off"]) {
|
||||
vi.stubEnv("ENCRYPTION_KEY_CHECK", value);
|
||||
setKeyMismatchState(null);
|
||||
const errors = vi.spyOn(console, "error").mockImplementation(() => {});
|
||||
expect(await runBootKeyCheck(prisma)).toBe(true);
|
||||
errors.mockRestore();
|
||||
expect(isKeyMismatch()).toBe(true);
|
||||
expect(getKeyMismatchWarning()).toBeNull();
|
||||
}
|
||||
expect(await prisma.encryptionKeyCanary.count()).toBe(0);
|
||||
});
|
||||
});
|
||||
|
||||
describe("retiring a key id after rotation (real Postgres)", () => {
|
||||
it("removes the canary of a key id only once no ciphertext remains under it", async () => {
|
||||
const prisma = getPrismaClient();
|
||||
vi.stubEnv("ENCRYPTION_KEYS", JSON.stringify({ v1: KEY_A, v2: KEY_B }));
|
||||
vi.stubEnv("ENCRYPTION_ACTIVE_KEY_ID", "v1");
|
||||
vi.stubEnv("ENCRYPTION_KEY", "");
|
||||
_resetCryptoCacheForTests();
|
||||
const userId = await seedUser(
|
||||
"rotating",
|
||||
new Date("2024-01-01T00:00:00Z"),
|
||||
sealNow,
|
||||
);
|
||||
expect(await runBootKeyCheck(prisma)).toBe(false);
|
||||
expect(await prisma.encryptionKeyCanary.count()).toBe(2);
|
||||
|
||||
// Rotation to v2 has not finished: one Bytes value still under v1.
|
||||
vi.stubEnv("ENCRYPTION_ACTIVE_KEY_ID", "v2");
|
||||
_resetCryptoCacheForTests();
|
||||
await prisma.user.update({
|
||||
where: { id: userId },
|
||||
data: {
|
||||
codexAccessTokenEncrypted: encrypt("rotated"),
|
||||
codexRefreshTokenEncrypted: encrypt("rotated"),
|
||||
},
|
||||
});
|
||||
await prisma.userHealthProfile.update({
|
||||
where: { userId },
|
||||
data: { aboutMeEncrypted: encryptToBytes("rotated") },
|
||||
});
|
||||
const partial = await retireCanariesWithoutData(prisma);
|
||||
expect(partial.removed).toEqual([]);
|
||||
expect(partial.remaining).toEqual({
|
||||
v1: ["UserHealthProfile.conditionsEncrypted"],
|
||||
});
|
||||
expect(await prisma.encryptionKeyCanary.count()).toBe(2);
|
||||
|
||||
await prisma.userHealthProfile.update({
|
||||
where: { userId },
|
||||
data: { conditionsEncrypted: encryptToBytes("rotated") },
|
||||
});
|
||||
const done = await retireCanariesWithoutData(prisma);
|
||||
expect(done.removed).toEqual(["v1"]);
|
||||
const left = await prisma.encryptionKeyCanary.findMany();
|
||||
expect(left.map((r) => r.keyId)).toEqual(["v2"]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("routes outside apiHandler refuse while the key does not match", () => {
|
||||
beforeEach(() => {
|
||||
setKeyMismatchState({ keyIds: ["v1"], detectedAt: "2026-10-03T00:00:00Z" });
|
||||
});
|
||||
|
||||
it("/mcp answers 503 encryption.key_mismatch before auth", async () => {
|
||||
const { POST } = await import("@/app/mcp/route");
|
||||
const res = await POST(
|
||||
new Request("http://localhost/mcp", {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"content-type": "application/json",
|
||||
authorization: "Bearer hlk_whatever",
|
||||
},
|
||||
body: JSON.stringify({ jsonrpc: "2.0", id: 1, method: "tools/list" }),
|
||||
}),
|
||||
);
|
||||
expect(res.status).toBe(503);
|
||||
expect((await res.json()).meta).toEqual({
|
||||
errorCode: "encryption.key_mismatch",
|
||||
});
|
||||
});
|
||||
|
||||
it("the MCP OAuth endpoints refuse in their RFC shape", async () => {
|
||||
const { POST: token } = await import("@/app/api/mcp/oauth/token/route");
|
||||
const { POST: register } =
|
||||
await import("@/app/api/mcp/oauth/register/route");
|
||||
const { GET: authorizeGet, POST: authorizePost } =
|
||||
await import("@/app/api/mcp/oauth/authorize/route");
|
||||
const responses = [
|
||||
await token(
|
||||
new NextRequest("http://localhost/api/mcp/oauth/token", {
|
||||
method: "POST",
|
||||
headers: { "content-type": "application/x-www-form-urlencoded" },
|
||||
body: "grant_type=authorization_code&code=x",
|
||||
}),
|
||||
),
|
||||
await register(
|
||||
post("/api/mcp/oauth/register", {
|
||||
redirect_uris: ["https://client.example/cb"],
|
||||
}),
|
||||
),
|
||||
await authorizeGet(
|
||||
new NextRequest(
|
||||
"http://localhost/api/mcp/oauth/authorize?response_type=code",
|
||||
),
|
||||
),
|
||||
await authorizePost(
|
||||
new NextRequest("http://localhost/api/mcp/oauth/authorize", {
|
||||
method: "POST",
|
||||
headers: { "content-type": "application/x-www-form-urlencoded" },
|
||||
body: "decision=approve",
|
||||
}),
|
||||
),
|
||||
];
|
||||
for (const res of responses) {
|
||||
expect(res.status).toBe(503);
|
||||
const body = await res.json();
|
||||
expect(body.error).toBe("temporarily_unavailable");
|
||||
expect(body.error_description).toContain("encryption key");
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe("encryption key backup step (real Postgres)", () => {
|
||||
it("is due, refuses a stale confirmation, records the right one, and is due again after a re-key", async () => {
|
||||
const prisma = getPrismaClient();
|
||||
|
||||
Reference in new issue
Block a user