mirror of
https://github.com/MBombeck/HealthLog.git
synced 2026-11-15 23:12:04 +00:00
feat(admin): replace the key backup step with a recovery kit
The step confirmed one key, the active application key. A restore needs more: off-host copies are sealed under BACKUP_ENCRYPTION_KEY, a copy written before a rotation needs the retired application key it was written under, and every session, token and share link hangs off API_TOKEN_HMAC_KEY. An operator could do everything the step asked and still be unable to read a single backup. The recovery kit lists every key a restore of this server needs, by id and fingerprint, never key material: - each id in ENCRYPTION_KEYS, plus any retired id a stored copy (innerKeyIds) or an off-host object inside the bucket's retention (OffhostBackupKeyUse) still lists, also once the server dropped it; - BACKUP_ENCRYPTION_KEY and each BACKUP_ENCRYPTION_PREVIOUS_KEYS entry, named by the id their objects record; - API_TOKEN_HMAC_KEY. Confirming records the set the admin was shown and refuses a stale one with 409 recovery.kit.stale. The kit is due again as soon as a key it lists is not in that set (a rotation, a new backup key, a re-keyed install); a key that merely drops out does not make it due. "Check my copy" now checks any key in the kit. Migration 0487 carries a confirmation given under the old step forward as a kit holding that key, so that part stays confirmed, and drops the four app_settings columns. It also creates the two tables the readiness evaluation uses (restore_drill_outcomes, recovery_readiness_notices). The routes move to /api/admin/recovery-kit; the reader of the keys the backups still need moves out of the encryption status route into src/lib/recovery/backup-key-needs.ts so the kit and the status view share one definition.
This commit is contained in:
1 parent
953c64f7a4
commit
d3c672cc3f
56 files changed
+3042
-1611
No files matched your search
@@ -0,0 +1,101 @@
|
||||
-- Recovery readiness: what the admins were last told, what the restore drill
|
||||
-- last found, and the recovery kit confirmation.
|
||||
--
|
||||
-- `recovery_readiness_notices` holds, per readiness signal, the state the
|
||||
-- admins were last notified about and when. It is the whole memory of the
|
||||
-- hourly evaluation: a state change notifies once, a failing signal is
|
||||
-- reminded at most once a day, and a recovery is announced once. It also
|
||||
-- remembers since when a signal has applied, so a dump switched on by an
|
||||
-- upgrade is not called failing before its first run had a chance.
|
||||
--
|
||||
-- `restore_drill_outcomes` keeps the monthly drill's last result for as long
|
||||
-- as it is the last one. pg-boss forgets a failed run after seven days and
|
||||
-- the admin failing-jobs card looks back 72 hours, so a failed monthly drill
|
||||
-- was on screen three days in thirty.
|
||||
--
|
||||
-- `recovery_kit_confirmations` replaces the four `encryption_key_backup_*`
|
||||
-- columns on `app_settings`. Those confirmed one key, the active application
|
||||
-- key; the kit confirms the set of every key a restore needs, by id and
|
||||
-- fingerprint. A confirmation given under the old step is carried over as a
|
||||
-- kit that holds that one key, so an operator who confirmed the active key
|
||||
-- keeps that part confirmed and is asked only for what the old step never
|
||||
-- covered.
|
||||
--
|
||||
-- A backup file written by a release that still had the four columns carries
|
||||
-- them in its instance settings; the restore writes an explicit list of
|
||||
-- settings and ignores them.
|
||||
--
|
||||
-- The whole file is one statement, so it applies whole or not at all, and
|
||||
-- running it again changes nothing. The drop takes an ACCESS EXCLUSIVE lock
|
||||
-- on `app_settings` for a catalogue update only; the wait is bounded.
|
||||
--
|
||||
-- Reversibility: re-adding the four nullable columns restores the old shape;
|
||||
-- the old step then reads as due until it is confirmed again.
|
||||
--
|
||||
-- @retry-on-lock-timeout
|
||||
|
||||
DO $migration$
|
||||
BEGIN
|
||||
PERFORM set_config('lock_timeout', '5s', true);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS "recovery_readiness_notices" (
|
||||
"signal" TEXT NOT NULL,
|
||||
"notified_state" TEXT,
|
||||
"notified_at" TIMESTAMP(3),
|
||||
"watched_since" TIMESTAMP(3),
|
||||
CONSTRAINT "recovery_readiness_notices_pkey" PRIMARY KEY ("signal"),
|
||||
CONSTRAINT "recovery_readiness_notices_state_check"
|
||||
CHECK ("notified_state" IS NULL OR "notified_state" IN ('ok', 'attention', 'failing'))
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS "restore_drill_outcomes" (
|
||||
"id" TEXT NOT NULL DEFAULT 'singleton',
|
||||
"last_run_at" TIMESTAMP(3) NOT NULL,
|
||||
"last_success_at" TIMESTAMP(3),
|
||||
"last_failure_at" TIMESTAMP(3),
|
||||
"last_failure_reason" TEXT,
|
||||
"accounts_checked" INTEGER NOT NULL DEFAULT 0,
|
||||
"accounts_failed" INTEGER NOT NULL DEFAULT 0,
|
||||
CONSTRAINT "restore_drill_outcomes_pkey" PRIMARY KEY ("id")
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS "recovery_kit_confirmations" (
|
||||
"id" TEXT NOT NULL DEFAULT 'singleton',
|
||||
"confirmed_at" TIMESTAMP(3) NOT NULL,
|
||||
"confirmed_by_user_id" TEXT,
|
||||
"entries" JSONB NOT NULL,
|
||||
CONSTRAINT "recovery_kit_confirmations_pkey" PRIMARY KEY ("id")
|
||||
);
|
||||
|
||||
IF EXISTS (
|
||||
SELECT 1 FROM information_schema.columns
|
||||
WHERE table_schema = current_schema()
|
||||
AND table_name = 'app_settings'
|
||||
AND column_name = 'encryption_key_backup_confirmed_at'
|
||||
) THEN
|
||||
INSERT INTO "recovery_kit_confirmations"
|
||||
("id", "confirmed_at", "confirmed_by_user_id", "entries")
|
||||
SELECT
|
||||
'singleton',
|
||||
"encryption_key_backup_confirmed_at",
|
||||
"encryption_key_backup_confirmed_by_user_id",
|
||||
jsonb_build_array(jsonb_build_object(
|
||||
'kind', 'application',
|
||||
'id', "encryption_key_backup_confirmed_key_id",
|
||||
'fingerprint', "encryption_key_backup_confirmed_fingerprint"
|
||||
))
|
||||
FROM "app_settings"
|
||||
WHERE "id" = 'singleton'
|
||||
AND "encryption_key_backup_confirmed_at" IS NOT NULL
|
||||
AND "encryption_key_backup_confirmed_key_id" IS NOT NULL
|
||||
AND "encryption_key_backup_confirmed_fingerprint" IS NOT NULL
|
||||
ON CONFLICT ("id") DO NOTHING;
|
||||
|
||||
ALTER TABLE "app_settings"
|
||||
DROP COLUMN IF EXISTS "encryption_key_backup_confirmed_at",
|
||||
DROP COLUMN IF EXISTS "encryption_key_backup_confirmed_key_id",
|
||||
DROP COLUMN IF EXISTS "encryption_key_backup_confirmed_fingerprint",
|
||||
DROP COLUMN IF EXISTS "encryption_key_backup_confirmed_by_user_id";
|
||||
END IF;
|
||||
END
|
||||
$migration$;
|
||||
+64
-15
@@ -5469,21 +5469,6 @@ model AppSettings {
|
||||
documentMaxFileBytes Int @default(26214400) @map("document_max_file_bytes")
|
||||
documentQuotaBytes BigInt @default(1073741824) @map("document_quota_bytes")
|
||||
|
||||
// The operator's confirmation that the encryption key is backed up, bound to
|
||||
// the key it was given for: the active key id and its fingerprint (the first
|
||||
// twelve hex characters of SHA-256 over the raw key bytes). A rotation or a
|
||||
// re-keyed install changes either one, and the step is due again. Never the
|
||||
// key itself. Written by `POST /api/admin/encryption/key-backup/confirm`;
|
||||
// consumer: `readKeyBackupStatus` (the admin step, the overview tile and the
|
||||
// dashboard banner for admins).
|
||||
encryptionKeyBackupConfirmedAt DateTime? @map("encryption_key_backup_confirmed_at")
|
||||
encryptionKeyBackupConfirmedKeyId String? @map("encryption_key_backup_confirmed_key_id")
|
||||
encryptionKeyBackupConfirmedFingerprint String? @map("encryption_key_backup_confirmed_fingerprint")
|
||||
/// Plain id, not a relation: the instance row must not become user-scoped,
|
||||
/// and a deleted admin leaves the confirmation standing. Consumer: the admin
|
||||
/// step names who confirmed when the account still exists.
|
||||
encryptionKeyBackupConfirmedByUserId String? @map("encryption_key_backup_confirmed_by_user_id")
|
||||
|
||||
/// When this instance first stored a discovery finding under the seasonally
|
||||
/// adjusted correlation engine (v1.42). Stamped once, by the first
|
||||
/// `syncAcceptedPatterns` run of a discovery family, before it writes; null
|
||||
@@ -5906,6 +5891,70 @@ model OffhostBackupKeyUse {
|
||||
@@map("offhost_backup_key_use")
|
||||
}
|
||||
|
||||
/// The monthly restore drill's last result, one row for the instance. pg-boss
|
||||
/// forgets a failed run after seven days and the failing-jobs card looks back
|
||||
/// 72 hours, so the drill's verdict lives here for as long as it is the last
|
||||
/// one. Written by `recordRestoreDrillOutcome` at the end of every drill that
|
||||
/// had something to check; consumer: `readRecoveryReadiness` (the readiness
|
||||
/// tile and the hourly evaluation).
|
||||
model RestoreDrillOutcome {
|
||||
id String @id @default("singleton")
|
||||
/// When the last drill finished. Consumer: the tile's drill row.
|
||||
lastRunAt DateTime @map("last_run_at")
|
||||
/// When a drill last passed every object it checked. A failure stays the
|
||||
/// verdict until a later success. Consumer: the readiness model.
|
||||
lastSuccessAt DateTime? @map("last_success_at")
|
||||
/// When a drill last failed. Consumer: the readiness model.
|
||||
lastFailureAt DateTime? @map("last_failure_at")
|
||||
/// One line naming why, already redacted. Consumer: the tile's drill row.
|
||||
lastFailureReason String? @map("last_failure_reason")
|
||||
/// Objects the last drill checked and how many of them failed. Consumer:
|
||||
/// the tile's drill row.
|
||||
accountsChecked Int @default(0) @map("accounts_checked")
|
||||
accountsFailed Int @default(0) @map("accounts_failed")
|
||||
|
||||
@@map("restore_drill_outcomes")
|
||||
}
|
||||
|
||||
/// Per readiness signal, the state the admins were last notified about. The
|
||||
/// hourly evaluation compares the current state against it: a change to
|
||||
/// attention or failing notifies once, a failing signal is reminded at most
|
||||
/// once a day, and a return to ok is announced once. The update is a
|
||||
/// compare-and-set on both columns, so two workers never send the same
|
||||
/// notice. Written and read by `notifyReadinessChanges`.
|
||||
model RecoveryReadinessNotice {
|
||||
/// The signal's id (`database_dump`, `weekly_copy`, …).
|
||||
signal String @id
|
||||
/// `ok`, `attention` or `failing`; null until anything was announced.
|
||||
notifiedState String? @map("notified_state")
|
||||
notifiedAt DateTime? @map("notified_at")
|
||||
/// Since when the evaluation has seen the signal apply (not `inactive`);
|
||||
/// cleared when it stops applying. Consumer: the readiness model, which
|
||||
/// judges a dump that has never been attempted against it.
|
||||
watchedSince DateTime? @map("watched_since")
|
||||
|
||||
@@map("recovery_readiness_notices")
|
||||
}
|
||||
|
||||
/// The recovery kit an admin confirmed: every key a restore needs, by kind,
|
||||
/// id and fingerprint, never the key itself. The kit is due again as soon as
|
||||
/// a key it needs is not in this set (a rotation, a new backup key). Written
|
||||
/// by `POST /api/admin/recovery-kit/confirm`; consumer: `readRecoveryKit`
|
||||
/// (the kit card, the dashboard banner for admins and the readiness model).
|
||||
model RecoveryKitConfirmation {
|
||||
id String @id @default("singleton")
|
||||
confirmedAt DateTime @map("confirmed_at")
|
||||
/// Plain id, not a relation: the instance row must not become user-scoped,
|
||||
/// and a deleted admin leaves the confirmation standing. Consumer: the kit
|
||||
/// card names who confirmed while the account still exists.
|
||||
confirmedByUserId String? @map("confirmed_by_user_id")
|
||||
/// `[{ kind, id, fingerprint }]`, validated on read
|
||||
/// (`parseKitEntries`). Consumer: `assessRecoveryKit`.
|
||||
entries Json
|
||||
|
||||
@@map("recovery_kit_confirmations")
|
||||
}
|
||||
|
||||
/// An account whose off-host copies have to leave the bucket: it was deleted,
|
||||
/// or its data was wiped. Written in the same transaction as the deletion, so
|
||||
/// the request cannot be lost between the two, and deleted once every object
|
||||
|
||||
Reference in new issue
Block a user