fix(backup): notice a backup run the process died under, and take its account last

A weekly or nightly backup job whose process died (a restart, a
container out of memory) stayed active until its two- or four-hour
expiry, and its retry walked the accounts in the same order: the account
that killed it has the oldest copy, so it led again, and every account
after it went without a copy for as long as that record stayed too
large.

Both passes now send with a five-minute pg-boss heartbeat, so a dead run
fails within minutes and its retry picks it up. Each pass records when it
started and finished every account (backup_pass_attempts, migration
0373); an account whose last attempt never finished goes behind all the
others. The backups page shows when the run in progress started and names
an account a run died under, for the weekly and the off-host card, and a
heartbeat timeout reads as what it means.
This commit is contained in:
Marc-André Bombeck committed 2026-10-04 15:56:56 +02:00
1 parent d55e329c48
commit f18e6ed34b
31 files changed
+734 -19

No files matched your search

+8
View File
@@ -9,6 +9,7 @@
* both sides a route-handler-independent shared type home; the route
* keeps owning the HTTP contract, the component keeps owning the UI.
*/
import type { BackupPassActivity } from "@/lib/jobs/backup-pass-attempts";
import type { BackupScheduleStatus } from "@/lib/jobs/backup-schedule-status";
import type { OffhostBackupFreshness } from "@/lib/jobs/offhost-backup-freshness";
@@ -54,6 +55,8 @@ export interface OffhostAccountRow {
export interface OffhostBackupOverview {
/** Whether this host has the variables the nightly job needs. */
configured: boolean;
/** The run in progress, and accounts a run died under. */
activity: BackupPassActivity;
/** Hours between two scheduled runs — the number behind the verdicts. */
periodHours: number;
/** One row per account. Empty when off-host backup is not configured. */
@@ -87,6 +90,11 @@ export interface BackupsList {
* made six weeks ago and one made on Sunday look identical in a table.
*/
schedule: BackupScheduleStatus;
/**
* The weekly run in progress, and accounts a run died under: a start
* without a finish, which a crash leaves and a caught failure does not.
*/
scheduleActivity: BackupPassActivity;
/**
* The off-host leg. A weekly row in this table says the copy that lives in
* this database is current; it says nothing about whether anything reached