mirror of
https://github.com/MBombeck/HealthLog.git
synced 2026-11-10 13:22:14 +00:00
Marc-André Bombeck
47e9f7e81c
Restoring a large account was one request: 94 s for 1.25 million readings.
A reverse proxy that cuts a request at 60 s (nginx's default) showed the
operator an error for a restore that went on to finish on the server.
The restore request now checks the typed confirmation, that the backup
exists and that its stored copy opens, admits one job for the account and
answers 202 with its id. The `backup-restore` queue runs the restore and
records its phase, counts and outcome on a `backup_restore_jobs` row
(migration 0347), which `GET /api/admin/backups/restores` and
`GET /api/admin/backups/restores/{id}` read back.
What a restore checks and writes is unchanged. The body moved from the
route into `restoreBackup` (src/lib/export/restore-backup.ts): every
refusal still happens before the first delete and the replacement is still
one transaction, so a failure part-way leaves the account as it was. While
it runs the account keeps showing its current data, and other writes are
neither refused nor held, as before.
- One restore per account: a partial unique index admits one queued or
running job, and a second request gets 409 `backup.restore.active` naming
the running job. A per-account advisory lock inside the transaction holds
even if a worker thought gone is still writing.
- Time budget: the job's expiry is two hours and a file whose transaction
limit would run past three quarters of it is refused before anything is
deleted (`jobDeadline`, beside `jobBudget`).
- Resuming: the job writes a heartbeat every two seconds. A boot sweep
queues a job whose worker stopped again from the start, at most twice,
then fails it as interrupted. The admission check releases an abandoned
job so it never holds the account's slot.
- The staged file is the stored `DataBackup` row, already encrypted at
rest. It stays whatever the outcome, so a retry needs no new upload. The
job records a digest of the copy and refuses one that was replaced while
it waited.
- Failures carry a stable code and a sentence that says what to do.
- The console follows the job: phase and counts while it runs, the outcome
after, and after a reload it finds the job again from the server. A poll
that fails on the network is never shown as a failed restore. A web
process separate from the worker evicts the account's caches when it
first reports the restore finished. The success line goes through the
outcome module, so an empty file is not drawn as a tick.
The model is registered in the backup plan (not carried) and the wipe plan
(wiped with the account); the queue in the maintenance registrar, the
subsystem surface and the job-fact allowlist; the new code in the error
catalogue; the routes in the OpenAPI contract. The tests that pin restore
semantics now drive the real route and the real job through one helper, and
`backup-restore-job.test.ts` pins the asynchronous contract itself.
91 lines
3.1 KiB
TypeScript
91 lines
3.1 KiB
TypeScript
/**
|
|
* Drive a queued restore to its end, the way the worker does, in one call.
|
|
*
|
|
* `POST /api/admin/backups/[id]/restore` answers 202 and leaves the restore to
|
|
* the `backup-restore` queue. The tests that pin what a restore checks and
|
|
* writes are about the restore, not the queue, so they call this instead of
|
|
* the route: it POSTs through the real route, runs the job the route queued
|
|
* with the real handler body, and answers the way the synchronous route did
|
|
* (200 with the report, or the refusal's status and message), read back from
|
|
* the job's row. Every hop in between is production code; only the pg-boss
|
|
* delivery is replaced by a direct call.
|
|
*
|
|
* The asynchronous contract itself (202, the status route, 409, resuming) is
|
|
* pinned by `backup-restore-job.test.ts`, which does not use this.
|
|
*/
|
|
import type { PgBoss } from "pg-boss";
|
|
|
|
import { POST as queueRestore } from "@/app/api/admin/backups/[id]/restore/route";
|
|
import { getGlobalBoss, setGlobalBoss } from "@/lib/jobs/boss-instance";
|
|
import {
|
|
readBackupRestoreJob,
|
|
runBackupRestoreJob,
|
|
} from "@/lib/jobs/backup-restore";
|
|
|
|
/** The status the synchronous route answered each refusal with. */
|
|
const LEGACY_STATUS: Record<string, number> = {
|
|
backup_not_found: 404,
|
|
backup_changed: 409,
|
|
"backup.payload.undecryptable": 422,
|
|
schema_invalid: 422,
|
|
incompatible_schema_version: 422,
|
|
owner_mismatch: 409,
|
|
owner_not_found: 422,
|
|
"backup.section.missing": 422,
|
|
document_ciphertext_missing: 422,
|
|
time_budget: 503,
|
|
transaction_failed: 500,
|
|
};
|
|
|
|
/** A queue that accepts every send, for a test with no worker. */
|
|
export const acceptingBoss = {
|
|
send: async () => "test-boss-job",
|
|
complete: async () => ({}),
|
|
} as unknown as PgBoss;
|
|
|
|
export async function POST(
|
|
request: Parameters<typeof queueRestore>[0],
|
|
context: { params: Promise<{ id: string }> },
|
|
): Promise<Response> {
|
|
// A file that mocks the queue already has one; otherwise stand one in for
|
|
// the duration of the request only, so nothing else in the file sees it.
|
|
const previous = getGlobalBoss();
|
|
if (!previous) setGlobalBoss(acceptingBoss);
|
|
let queued: Response;
|
|
try {
|
|
queued = await queueRestore(request, context);
|
|
} finally {
|
|
if (!previous) setGlobalBoss(null);
|
|
}
|
|
if (queued.status !== 202) return queued;
|
|
|
|
const { data } = (await queued.json()) as { data: { jobId: string } };
|
|
try {
|
|
await runBackupRestoreJob(data.jobId);
|
|
} catch {
|
|
// The job recorded the failure on its row before rethrowing; the
|
|
// synchronous route turned the same throw into a 500 envelope.
|
|
}
|
|
const job = await readBackupRestoreJob(data.jobId);
|
|
if (!job) throw new Error(`restore job ${data.jobId} vanished`);
|
|
|
|
if (job.status === "succeeded" && job.result) {
|
|
return Response.json(
|
|
{ data: { restored: true, ...job.result }, error: null },
|
|
{ status: 200 },
|
|
);
|
|
}
|
|
const failure = job.failure ?? { code: "unexpected", message: "failed" };
|
|
return Response.json(
|
|
{
|
|
data: null,
|
|
error: failure.message,
|
|
meta: {
|
|
errorCode: failure.code,
|
|
...(failure.sections ? { sections: failure.sections } : {}),
|
|
},
|
|
},
|
|
{ status: LEGACY_STATUS[failure.code] ?? 500 },
|
|
);
|
|
}
|