mirror of
https://github.com/MBombeck/HealthLog.git
synced 2026-11-07 04:12:04 +00:00
fix(backups): answer the restore preview from counts kept with the copy
The restore dialog asked GET /api/admin/backups/{id}/summary for the
contents of a copy, and the route read the whole stored copy to answer:
every piece opened, every measurement parsed, the rest validated. On an
account of 1.8 million readings on slow hardware that took about two
minutes. The dialog stopped waiting after 15 s and said the contents
could not be read, and its automatic retry made the server read the
whole copy a second time.
The copy passes through the process once anyway while it is written, so
the preview is worked out there and kept on the row (DataBackup.preview,
migration 0364): the summarizeBackup counts, the schema version, and
the inner key ids with the sections they sit in and one sample value
each. Every creation path goes through storeBackupBlob (the weekly and
manual pass, and an upload), so every new copy carries one. The column
is plain JSON: counts and dates only, and the samples are ciphertext
that stands for what is inside the copy, which a key rotation must not
re-seal.
The route answers from it without opening the copy. Both verdicts are
taken again on every read, from what the preview keeps: whether this
release restores the schema version, and whether this server holds the
keys the copy needs. The pieces are still counted against the row, so
a copy that lost one is refused as the restore and the download refuse
it. A preview names the copy it describes and is ignored once the copy
changes; converting a single stored value into pieces carries it over.
A copy written before this has no preview. It is read whole once, as
before, and the result is kept on the row, so it is slow only the first
time; concurrent requests for the same copy share the one read.
Only a file that fails the schema answers 422 "Backup payload failed
schema validation". Read refusals keep their codes; anything else is
the server failing and answers 500.
The dialog no longer retries the preview, waits up to 55 s, and when it
stops waiting on an older copy it says the copy is still being read and
to open the dialog again in a few minutes.
Refs #1031
This commit is contained in:
Marc-André Bombeck committed
1 parent
ae6a5ac693
commit
a2fceb5585
19 files changed
+1605
-101
No files matched your search
@@ -7291,6 +7291,7 @@
|
||||
"previewTitle": "Inhalt der Sicherung",
|
||||
"previewLoading": "Inhalt wird gelesen…",
|
||||
"previewUnavailable": "Inhalt konnte nicht gelesen werden — die Wiederherstellung bleibt möglich.",
|
||||
"previewStillReading": "Diese Sicherung stammt aus einer früheren Version, deshalb wird ihr Inhalt vollständig gelesen. Bei einem großen Datenbestand dauert das eine Weile. Öffne diesen Dialog in ein paar Minuten noch einmal, dann steht er hier. Die Wiederherstellung hängt nicht davon ab.",
|
||||
"previewEmpty": "Die Datei enthält keine Datensätze.",
|
||||
"previewOther": "Weitere Datensätze",
|
||||
"previewMeasurements": "Messwerte",
|
||||
|
||||
@@ -7291,6 +7291,7 @@
|
||||
"previewTitle": "What this backup contains",
|
||||
"previewLoading": "Reading contents…",
|
||||
"previewUnavailable": "Contents could not be read — restoring is still possible.",
|
||||
"previewStillReading": "This backup was made by an earlier version, so its contents are being read in full, which takes a while for a large record. Open this dialog again in a few minutes to see them. Restoring does not depend on it.",
|
||||
"previewEmpty": "The file contains no records.",
|
||||
"previewOther": "Other records",
|
||||
"previewMeasurements": "Measurements",
|
||||
|
||||
@@ -7291,6 +7291,7 @@
|
||||
"previewTitle": "Contenido de la copia",
|
||||
"previewLoading": "Leyendo el contenido…",
|
||||
"previewUnavailable": "No se pudo leer el contenido — restaurar sigue siendo posible.",
|
||||
"previewStillReading": "Esta copia se hizo con una versión anterior, así que su contenido se está leyendo completo, y con un historial grande eso lleva un rato. Vuelve a abrir este diálogo en unos minutos para verlo. Restaurar no depende de ello.",
|
||||
"previewEmpty": "El archivo no contiene registros.",
|
||||
"previewOther": "Otros registros",
|
||||
"previewMeasurements": "Mediciones",
|
||||
|
||||
@@ -7291,6 +7291,7 @@
|
||||
"previewTitle": "Contenu de la sauvegarde",
|
||||
"previewLoading": "Lecture du contenu…",
|
||||
"previewUnavailable": "Le contenu n'a pas pu être lu — la restauration reste possible.",
|
||||
"previewStillReading": "Cette sauvegarde a été créée par une version antérieure, son contenu est donc lu en entier, ce qui prend un moment pour un gros dossier. Rouvrez cette fenêtre dans quelques minutes pour le voir. La restauration n'en dépend pas.",
|
||||
"previewEmpty": "Le fichier ne contient aucun enregistrement.",
|
||||
"previewOther": "Autres enregistrements",
|
||||
"previewMeasurements": "Mesures",
|
||||
|
||||
@@ -7291,6 +7291,7 @@
|
||||
"previewTitle": "Contenuto del backup",
|
||||
"previewLoading": "Lettura del contenuto…",
|
||||
"previewUnavailable": "Impossibile leggere il contenuto — il ripristino resta possibile.",
|
||||
"previewStillReading": "Questo backup è stato creato da una versione precedente, quindi il suo contenuto viene letto per intero, e con un archivio grande ci vuole un po'. Riapri questa finestra tra qualche minuto per vederlo. Il ripristino non dipende da questo.",
|
||||
"previewEmpty": "Il file non contiene record.",
|
||||
"previewOther": "Altri record",
|
||||
"previewMeasurements": "Misurazioni",
|
||||
|
||||
@@ -7270,6 +7270,7 @@
|
||||
"previewTitle": "이 백업에 담긴 내용",
|
||||
"previewLoading": "내용을 읽는 중…",
|
||||
"previewUnavailable": "내용을 읽지 못했어요 — 복원은 그래도 할 수 있어요.",
|
||||
"previewStillReading": "이 백업은 이전 버전에서 만들어져서 내용을 처음부터 끝까지 읽고 있어요. 기록이 많으면 시간이 좀 걸려요. 몇 분 뒤에 이 창을 다시 열면 내용이 보여요. 복원은 이것과 상관없이 할 수 있어요.",
|
||||
"previewEmpty": "파일에 기록이 없어요.",
|
||||
"previewOther": "기타 기록",
|
||||
"previewMeasurements": "측정",
|
||||
|
||||
@@ -7291,6 +7291,7 @@
|
||||
"previewTitle": "Zawartość kopii",
|
||||
"previewLoading": "Odczytywanie zawartości…",
|
||||
"previewUnavailable": "Nie udało się odczytać zawartości — przywracanie jest nadal możliwe.",
|
||||
"previewStillReading": "Ta kopia powstała we wcześniejszej wersji, więc jej zawartość jest odczytywana w całości, co przy dużych danych chwilę trwa. Otwórz to okno ponownie za kilka minut, aby ją zobaczyć. Przywracanie od tego nie zależy.",
|
||||
"previewEmpty": "Plik nie zawiera rekordów.",
|
||||
"previewOther": "Inne rekordy",
|
||||
"previewMeasurements": "Pomiary",
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
-- What the restore preview shows for a stored backup, worked out while the
|
||||
-- copy is written, so the preview no longer reads the whole copy inside a web
|
||||
-- request. Nullable: a copy written before this release has none until its
|
||||
-- first preview has read it once.
|
||||
ALTER TABLE "data_backups" ADD COLUMN IF NOT EXISTS "preview" JSONB;
|
||||
@@ -5032,6 +5032,18 @@ model DataBackup {
|
||||
/// Whether `innerKeyIds` was recorded; false on every copy written before
|
||||
/// v1.39.3. Consumer: the same status read (`backups.unrecorded`).
|
||||
innerKeyIdsRecorded Boolean @default(false) @map("inner_key_ids_recorded")
|
||||
/// What the restore preview shows for this copy, worked out while the copy
|
||||
/// was written (`src/lib/export/backup-preview.ts`): the section counts, the
|
||||
/// schema version, and the inner key ids with the sections they sit in and
|
||||
/// one sample value each, so the key verdict is taken again against this
|
||||
/// server's keys every time it is read. It names the copy it describes and
|
||||
/// is ignored once the copy changes. Plain JSON on purpose: counts and dates
|
||||
/// only, and the samples are ciphertext already, which a key rotation must
|
||||
/// not re-seal because they stand for what is inside the copy. NULL for a
|
||||
/// copy written before this column existed, until its first preview has
|
||||
/// read it whole.
|
||||
/// Consumer: `GET /api/admin/backups/{id}/summary`.
|
||||
preview Json?
|
||||
createdAt DateTime @default(now()) @map("created_at")
|
||||
|
||||
user User @relation(fields: [userId], references: [id], onDelete: Cascade)
|
||||
|
||||
@@ -1,11 +1,14 @@
|
||||
/**
|
||||
* v1.37.20 — restore preview: the summary read decrypts and validates the
|
||||
* stored file through the SAME helpers the restore uses, and only ever
|
||||
* reads. Watched red: written against the route before its annotate landed;
|
||||
* the not-found and decrypt-failure arms were asserted first.
|
||||
* Restore preview. A copy written with a preview is answered from it without
|
||||
* opening the copy (#1031); a copy from before previews is read whole once,
|
||||
* through the same helpers the restore uses, and what that read found is kept
|
||||
* on the row. The verdicts (schema version, keys) are taken again on every
|
||||
* read. Only a file that fails the schema is a 422 with that message; any
|
||||
* other failure is the server's, not the file's.
|
||||
*/
|
||||
import { describe, it, expect, vi, beforeEach } from "vitest";
|
||||
import { NextRequest } from "next/server";
|
||||
import { ZodError } from "zod/v4";
|
||||
|
||||
vi.mock("@/lib/api-handler", () => ({
|
||||
apiHandler: <T extends (...args: unknown[]) => unknown>(fn: T) => fn,
|
||||
@@ -23,9 +26,15 @@ vi.mock("@/lib/api-handler", () => ({
|
||||
vi.mock("@/lib/logging/context", () => ({ annotate: vi.fn() }));
|
||||
|
||||
const findUniqueMock = vi.fn();
|
||||
const updateManyMock = vi.fn();
|
||||
const chunkCountMock = vi.fn();
|
||||
vi.mock("@/lib/db", () => ({
|
||||
prisma: {
|
||||
dataBackup: { findUnique: (...a: unknown[]) => findUniqueMock(...a) },
|
||||
dataBackup: {
|
||||
findUnique: (...a: unknown[]) => findUniqueMock(...a),
|
||||
updateMany: (...a: unknown[]) => updateManyMock(...a),
|
||||
},
|
||||
dataBackupChunk: { count: (...a: unknown[]) => chunkCountMock(...a) },
|
||||
},
|
||||
}));
|
||||
|
||||
@@ -34,16 +43,32 @@ const decryptMock = vi.fn();
|
||||
// decrypts anything, so a stub of this module has to answer that too.
|
||||
vi.mock("@/lib/crypto", () => ({
|
||||
decrypt: (...a: unknown[]) => decryptMock(...a),
|
||||
decryptBytes: vi.fn(),
|
||||
getConfiguredKeyIds: () => ["v1"],
|
||||
isStreamCiphertext: () => false,
|
||||
decryptStream: vi.fn(),
|
||||
}));
|
||||
|
||||
const openSpy = vi.fn();
|
||||
vi.mock("@/lib/export/stored-backup", async (importOriginal) => {
|
||||
const actual =
|
||||
await importOriginal<typeof import("@/lib/export/stored-backup")>();
|
||||
return {
|
||||
...actual,
|
||||
openStoredBackup: (...a: Parameters<typeof actual.openStoredBackup>) => {
|
||||
openSpy(...a);
|
||||
return actual.openStoredBackup(...a);
|
||||
},
|
||||
};
|
||||
});
|
||||
|
||||
const parseMock = vi.fn();
|
||||
const summarizeMock = vi.fn();
|
||||
const compatibleMock = vi.fn((version: string) => version !== "99");
|
||||
vi.mock("@/lib/validations/backup", () => ({
|
||||
parseBackupPayload: (...a: unknown[]) => parseMock(...a),
|
||||
summarizeBackup: (...a: unknown[]) => summarizeMock(...a),
|
||||
isCompatibleSchemaVersion: vi.fn(() => true),
|
||||
isCompatibleSchemaVersion: (version: string) => compatibleMock(version),
|
||||
// The measurements are read as a stream and counted one by one.
|
||||
BACKUP_SCHEMA_VERSION: "2",
|
||||
backupMeasurementSchema: {
|
||||
@@ -52,52 +77,201 @@ vi.mock("@/lib/validations/backup", () => ({
|
||||
}));
|
||||
|
||||
import { GET } from "../route";
|
||||
import { storedCopyIdentity } from "@/lib/export/backup-preview";
|
||||
|
||||
const request = new NextRequest(
|
||||
"http://localhost/api/admin/backups/b1/summary",
|
||||
);
|
||||
const params = { params: Promise.resolve({ id: "b1" }) };
|
||||
const params = () => ({ params: Promise.resolve({ id: "b1" }) });
|
||||
|
||||
const ROW = {
|
||||
id: "b1",
|
||||
userId: "u1",
|
||||
data: "ciphertext",
|
||||
chunkCount: null,
|
||||
chunkStreamId: null,
|
||||
createdAt: new Date("2026-08-01T00:00:00Z"),
|
||||
user: { id: "u1", username: "self-hoster" },
|
||||
};
|
||||
|
||||
function storedPreview(overrides: Record<string, unknown> = {}) {
|
||||
return {
|
||||
version: 1,
|
||||
copy: storedCopyIdentity(ROW),
|
||||
summary: {
|
||||
schemaVersion: "2",
|
||||
userId: "u1",
|
||||
exportedAt: "2026-08-01T00:00:00.000Z",
|
||||
measurements: 1_800_000,
|
||||
moodEntries: 3,
|
||||
},
|
||||
keys: [],
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks();
|
||||
findUniqueMock.mockResolvedValue({
|
||||
id: "b1",
|
||||
userId: "u1",
|
||||
data: "ciphertext",
|
||||
chunkCount: null,
|
||||
chunkStreamId: null,
|
||||
createdAt: new Date("2026-08-01T00:00:00Z"),
|
||||
user: { id: "u1", username: "self-hoster" },
|
||||
});
|
||||
findUniqueMock.mockResolvedValue({ ...ROW, preview: null });
|
||||
updateManyMock.mockResolvedValue({ count: 1 });
|
||||
// Twelve measurements in the file: the preview counts them from the stream.
|
||||
decryptMock.mockReturnValue(
|
||||
JSON.stringify({
|
||||
measurements: Array.from({ length: 12 }, (_, i) => ({ id: `m${i}` })),
|
||||
}),
|
||||
);
|
||||
parseMock.mockReturnValue({ schemaVersion: 3 });
|
||||
summarizeMock.mockReturnValue({ measurements: 0, moodEntries: 3 });
|
||||
parseMock.mockReturnValue({ schemaVersion: "2" });
|
||||
summarizeMock.mockReturnValue({
|
||||
schemaVersion: "2",
|
||||
userId: "u1",
|
||||
exportedAt: "2026-08-01T00:00:00.000Z",
|
||||
measurements: 0,
|
||||
moodEntries: 3,
|
||||
});
|
||||
});
|
||||
|
||||
describe("GET /api/admin/backups/[id]/summary", () => {
|
||||
it("answers the file's counts without writing anything", async () => {
|
||||
const res = await GET(request, params);
|
||||
describe("GET /api/admin/backups/[id]/summary, from the stored preview", () => {
|
||||
it("answers from the preview without opening the copy", async () => {
|
||||
findUniqueMock.mockResolvedValue({ ...ROW, preview: storedPreview() });
|
||||
const res = await GET(request, params());
|
||||
expect(res.status).toBe(200);
|
||||
const body = await res.json();
|
||||
expect(body.data.summary).toEqual({ measurements: 12, moodEntries: 3 });
|
||||
expect(body.data.summary).toMatchObject({
|
||||
measurements: 1_800_000,
|
||||
moodEntries: 3,
|
||||
});
|
||||
expect(body.data.owner).toBe("self-hoster");
|
||||
expect(openSpy).not.toHaveBeenCalled();
|
||||
expect(decryptMock).not.toHaveBeenCalled();
|
||||
expect(updateManyMock).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("takes the schema verdict again against this release", async () => {
|
||||
findUniqueMock.mockResolvedValue({
|
||||
...ROW,
|
||||
preview: storedPreview({
|
||||
summary: {
|
||||
schemaVersion: "99",
|
||||
userId: "u1",
|
||||
exportedAt: "2026-08-01T00:00:00.000Z",
|
||||
measurements: 1,
|
||||
},
|
||||
}),
|
||||
});
|
||||
const res = await GET(request, params());
|
||||
expect(res.status).toBe(422);
|
||||
expect(openSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("takes the key verdict again against this server's keys", async () => {
|
||||
findUniqueMock.mockResolvedValue({
|
||||
...ROW,
|
||||
preview: storedPreview({
|
||||
keys: [
|
||||
{
|
||||
keyId: "v7",
|
||||
count: 4,
|
||||
sections: ["moodEntries"],
|
||||
sample: { value: "v7.AAAA", form: "string" },
|
||||
},
|
||||
// Only in the instance settings, which the restore leaves out.
|
||||
{
|
||||
keyId: "v8",
|
||||
count: 1,
|
||||
sections: ["appSettings"],
|
||||
sample: null,
|
||||
},
|
||||
],
|
||||
}),
|
||||
});
|
||||
const res = await GET(request, params());
|
||||
expect(res.status).toBe(422);
|
||||
expect(await res.json()).toMatchObject({
|
||||
meta: { errorCode: "backup.key.missing", keyIds: ["v7"] },
|
||||
});
|
||||
expect(openSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("still counts the pieces, and refuses a copy that lost one", async () => {
|
||||
const chunked = {
|
||||
...ROW,
|
||||
data: null,
|
||||
chunkCount: 3,
|
||||
chunkStreamId: "s1",
|
||||
};
|
||||
findUniqueMock.mockResolvedValue({
|
||||
...chunked,
|
||||
preview: storedPreview({ copy: storedCopyIdentity(chunked) }),
|
||||
});
|
||||
chunkCountMock.mockResolvedValue(3);
|
||||
expect((await GET(request, params())).status).toBe(200);
|
||||
|
||||
chunkCountMock.mockResolvedValue(2);
|
||||
const res = await GET(request, params());
|
||||
expect(res.status).toBe(422);
|
||||
expect(await res.json()).toMatchObject({
|
||||
meta: { errorCode: "backup.payload.undecryptable" },
|
||||
});
|
||||
expect(openSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("ignores a preview that describes another copy", async () => {
|
||||
findUniqueMock.mockResolvedValue({
|
||||
...ROW,
|
||||
preview: storedPreview({ copy: "chunks:other:3" }),
|
||||
});
|
||||
const res = await GET(request, params());
|
||||
expect(res.status).toBe(200);
|
||||
expect((await res.json()).data.summary.measurements).toBe(12);
|
||||
expect(openSpy).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
});
|
||||
|
||||
describe("GET /api/admin/backups/[id]/summary, a copy without a preview", () => {
|
||||
it("reads the copy once and keeps what it found on the row", async () => {
|
||||
const res = await GET(request, params());
|
||||
expect(res.status).toBe(200);
|
||||
const body = await res.json();
|
||||
expect(body.data.summary).toMatchObject({
|
||||
measurements: 12,
|
||||
moodEntries: 3,
|
||||
});
|
||||
expect(openSpy).toHaveBeenCalledTimes(1);
|
||||
await vi.waitFor(() => expect(updateManyMock).toHaveBeenCalledTimes(1));
|
||||
const [call] = updateManyMock.mock.calls[0] as [
|
||||
{ where: Record<string, unknown>; data: { preview: { copy: string } } },
|
||||
];
|
||||
// Only onto the copy it was read from.
|
||||
expect(call.where).toEqual({
|
||||
id: "b1",
|
||||
chunkStreamId: null,
|
||||
chunkCount: null,
|
||||
});
|
||||
expect(call.data.preview.copy).toBe(storedCopyIdentity(ROW));
|
||||
});
|
||||
|
||||
it("shares one read between requests for the same copy", async () => {
|
||||
const [a, b] = await Promise.all([
|
||||
GET(request, params()),
|
||||
GET(request, params()),
|
||||
]);
|
||||
expect(a.status).toBe(200);
|
||||
expect(b.status).toBe(200);
|
||||
expect(openSpy).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("404s an unknown backup id", async () => {
|
||||
findUniqueMock.mockResolvedValue(null);
|
||||
await expect(GET(request, params)).rejects.toMatchObject({ status: 404 });
|
||||
await expect(GET(request, params())).rejects.toMatchObject({
|
||||
status: 404,
|
||||
});
|
||||
});
|
||||
|
||||
it("refuses a file it cannot decrypt with the documented 422", async () => {
|
||||
decryptMock.mockImplementation(() => {
|
||||
throw new Error("bad key");
|
||||
});
|
||||
const res = await GET(request, params);
|
||||
const res = await GET(request, params());
|
||||
// Bad stored input — a key dropped from `ENCRYPTION_KEYS`, or bytes that
|
||||
// are not the ones written — and not a fault in this process, so it does
|
||||
// not answer 500 and does not reach the error reporter as one.
|
||||
@@ -105,13 +279,42 @@ describe("GET /api/admin/backups/[id]/summary", () => {
|
||||
expect(await res.json()).toMatchObject({
|
||||
meta: { errorCode: "backup.payload.undecryptable" },
|
||||
});
|
||||
expect(updateManyMock).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("422s a file that fails schema validation", async () => {
|
||||
parseMock.mockImplementation(() => {
|
||||
throw new Error("nope");
|
||||
throw new ZodError([]);
|
||||
});
|
||||
const res = await GET(request, params);
|
||||
const res = await GET(request, params());
|
||||
expect(res.status).toBe(422);
|
||||
expect((await res.json()).error).toBe(
|
||||
"Backup payload failed schema validation",
|
||||
);
|
||||
expect(updateManyMock).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("422s a file that is not JSON", async () => {
|
||||
decryptMock.mockReturnValue("{ not json");
|
||||
const res = await GET(request, params());
|
||||
expect(res.status).toBe(422);
|
||||
expect((await res.json()).error).toBe(
|
||||
"Backup payload failed schema validation",
|
||||
);
|
||||
});
|
||||
|
||||
it("does not call a server failure a schema failure", async () => {
|
||||
parseMock.mockImplementation(() => {
|
||||
throw new Error("out of memory");
|
||||
});
|
||||
// Thrown to the handler, which answers 500 and reports it.
|
||||
await expect(GET(request, params())).rejects.toThrow("out of memory");
|
||||
expect(updateManyMock).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("still answers when keeping the preview fails", async () => {
|
||||
updateManyMock.mockRejectedValue(new Error("db gone"));
|
||||
const res = await GET(request, params());
|
||||
expect(res.status).toBe(200);
|
||||
});
|
||||
});
|
||||
@@ -4,33 +4,57 @@
|
||||
*
|
||||
* The restore dialog used to ask for the typed confirmation on nothing but
|
||||
* a filename and a date; the counts only appeared in the audit row after
|
||||
* the rows were already replaced. This read decrypts and schema-validates
|
||||
* the stored file exactly the way the restore itself does (same decrypt,
|
||||
* same `parseBackupPayload`, same compatibility gate) and answers with the
|
||||
* `summarizeBackup` counts — so what the operator confirms is what the
|
||||
* restore will write, derived from the same code path. Read-only; nothing
|
||||
* is touched.
|
||||
* the rows were already replaced. This answers with the `summarizeBackup`
|
||||
* counts of the stored copy, and refuses the way the restore would refuse:
|
||||
* a schema version this release cannot restore, or a key the copy's inner
|
||||
* ciphertext needs and this server does not have.
|
||||
*
|
||||
* The counts are worked out while the copy is written and kept on its row
|
||||
* (`DataBackup.preview`, `src/lib/export/backup-preview.ts`), so this answers
|
||||
* without opening the copy. Reading the copy here took two minutes for an
|
||||
* account of 1.8 million readings on slow hardware, long after the dialog had
|
||||
* given up (#1031). Both verdicts are taken again on every read, from what
|
||||
* the preview keeps, because the release and the server's keys can both
|
||||
* change after the copy was written. The copy's pieces are still counted
|
||||
* against the row (no piece is read), so a copy that lost one since is
|
||||
* refused here, as the restore and the download refuse it.
|
||||
*
|
||||
* A copy written before previews existed has none. That one is read whole,
|
||||
* once: the preview worked out from it is stored on the row, so the next
|
||||
* request answers at once, and concurrent requests for the same copy share
|
||||
* the one read.
|
||||
*/
|
||||
import { NextRequest } from "next/server";
|
||||
import { ZodError } from "zod/v4";
|
||||
|
||||
import { Prisma } from "@/generated/prisma/client";
|
||||
import { prisma } from "@/lib/db";
|
||||
import { apiHandler, HttpError, requireAdmin } from "@/lib/api-handler";
|
||||
import { apiError, apiSuccess } from "@/lib/api-response";
|
||||
import {
|
||||
checkStoredBackupShape,
|
||||
isStoredBackupReadError,
|
||||
openStoredBackup,
|
||||
storedBackupRefusal,
|
||||
STORED_BACKUP_SELECT,
|
||||
type StoredBackupRef,
|
||||
} from "@/lib/export/stored-backup";
|
||||
import {
|
||||
assessBackupKeys,
|
||||
BACKUP_KEY_MISSING_CODE,
|
||||
BackupKeyIdCollector,
|
||||
describeBackupKeyProblem,
|
||||
} from "@/lib/export/backup-key-ids";
|
||||
import { BackupJsonError } from "@/lib/export/backup-json-scan";
|
||||
import {
|
||||
buildBackupPreview,
|
||||
storedCopyIdentity,
|
||||
storedPreviewFor,
|
||||
type BackupPreview,
|
||||
} from "@/lib/export/backup-preview";
|
||||
import {
|
||||
readStreamedBackup,
|
||||
type BackupSource,
|
||||
type StreamedBackup,
|
||||
StreamedBackupInvalidError,
|
||||
} from "@/lib/export/streamed-backup";
|
||||
import { annotate } from "@/lib/logging/context";
|
||||
import {
|
||||
@@ -43,6 +67,114 @@ export const dynamic = "force-dynamic";
|
||||
|
||||
type RouteParams = { params: Promise<{ id: string }> };
|
||||
|
||||
/** Why a copy could not be summarised, as the route answers it. */
|
||||
type Refusal = { status: number; message: string; code?: string };
|
||||
|
||||
type ReadOutcome = { preview: BackupPreview } | { refusal: Refusal };
|
||||
|
||||
/**
|
||||
* Reads of a copy without a preview that are running now, by copy. A second
|
||||
* request for the same copy (the dialog opened again while the first read is
|
||||
* still going) waits for the first instead of reading it all again.
|
||||
*/
|
||||
const inFlight = new Map<string, Promise<ReadOutcome>>();
|
||||
|
||||
/** Whether a failure is the file itself not being a backup this reads. */
|
||||
function isPayloadInvalid(err: unknown): boolean {
|
||||
return (
|
||||
err instanceof BackupJsonError ||
|
||||
err instanceof StreamedBackupInvalidError ||
|
||||
err instanceof ZodError
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Read a copy whole and work out its preview: the way every preview was
|
||||
* taken before previews were stored. Throws for anything that is neither a
|
||||
* read refusal nor a file that fails the schema, which the handler answers
|
||||
* as a server fault.
|
||||
*/
|
||||
async function readPreview(
|
||||
backup: StoredBackupRef,
|
||||
copy: string,
|
||||
): Promise<ReadOutcome> {
|
||||
const refused = (err: unknown): ReadOutcome => {
|
||||
const refusal = storedBackupRefusal(err);
|
||||
return {
|
||||
refusal: {
|
||||
status: refusal.status,
|
||||
message: refusal.message,
|
||||
code: refusal.code,
|
||||
},
|
||||
};
|
||||
};
|
||||
|
||||
// Opened and read as a stream: a large record's JSON is longer than any
|
||||
// string V8 can hold (#1031).
|
||||
let source;
|
||||
try {
|
||||
source = await openStoredBackup(prisma, backup);
|
||||
} catch (err) {
|
||||
// Same refusal the restore and the download give, for the same reason:
|
||||
// a copy this instance cannot open is bad stored input, and the preview
|
||||
// is the first place an operator meets it.
|
||||
return refused(err);
|
||||
}
|
||||
|
||||
try {
|
||||
const streamed = await readStreamedBackup(source);
|
||||
const payload = parseBackupPayload(streamed.raw);
|
||||
const summary = {
|
||||
...summarizeBackup(payload),
|
||||
measurements: streamed.measurementCount,
|
||||
};
|
||||
return { preview: buildBackupPreview(copy, summary, streamed.keys) };
|
||||
} catch (err) {
|
||||
if (isStoredBackupReadError(err)) return refused(err);
|
||||
// Only a file that is not a backup this release reads is a 422. Anything
|
||||
// else is this server failing, not the file, and is answered as such.
|
||||
if (isPayloadInvalid(err)) {
|
||||
return {
|
||||
refusal: {
|
||||
status: 422,
|
||||
message: "Backup payload failed schema validation",
|
||||
},
|
||||
};
|
||||
}
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Keep a preview worked out by reading the copy, so the next request answers
|
||||
* without reading it. Written only while the row still holds the copy it was
|
||||
* read from; a failure here costs the next request the same read, nothing
|
||||
* more, so it is noted and not raised.
|
||||
*/
|
||||
async function keepPreview(
|
||||
backup: StoredBackupRef,
|
||||
preview: BackupPreview,
|
||||
): Promise<void> {
|
||||
try {
|
||||
await prisma.dataBackup.updateMany({
|
||||
where: {
|
||||
id: backup.id,
|
||||
chunkStreamId: backup.chunkStreamId,
|
||||
chunkCount: backup.chunkCount,
|
||||
},
|
||||
data: { preview: preview as unknown as Prisma.InputJsonValue },
|
||||
});
|
||||
} catch (err) {
|
||||
annotate({
|
||||
action: { name: "admin.backups.summary.keep_failed" },
|
||||
meta: {
|
||||
backupId: backup.id,
|
||||
error: err instanceof Error ? err.name : "unknown",
|
||||
},
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
export const GET = apiHandler(
|
||||
async (_request: NextRequest, { params }: RouteParams) => {
|
||||
const { user: admin } = await requireAdmin();
|
||||
@@ -53,6 +185,7 @@ export const GET = apiHandler(
|
||||
where: { id },
|
||||
select: {
|
||||
...STORED_BACKUP_SELECT,
|
||||
preview: true,
|
||||
createdAt: true,
|
||||
user: { select: { id: true, username: true } },
|
||||
},
|
||||
@@ -61,41 +194,58 @@ export const GET = apiHandler(
|
||||
throw new HttpError(404, "Backup not found");
|
||||
}
|
||||
|
||||
// Opened and read as a stream: a large record's JSON is longer than any
|
||||
// string V8 can hold (#1031).
|
||||
let source: BackupSource;
|
||||
try {
|
||||
source = await openStoredBackup(prisma, backup);
|
||||
} catch (err) {
|
||||
// Same refusal the restore and the download give, for the same reason:
|
||||
// a copy this instance cannot open is bad stored input, and the preview
|
||||
// is the first place an operator meets it.
|
||||
const refusal = storedBackupRefusal(err);
|
||||
return apiError(refusal.message, refusal.status, {
|
||||
errorCode: refusal.code,
|
||||
});
|
||||
}
|
||||
|
||||
let payload;
|
||||
let measurementCount: number;
|
||||
let streamed: StreamedBackup;
|
||||
try {
|
||||
streamed = await readStreamedBackup(source);
|
||||
measurementCount = streamed.measurementCount;
|
||||
payload = parseBackupPayload(streamed.raw);
|
||||
} catch (err) {
|
||||
if (isStoredBackupReadError(err)) {
|
||||
let preview = storedPreviewFor(backup.preview, backup);
|
||||
const fromStore = preview !== null;
|
||||
if (preview) {
|
||||
// The counts were kept; the copy's pieces are still counted, so a copy
|
||||
// that lost one since is refused here as the restore would refuse it.
|
||||
// Tampering inside a piece is found by the restore's own open, before
|
||||
// it deletes anything.
|
||||
try {
|
||||
await checkStoredBackupShape(prisma, backup);
|
||||
} catch (err) {
|
||||
if (!isStoredBackupReadError(err)) throw err;
|
||||
const refusal = storedBackupRefusal(err);
|
||||
return apiError(refusal.message, refusal.status, {
|
||||
errorCode: refusal.code,
|
||||
return refusalResponse({
|
||||
status: refusal.status,
|
||||
message: refusal.message,
|
||||
code: refusal.code,
|
||||
});
|
||||
}
|
||||
return apiError("Backup payload failed schema validation", 422);
|
||||
}
|
||||
if (!preview) {
|
||||
const copy = storedCopyIdentity(backup);
|
||||
if (copy === null) {
|
||||
// A row holding both forms, or neither: the open refuses it, with
|
||||
// the reason.
|
||||
const outcome = await readPreview(backup, "");
|
||||
if ("refusal" in outcome) return refusalResponse(outcome.refusal);
|
||||
throw new Error("A stored backup with no nameable copy was read");
|
||||
}
|
||||
const flight = `${backup.id}:${copy}`;
|
||||
let pending = inFlight.get(flight);
|
||||
if (!pending) {
|
||||
pending = readPreview(backup, copy).finally(() => {
|
||||
inFlight.delete(flight);
|
||||
});
|
||||
inFlight.set(flight, pending);
|
||||
// Kept whatever the verdicts below say: they are taken again on
|
||||
// every read, and the counts do not change.
|
||||
void pending.then(
|
||||
(outcome) =>
|
||||
"preview" in outcome ? keepPreview(backup, outcome.preview) : null,
|
||||
() => null,
|
||||
);
|
||||
}
|
||||
const outcome = await pending;
|
||||
if ("refusal" in outcome) return refusalResponse(outcome.refusal);
|
||||
preview = outcome.preview;
|
||||
}
|
||||
|
||||
if (!isCompatibleSchemaVersion(payload.schemaVersion)) {
|
||||
const { summary } = preview;
|
||||
if (!isCompatibleSchemaVersion(summary.schemaVersion)) {
|
||||
return apiError(
|
||||
`Backup schema version ${payload.schemaVersion} is not restorable by this release`,
|
||||
`Backup schema version ${summary.schemaVersion} is not restorable by this release`,
|
||||
422,
|
||||
);
|
||||
}
|
||||
@@ -105,9 +255,10 @@ export const GET = apiHandler(
|
||||
// does not have. The instance settings are left out here, as the restore
|
||||
// leaves them out unless asked; the restore itself checks them when they
|
||||
// are asked for.
|
||||
const keyVerdict = assessBackupKeys(streamed.keys, {
|
||||
ignoreSections: new Set(["appSettings"]),
|
||||
});
|
||||
const keyVerdict = assessBackupKeys(
|
||||
BackupKeyIdCollector.fromStored(preview.keys),
|
||||
{ ignoreSections: new Set(["appSettings"]) },
|
||||
);
|
||||
const keyProblem = describeBackupKeyProblem(keyVerdict);
|
||||
if (keyProblem) {
|
||||
return apiError(keyProblem, 422, {
|
||||
@@ -116,14 +267,9 @@ export const GET = apiHandler(
|
||||
});
|
||||
}
|
||||
|
||||
const summary = {
|
||||
...summarizeBackup(payload),
|
||||
measurements: measurementCount,
|
||||
};
|
||||
|
||||
annotate({
|
||||
action: { name: "admin.backups.summary" },
|
||||
meta: { backupId: id, ownerId: backup.userId },
|
||||
meta: { backupId: id, ownerId: backup.userId, fromStore },
|
||||
});
|
||||
|
||||
return apiSuccess({
|
||||
@@ -133,3 +279,11 @@ export const GET = apiHandler(
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
function refusalResponse(refusal: Refusal) {
|
||||
return apiError(
|
||||
refusal.message,
|
||||
refusal.status,
|
||||
refusal.code ? { errorCode: refusal.code } : undefined,
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,136 @@
|
||||
/**
|
||||
* The restore dialog's contents box asks once and says honestly why it has
|
||||
* nothing to show.
|
||||
*
|
||||
* A copy from an earlier version is read whole on the server, which took two
|
||||
* minutes for a large record; the dialog stopped waiting, retried, and the
|
||||
* server read the whole copy a second time (#1031). Pinned here: the preview
|
||||
* query is never retried and carries its own timeout, and a timeout reads as
|
||||
* "still being read", not as "could not be read".
|
||||
*
|
||||
* Mutation check: drop `retry: false` and the first case goes red; classify a
|
||||
* timeout as unavailable and the second goes red.
|
||||
*/
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import { renderToStaticMarkup } from "react-dom/server";
|
||||
|
||||
import { ApiError } from "@/lib/api/api-fetch";
|
||||
import { queryKeys } from "@/lib/query-keys";
|
||||
|
||||
const mocks = vi.hoisted(() => ({
|
||||
queries: [] as Array<Record<string, unknown>>,
|
||||
apiGet: vi.fn(async () => ({ summary: {} })),
|
||||
}));
|
||||
|
||||
vi.mock("next/navigation", () => ({
|
||||
useRouter: () => ({ push: vi.fn(), replace: vi.fn() }),
|
||||
usePathname: () => "/admin/backups",
|
||||
}));
|
||||
|
||||
vi.mock("sonner", () => ({ toast: { success: vi.fn(), error: vi.fn() } }));
|
||||
|
||||
vi.mock("@/lib/api/api-fetch", async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import("@/lib/api/api-fetch")>();
|
||||
return { ...actual, apiGet: mocks.apiGet };
|
||||
});
|
||||
|
||||
vi.mock("@tanstack/react-query", () => ({
|
||||
useQuery: (options: Record<string, unknown>) => {
|
||||
mocks.queries.push(options);
|
||||
return {
|
||||
data: {
|
||||
rows: [
|
||||
{
|
||||
id: "b1",
|
||||
userId: "u1",
|
||||
username: "self-hoster",
|
||||
type: "WEEKLY_AUTO",
|
||||
sizeBytes: 1024,
|
||||
createdAt: "2026-09-01T03:00:00.000Z",
|
||||
},
|
||||
],
|
||||
retentionDays: 30,
|
||||
schedule: {
|
||||
lastSuccessAt: "2026-09-01T03:00:00.000Z",
|
||||
lastSuccessAgeDays: 2,
|
||||
staleAfterDays: 10,
|
||||
stale: false,
|
||||
lastRun: null,
|
||||
lastRunFailed: false,
|
||||
},
|
||||
},
|
||||
isLoading: false,
|
||||
isError: false,
|
||||
};
|
||||
},
|
||||
useQueryClient: () => ({ invalidateQueries: vi.fn() }),
|
||||
useMutation: () => ({
|
||||
mutate: vi.fn(),
|
||||
mutateAsync: vi.fn(),
|
||||
isPending: false,
|
||||
isError: false,
|
||||
error: null,
|
||||
variables: undefined,
|
||||
}),
|
||||
}));
|
||||
|
||||
vi.mock("@/hooks/use-auth", () => ({
|
||||
useAuth: () => ({
|
||||
user: { id: "u1", username: "testuser", role: "ADMIN" },
|
||||
isAuthenticated: true,
|
||||
isLoading: false,
|
||||
refetch: vi.fn(),
|
||||
}),
|
||||
}));
|
||||
|
||||
import { I18nProvider } from "@/lib/i18n/context";
|
||||
import { BackupsSection, restorePreviewFailure } from "../backups-section";
|
||||
|
||||
describe("restore preview query", () => {
|
||||
it("is never retried and waits a bounded time", async () => {
|
||||
renderToStaticMarkup(
|
||||
<I18nProvider initialLocale="en">
|
||||
<BackupsSection />
|
||||
</I18nProvider>,
|
||||
);
|
||||
const key = JSON.stringify(queryKeys.adminBackupSummary("b1"));
|
||||
const preview = mocks.queries.find(
|
||||
(options) => JSON.stringify(options.queryKey) === key,
|
||||
);
|
||||
expect(preview).toBeDefined();
|
||||
expect(preview!.retry).toBe(false);
|
||||
|
||||
await (preview!.queryFn as () => Promise<unknown>)();
|
||||
const [path, init] = mocks.apiGet.mock.calls[0] as unknown as [
|
||||
string,
|
||||
RequestInit,
|
||||
];
|
||||
expect(path).toBe("/api/admin/backups/b1/summary");
|
||||
expect(init.signal).toBeInstanceOf(AbortSignal);
|
||||
});
|
||||
});
|
||||
|
||||
describe("restorePreviewFailure", () => {
|
||||
it("reads a timeout as a copy still being read", () => {
|
||||
const timeout = new DOMException("signal timed out", "TimeoutError");
|
||||
expect(restorePreviewFailure(timeout)).toBe("stillReading");
|
||||
});
|
||||
|
||||
it("names a missing key", () => {
|
||||
expect(
|
||||
restorePreviewFailure(
|
||||
new ApiError("key", 422, {
|
||||
errorCode: "backup.key.missing",
|
||||
keyIds: ["v1"],
|
||||
}),
|
||||
),
|
||||
).toBe("keyMissing");
|
||||
});
|
||||
|
||||
it("calls everything else unavailable", () => {
|
||||
expect(restorePreviewFailure(new ApiError("boom", 500))).toBe(
|
||||
"unavailable",
|
||||
);
|
||||
expect(restorePreviewFailure(new TypeError("offline"))).toBe("unavailable");
|
||||
});
|
||||
});
|
||||
@@ -90,6 +90,39 @@ import {
|
||||
* Mirrors the wipe dialog's pattern but adds the typed gate because the
|
||||
* blast radius is bigger (re-creates rows, not just deletes).
|
||||
*/
|
||||
/**
|
||||
* How long the restore dialog waits for the contents of a copy. A copy's
|
||||
* counts are kept with it and come back at once; this only runs out on a copy
|
||||
* from an earlier version that the server is still reading whole, and stays
|
||||
* under the minute a reverse proxy commonly allows a request.
|
||||
*/
|
||||
const PREVIEW_TIMEOUT_MS = 55_000;
|
||||
|
||||
/**
|
||||
* What the restore dialog says when the contents did not come back: the key
|
||||
* the copy needs and this server lacks, a copy from an earlier version still
|
||||
* being read when the dialog stopped waiting, or anything else.
|
||||
*/
|
||||
export function restorePreviewFailure(
|
||||
error: unknown,
|
||||
): "keyMissing" | "stillReading" | "unavailable" {
|
||||
if (
|
||||
error instanceof ApiError &&
|
||||
error.meta?.errorCode === "backup.key.missing"
|
||||
) {
|
||||
return "keyMissing";
|
||||
}
|
||||
// `AbortSignal.timeout` rejects the fetch with a DOMException of this name.
|
||||
if (
|
||||
typeof error === "object" &&
|
||||
error !== null &&
|
||||
(error as { name?: unknown }).name === "TimeoutError"
|
||||
) {
|
||||
return "stillReading";
|
||||
}
|
||||
return "unavailable";
|
||||
}
|
||||
|
||||
/**
|
||||
* v1.37.20 — the sections a restore preview names, in render order. Each
|
||||
* entry sums one or more `BackupSummary` counters under one human label;
|
||||
@@ -151,18 +184,25 @@ function RestoreRowDialog({
|
||||
const [withInstanceSettings, setWithInstanceSettings] = useState(false);
|
||||
|
||||
// v1.37.20 — restore preview: fetch what the file contains the moment the
|
||||
// dialog opens, so the typed confirmation is an informed one. Derived from
|
||||
// the same decrypt + schema path the restore itself runs. A failed preview
|
||||
// never blocks the restore — it states its own absence instead.
|
||||
// dialog opens, so the typed confirmation is an informed one. A failed
|
||||
// preview never blocks the restore — it states its own absence instead.
|
||||
//
|
||||
// Never retried. The server answers from counts kept with the copy, at
|
||||
// once; only a copy from an earlier version is read whole, and that read
|
||||
// goes on (and is kept) after this request gives up. A retry would only
|
||||
// start the same read a second time (#1031).
|
||||
const preview = useQuery({
|
||||
queryKey: queryKeys.adminBackupSummary(row.id),
|
||||
queryFn: () =>
|
||||
apiGet<{ summary: Record<string, unknown> }>(
|
||||
`/api/admin/backups/${row.id}/summary`,
|
||||
{ signal: AbortSignal.timeout(PREVIEW_TIMEOUT_MS) },
|
||||
),
|
||||
enabled: open,
|
||||
staleTime: 60_000,
|
||||
retry: false,
|
||||
});
|
||||
const previewFailure = restorePreviewFailure(preview.error);
|
||||
const numericSummary: Record<string, number> = {};
|
||||
if (preview.data?.summary) {
|
||||
for (const [key, value] of Object.entries(preview.data.summary)) {
|
||||
@@ -239,17 +279,21 @@ function RestoreRowDialog({
|
||||
</p>
|
||||
)}
|
||||
{preview.isError &&
|
||||
(preview.error instanceof ApiError &&
|
||||
preview.error.meta?.errorCode === "backup.key.missing" ? (
|
||||
(previewFailure === "keyMissing" &&
|
||||
preview.error instanceof ApiError ? (
|
||||
// The one preview failure the restore would repeat: say which
|
||||
// key, so the operator can put it back before trying.
|
||||
<p role="alert" className="text-destructive">
|
||||
{t("admin.section.backups.previewKeyMissing", {
|
||||
keys: Array.isArray(preview.error.meta.keyIds)
|
||||
keys: Array.isArray(preview.error.meta?.keyIds)
|
||||
? preview.error.meta.keyIds.map(String).join(", ")
|
||||
: "",
|
||||
})}
|
||||
</p>
|
||||
) : previewFailure === "stillReading" ? (
|
||||
<p className="text-muted-foreground">
|
||||
{t("admin.section.backups.previewStillReading")}
|
||||
</p>
|
||||
) : (
|
||||
<p className="text-muted-foreground">
|
||||
{t("admin.section.backups.previewUnavailable")}
|
||||
|
||||
@@ -0,0 +1,222 @@
|
||||
/**
|
||||
* The preview worked out while a copy is written has to say what the preview
|
||||
* route used to derive by reading the stored copy: the same counts, the same
|
||||
* key uses. Held here against that derivation over the same file, fed in
|
||||
* pieces that split values, strings and multi-byte characters.
|
||||
*/
|
||||
import { afterEach, describe, expect, it } from "vitest";
|
||||
|
||||
import { _resetCryptoCacheForTests, encrypt } from "@/lib/crypto";
|
||||
|
||||
import {
|
||||
BackupKeyIdCollector,
|
||||
assessBackupKeys,
|
||||
} from "@/lib/export/backup-key-ids";
|
||||
import {
|
||||
buildBackupPreview,
|
||||
createBackupPreviewScanner,
|
||||
storedCopyIdentity,
|
||||
storedPreviewFor,
|
||||
} from "@/lib/export/backup-preview";
|
||||
import { backupPayloadSchema, summarizeBackup } from "@/lib/validations/backup";
|
||||
|
||||
const ts = "2026-07-19T07:00:00.000Z";
|
||||
/** A bytes-column value holding the string codec under key id `v3`. */
|
||||
const sealedV3 = Buffer.from(`v3.${"QUJD".repeat(20)}`, "utf8").toString(
|
||||
"base64",
|
||||
);
|
||||
|
||||
const FILE = {
|
||||
schemaVersion: "2",
|
||||
exportedAt: ts,
|
||||
userId: "u1",
|
||||
measurements: [1, 2, 3].map((i) => ({
|
||||
id: `m${i}`,
|
||||
type: "WEIGHT",
|
||||
value: 70 + i,
|
||||
unit: "kg",
|
||||
measuredAt: ts,
|
||||
source: "MANUAL",
|
||||
notes: "Grüße, ☕",
|
||||
...(i === 2 ? { notesEncrypted: sealedV3 } : {}),
|
||||
})),
|
||||
intakeEvents: [
|
||||
{ medication: "Example", scheduledFor: ts, source: "WEB" },
|
||||
{ medication: "Example", scheduledFor: ts, source: "WEB" },
|
||||
],
|
||||
moodEntries: [
|
||||
{
|
||||
date: "2026-05-08",
|
||||
mood: "GUT",
|
||||
score: 4,
|
||||
source: "MOODLOG",
|
||||
loggedAt: ts,
|
||||
},
|
||||
],
|
||||
labResults: [{ analyte: "HbA1c", unit: "%", takenAt: ts, source: "MANUAL" }],
|
||||
illnessEpisodes: [
|
||||
{
|
||||
id: "ep-1",
|
||||
label: "Cold",
|
||||
type: "INFECTION",
|
||||
lifecycle: "ACUTE",
|
||||
onsetAt: ts,
|
||||
dayLogs: [
|
||||
{ date: "2026-04-01", symptoms: [] },
|
||||
{ date: "2026-04-02", symptoms: [] },
|
||||
],
|
||||
},
|
||||
],
|
||||
};
|
||||
|
||||
async function scan(pieces: Array<string | Uint8Array>) {
|
||||
const scanner = createBackupPreviewScanner();
|
||||
for (const piece of pieces) await scanner.feed(piece);
|
||||
return scanner.finish();
|
||||
}
|
||||
|
||||
function split(text: string, size: number): Uint8Array[] {
|
||||
const bytes = Buffer.from(text, "utf8");
|
||||
const out: Uint8Array[] = [];
|
||||
for (let at = 0; at < bytes.length; at += size) {
|
||||
out.push(bytes.subarray(at, at + size));
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
describe("createBackupPreviewScanner", () => {
|
||||
const json = JSON.stringify(FILE);
|
||||
// What the preview route derived from a stored copy.
|
||||
const expected = summarizeBackup(backupPayloadSchema.parse(FILE));
|
||||
const expectedKeys = new BackupKeyIdCollector();
|
||||
expectedKeys.visit(FILE);
|
||||
|
||||
it.each([1, 7, 64, 100_000])(
|
||||
"counts what the full read counts, in pieces of %i bytes",
|
||||
async (size) => {
|
||||
const result = await scan(split(json, size));
|
||||
expect(result).not.toBeNull();
|
||||
expect(result!.summary).toEqual(expected);
|
||||
expect(result!.summary.measurements).toBe(3);
|
||||
expect(result!.summary.intakeEvents).toBe(2);
|
||||
expect(result!.summary.moodEntries).toBe(1);
|
||||
expect(result!.summary.illnessDayLogs).toBe(2);
|
||||
expect(result!.keys.toStored()).toEqual(expectedKeys.toStored());
|
||||
expect(result!.keys.keyIds()).toEqual(["v3"]);
|
||||
},
|
||||
);
|
||||
|
||||
it("takes text pieces the way the weekly writer hands them over", async () => {
|
||||
const result = await scan([json.slice(0, 50), json.slice(50)]);
|
||||
expect(result!.summary).toEqual(expected);
|
||||
});
|
||||
|
||||
it("answers null for a file that is not a backup, and never holds the writer", async () => {
|
||||
expect(await scan(["{ not json", "more", "and more"])).toBeNull();
|
||||
expect(await scan([JSON.stringify({ schemaVersion: 2 })])).toBeNull();
|
||||
expect(await scan([json.slice(0, 40)])).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe("storedPreviewFor", () => {
|
||||
const chunked = { data: null, chunkCount: 4, chunkStreamId: "s1" };
|
||||
const preview = () =>
|
||||
buildBackupPreview(
|
||||
storedCopyIdentity(chunked)!,
|
||||
summarizeBackup(backupPayloadSchema.parse(FILE)),
|
||||
new BackupKeyIdCollector(),
|
||||
);
|
||||
|
||||
it("reads a preview of the copy the row holds", () => {
|
||||
const stored = JSON.parse(JSON.stringify(preview()));
|
||||
expect(storedPreviewFor(stored, chunked)?.summary.measurements).toBe(3);
|
||||
});
|
||||
|
||||
it("ignores a preview of another copy", () => {
|
||||
expect(
|
||||
storedPreviewFor(preview(), { ...chunked, chunkStreamId: "s2" }),
|
||||
).toBeNull();
|
||||
expect(
|
||||
storedPreviewFor(preview(), { ...chunked, chunkCount: 5 }),
|
||||
).toBeNull();
|
||||
});
|
||||
|
||||
it("describes nothing on a row holding both forms, or none", () => {
|
||||
expect(storedPreviewFor(preview(), { ...chunked, data: "x" })).toBeNull();
|
||||
expect(
|
||||
storedPreviewFor(preview(), {
|
||||
data: null,
|
||||
chunkCount: null,
|
||||
chunkStreamId: null,
|
||||
}),
|
||||
).toBeNull();
|
||||
});
|
||||
|
||||
it("ignores a malformed preview", () => {
|
||||
expect(storedPreviewFor({ version: 1 }, chunked)).toBeNull();
|
||||
expect(storedPreviewFor(null, chunked)).toBeNull();
|
||||
});
|
||||
|
||||
it("names a single stored value by its length and head", () => {
|
||||
const a = storedCopyIdentity({
|
||||
data: "v1.aaaa",
|
||||
chunkCount: null,
|
||||
chunkStreamId: null,
|
||||
});
|
||||
const b = storedCopyIdentity({
|
||||
data: "v1.aaab",
|
||||
chunkCount: null,
|
||||
chunkStreamId: null,
|
||||
});
|
||||
expect(a).toMatch(/^value:[0-9a-f]{64}$/);
|
||||
expect(a).not.toBe(b);
|
||||
});
|
||||
});
|
||||
|
||||
describe("BackupKeyIdCollector.toStored / fromStored", () => {
|
||||
const saved = {
|
||||
key: process.env.ENCRYPTION_KEY,
|
||||
keys: process.env.ENCRYPTION_KEYS,
|
||||
active: process.env.ENCRYPTION_ACTIVE_KEY_ID,
|
||||
};
|
||||
function useKeys(keys: Record<string, string>, active: string) {
|
||||
delete process.env.ENCRYPTION_KEY;
|
||||
process.env.ENCRYPTION_KEYS = JSON.stringify(keys);
|
||||
process.env.ENCRYPTION_ACTIVE_KEY_ID = active;
|
||||
_resetCryptoCacheForTests();
|
||||
}
|
||||
afterEach(() => {
|
||||
for (const [name, value] of [
|
||||
["ENCRYPTION_KEY", saved.key],
|
||||
["ENCRYPTION_KEYS", saved.keys],
|
||||
["ENCRYPTION_ACTIVE_KEY_ID", saved.active],
|
||||
] as const) {
|
||||
if (value === undefined) delete process.env[name];
|
||||
else process.env[name] = value;
|
||||
}
|
||||
_resetCryptoCacheForTests();
|
||||
});
|
||||
|
||||
it("keeps what the verdict needs, so it can be taken again later", () => {
|
||||
// Written under `old` and `other` on the host that made the copy.
|
||||
useKeys({ old: "11".repeat(32), other: "22".repeat(32) }, "old");
|
||||
const collector = new BackupKeyIdCollector();
|
||||
collector.add("moodEntries", encrypt("a note"));
|
||||
process.env.ENCRYPTION_ACTIVE_KEY_ID = "other";
|
||||
_resetCryptoCacheForTests();
|
||||
collector.add("labResults", encrypt("another note"));
|
||||
collector.add("appSettings", encrypt("a setting"));
|
||||
const stored = JSON.parse(JSON.stringify(collector.toStored()));
|
||||
const options = { ignoreSections: new Set(["appSettings"]) };
|
||||
|
||||
// Later, on a host that dropped `old` and holds different material
|
||||
// under `other`.
|
||||
useKeys({ other: "33".repeat(32) }, "other");
|
||||
const back = BackupKeyIdCollector.fromStored(stored);
|
||||
const verdict = assessBackupKeys(back, options);
|
||||
expect(verdict.missing).toEqual(["old"]);
|
||||
expect(verdict.unreadable).toEqual(["other"]);
|
||||
expect(verdict).toEqual(assessBackupKeys(collector, options));
|
||||
expect(back.toStored()).toEqual(collector.toStored());
|
||||
});
|
||||
});
|
||||
@@ -185,6 +185,43 @@ export class BackupKeyIdCollector {
|
||||
entries(): ReadonlyMap<string, KeyUse> {
|
||||
return this.uses;
|
||||
}
|
||||
|
||||
/**
|
||||
* What was collected, as plain data, for a copy's stored preview
|
||||
* (`backup-preview.ts`): enough to hold it against this host's keys again
|
||||
* later without reading the copy.
|
||||
*/
|
||||
toStored(): StoredKeyUse[] {
|
||||
return [...this.uses.entries()]
|
||||
.sort(([a], [b]) => a.localeCompare(b))
|
||||
.map(([keyId, use]) => ({
|
||||
keyId,
|
||||
count: use.count,
|
||||
sections: [...use.sections].sort(),
|
||||
sample: use.sample ? { ...use.sample } : null,
|
||||
}));
|
||||
}
|
||||
|
||||
/** The collector {@link toStored} was taken from. */
|
||||
static fromStored(stored: readonly StoredKeyUse[]): BackupKeyIdCollector {
|
||||
const collector = new BackupKeyIdCollector();
|
||||
for (const entry of stored) {
|
||||
collector.uses.set(entry.keyId, {
|
||||
count: entry.count,
|
||||
sections: new Set(entry.sections),
|
||||
sample: entry.sample ? { ...entry.sample } : null,
|
||||
});
|
||||
}
|
||||
return collector;
|
||||
}
|
||||
}
|
||||
|
||||
/** One key id a copy needs, as {@link BackupKeyIdCollector.toStored} keeps it. */
|
||||
export interface StoredKeyUse {
|
||||
keyId: string;
|
||||
count: number;
|
||||
sections: string[];
|
||||
sample: { value: string; form: InnerCiphertextForm } | null;
|
||||
}
|
||||
|
||||
export interface BackupKeyAssessment {
|
||||
|
||||
@@ -0,0 +1,223 @@
|
||||
/**
|
||||
* What the restore preview shows for a stored backup, worked out while the
|
||||
* copy is written and kept on its row (`DataBackup.preview`).
|
||||
*
|
||||
* Why. The preview used to read the whole stored copy inside the request that
|
||||
* asked for it: open every piece, parse every measurement, validate the rest.
|
||||
* For an account of 1.8 million readings on slow hardware that took two
|
||||
* minutes, the dialog gave up long before, and the operator was told the
|
||||
* contents could not be read (#1031). The copy passes through this process
|
||||
* once anyway, on its way into the pieces, so the counts are taken there.
|
||||
*
|
||||
* What is kept. The `summarizeBackup` counts of the copy, its schema version,
|
||||
* and the inner key ids with the sections they sit in and one sample value
|
||||
* each. The verdicts are not kept: whether this release can restore the
|
||||
* schema version, and whether this server holds the keys the copy needs, can
|
||||
* both change after the copy was written, so the route takes them again from
|
||||
* these inputs on every read.
|
||||
*
|
||||
* Which copy. A preview names the copy it describes (`copy`), and a reader
|
||||
* ignores one that names another: a copy replaced by a writer that does not
|
||||
* know about previews (an older release, after a downgrade) must not be
|
||||
* described by the numbers of the one it replaced.
|
||||
*/
|
||||
import { createHash } from "node:crypto";
|
||||
import { z } from "zod/v4";
|
||||
|
||||
import {
|
||||
BackupKeyIdCollector,
|
||||
type StoredKeyUse,
|
||||
} from "@/lib/export/backup-key-ids";
|
||||
import { scanBackupJson } from "@/lib/export/backup-json-scan";
|
||||
import type { StoredBackupRef } from "@/lib/export/stored-backup";
|
||||
import {
|
||||
backupPayloadSchema,
|
||||
summarizeBackup,
|
||||
type BackupSummary,
|
||||
} from "@/lib/validations/backup";
|
||||
|
||||
export interface BackupPreview {
|
||||
version: 1;
|
||||
/** The copy this describes (`storedCopyIdentity`). */
|
||||
copy: string;
|
||||
summary: BackupSummary;
|
||||
keys: StoredKeyUse[];
|
||||
}
|
||||
|
||||
const storedPreviewSchema = z.object({
|
||||
version: z.literal(1),
|
||||
copy: z.string().min(1),
|
||||
summary: z
|
||||
.object({
|
||||
schemaVersion: z.string(),
|
||||
userId: z.string(),
|
||||
exportedAt: z.string(),
|
||||
measurements: z.number(),
|
||||
})
|
||||
.catchall(z.union([z.string(), z.number()])),
|
||||
keys: z.array(
|
||||
z.object({
|
||||
keyId: z.string(),
|
||||
count: z.number().int().nonnegative(),
|
||||
sections: z.array(z.string()),
|
||||
sample: z
|
||||
.object({
|
||||
value: z.string(),
|
||||
form: z.enum(["string", "bytes-string", "binary"]),
|
||||
})
|
||||
.nullable(),
|
||||
}),
|
||||
),
|
||||
});
|
||||
|
||||
/**
|
||||
* How a single stored value is told apart from another: its length and its
|
||||
* first characters, which carry the key id and the random nonce of the
|
||||
* envelope. Hashed, so the preview does not repeat the envelope's head.
|
||||
*/
|
||||
export function singleValueIdentity(length: number, head: string): string {
|
||||
return `value:${createHash("sha256").update(`${length}:${head}`).digest("hex")}`;
|
||||
}
|
||||
|
||||
/** How many leading characters of a single stored value the identity reads. */
|
||||
export const SINGLE_VALUE_IDENTITY_HEAD = 128;
|
||||
|
||||
/**
|
||||
* The name of the copy a row holds now, or null for a row nothing may
|
||||
* describe: one holding both forms, or neither.
|
||||
*/
|
||||
export function storedCopyIdentity(
|
||||
backup: Pick<StoredBackupRef, "data" | "chunkCount" | "chunkStreamId">,
|
||||
): string | null {
|
||||
if (backup.data != null && backup.chunkStreamId != null) return null;
|
||||
if (backup.chunkStreamId != null) {
|
||||
return `chunks:${backup.chunkStreamId}:${backup.chunkCount ?? ""}`;
|
||||
}
|
||||
if (backup.data != null) {
|
||||
return singleValueIdentity(
|
||||
backup.data.length,
|
||||
backup.data.slice(0, SINGLE_VALUE_IDENTITY_HEAD),
|
||||
);
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
export function buildBackupPreview(
|
||||
copy: string,
|
||||
summary: BackupSummary,
|
||||
keys: BackupKeyIdCollector,
|
||||
): BackupPreview {
|
||||
return { version: 1, copy, summary, keys: keys.toStored() };
|
||||
}
|
||||
|
||||
/**
|
||||
* The stored preview of a row, when it describes the copy the row holds now;
|
||||
* null when there is none, it is malformed, or it describes another copy.
|
||||
*/
|
||||
export function storedPreviewFor(
|
||||
stored: unknown,
|
||||
backup: Pick<StoredBackupRef, "data" | "chunkCount" | "chunkStreamId">,
|
||||
): BackupPreview | null {
|
||||
const copy = storedCopyIdentity(backup);
|
||||
if (copy === null || stored == null) return null;
|
||||
const parsed = storedPreviewSchema.safeParse(stored);
|
||||
if (!parsed.success || parsed.data.copy !== copy) return null;
|
||||
return parsed.data as unknown as BackupPreview;
|
||||
}
|
||||
|
||||
/**
|
||||
* The sections that grow with a record. Their elements are counted and their
|
||||
* key ids read as they pass, and never kept, so the scan holds no more of the
|
||||
* copy than the writer itself does (`full-backup-stream.ts`).
|
||||
*/
|
||||
const STREAMED = new Set(["measurements", "intakeEvents", "moodEntries"]);
|
||||
|
||||
export interface BackupPreviewScanner {
|
||||
/**
|
||||
* Hand over the next piece of the JSON. Resolves once the scan has taken
|
||||
* it, so a writer never runs more than a piece ahead of the scan.
|
||||
*/
|
||||
feed(piece: string | Uint8Array): Promise<void>;
|
||||
/**
|
||||
* The counts and key uses, once every piece has been fed; null when the
|
||||
* JSON was not a backup the schema accepts. Never rejects: a copy is
|
||||
* stored whether or not its preview could be worked out.
|
||||
*/
|
||||
finish(): Promise<{
|
||||
summary: BackupSummary;
|
||||
keys: BackupKeyIdCollector;
|
||||
} | null>;
|
||||
}
|
||||
|
||||
/**
|
||||
* Work out a preview from a copy's JSON as it is written. The counts are the
|
||||
* ones the preview route derives from the stored copy (`summarizeBackup` over
|
||||
* the parsed file), the three bulk sections counted rather than parsed.
|
||||
*/
|
||||
export function createBackupPreviewScanner(): BackupPreviewScanner {
|
||||
const queue: Array<{ chunk: Uint8Array; taken: () => void }> = [];
|
||||
let ended = false;
|
||||
let stopped = false;
|
||||
let wake: (() => void) | null = null;
|
||||
const signal = () => {
|
||||
const w = wake;
|
||||
wake = null;
|
||||
w?.();
|
||||
};
|
||||
|
||||
async function* pieces(): AsyncGenerator<Uint8Array> {
|
||||
for (;;) {
|
||||
const next = queue.shift();
|
||||
if (next) {
|
||||
next.taken();
|
||||
yield next.chunk;
|
||||
continue;
|
||||
}
|
||||
if (ended) return;
|
||||
await new Promise<void>((resolve) => {
|
||||
wake = resolve;
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
const keys = new BackupKeyIdCollector();
|
||||
const result = scanBackupJson(pieces(), {
|
||||
streamKeys: STREAMED,
|
||||
onElement: (key, element) => keys.visit(element, key),
|
||||
})
|
||||
.then(({ document, streamedCounts }) => {
|
||||
keys.visit(document);
|
||||
const parsed = backupPayloadSchema.safeParse(document);
|
||||
if (!parsed.success) return null;
|
||||
const summary: BackupSummary = {
|
||||
...summarizeBackup(parsed.data),
|
||||
measurements: streamedCounts.measurements ?? 0,
|
||||
intakeEvents: streamedCounts.intakeEvents ?? 0,
|
||||
moodEntries: streamedCounts.moodEntries ?? 0,
|
||||
};
|
||||
return { summary, keys };
|
||||
})
|
||||
.catch(() => null)
|
||||
.finally(() => {
|
||||
// A scan that stopped early takes nothing more: release the writer.
|
||||
stopped = true;
|
||||
for (const waiting of queue.splice(0)) waiting.taken();
|
||||
});
|
||||
|
||||
return {
|
||||
feed(piece) {
|
||||
if (stopped) return Promise.resolve();
|
||||
const chunk =
|
||||
typeof piece === "string" ? Buffer.from(piece, "utf8") : piece;
|
||||
return new Promise<void>((taken) => {
|
||||
queue.push({ chunk, taken });
|
||||
signal();
|
||||
});
|
||||
},
|
||||
finish() {
|
||||
ended = true;
|
||||
signal();
|
||||
return result;
|
||||
},
|
||||
};
|
||||
}
|
||||
@@ -23,7 +23,7 @@
|
||||
* this module exists to remove. A model create is planned once for its shape
|
||||
* and keeps nothing.
|
||||
*/
|
||||
import type { PrismaClient } from "@/generated/prisma/client";
|
||||
import { Prisma, type PrismaClient } from "@/generated/prisma/client";
|
||||
|
||||
import { isStreamCiphertext, openStreamDecryptor } from "@/lib/crypto";
|
||||
import {
|
||||
@@ -35,6 +35,14 @@ import {
|
||||
type PackBackupOptions,
|
||||
} from "@/lib/export/backup-blob";
|
||||
import { createBackupKeyIdTextScanner } from "@/lib/export/backup-key-ids";
|
||||
import {
|
||||
buildBackupPreview,
|
||||
createBackupPreviewScanner,
|
||||
SINGLE_VALUE_IDENTITY_HEAD,
|
||||
singleValueIdentity,
|
||||
storedCopyIdentity,
|
||||
type BackupPreview,
|
||||
} from "@/lib/export/backup-preview";
|
||||
import {
|
||||
BackupIntegrityError,
|
||||
newChunkStreamId,
|
||||
@@ -118,6 +126,9 @@ export async function storeBackupBlob(
|
||||
// what is inside them, so this is what tells an operator how long a
|
||||
// retired key is still needed (`GET /api/admin/encryption/status`).
|
||||
const keyScanner = createBackupKeyIdTextScanner();
|
||||
// And what the restore preview shows, from the same pass, so the
|
||||
// preview never has to read the copy back (`backup-preview.ts`).
|
||||
const previewScanner = createBackupPreviewScanner();
|
||||
const { chunks, bytes } = await packBackupChunks(
|
||||
async (sealed, seq) => {
|
||||
await tx.dataBackupChunk.create({
|
||||
@@ -129,10 +140,12 @@ export async function storeBackupBlob(
|
||||
(write) =>
|
||||
producer(async (chunk) => {
|
||||
keyScanner.feed(chunk);
|
||||
await previewScanner.feed(chunk);
|
||||
await write(chunk);
|
||||
}),
|
||||
options,
|
||||
);
|
||||
const scanned = await previewScanner.finish();
|
||||
|
||||
// Again at the end: a restore may have been queued while this ran. The
|
||||
// window left is the few milliseconds to the commit, and a restore
|
||||
@@ -147,6 +160,18 @@ export async function storeBackupBlob(
|
||||
chunkStreamId: streamId,
|
||||
innerKeyIds: keyScanner.keyIds(),
|
||||
innerKeyIdsRecorded: true,
|
||||
// Prisma's JSON null clears a preview left by the previous copy.
|
||||
preview: scanned
|
||||
? (buildBackupPreview(
|
||||
storedCopyIdentity({
|
||||
data: null,
|
||||
chunkCount: chunks,
|
||||
chunkStreamId: streamId,
|
||||
})!,
|
||||
scanned.summary,
|
||||
scanned.keys,
|
||||
) as unknown as Prisma.InputJsonValue)
|
||||
: Prisma.DbNull,
|
||||
createdAt: new Date(),
|
||||
},
|
||||
});
|
||||
@@ -156,6 +181,17 @@ export async function storeBackupBlob(
|
||||
);
|
||||
}
|
||||
|
||||
/** A single value's stored preview, when it describes that value. */
|
||||
function previewOfSingleValue(
|
||||
stored: unknown,
|
||||
len: number,
|
||||
head: string,
|
||||
): BackupPreview | null {
|
||||
if (!stored || typeof stored !== "object") return null;
|
||||
const preview = stored as BackupPreview;
|
||||
return preview.copy === singleValueIdentity(len, head) ? preview : null;
|
||||
}
|
||||
|
||||
/**
|
||||
* The first characters of a single stored value, which carry its key id, or
|
||||
* null when the row no longer holds one. For the key-rotation scan, which must
|
||||
@@ -208,10 +244,16 @@ export async function convertSingleValueBackup(
|
||||
`SET LOCAL idle_in_transaction_session_timeout = '${STORE_IDLE_TIMEOUT}'`,
|
||||
);
|
||||
const [row] = await tx.$queryRaw<
|
||||
Array<{ head: string; len: number; stream_id: string | null }>
|
||||
Array<{
|
||||
head: string;
|
||||
len: number;
|
||||
stream_id: string | null;
|
||||
preview: unknown;
|
||||
}>
|
||||
>`
|
||||
SELECT left(data, 128) AS head, length(data)::int AS len,
|
||||
chunk_stream_id AS stream_id
|
||||
SELECT left(data, ${SINGLE_VALUE_IDENTITY_HEAD}::int) AS head,
|
||||
length(data)::int AS len,
|
||||
chunk_stream_id AS stream_id, preview
|
||||
FROM data_backups
|
||||
WHERE id = ${id} AND data IS NOT NULL
|
||||
FOR UPDATE
|
||||
@@ -272,9 +314,28 @@ export async function convertSingleValueBackup(
|
||||
// limit on writing a new one.
|
||||
{ maxBytes: Number.MAX_SAFE_INTEGER },
|
||||
);
|
||||
// The content is the same copy in another form, so a preview of it
|
||||
// still holds; it is moved over to the new form's name.
|
||||
const preview = previewOfSingleValue(row.preview, row.len, row.head);
|
||||
await tx.dataBackup.update({
|
||||
where: { id },
|
||||
data: { data: null, chunkCount: chunks, chunkStreamId: streamId },
|
||||
data: {
|
||||
data: null,
|
||||
chunkCount: chunks,
|
||||
chunkStreamId: streamId,
|
||||
...(preview
|
||||
? {
|
||||
preview: {
|
||||
...preview,
|
||||
copy: storedCopyIdentity({
|
||||
data: null,
|
||||
chunkCount: chunks,
|
||||
chunkStreamId: streamId,
|
||||
})!,
|
||||
} as unknown as Prisma.InputJsonValue,
|
||||
}
|
||||
: {}),
|
||||
},
|
||||
});
|
||||
return "converted";
|
||||
},
|
||||
|
||||
@@ -141,6 +141,71 @@ async function* openedChunks(
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Refuse a chunked copy whose row does not list as many pieces as are stored:
|
||||
* one missing, or one this code never wrote. A count, without reading a
|
||||
* piece; `openStoredBackup` goes on to authenticate every piece.
|
||||
*/
|
||||
async function checkStoredBackupPieceCount(
|
||||
prisma: ChunkReader,
|
||||
id: string,
|
||||
streamId: string,
|
||||
count: number,
|
||||
): Promise<void> {
|
||||
if (count < 1) {
|
||||
throw new BackupIntegrityError("The stored copy lists no pieces.");
|
||||
}
|
||||
// Pieces beyond the count would be ignored by the reads, but they are not
|
||||
// something this code ever writes, so their presence is refused too.
|
||||
const stored = await prisma.dataBackupChunk.count({
|
||||
where: { backupId: id },
|
||||
});
|
||||
if (stored !== count) {
|
||||
const now = await prisma.dataBackup.findUnique({
|
||||
where: { id },
|
||||
select: { chunkStreamId: true },
|
||||
});
|
||||
if (now?.chunkStreamId !== streamId) throw new BackupReplacedError();
|
||||
throw new BackupIntegrityError(
|
||||
`The stored copy lists ${count} pieces but ${stored} are stored.`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The checks on a stored copy that need no piece read: both forms, a count
|
||||
* without a stream id or the reverse, no content, and the pieces a chunked
|
||||
* copy lists against those stored. Throws what `openStoredBackup` would
|
||||
* throw for the same row. For a reader that answers from what was kept about
|
||||
* the copy (the restore preview) and must still not describe a copy the
|
||||
* restore would refuse on sight.
|
||||
*/
|
||||
export async function checkStoredBackupShape(
|
||||
prisma: ChunkReader,
|
||||
backup: StoredBackupRef,
|
||||
): Promise<void> {
|
||||
if (backup.data != null && backup.chunkStreamId != null) {
|
||||
throw new BackupFormsConflictError();
|
||||
}
|
||||
if ((backup.chunkStreamId == null) !== (backup.chunkCount == null)) {
|
||||
throw new BackupIntegrityError(
|
||||
"The stored copy lists pieces without saying how many, or the reverse.",
|
||||
);
|
||||
}
|
||||
if (backup.chunkStreamId == null || backup.chunkCount == null) {
|
||||
if (backup.data == null) {
|
||||
throw new BackupIntegrityError("The stored copy has no content.");
|
||||
}
|
||||
return;
|
||||
}
|
||||
await checkStoredBackupPieceCount(
|
||||
prisma,
|
||||
backup.id,
|
||||
backup.chunkStreamId,
|
||||
backup.chunkCount,
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Open a stored copy as a source of its JSON bytes, openable as many times as
|
||||
* the caller needs. Rejects when the copy cannot be read whole: a key that is
|
||||
@@ -166,24 +231,7 @@ export async function openStoredBackup(
|
||||
return openBackupBlob(backup.data);
|
||||
}
|
||||
const { id, chunkStreamId: streamId, chunkCount: count } = backup;
|
||||
if (count < 1) {
|
||||
throw new BackupIntegrityError("The stored copy lists no pieces.");
|
||||
}
|
||||
// Pieces beyond the count would be ignored by the reads below, but they are
|
||||
// not something this code ever writes, so their presence is refused too.
|
||||
const stored = await prisma.dataBackupChunk.count({
|
||||
where: { backupId: id },
|
||||
});
|
||||
if (stored !== count) {
|
||||
const now = await prisma.dataBackup.findUnique({
|
||||
where: { id },
|
||||
select: { chunkStreamId: true },
|
||||
});
|
||||
if (now?.chunkStreamId !== streamId) throw new BackupReplacedError();
|
||||
throw new BackupIntegrityError(
|
||||
`The stored copy lists ${count} pieces but ${stored} are stored.`,
|
||||
);
|
||||
}
|
||||
await checkStoredBackupPieceCount(prisma, id, streamId, count);
|
||||
// The whole copy, checked once before anyone reads a byte of it.
|
||||
for await (const piece of openedChunks(prisma, id, streamId, count)) {
|
||||
void piece;
|
||||
|
||||
@@ -0,0 +1,352 @@
|
||||
/**
|
||||
* The restore preview is worked out while a copy is written and answered from
|
||||
* the row, without opening the copy (#1031).
|
||||
*
|
||||
* Both ends and the pipe: the copies are written by the real weekly job and
|
||||
* the real upload route, the preview route reads them back, and what it
|
||||
* answers is held against a full read of the stored copy, which is how the
|
||||
* preview used to be taken. A copy from before previews is read whole once
|
||||
* and its preview kept; a key dropped after the copy was written is still
|
||||
* named, because the verdict is taken again at every read.
|
||||
*
|
||||
* Mutations that must turn this red: drop the preview scanner from
|
||||
* `storeBackupBlob` (no preview on the row, the route opens the copy), or
|
||||
* answer from the stored verdict instead of taking it again (the dropped key
|
||||
* is not named).
|
||||
*/
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
|
||||
import { _resetCryptoCacheForTests, encrypt } from "@/lib/crypto";
|
||||
import { Prisma } from "@/generated/prisma/client";
|
||||
|
||||
import { cookieJar, headerJar } from "./mock-next-headers";
|
||||
import { getPrismaClient, truncateAllTables } from "./setup";
|
||||
import { readStoredBackup } from "./stored-backup-read";
|
||||
|
||||
vi.mock("next/headers", async () => {
|
||||
const { cookieJar, headerJar } = await import("./mock-next-headers");
|
||||
return {
|
||||
headers: vi.fn(async () => ({
|
||||
get: (name: string) => headerJar.get(name.toLowerCase()) ?? null,
|
||||
})),
|
||||
cookies: vi.fn(async () => ({
|
||||
get: (name: string) => {
|
||||
const value = cookieJar.get(name);
|
||||
return value ? { name, value } : undefined;
|
||||
},
|
||||
set: (name: string, value: string) => {
|
||||
cookieJar.set(name, value);
|
||||
},
|
||||
delete: (name: string) => {
|
||||
cookieJar.delete(name);
|
||||
},
|
||||
})),
|
||||
};
|
||||
});
|
||||
|
||||
vi.mock("@/lib/db-compat", () => ({
|
||||
ensureDbCompatibility: vi.fn().mockResolvedValue(undefined),
|
||||
}));
|
||||
|
||||
const opened = vi.hoisted(() => ({ count: 0 }));
|
||||
vi.mock("@/lib/export/stored-backup", async (importOriginal) => {
|
||||
const actual =
|
||||
await importOriginal<typeof import("@/lib/export/stored-backup")>();
|
||||
return {
|
||||
...actual,
|
||||
openStoredBackup: (...a: Parameters<typeof actual.openStoredBackup>) => {
|
||||
opened.count += 1;
|
||||
return actual.openStoredBackup(...a);
|
||||
},
|
||||
};
|
||||
});
|
||||
|
||||
const saved = {
|
||||
key: process.env.ENCRYPTION_KEY,
|
||||
keys: process.env.ENCRYPTION_KEYS,
|
||||
active: process.env.ENCRYPTION_ACTIVE_KEY_ID,
|
||||
};
|
||||
|
||||
function useKeys(keys: Record<string, string>, active: string) {
|
||||
process.env.ENCRYPTION_KEY = "";
|
||||
process.env.ENCRYPTION_KEYS = JSON.stringify(keys);
|
||||
process.env.ENCRYPTION_ACTIVE_KEY_ID = active;
|
||||
_resetCryptoCacheForTests();
|
||||
}
|
||||
const OLD = "11".repeat(32);
|
||||
const CUR = "22".repeat(32);
|
||||
|
||||
beforeEach(async () => {
|
||||
await truncateAllTables(getPrismaClient());
|
||||
cookieJar.clear();
|
||||
headerJar.clear();
|
||||
opened.count = 0;
|
||||
useKeys({ old: OLD, cur: CUR }, "cur");
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
process.env.ENCRYPTION_KEY = saved.key ?? "";
|
||||
process.env.ENCRYPTION_KEYS = saved.keys ?? "";
|
||||
process.env.ENCRYPTION_ACTIVE_KEY_ID = saved.active ?? "";
|
||||
_resetCryptoCacheForTests();
|
||||
});
|
||||
|
||||
async function seedAdmin() {
|
||||
const prisma = getPrismaClient();
|
||||
const admin = await prisma.user.create({
|
||||
data: {
|
||||
username: "preview-admin",
|
||||
email: "preview-admin@example.test",
|
||||
role: "ADMIN",
|
||||
},
|
||||
});
|
||||
const session = await prisma.session.create({
|
||||
data: { userId: admin.id, expiresAt: new Date(Date.now() + 60_000) },
|
||||
});
|
||||
cookieJar.set("healthlog_session", session.id);
|
||||
return admin;
|
||||
}
|
||||
|
||||
async function preview(id: string) {
|
||||
const { GET } = await import("@/app/api/admin/backups/[id]/summary/route");
|
||||
const res = await GET(
|
||||
new Request(`http://localhost/api/admin/backups/${id}/summary`) as never,
|
||||
{ params: Promise.resolve({ id }) },
|
||||
);
|
||||
return {
|
||||
status: res.status,
|
||||
body: (await res.json()) as {
|
||||
data: { summary: Record<string, unknown> } | null;
|
||||
error: string | null;
|
||||
meta?: { errorCode?: string; keyIds?: string[] };
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/** The counts a full read of the stored copy gives: the old preview. */
|
||||
async function fullReadSummary(id: string) {
|
||||
const { parseBackupPayload, summarizeBackup } =
|
||||
await import("@/lib/validations/backup");
|
||||
const opensBefore = opened.count;
|
||||
const payload = parseBackupPayload(
|
||||
JSON.parse(await readStoredBackup(getPrismaClient(), id)),
|
||||
);
|
||||
opened.count = opensBefore;
|
||||
return summarizeBackup(payload);
|
||||
}
|
||||
|
||||
describe("restore preview kept with the copy", () => {
|
||||
it("the weekly backup writes it, and the preview answers from it", async () => {
|
||||
const prisma = getPrismaClient();
|
||||
const admin = await seedAdmin();
|
||||
await prisma.measurement.createMany({
|
||||
data: Array.from({ length: 25 }, (_, i) => ({
|
||||
userId: admin.id,
|
||||
type: "PULSE" as const,
|
||||
value: 60 + i,
|
||||
unit: "bpm",
|
||||
measuredAt: new Date(Date.UTC(2026, 8, 1, 0, i)),
|
||||
source: "MANUAL" as const,
|
||||
})),
|
||||
});
|
||||
|
||||
const { handleDataBackup } =
|
||||
await import("@/lib/jobs/reminder/backup-handlers");
|
||||
await handleDataBackup([]);
|
||||
const row = await prisma.dataBackup.findFirstOrThrow({
|
||||
where: { userId: admin.id, type: "WEEKLY_AUTO" },
|
||||
});
|
||||
expect(row.preview).toMatchObject({
|
||||
version: 1,
|
||||
copy: `chunks:${row.chunkStreamId}:${row.chunkCount}`,
|
||||
summary: { measurements: 25 },
|
||||
});
|
||||
|
||||
const res = await preview(row.id);
|
||||
expect(res.status).toBe(200);
|
||||
expect(opened.count).toBe(0);
|
||||
expect(res.body.data!.summary).toEqual(await fullReadSummary(row.id));
|
||||
expect(res.body.data!.summary.measurements).toBe(25);
|
||||
});
|
||||
|
||||
it("an upload writes it, and a key dropped later is still named", async () => {
|
||||
const prisma = getPrismaClient();
|
||||
const admin = await seedAdmin();
|
||||
// A note written under `old`, as a copy taken before a rotation carries.
|
||||
useKeys({ old: OLD, cur: CUR }, "old");
|
||||
const note = Buffer.from(encrypt("before"), "utf8").toString("base64");
|
||||
useKeys({ old: OLD, cur: CUR }, "cur");
|
||||
const file = JSON.stringify({
|
||||
schemaVersion: "2",
|
||||
exportedAt: "2026-09-20T00:00:00.000Z",
|
||||
userId: admin.id,
|
||||
measurements: Array.from({ length: 30 }, (_, i) => ({
|
||||
id: `m-${i}`,
|
||||
type: "PULSE",
|
||||
value: 60 + (i % 30),
|
||||
unit: "bpm",
|
||||
measuredAt: new Date(Date.UTC(2026, 8, 1, 0, i)).toISOString(),
|
||||
source: "MANUAL",
|
||||
...(i === 3 ? { notesEncrypted: note } : {}),
|
||||
})),
|
||||
});
|
||||
|
||||
const { POST } = await import("@/app/api/admin/backups/upload/route");
|
||||
const uploaded = await POST(
|
||||
new Request("http://localhost/api/admin/backups/upload", {
|
||||
method: "POST",
|
||||
headers: { "content-type": "application/json" },
|
||||
body: file,
|
||||
}) as never,
|
||||
);
|
||||
expect(uploaded.status).toBe(201);
|
||||
const { data: upload } = (await uploaded.json()) as {
|
||||
data: { id: string; summary: Record<string, unknown> };
|
||||
};
|
||||
const row = await prisma.dataBackup.findUniqueOrThrow({
|
||||
where: { id: upload.id },
|
||||
});
|
||||
expect(row.preview).toMatchObject({ summary: { measurements: 30 } });
|
||||
|
||||
const res = await preview(upload.id);
|
||||
expect(res.status).toBe(200);
|
||||
expect(opened.count).toBe(0);
|
||||
expect(res.body.data!.summary).toEqual(upload.summary);
|
||||
expect(res.body.data!.summary).toEqual(await fullReadSummary(upload.id));
|
||||
|
||||
// The operator drops `old` after the copy was written.
|
||||
useKeys({ cur: CUR }, "cur");
|
||||
const refused = await preview(upload.id);
|
||||
expect(refused.status).toBe(422);
|
||||
expect(refused.body.meta).toMatchObject({
|
||||
errorCode: "backup.key.missing",
|
||||
keyIds: ["old"],
|
||||
});
|
||||
expect(opened.count).toBe(0);
|
||||
});
|
||||
|
||||
it("a copy from before previews is read whole once, then answered from the row", async () => {
|
||||
const prisma = getPrismaClient();
|
||||
const admin = await seedAdmin();
|
||||
await prisma.measurement.createMany({
|
||||
data: Array.from({ length: 5 }, (_, i) => ({
|
||||
userId: admin.id,
|
||||
type: "PULSE" as const,
|
||||
value: 60 + i,
|
||||
unit: "bpm",
|
||||
measuredAt: new Date(Date.UTC(2026, 8, 1, 0, i)),
|
||||
source: "MANUAL" as const,
|
||||
})),
|
||||
});
|
||||
const { handleDataBackup } =
|
||||
await import("@/lib/jobs/reminder/backup-handlers");
|
||||
await handleDataBackup([]);
|
||||
const row = await prisma.dataBackup.findFirstOrThrow({
|
||||
where: { userId: admin.id },
|
||||
});
|
||||
// As an earlier release left it.
|
||||
await prisma.dataBackup.update({
|
||||
where: { id: row.id },
|
||||
data: { preview: Prisma.DbNull },
|
||||
});
|
||||
|
||||
const first = await preview(row.id);
|
||||
expect(first.status).toBe(200);
|
||||
expect(opened.count).toBe(1);
|
||||
expect(first.body.data!.summary.measurements).toBe(5);
|
||||
await vi.waitFor(async () => {
|
||||
const kept = await prisma.dataBackup.findUniqueOrThrow({
|
||||
where: { id: row.id },
|
||||
});
|
||||
expect(kept.preview).toMatchObject({ summary: { measurements: 5 } });
|
||||
});
|
||||
|
||||
const second = await preview(row.id);
|
||||
expect(second.status).toBe(200);
|
||||
expect(opened.count).toBe(1);
|
||||
expect(second.body.data!.summary).toEqual(first.body.data!.summary);
|
||||
});
|
||||
|
||||
it("a replaced copy is described by its own preview", async () => {
|
||||
const prisma = getPrismaClient();
|
||||
const admin = await seedAdmin();
|
||||
const { handleDataBackup } =
|
||||
await import("@/lib/jobs/reminder/backup-handlers");
|
||||
await handleDataBackup([]);
|
||||
await prisma.measurement.create({
|
||||
data: {
|
||||
userId: admin.id,
|
||||
type: "PULSE",
|
||||
value: 61,
|
||||
unit: "bpm",
|
||||
measuredAt: new Date(Date.UTC(2026, 8, 2)),
|
||||
source: "MANUAL",
|
||||
},
|
||||
});
|
||||
await handleDataBackup([]);
|
||||
const row = await prisma.dataBackup.findFirstOrThrow({
|
||||
where: { userId: admin.id },
|
||||
});
|
||||
expect(row.preview).toMatchObject({
|
||||
copy: `chunks:${row.chunkStreamId}:${row.chunkCount}`,
|
||||
summary: { measurements: 1 },
|
||||
});
|
||||
const res = await preview(row.id);
|
||||
expect(res.body.data!.summary.measurements).toBe(1);
|
||||
expect(opened.count).toBe(0);
|
||||
});
|
||||
|
||||
it("a single stored value keeps its preview when it is converted to pieces", async () => {
|
||||
const prisma = getPrismaClient();
|
||||
const admin = await seedAdmin();
|
||||
// The oldest stored form: the whole file as one encrypted value.
|
||||
const row = await prisma.dataBackup.create({
|
||||
data: {
|
||||
userId: admin.id,
|
||||
type: "MANUAL_UPLOAD_1",
|
||||
data: encrypt(
|
||||
JSON.stringify({
|
||||
schemaVersion: "1",
|
||||
exportedAt: "2026-05-09T10:00:00.000Z",
|
||||
userId: admin.id,
|
||||
measurements: [
|
||||
{
|
||||
type: "WEIGHT",
|
||||
value: 80,
|
||||
unit: "kg",
|
||||
measuredAt: "2026-05-08T07:00:00.000Z",
|
||||
source: "MANUAL",
|
||||
},
|
||||
],
|
||||
}),
|
||||
),
|
||||
},
|
||||
});
|
||||
|
||||
const first = await preview(row.id);
|
||||
expect(first.status).toBe(200);
|
||||
expect(opened.count).toBe(1);
|
||||
await vi.waitFor(async () => {
|
||||
const kept = await prisma.dataBackup.findUniqueOrThrow({
|
||||
where: { id: row.id },
|
||||
});
|
||||
expect(kept.preview).toMatchObject({ summary: { measurements: 1 } });
|
||||
});
|
||||
|
||||
const { convertSingleValueBackup } =
|
||||
await import("@/lib/export/store-backup-blob");
|
||||
expect(await convertSingleValueBackup(prisma, row.id)).toBe("converted");
|
||||
const converted = await prisma.dataBackup.findUniqueOrThrow({
|
||||
where: { id: row.id },
|
||||
});
|
||||
expect(converted.preview).toMatchObject({
|
||||
copy: `chunks:${converted.chunkStreamId}:${converted.chunkCount}`,
|
||||
summary: { measurements: 1 },
|
||||
});
|
||||
|
||||
const second = await preview(row.id);
|
||||
expect(second.status).toBe(200);
|
||||
expect(opened.count).toBe(1);
|
||||
expect(second.body.data!.summary).toEqual(first.body.data!.summary);
|
||||
});
|
||||
});
|
||||
Reference in new issue
Block a user