perf(backup): take the weekly copy's preview from the writer, not a second parse

Since v1.39.6 the preview scanner parsed the JSON on its way into the
pieces, kept every section but the three bulk tables, and validated the
whole document at the end: a second copy of the record beside the
writer's own, inside the weekly job. On an account with a document vault
that is most of the record.

The writer now shows each section and each bulk row to an observer just
before writing it. The preview collects key uses from that and takes its
counts from the writer's own FullBackupCounts; nothing is parsed or kept.
An upload hands over the summary and key uses it has already validated.
The round trip pins the writer's counts and key uses to a full read of
the copy on an account with a row in every model, and a memory test holds
what the pass keeps once the producer is done.

Measured on 70,000 measurements, 3,000 mood entries and 40 one-megabyte
documents under --max-old-space-size=524: held after the producer
56 MB before, 3 MB after; peak heapUsed 398 MB before, 334 MB after.
This commit is contained in:
Marc-André Bombeck committed 2026-10-04 15:56:56 +02:00
1 parent 5345c8be1e
commit d55e329c48
10 files changed
+426 -144

No files matched your search

+13 -1
View File
@@ -221,6 +221,9 @@ export const POST = apiHandler(async (request: NextRequest) => {
let measurementCount = 0;
let owner: { id: string; username: string } | null = null;
// Filled while the file is read; the preview is the summary this route
// answers with and the key uses it already holds (`backup-preview.ts`).
const keys = new BackupKeyIdCollector();
let created: { id: string };
try {
created = await storeBackupBlob(
@@ -229,6 +232,16 @@ export const POST = apiHandler(async (request: NextRequest) => {
userId: admin.id,
type: uploadType,
ownerAfterRead: () => owner!.id,
preview: () =>
payload
? {
summary: {
...summarizeBackup(payload),
measurements: measurementCount,
},
keys,
}
: null,
},
async (write) => {
// One pass: every chunk is stored as it is checked, and a check that
@@ -240,7 +253,6 @@ export const POST = apiHandler(async (request: NextRequest) => {
}
}
let firstWithoutId: number | null = null;
const keys = new BackupKeyIdCollector();
let scanned;
try {
scanned = await scanBackupJson(storedAsRead(), {