An account of 1.25 million measurements could not export its data or be
restored from any backup (#1031). Measured in a 1 GB container:
- The full export and the passphrase export built the whole payload and
its JSON string in memory; the app ran out of heap and restarted.
- The weekly backup's JSON is 662 MB, longer than any string V8 can hold,
so `unpackBackupBlob` threw on every read. Restore, download and the
restore preview all answered "could not be decrypted".
- Upload accepted 10 MB, a fraction of a large export.
Every reader and writer of a backup file now streams it:
- `scanBackupJson` reads a backup's bytes once, keeps every section but
the named bulk arrays, and hands their elements out one at a time.
`readStreamedBackup` uses it twice: a first pass that validates each
measurement against the element schema and keeps the rest of the file,
and a second pass that hands the measurements to the writer in batches.
Stored copies and off-host objects are authenticated whole on their
compressed ciphertext before any plaintext is read; only the
decompression is streamed.
- Restore writes the measurements with one `INSERT … FROM unnest` per
batch. Prisma caches a query plan per `createMany` row count, and one
1 000-row plan kept about 145 MB alive, which alone put the restore past
the heap limit. The transaction budget grows with the measurement count.
The awards and ECG sections are handed the ids they reference that were
written, not every written id.
- The export routes write the file into the response as it is produced.
The passphrase export spools its ciphertext to a temporary file because
the HLX1 format puts the tag first; the format is unchanged, so existing
archives open as before.
- Upload accepts the raw body, JSON or gzip, up to the 512 MB request
ceiling, checks it as it arrives and stores it in the same pass; the admin
page compresses the file in the browser before sending it. Multipart stays
for small files.
- Download streams the stored JSON; the preview counts measurements from
the stream. The monthly restore drill and `scripts/restore-backup.ts`
read the off-host object the same way; the script can write `.json.gz`.
On the seeded account in a 1 GB container: export 25 s at 389 MB peak,
passphrase export 25 s, both uploads 19 s, restore of the weekly backup
94 s at 554 MB peak, and export, upload and restore round-trip to exactly
the same rows. The encrypted-column guard now also recognises the raw SQL
statement that writes `DataBackup.data`.
A disaster-recovery snapshot carries the singleton `app_settings` row —
registration, the MFA requirement, the default locale and timezone, module
availability, the notification and AI configuration, the document size cap and
quota — and the restore wrote it back as a side effect of restoring one
account. An operator putting a single record back on Wednesday reconfigured the
whole host from Monday's file, silently, with nothing on screen saying so.
Rebuilding a host from a snapshot is a real case, so the write stays: it is now
`restoreInstanceSettings: true` on the request and a checkbox beside the typed
gate, off unless the operator ticks it. Restoring an account and reconfiguring
the installation are two decisions, so they are two answers.
The 422 confirmation sentence and the dialog copy drop the instance-wide claim;
the opt-in carries it instead, in all seven catalogues. Integration coverage
runs both arms: without the flag the settings row keeps its own values, with it
the file's values land.
The weekly pass can stop and leave no trace where anyone looks. The
handler's own error never runs when the job is killed rather than thrown;
`recordError()` increments an unnamed counter; and the admin status page's
failing-queue panel looks back 72 hours, which for a weekly queue means a
permanent failure is on screen three days in seven. The backups page
itself showed an ordinary row with an ordinary timestamp, because a copy
made six weeks ago reads exactly like Sunday's until somebody subtracts
the dates.
The listing now carries two facts that fail at different times: the age of
the newest scheduled copy, which never leaves any retention window, and
the terminal state of the last scheduled run, which is the only place the
reason is written down. `readLastQueueRun` takes the newest run rather
than a window, so a weekly queue's outcome stays readable until the next
one replaces it, and it unwraps the value pg-boss stores a thrown error
under, so the page says "job timed out" instead of raw JSON. The
failing-queue reader gets the same unwrap.
Manual uploads are excluded from the cadence judgement by the shape of the
input, so an ad-hoc upload cannot make a dead cron look alive, and a fresh
instance is not called overdue for having no copy yet.
The listing also stops reading every blob into the web process to measure
it; `octet_length` answers that in the database, which on an instance with
large records was its own way of running out of memory.
The client bundle carries only the active locale — resolved on the
server and handed to the provider with no hydration flash, halving the
dashboard route to under a megabyte parsed (288 KiB gzip). The auth
shell renders the app chrome immediately instead of a spinner while the
session resolves in parallel, the widgets request folds into the
snapshot it duplicated, and insight pills prefetch their sub-pages —
a first visit to a sub-page lands in about ninety milliseconds instead
of nearly a second.
Bumps version to 1.4.15 and adds the corresponding CHANGELOG entry
covering backup completeness, achievements UI, onboarding tour,
doctor-report v2, auto-deploy on GHCR push, empty-states sweep,
notification + integration status UI, AI hallucination guardrails,
mobile UX fixes, and the BD-Zielbereich percentage fix.
Includes the formatting drift left over by the v1.4.15 development
phases (Prettier sweep on touched files) and a typecheck fix in
`dashboard-layout.test.ts` so `pnpm typecheck`, `pnpm lint`,
`pnpm format:check`, `pnpm test`, and `pnpm test:integration` all
pass green on the release tag.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces blank rectangles in 6 admin sub-routes with the EmptyState
primitive. Each surface now shows an icon, localized title +
description, and (where it makes sense) a primary action so a fresh
admin always knows what to do next.
- /admin/users — Users icon, filter-aware copy, "Show all users" reset
CTA when a filter is active.
- /admin/backups — Database icon, "Backup now" duplicate CTA inside the
empty card so the action is right next to the explanation.
- /admin/login-overview — ScrollText icon, splits "no auth events" from
the safer "no failed sign-ins"; reset CTA on the failed-only filter.
- /admin/api-tokens — Key icon, copy points to native client sign-in
and Settings -> Account where tokens are issued.
- /admin/feedback — Inbox icon, compact-size, per-tab description.
- /admin overview audit-preview — compact + plain so it nests inside
the parent card without doubling the dashed border.
New unit tests for the users + backups paths assert the primitive
mounts (role=status / dashed border), the localized strings render,
and the reset/CTA actions appear only in the right contexts.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Marc-André Bombeck