feat(import): refuse an Apple Health import the heap cannot carry

Before the multi-minute unpack starts, compare the archive's declared
uncompressed export.xml size against the V8 heap limit. The streaming
parse keeps resident state span-bounded, so the model is generous —
a fixed 192 MiB base plus 1/64 of the XML size; the 8 GiB extraction
cap needs ~320 MiB, which any default heap clears. The check only
refuses the truly impossible (an operator-pinned tiny heap against a
huge export) and does so with an actionable failureReason naming
NODE_OPTIONS=--max-old-space-size, instead of an OOM-killed worker
and a job stuck in a running state (#775).

Document the memory model, the insufficient_memory refusal, and the
interrupted-import reconcile in the self-hosting scaling guide.
This commit is contained in:
Marc-André Bombeck committed 2026-08-13 06:16:44 +02:00
1 parent dd02f983cb
commit 50ff90a409
4 files changed
+179 -1

No files matched your search

+30
View File
@@ -56,6 +56,36 @@ know:
Without a shared staging volume, imports in split mode cannot
complete.
## Apple Health import memory
The import parses `export.xml` as a stream: the archive is never
buffered whole, records flush to the database in fixed-size batches,
and what stays resident scales with the data's calendar span, not its
record count. Multi-year exports with millions of records parse within
the default Node.js heap.
Before parsing starts, the worker compares the archive's declared
uncompressed size against the runtime heap limit. If the limit is too
low to carry the export — realistically only when
`--max-old-space-size` was pinned far below the default — the import
fails immediately with a `failureReason` starting with
`insufficient_memory`, naming the required and available amounts.
Raise the limit on the container that runs the worker and re-upload:
```yaml
# docker-compose.yml
services:
app:
environment:
NODE_OPTIONS: --max-old-space-size=1024
```
An import interrupted by a container restart or an out-of-memory kill
does not stay stuck: the worker reconciles non-terminal import jobs at
boot and every 15 minutes, flipping orphaned rows to `failed`
(`interrupted_by_restart`) so the UI offers a retry instead of an
endless spinner.
## Healthchecks
- `app` (web) healthcheck: `wget /api/version` every 30s.