mirror of
https://github.com/MBombeck/HealthLog.git
synced 2026-11-08 12:32:43 +00:00
Marc-André Bombeck
3118cf240c
The compression that landed last release was real and was measured in the wrong place. On a development machine with a multi-gigabyte heap the pass finished; the container the job runs in is capped at 1 GB, which gives V8 a 524 MB limit, and there the same pass died of memory about fifteen seconds in. Because the job shares the application process, that was not one failed backup — it was a restart, and every signed-in session on the host went with it. Reproduced before touching anything, under `--max-old-space-size=450` against a seeded account of 445 000 measurements, 30 000 mood entries and 60 000 intake events: `FATAL ERROR: Reached heap limit` at 34 s. The same fixture doubled to 890 000 measurements now completes inside a 296 MB limit, holding 16 MB of live heap where the old writer held 168 MB. The document is written incrementally now. `buildFullBackupPayload` takes a `deferBulk` mode that declares the three unbounded tables instead of reading them, and `streamFullBackupJson` walks the payload in key order, pulls those tables through page by page, and releases every other section as soon as its JSON reaches the sink. `packBackupBlobStreaming` feeds that into gzip and an AES-256-GCM stream, so nothing between the database rows and the stored value ever exists whole. Both writers go through the same per-row serialisers, and the round-trip suite pins their output byte for byte rather than trusting that they agree. The intake and mood reads are ordered totally now. A keyset cursor needs it, and a partial order also let two exports of one account differ over a tie, which every multi-dose day has. The streamed ciphertext is a new shape at rest, because GCM only produces its tag once the last block is in and a leading tag is what makes the old layout unstreamable. It carries a marker no older writer can produce, so the two families are disjoint by construction rather than by sniffing. All three stored shapes — plain, compressed, streamed — restore, proved both ways. The writer also watches its own live heap and stops at 80 % of V8's limit. An account too large for its host is now one failed backup, counted in the run's meta, instead of a restart for everybody on it. The stored column is the ceiling that is left: `data_backups.data` is a single `text` value, so the finished artifact has to exist as one string, and that is the only part of the job still growing with the record. The operator runbook says so, with the number at which it starts to matter.
docs/ — internal reference + operator playbooks
The user-facing documentation lives at https://docs.healthlog.dev — that's the canonical surface for self-hosting guides, integration walkthroughs, AI-provider setup, and the doctor-report workflow.
The files in this directory are the internal operator playbooks, audit notes, and machine-readable specs that ship alongside the source:
api/— the OpenAPI 3.1 spec for the native-client subset (iOS DTO codegen target). Seeapi/README.mdfor the preview commands. Also home to the MCP-server contract:api/mcp-capabilities.md(tools, resources, prompts, write model) andapi/mcp-skills.md(building a connector / skill).ops/— operator runbooks: deploy, backup / restore, encryption-key rotation, password reset and account recovery, env checks, TLS pinning, data repair. These are what an operator reaches for during an incident.self-hosting/— horizontal-scaling notes (HEALTHLOG_PROCESS_TYPE=web|worker|all) and the deploy-pipeline recipe. Routine install steps stay ondocs.healthlog.dev. Includesself-hosting/mcp.md— enabling the off-by-default MCP connector and pointing Claude.ai / ChatGPT / Claude Desktop at it, andself-hosting/sso.md— OIDC SSO login: IdP setup, the identity-pinning security model,OIDC_ONLYconsequences, and the break-glass runbook. Alsoself-hosting/geolite2.md— bringing your own offline GeoLite2 databases viaGEOLITE2_DIRinstead of sending login IPs to a third-party lookup. Andself-hosting/account-sharing.md— letting one account on the instance open another's health record: the three access levels, the eight sections a grant can be narrowed to, managed profiles for somebody with no login and the guardians who look after them, what revocation does and does not undo, and which trust properties of the instance are unchanged by any of it.migration/— migration notes. Currently only v1.3 to v1.4; later releases are covered byCHANGELOG.mdand https://docs.healthlog.dev.integrations/— per-integration reference (Withings, Fitbit, Google Health, WHOOP, Apple Health, AI providers, data import).adr/— architecture decision records.security/— threat-model notes.diagrams/— architecture diagrams rendered through docs.healthlog.dev.releases/— release imagery.apple-store-connect-checklist.md— the iOS submission gate.codex-protocol-spec.md— Codex / ChatGPT OAuth protocol reverse-engineering notes.ui-guidelines.md— UI / a11y house style.
If you're a self-hoster looking to spin up an instance, start at https://docs.healthlog.dev or jump to the Quick Start in the project README.