Every finding behind this work is a two-ended change whose ends shipped in
different releases with nothing holding them together, and every gate the repo
runs proves one end: typecheck proves the export exists, knip proves it is
imported, the OpenAPI check proves the schema matches. None of them notice when
the second end never arrives.
Two questions here can be answered exactly, so they become tests.
The account endpoint rides every app boot and carries the user's preferences.
Its field names are read out of the response literal and each one must be read
by a client module — property access or destructuring, comments stripped,
because a doc comment claiming a consumer is what the remembered practice name
had for its whole dead life. That field would have failed this on the day it
shipped. One field is allowlisted today with its reason written out: the
insurer institution number has no web surface at all, which the entry keeps
visible rather than papering over.
The measurement backup select is now a named constant, compared field by field
against the model in the schema. Both sides are literals, so the comparison is
exact — no name heuristics, nothing that can drift towards accepting. A column
added to the model and forgotten by the backup fails at the commit that adds
it, which is how the aggregation provenance went missing. Scoped to the one
model; extend it as each further model becomes backup-relevant.
The disaster-recovery backup selected every Measurement scalar except the
aggregation fields, so a restore brought every `stats:` aggregate back with no
provenance and the Apple Health authority ladder came back flat: an export.xml
source-day estimate could then overwrite a native HealthKit statistic.
`aggregationProvenance` now rides the select, the disaster-recovery
serialization, the payload schema and the restore mapping, following
`externalSourceVersion` exactly — including staying out of the reduced
portable shape, which carries the human-readable subset only. The schema field
is optional, so every backup written before this parses unchanged.
Restoring one of those older backups no longer recreates the ambiguity
migration 0263 repaired: an Apple Health row under a `stats:` identity with no
provenance comes back as LEGACY_UNKNOWN, the sentinel the cumulative drain
already writes in that situation, so a native sync or archive reprocess can
repair it. The gate is scoped as 0263 scoped its backfill — other providers
mint `stats:` identities for their cumulative dailies and have never carried
provenance, and ordinary point rows keep NULL.
Merge the eight reviewed audit fixes that were cut a week ago and never
landed. Main has moved on since, so each fix was re-checked against the
current tree and each of the five conflicts resolved by understanding
both sides.
Kept, still open on main:
- Proxy early exits now carry the transport security header subset via a
shared helper. Reconciled with main's /api/health worker-only carve-out.
- Batch measurement ingest bounds the entry instant so a future-dated row
can no longer win "latest reading" or poison the rollup buckets.
- Web Push dials through the pinned safeFetch dispatcher with
requirePublicHost, closing the last user-controlled-host egress.
- Mood bulk upsert and its tag/factor links run in one transaction, so a
rejected rating rolls the mood row back instead of leaving a tagless row.
- Full backup carries NutrientIntakeDay with its unit and source. Folded
into main's portable-export / disaster-recovery split.
- Live daily-series fallback restores the min/max spread band the tiered
read already emitted.
- Coach stream aborts its in-flight reader on unmount and resets send state
on a conversation switch.
- Manual water writes share the day-bound helper, nutrient add clamps after
the increment, and the thiamin cap drops to a real supplement ceiling.
- Compliance point window widens symmetrically instead of wrapping overnight.
- Oura, Polar and Fitbit token helpers classify an empty or non-JSON 2xx
body as a handled transient error.
- WebAuthn relying-party fallback to localhost is gated to development.
- Chart-token strip list is derived from the allowlist and pinned by a test.
- Regression tests pin the snapshot-cache tenant key, the GlitchTip
path-secret redaction and the Withings credentials delete.
Superseded by main, tests re-pointed at the current shape:
- The intake CSV import was rewritten on main to run through a background
job with a single-active-job admission gate. The future-date bound is
re-applied at request validation and the per-user limiter is kept for
parity with the bulk twin; the old synchronous-write test is replaced.
The water quick-add validated calendar realism but never the upper day bound the
batch route applies, so a far-future day was accepted and — every read filters
`day >= since` with no ceiling — became the permanent latest day on the settings
card and the dashboard tile. Hoist the bound into one shared helper both routes
use.
`mode: "add"` bounded a single request to the catalog's plausible daily max but
nothing across requests, so repeated quick-adds drove the stored total
arbitrarily high while the batch route enforced the same cap strictly. Clamp
after the increment so the write stays atomic.
Thiamin's cap sat at exactly 1000 mg while its PRI is exactly 1.0 mg, so a
microgram-read-as-milligram sample landed on precisely the cap and the `>`
comparison let it through. 500 still clears any real supplement dose.
NutrientIntakeDay was in no export path, which contradicted the schema's own
reason for denormalising the unit column. Carry it in the full backup with its
unit and source — source is part of the composite key, so without it a restore
cannot tell a manual entry from a synced day total.
The full-backup export (plaintext + passphrase-encrypted, and the weekly
auto-backup) only ever covered measurements, medications, mood, and cycle
data. Labs, illness episodes, allergies, family history, and workouts were
silently missing, and the export UI implied "everything" was included.
Adds a shared records-backup section (src/lib/export/records-backup.ts) that
both the on-demand export routes and the weekly data-backup worker read from:
lab results + the biomarker catalog, illness episodes (including flares via
parentConditionId) with their day-logs, allergies, family history, and
workout summaries. Every read is owner-scoped and soft-delete-filtered;
encrypted free-text columns decrypt fail-soft so one bad row never aborts
the whole backup.
Document original files and workout GPS/per-sample time series stay
deliberately out of the JSON payload (size). The payload now carries an
explicit manifest disclosing exactly that, and the export UI states it in
plain text instead of implying completeness. These new domains are
export-only for now — admin restore still only recreates the original
measurements/medications/mood/cycle domains; the payload validates and
reports counts for the rest, but restoring a snapshot does not yet recreate
them.
Also fixes the admin backup-upload total to count every domain, not just
the original four.
Platform-wide hardening: the failure classes found live this week hunted down
across every integration.
Fitbit carried the same tombstone wedge fixed for Google — probe now matches
soft-deleted rows and the update resurrects them; the comment describing a
partial unique index that never existed is corrected. Sleep segment ids are
stable across re-scoring for Withings (session+segment-start), WHOOP/Polar
(one segment per stage, no positional index) and Oura (run-start keyed), each
with a session-scoped sweep that soft-deletes rows a re-score orphaned — so
legacy and affected nights heal as they are re-read.
One platform rule for deleted rows: source-owned readings resurrect on
re-import (Withings, WHOOP, Oura, Polar, Nightscout, mood webhook, CSV
re-imports, Apple stats day-totals); Apple sample-grain deletions stick per
the paired-client contract, and the mood bulk route's tombstone match is now
a true no-op reported as duplicate. Import counts reconcile against actual
createMany counts.
Google lifecycle: a dead token registers on the cycle's hard-fail ledger, so
an expired connection parks instead of stamping success hourly; backfill and
sleep-repair markers stamp only after a clean pass (pg-boss retries are now
real); failed measurement writes hold the watermark; no-op updates are
skipped; mapping.md tells the current truth.
Scale: doctor-report reads only consumed columns and enabled sections;
exports and backups keyset-paginate; achievements tally via SQL buckets;
series charts day-bucket dense kinds beyond 90 days; BP pairing is a linear
two-pointer; distinct-type scans run server-side; the graded-series cold
fallback is bounded; the fitbit defer tracker loops instead of spreading.
Add an opt-in to download the full backup as an encrypted archive instead
of a plaintext JSON file. The user supplies a passphrase over the request;
the server derives a 256-bit key with Argon2id and seals the export bytes
with AES-256-GCM into a self-describing HLX1 archive (magic, version, KDF
params, salt, iv, tag, ciphertext), mirroring the off-host backup envelope.
POST /api/export/encrypted returns the binary archive and is step-up gated
for accounts with a second factor; single-factor accounts use a normal
session. The passphrase is never stored and never logged, and there is no
server-side recovery — the UI states this. A standalone decryptor
(scripts/decrypt-export.ts) opens the archive with the passphrase outside
the app. The plaintext full-backup route now shares one payload builder so
both variants emit the same restore-compatible shape.
Marc-André Bombeck