Platform-wide hardening: the failure classes found live this week hunted down
across every integration.
Fitbit carried the same tombstone wedge fixed for Google — probe now matches
soft-deleted rows and the update resurrects them; the comment describing a
partial unique index that never existed is corrected. Sleep segment ids are
stable across re-scoring for Withings (session+segment-start), WHOOP/Polar
(one segment per stage, no positional index) and Oura (run-start keyed), each
with a session-scoped sweep that soft-deletes rows a re-score orphaned — so
legacy and affected nights heal as they are re-read.
One platform rule for deleted rows: source-owned readings resurrect on
re-import (Withings, WHOOP, Oura, Polar, Nightscout, mood webhook, CSV
re-imports, Apple stats day-totals); Apple sample-grain deletions stick per
the paired-client contract, and the mood bulk route's tombstone match is now
a true no-op reported as duplicate. Import counts reconcile against actual
createMany counts.
Google lifecycle: a dead token registers on the cycle's hard-fail ledger, so
an expired connection parks instead of stamping success hourly; backfill and
sleep-repair markers stamp only after a clean pass (pg-boss retries are now
real); failed measurement writes hold the watermark; no-op updates are
skipped; mapping.md tells the current truth.
Scale: doctor-report reads only consumed columns and enabled sections;
exports and backups keyset-paginate; achievements tally via SQL buckets;
series charts day-bucket dense kinds beyond 90 days; BP pairing is a linear
two-pointer; distinct-type scans run server-side; the graded-series cold
fallback is bounded; the fitbit defer tracker loops instead of spreading.
The CSV importer ran a serial prisma.upsert/create per accepted row, so
an N-row file cost N round-trips and a few-thousand-row import could
approach the request timeout. Adopt the /measurements/batch shape:
pre-fetch existing rows under the IMPORT source in one query per
dedup-key shape (externalId and natural-key paths), partition into bulk
createMany + grouped updateMany, and reconstruct the per-line
inserted/updated/duplicate envelope from the partition. In-file repeats
and re-import idempotency preserved; response contract unchanged.
Lay the account-security and data-sovereignty substrate for v1.23. No
user-facing behaviour yet — schema, crypto registry, redaction, and the
note-encryption data path only.
Schema + migrations:
- 0199 — User totp_secret_encrypted / totp_confirmed_at / mfa_enforced /
passkey_upgrade_nudge_dismissed; Session mfa_verified_at / last_active_at;
Passkey last_used_at. All additive, nullable / defaulted.
- 0200 — mfa_challenges (single-use step-up ticket; only the ticket hash is
stored, so no session or token is ever minted before the second factor),
mfa_recovery_codes (Argon2id at rest), webauthn_mfa_credentials (second-factor
keys, kept separate from passkeys, which remain the passwordless-primary set).
- 0201 — mood_entries.note_encrypted + measurements.notes_encrypted (BYTEA),
matching the existing free-text encrypted-note convention.
Free-text health notes:
- New writes encrypt the mood / measurement note into the *_encrypted column and
null the legacy plaintext column; reads prefer the ciphertext and fall back to
plaintext only for not-yet-migrated rows, via a single read/write helper.
- A boot-time job migrates existing plaintext rows in an idempotent,
transactional, fail-closed pass (encrypt, then null the plaintext, per row in
one transaction; safe to re-run; a row with content is never left both-null).
- The legacy plaintext columns are kept this release and dropped in a follow-up
once the backfill reports zero remaining rows — the same boundary the
key-rotation playbook uses before dropping a legacy key.
Crypto coverage:
- Register the three new encrypted columns and wire them into the key-rotation
script so a rotation never orphans them.
- Extend the observability and log redaction denylists so TOTP secrets, MFA /
recovery / backup codes, and export passphrases never reach a wide event.
One-shot prettier pass so the new format:check gate starts from a clean
baseline. Formatting only — no behaviour change. Generator-owned and
vendored artifacts stay untouched via .prettierignore.
A self-hoster migrating from a spreadsheet or a meter export overwhelmingly
has CSV, which neither the JSON nor the Apple Health import covers. Add
`POST /api/import/csv` reusing the existing write loop and rollup re-fold.
The documented header is `type,value,unit,measuredAt[,glucoseContext,notes,
externalId]`, order-independent. measuredAt must carry an explicit ISO-8601
offset — a row without one is skipped rather than guessed — and is bounded by
the shared entry-instant validator. Glucose accepts mmol/L (converted to
canonical mg/dL) and weight accepts lb (converted to kg); any other
non-canonical unit is skipped, never mis-stored. An optional externalId
column makes a re-upload idempotent; without it a re-upload duplicates.
The response carries a per-row status envelope so a 2000-row file with three
bad rows imports 1997 and names the three. `?dryRun=1` validates and previews
the outcome without writing. The parse, validation, and unit conversion live
in a pure module so they are unit-tested without a database.
Settings -> Export & Import gains a third CSV card mirroring the JSON and
Apple Health cards, with upload, paste, preview, and a downloadable example.
The new route is documented in the OpenAPI contract.
Marc-André Bombeck