HealthKit's basalEnergyBurned sat in the deferred set, so the batch route
refused it per entry and acceptedTypes left it out. It now maps to a new
BASAL_ENERGY_BURNED type in kcal (migration 0493), beside active energy
and never added to it.
It is cumulative like active energy: summed per day on the read path and
in the rollups, drained into one daily row, imported from export.zip,
listed and charted with its own label in every locale, part of the report
selection and the backup, and readable over MCP. It carries no personal
record direction and feeds no score, insight or Coach snapshot.
Health Connect is left unmapped on purpose: its basal metabolic rate is a
rate at one instant, not a day's energy. No other integration reports a
resting figure of its own.
Refs #1340
Both wearables wrote their period days with a bare create and so skipped
the seam every other cycle writer goes through: a first bleeding day never
opened its cycle and no day was attributed to one. An account whose
periods came from a watch or a ring held day-logs only, with no cycle
start, no lengths and no forecast.
The create now goes through one shared helper that keeps the
create-only-if-absent rule and then runs the same attribution and
`ensureCycleForBleedingDay` inference as the shared upsert. The hourly
legs and the newest-first Garmin backfill both use it; a finished Garmin
period also stamps its last day on a cycle that has no period end yet.
Rows imported before this change are repaired by a one-shot job sent from
boot discovery while candidates exist. It opens or attributes cycles
oldest first per account, touches no day-log content, and converges to an
empty candidate set.
Refs #1333
Garmin Connect now offers a Personal API: a person creates a key for
their own account and any client can buy short-lived tokens with it.
HealthLog takes that key in a new Garmin card, checks it with Garmin
before storing it sealed on the user row, and polls every hour.
- Transport: key exchange with a per-process token cache keyed on the
key, one re-exchange on a refused token and then reauth_required, the
429 Retry-After carried to the caller, the China host tried when the
global one does not know the key, answers read with a byte cap, all
egress through safeFetch to the two fixed hosts.
- Sync: daily summaries, sleep summaries, HRV, weigh-ins and the
activity search, chunked to each resource's documented range, read one
after another and stopped at the first rate limit; today's menstrual
cycle summary adds logged period days the person has not logged.
- Backfill: a resumable walk through the shared admission lane that
learns where the history starts from the daily summaries and the
activities, waits out Garmin's rate limit and stamps its marker.
- Types: GARMIN source; BODY_BATTERY and STRESS_LEVEL for Garmin's own
day scores, kept apart from the computed STRESS_SCORE; TRAINING_READINESS
and ACUTE_TRAINING_LOAD for figures a bridge pushes through the ingest
token. All four chart on the recovery page and sit on the ladders,
units, ranges, labels and report groups like the device-native scores.
- Key expiry: learned from a token capped short of a day, else estimated
from the validity picked at connect; the card warns a week ahead and
asks for a new key once Garmin refuses the old one.
- Ledger, health, freshness, watchdog, backup classes, wipe plan,
encrypted-column registry, rotation script, OpenAPI and all ten
locales carry the new integration. The old "no Garmin connector" note
is gone.
The status prompts carry the unattended flag for background calls, the
queue list holds the lab date repair, and an account whose only way to
confirm is its identity provider is now offered that way.
A full rollup backfill that failed on every boot showed only on the
admin page's card of failing background jobs. From the account's side
the only symptom was empty dashboard cards, and nothing pointed the
operator at the card (#1298).
A new hourly evaluation reads the newest terminal runs of every queue
whose failure has no account-facing surface (the system audience in
subsystem-surface.ts) from pg-boss's own ledger. A failed run adds its
attempts, retries included, and the first successful run ends the
streak. Three failed attempts in a row send every admin a SYSTEM_ALERT
once, a reminder at most once a day while the queue keeps failing, and
one notice when it runs successfully again: the decision the backup
health notices use, with the same compare-and-set claim, so two workers
never send a notice twice.
The memory is one row per announced queue in system_job_alert_notices
(migration 0490). The notice names the queue, the streak and the last
error, and opens the admin status page.
The translation files merge key by key (no key changed on both sides), the
backups list carries both the database dump and the pre-wipe copy window,
and the backup-restore doc keeps the shorter rewrite plus the dump in the
bucket and a lifecycle rule that covers database/.
Deleting an account, a guardian deleting a managed profile, and "Delete
all data" were final the moment they were confirmed: the account row
cascaded into every table and every stored copy, and the off-host objects
were purged in the same breath. Nothing short of rolling the whole
instance back could return one account.
ACCOUNT_DELETION_GRACE_DAYS (14 by default, at most 30, 0 erases at once
as before) now sets a window:
- Account and managed-profile deletion lock the account (migration 0488:
deletion_requested_at, deletion_purge_at, deletion_requested_by_id).
Every session, refresh token, API token, connector, share link and
step-up elevation is revoked in the same transaction, and the record,
its stored copies and its off-host objects are left as they are.
- Sign-in is refused with 403 auth.account.deletion_pending and the
erasure date, after the credential was proved and before a second
factor is asked for. The session and Bearer resolvers and the acting-
account resolver refuse a pending account too, so a credential minted
in the moment before the lock is useless.
- POST /api/admin/users/{id}/undo-deletion (cookie-only, audit-logged)
clears the lock. The admin users list reports the pending state.
- The hourly account-deletion-purge job erases accounts whose window has
ended, exactly as the immediate deletion did, with the off-host purge
request committed alongside. The Guardian invariant is decided again;
a refused erasure stays pending and is reported.
- A wipe first stores the whole record as a PRE_WIPE_<ms> copy through
the writer the weekly copy uses, keeps it out of the wipe, and holds
its off-host purge back with a new not_before on the request. The
ordinary restore brings the record back; restoring that copy withdraws
the held purge. The purge job deletes the copy when its window ends.
Background work skips accounts pending deletion through one shared
predicate (src/lib/account-deletion/active-account.ts): every reminder,
digest, Coach, sync, backfill, rollup, score and backup enumeration
filters with it, and account-deletion-enumeration-guard.test.ts fails on
a query in background work that neither filters nor says why it need
not. Notification delivery and the AI capability loader refuse a pending
account as chokepoints behind it, and the Withings, WHOOP and Telegram
webhooks and the notification quick actions ignore one.
The privacy summary publishes the window. Two pieces of the reminder tick
and the off-host pass moved into modules of their own to keep the files
inside their size ratchet.
The step confirmed one key, the active application key. A restore needs
more: off-host copies are sealed under BACKUP_ENCRYPTION_KEY, a copy
written before a rotation needs the retired application key it was
written under, and every session, token and share link hangs off
API_TOKEN_HMAC_KEY. An operator could do everything the step asked and
still be unable to read a single backup.
The recovery kit lists every key a restore of this server needs, by id
and fingerprint, never key material:
- each id in ENCRYPTION_KEYS, plus any retired id a stored copy
(innerKeyIds) or an off-host object inside the bucket's retention
(OffhostBackupKeyUse) still lists, also once the server dropped it;
- BACKUP_ENCRYPTION_KEY and each BACKUP_ENCRYPTION_PREVIOUS_KEYS entry,
named by the id their objects record;
- API_TOKEN_HMAC_KEY.
Confirming records the set the admin was shown and refuses a stale one
with 409 recovery.kit.stale. The kit is due again as soon as a key it
lists is not in that set (a rotation, a new backup key, a re-keyed
install); a key that merely drops out does not make it due. "Check my
copy" now checks any key in the kit.
Migration 0487 carries a confirmation given under the old step forward
as a kit holding that key, so that part stays confirmed, and drops the
four app_settings columns. It also creates the two tables the readiness
evaluation uses (restore_drill_outcomes, recovery_readiness_notices).
The routes move to /api/admin/recovery-kit; the reader of the keys the
backups still need moves out of the encryption status route into
src/lib/recovery/backup-key-needs.ts so the kit and the status view
share one definition.
The per-account copies roll one account back on the same server; nothing
the app made could rebuild a lost one. A scheduled job now runs pg_dump
over the whole database (every schema, pg-boss included) into
DB_BACKUP_DIR as healthlog-<UTC time>-daily.dump, with a manifest beside
it: app version and build, newest migration, Postgres version, size and
SHA-256, exact row counts of every table, and the fingerprints of the
keys the database needs.
The counts read the same exported snapshot pg_dump reads, so a dump
taken while the app writes still matches its manifest. The archive is
written as .partial, listed back with pg_restore --list, and renamed
after its manifest; a failed run leaves nothing named like a good dump.
Retention removes only files this job wrote, oldest first, and never
the newest good one (DB_BACKUP_KEEP, default 7; two pre-migration dumps).
With the off-host bucket configured, each dump and its manifest go up
under database/ in the envelope the account copies use, streamed. Every
run lands in the new database_backup_runs ledger (migration 0486), and
getDatabaseBackupHealth() answers the last good dump, the last failure,
whether the directory is writable and whether off-host is configured.
The boot readiness summary says where dumps go, or that none can be kept.
The dump, naming, retention and restore rules live in one plain ESM
module so the image's own scripts can share them without tsx.
The backup carried neither the import ledger nor the tombstones of
deleted documents, so after a restore onto a fresh instance a token
importer stored every imported document the person had deleted on
purpose again.
The document filing section now carries documentImportKeys: the
ledger's rows plus the own and alias keys of the documents waiting in
Recently deleted, which still do not travel as documents. The restore
writes them back, minus any key a restored live document holds, and
counts them in the restore summary and the preview. DocumentImportKey
moves from the excluded models to the two-ended ones, and the round
trip, the column audit and the parity goldens carry a ledger row.
The per-account preview switch was kept for the topic overview pages,
which were dropped before they shipped. Its one flag was stored, served
and backed up, and read by nothing; the column-reader audit listed it as
a pending consumer that will not come.
Removed completely:
- `/api/settings/preview-features` (GET, PATCH) is retired: it answers
410 Gone with `meta.errorCode: route.retired` and `removedIn: 1.46.0`,
and the contract publishes it under Retired. The route, its validation
schema, the `PreviewFeatures` OpenAPI schemas and the settings module
are deleted.
- Migration 0485 drops `users.preview_features_json` under a bounded lock
wait, with the retry marker.
- The backup no longer writes the field. A file from v1.44 or v1.45 that
carries it still restores: the schema accepts it and no codec reads it,
so nothing is refused or written for it. The backup-parity goldens
lose the field and apply one setting fewer.
- The data-wipe plan entry and the sharing-surface actor entry go with
the route (frozen count 273 -> 272).
No client build reads the route: neither iOS repository nor any local
checkout references it.
A gateway that refuses reasoning_effort outright (LiteLLM in front of
Ollama: "does not support parameters: ['reasoning_effort']") in front of a
model that reasons without answering sent requests until something outside
stopped it. The answerless rule learned to ask with "none", the refusal of
that "none" was learned as "send no field" over it, the next request came
back answerless again, and each lesson undid the other.
Each call of the Local and the OpenAI-compatible gateway clients now carries
one request budget through every repeat (request-hops.ts): at most two
requests, after which the call ends with the failure it has. A structural
guard holds both clients to sending only through the budget and to handing
it to every repeat.
The refusal is now learned as what it is: an endpoint that refuses the
field, in front of a model already seen reasoning unasked, keeps both facts
(FIELD_REFUSED_REASONS_REGARDLESS), so it is never asked with "none" again
and keeps its thinking room. An answerless reply from an endpoint that
refuses the field is final. When the refused "none" was the repeat of an
answerless reply, the call fails as reasoning_only, which is what it ran
into. The refusal handling of both clients lives in wire-refusal.ts.
A status run that hits its bound now aborts the call it gave up on instead
of leaving it running detached.
Refs #1279
A course's end date is a calendar day and counts in full: the doses of that
day stay due, reminded and expected until local midnight. That is right for
an end date set in the editor and wrong for "End course", whose dialog
promises that nothing more is due.
A course now carries `endedAt` (migration 0482, additive, nullable), the
instant it was ended on its `endsOn` day, projected onto the medication next
to `startsOn` / `endsOn` by the one course writer. From that instant:
- the schedule engine emits no slot, so a dose later that day is not due,
not reminded, not projected onto the today list and never expected by
compliance, while a dose earlier that day stays inside the course;
- the course reads ENDED (`courseStatusAt` / `courseStatusOn`), so the dose
actions, `remindersActive` and the reminder gate stop at once, follow-ups
of an earlier dose included;
- every engine context builder, compliance context, dose history, Coach
snapshot, doctor report and the adherence storyline read it, through one
select and one picker for the medication window.
Any write that moves a course's `endsOn` clears the instant, so an end date
chosen in the editor keeps meaning the whole day. Backups carry it in both
purposes and a restore projects it again. `courseStatusAt` moves next to the
other course rules.
Refs #1280
pg-boss gives a send that names no retry options the queue's defaults:
two retries, at once, without a backoff. A job that reached a model and
failed then ran again immediately, twice, against the provider that had
just failed it. Two model-bound queues still relied on those defaults:
the Coach memory refresh send and the nudge cron. Both now ask for the
same two retries, a minute apart and backed off.
ai-job-retry-bound-guard holds the rule: every send to a queue whose jobs
reach a model names retryLimit (at most 3) and, unless it is 0,
retryBackoff with a retryDelay of at least 30 s; every such cron carries
the same in its schedule options. The queues come from the model-run
sites the expiry guard already keeps complete (now a shared helper), plus
the job files that call a provider directly, which the new guard keeps
complete itself. A planted case runs the checker over sources shaped like
the tree before this change. The queue golden records the nudge cron's
new options.
Refs #1279
A status note whose generation stalled or failed opened a fixed five-minute
negative window, and only the read path honoured it. The ingest
invalidator counted only a stored note as fresh, so every data sync
re-enqueued each note whose last generation had just failed, and the
worker generated it again whatever the window said. With a model that
never answers, every sync bought another round of calls per note.
The window now grows with the generations in a row that ended without a
note (failure_streak, migration 0480): five minutes, doubling per failure,
up to two hours. A written or re-dated note resets it. A failure inside an
open window neither moves nor counts it.
runStatusCompletion waits the window out before any provider work,
whichever path selected the note again (a sync, a warm pass, a queued
job), and the ingest invalidator no longer enqueues a note inside its
window. A derived-score assessment that fails now opens the same window.
The person's own regenerate closes the record's windows first, and saving
the AI provider settings closes them and forgets the provider's health
rows, so a fixed setup does not sit out a pause the old one earned.
The window code moves to status-negative-window.ts; the status parity
golden records that the mood note screened earlier in the same minute is
no longer generated again inside its window.
Refs #1279
A status run (a metric card, the batched assessment, a derived score, a
workout line, a Coach memory pass) is one model call plus the one repeat
the Local and gateway clients send when a thinking model answered with
reasoning alone, each with the person's response timeout. The run's outer
bound was a single call, so the repeat was cut off, and the open card
stopped polling after 48 s, so a note from a model that needs minutes
never reached the page. The bound is now the per-call timeout times
STATUS_MODEL_CALLS, and the card's poll ceiling follows the published
response timeout (fast for the first 48 s, every 15 s after), only while
the route reports the note in flight; the fallback text stays meanwhile.
The status queue is exclusive, so the longer poll cannot queue a second
generation beside a running one.
The morning refresh is one comprehensive generation of up to five calls
in a row, with no outer bound and pg-boss's fifteen-minute expiry, so a
large setting had pg-boss fail and retry it while it was still running.
It now runs behind the same bound as the warms, and the job carries an
expiry derived from that worst case. The per-metric note, the workout line
and the Coach memory refresh get expiries derived the same way.
queue-runtime.ts gains a bounded class for these jobs: model time read off
model-time.ts, an expiry above it, carried on every send. A new guard
names every caller of runStatusCompletion and generateComprehensiveInsight
with the queues that reach it, and holds each such queue to bounded above
its runs or long under a run lock.
Refs #1274
A reading sent with a measurements:write token keeps its EXTERNAL source
and now points at the token that carried it (measurements.ingest_token_id).
Every stored row answers a server-resolved appleHealthMirror: true for
WITHINGS, IMPORT and MANUAL, the token's choice for EXTERNAL, false
otherwise. The plain list accepts appleHealthMirror=true so a client can
page exactly those rows.
Migration 0476 adds the columns and links existing EXTERNAL readings only
where one measurement token provably sent them; 0477 builds the index
concurrently. Backups carry the link and a restore keeps it only for a
token the account still holds. A guard holds every EXTERNAL writer to
setting the link.
The JSON import refused the app's own files: the full backup writes an
empty note, tag list or glucose context as null and the JSON export as
an empty string, and the import schema read either as a value of the
wrong type. The file is now read in the exporter's terms before it is
validated: the export's envelope is opened, every null or empty field
is treated as absent, a backup row carrying a deletion is left out and
counted as skipped, and a unit the app converts on every other way in
is converted to the stored one.
The round trip also showed the portable backup dropping what a reading
itself says: glucose context, sleep stage, range bounds, rhythm and
device. They ride the portable file now, and a portable restore puts
them back. Sync bookkeeping stays out.
An integration test exports an account three ways (full backup, JSON
export, measurement CSV), imports each into a fresh account and compares
what the import carries.
The goldens held their fields in the order the code assigned them. They are now written with every object's keys sorted, so the text depends only on content and diffs and compresses well; the parsed content of every golden is unchanged. They carry no clock reading or generated id: the clock is pinned and the ids are fixed.
An administration now stores `wording`: `validated` (the published
validated version of the reader's language, the original for English),
`unvalidated_translation` (HealthLog's own translation) or `original_en`
(the English original, chosen by a reader of another language). The
client sends only `englishOriginal`; the server resolves the rest from
the wording registry, so a client cannot claim a validated wording the
registry does not hold. `locale` keeps the reader's language, which
still selects the crisis resources.
Additive migration 0475: a nullable column with no backfill, since which
text earlier rows showed depended on releases since replaced. Every read
returns the field, the OpenAPI contract carries it and `englishOriginal`,
and the disaster-recovery backup carries and restores it (the parity
goldens gain the column; their pinned timestamps move by one step
because the fill writes one more column).
The PHQ-9 item-9 crisis signpost is answered from the answers alone and
is proven in every wording through the real route.
A disaster-recovery restore wrote these rows stamped with the restore's own
time: the medication's updatedAt (moved again by the course window), the
category assignment, custom category labels, courses and composition, the
mood day context and tag links, illness and cycle symptom links, nutrient
days, and a flare's updatedAt after the parent pointer pass. The recovery
file now carries each stamp (additive fields: linkCreatedAt on a mood
entry, symptomLinkCreatedAt on a cycle day log, createdAt on an illness
symptom, categoryUpdatedAt on a medication, updatedAt where missing) and
the restore writes it back. A file without them restores as before.
A custom category label, a course note and a marker's context rode the
recovery file as readable text and were sealed again under the active key.
They now ride as ciphertext (labelEncrypted, noteEncrypted,
contextEncrypted) and come back as the same bytes; a file that carries the
readable value instead is sealed on the way in, as before. The portable
file keeps the readable values. The rotation script and the
encrypted-column writer guard already cover all three columns.
The parity golden copies change for exactly this: the new fields in the
recovery file, the restored stamps in place of <now>, and the three values
coming back under the key they were sealed with. The legacy readable coach
title and custom metric note are still sealed on restore, since the row
holds no ciphertext to carry. The round-trip column audit no longer
excludes the four link stamps.
Every record with a tombstone (allergies, labs, conditions, vaccinations,
life events, measurements and the rest) rides the disaster-recovery file
deleted rows included, so a restore gives back the account as it was. The
family history read only live entries for both files, so a restore lost
its deletions, and it carried the note as readable text that the restore
sealed again under the active key.
The disaster-recovery slice now carries every entry with its deletion and
the note as the ciphertext at rest, and the restore writes both back
verbatim; the portable file keeps the readable live entries. The parity
golden copies change for exactly that: the deleted entry, its note's key
and the shifted ids after it.
The parity harness normalised any timestamp written during the run to
<now>, and the seeded rows' createdAt and updatedAt were themselves written
during the run, so a restore that dropped a row's createdAt still matched
its golden copy. Every run-time timestamp of the seeded account is now
pinned to a fixed historical value, and the seeded snapshot is asserted to
hold none, so a lost createdAt shows as <now> against a historical value.
The account also carries a tombstone of every model that keeps them, a
flare under its condition and a superseded profile fact, so the second
restore pass resolves real pointers. The fixture is sealed under the
legacy key and a second key is active from the export on; sealed values
render as the key id they name and the text they open to, so a value the
restore re-encrypts instead of carrying over differs.
The golden copies are written from the backup code as it stood before the
export and restore moved into per-domain modules (64fa65b46), and the
current code matches them.
A table read without an order comes back in whatever order its last
updates left on disk, so the column fill could pick a different row of a
two-row table from run to run. The audit rows of a restore are now found
by the copy's id instead of by time, since the database clock and the
test's clock do not agree to the millisecond. The golden copies are
written again from the code before the restore and export moved.
Seed one account as far as the schema allows (a row of every two-ended
model, every column of those rows and of the account row, the instance
settings, a sharing grant), export it both ways and restore each file
into a fresh account, over the live account, and from a version-1 file
in the old shapes. The files, counts, parsed payload, preview, restore
outcome, reported phases, audit rows, wide-event fields and every row the
account holds afterwards are compared with golden copies after ids,
ciphertext and run-time timestamps are normalised. Refusals, skip reports
and the schema's issue list for malformed files are pinned the same way.
The seed moves out of the round-trip test into a shared fixture module
so both files fill the same account.
Marc-André Bombeck