A blood pressure reading is a <Correlation> wrapping its systolic and diastolic
records, and the marker can sit on the correlation instead of on each record.
The import only read it from a record's own metadata, so a marked correlation
would have come back in full.
A correlation's records are now held until it closes, then all kept or all left
out together: the marker on the correlation, before or after its records, or on
a record, applies. Tests cover each, the marker not spilling onto the next
record, and a self-closing record.
The fixture is invented, in the shape Apple's DTD documents, and says so in its
header and its file name. It shows the parser reads a file laid out like an
export; it cannot show that Apple writes a custom metadata key verbatim.
The iOS app writes manual entries into Apple Health, and mirrors Withings and
import rows into it, each stamped with dev.healthlog.app.origin = healthlog.
The server already holds those readings under their own source, so the same
sample arriving through an Apple Health export is a second copy: the series
shows it twice and the rollups count it twice.
The export import now leaves out a record carrying that marker. The marker is a
child <MetadataEntry>, read after the record's open tag, so a record is
committed at its close tag. As a fallback for manual entries mirrored before the
app set the marker, a record is also left out when a MANUAL row of the same
type and value sits within 2 s (same rule and tolerance as the sync path).
Withings and import rows are not matched by value and time: they reach Apple
Health only through the mirror, which always sets the marker.
The import summary says how many records were left out for this reason.
Fixes#1109
With every sealed column filled, the column round trip now also builds
the portable file of the same account and fails on any section that
carries a sealed column as stored ciphertext, unless the section is named
with a reason. The disaster-recovery file of the same account is the
floor, so a matcher that finds nothing cannot pass the check.
The account's own mood categories, tags and hidden set were written only
inside the restore's mood-entry branch, so a vocabulary set up before the
first entry did not come back. They are written whether or not the file
carries an entry.
A practitioner's note, a visit's reason, outcome and body site and a
vaccination's note travelled as stored ciphertext in a portable file, the
same defect the cycle and mood sections had. They now go through the
shared sealed-text helpers: readable in a portable file, sealed under the
receiving host's key on restore, ciphertext kept only where this host
opens it and otherwise reported as visitCiphertext or
vaccinationCiphertext. The practitioner address and phone ciphertext of a
disaster-recovery file gets the same check, since the key preflight now
leaves these sections to it.
The replay cell was keyed on (user, key, method, path). A client that sends
two different writes under one key got the first write's cached response for
the second, and the second was never saved. Native clients up to 1.1.0 post
the two halves of a manual blood pressure exactly that way, so the diastolic
value was silently dropped.
Each cell now stores a SHA-256 fingerprint of the request body (JSON with
recursively sorted keys, otherwise the raw bytes; scoped by method and path;
at most the first MiB is read). Lookup replays, or refuses as in flight, only
when the fingerprint matches. A different body, or a row written before the
column existed, runs the handler without claiming or caching and leaves the
existing cell for its own retries; a lost claim race re-checks the winner's
fingerprint the same way. The body itself is never stored.
Migration 0371 adds the nullable idempotency_keys.request_fingerprint column.
The OpenAPI parameter description, the client notes and the Home Assistant
guide say what a replay now requires. docs/ops gains a read-only query that
lists systolic readings left without their diastolic half, held to the
schema by an integration test.
The labels of the mood tags and mood categories an account created
travelled as the stored ciphertext in a portable file, so a restore onto
a host with another key refused the file or wrote labels nobody could
open. They now follow the cycle note: readable in a portable file, sealed
under the receiving host's key on restore, ciphertext kept only where
this host opens it and otherwise reported as moodLabelCiphertext.
The sealing helpers move to one module both sections share.
The row update (name, schedule, oneShot) committed before the course
write. When a concurrent course made the course write refuse, the client
got the refusal while the rest was already written. The schedule replace,
the row update, the pause era and the course write now commit together,
so a refusal leaves the medication unchanged. The rollup recompute runs
after the commit.
A portable file carried the day-log note, the sensitive-category envelope
and custom symptom labels as the stored ciphertext, so a restore onto a
host with another key either refused the file or wrote values nobody
could open. The portable file now carries them readable and the restore
seals them under the receiving host's key; a disaster-recovery file keeps
the ciphertext.
A file written before this still restores: each ciphertext value is
opened with this host's keys first, and one that does not open is kept
out of the row and reported in the skip summary as cycleCiphertext. The
key preflight leaves the cycle sections to that per-value check.
The probe decided on the eight oldest values alone. An install
reinstalled under a fresh key long ago holds old rows sealed under a key
that is gone and all current data under the key it runs with; it works
today, and the check would refuse it with 503 on every request.
When every oldest value fails, the newest values of the same columns are
now sampled in reverse order. One that opens makes the verdict
inconclusive (mixed): the process serves, warns and probes again at the
next start. Only old and newest values that all fail are a mismatch. A
newest-only open never seals; sealing stays with the oldest-first rule.
Every probe query now runs in its own transaction under a
transaction-local statement_timeout (2 s), so a large or locked table
ends that column with an error and the probe as incomplete instead of
holding up the boot. The newest order is the exact mirror of the oldest,
so an index on the timestamp serves both directions.
The module comment lists the three conditions that still yield a
mismatch.
At a first start there is no recorded key yet, so an operator facing a
refusal they are certain is false has no row to delete. With
ENCRYPTION_KEY_CHECK=warn (default enforce) the check still runs and logs
its full outcome, /api/health names a mismatch as a warning field without
failing, requests are served, and no key is recorded unless every key id
passes.
The variable is on the compose whitelist, in both example files, the env
manifest, the Unraid and Portainer templates and the published-env guard.
The log block, the mismatch page and the self-hosting pages offer it as the
last resort after the three real fixes, and say that it disables a safety
check.
The backfill and the restore of an older file read the creation day of
an end-only window in UTC while every reader of the window uses the
account's zone; both now use the account's zone (the default zone when
the stored one is unknown to Postgres). A window whose start lies after
its end, which the API refuses but older rows hold, becomes a one-day
course on the end day: every reader treated it as empty and ended, and
a swap would have invented expected doses. One shared rule
(courseFromWindow) serves the restore and the course writer.
apiHandler answers 503 encryption.key_mismatch before auth, but the remote
MCP endpoint and the OAuth authorize, token and register routes wrap in
withBackgroundEvent instead and never checked. An MCP write tool could seal
new rows under the wrong key, and those rows would open under it at the
next boot.
The refusal moves into one helper used by apiHandler, /mcp (same envelope)
and the OAuth routes (RFC 6749 temporarily_unavailable with the error code
as an extension member). A structural test fails when a route outside
apiHandler, or a server action, lacks the refusal in every exported handler
and is not on a short allowlist of routes that never touch the database.
The boot key check recorded whatever key was configured whenever its probe
found nothing to try. It only read string columns, so an install whose
ciphertext sat in Bytes columns (notes, documents, ECG, symptoms) read as
empty; a probe that ran out of time did the same; and rows written by a
process that served under a wrong key could open on the next boot and get
that key recorded as the right one.
The probe now reads every registered column, string and Bytes, in all four
stored layouts, and tries the oldest values first, ordered by the row's
updatedAt (else createdAt, else primary key). A key is recorded only when
the oldest value that opens is preceded by at most one that does not, or
when no column holds a single value under the key id. A time-out, an
unreadable table, oversized values or mixed results are inconclusive: the
process serves, records nothing and checks again at the next start.
When the first insert loses a race, the stored record is read back and must
open under this process's key, else the outcome is a mismatch.
A rotation that walked every column without errors removes the record of
each previous key id that no longer holds any value. The refusal log block
and the self-hosting pages explain how to reset the check for a key id.
The paragraph described the old values. It is deleted with them, its generation claim is detached the way a delete detaches it (the day's slot stays spent), and the workout's arrival is raised again so the paragraph is written from the corrected values through the ordinary path, under the same daily cap.
The history minted bands only from the latest course, so misses in an
earlier course counted in the rate and appeared nowhere in the history.
It now mints over the span of every course and keeps the slots on a
course day. The compliance ledger and the cadence missed tally drop the
gap slots before the reconstruction instead of after it, so a dose
logged between courses stays visible as an off-schedule take in both.
A medication with no course rows and no window is running since its
creation; a course added beside it projected the medication into ending,
which stopped its reminders and dropped its history from compliance. The
create now refuses that (medications.course.currentOpen; End course
writes the first course). A window the rows never recorded is
materialised as the first course before the new one is checked.
An iPhone build that predates custom categories decodes an unknown category
as OTHER and sends it back on every edit. A Bearer update sending OTHER over
a custom category now keeps the custom one; the web can still move it back.
A medication taken now and then (#1024) keeps one row and gains course
rows: calendar spans, both ends inclusive, an optional encrypted note, no
schedule or dose of their own. Courses never overlap and every course but
the latest has ended, so the latest course is always the one the
medication's startsOn/endsOn project. src/lib/medications/courses.ts is
the only writer of those two columns for a medication with courses; it
writes under a per-medication advisory lock and re-projects in the same
transaction, so reminders, the next-due line and intakeActionable keep
reading the columns and stay right.
- Migration 0368 adds medication_courses and backfills one course per
medication window (an end-only window starts on the creation day, or
on its end day when that lies earlier); a medication without a window
stays course-less. Idempotent, data-preserving.
- GET/POST /api/medications/{id}/courses and PATCH/DELETE
.../courses/{courseId} (owner-only); create seeds the first course
from a window; the medication PUT edits the latest course and refuses
a window that would break the rules (medications.course.*).
- List, detail, create and update carry courses, courseCount,
previousCourseEndedOn and canStartCourse, resolved on the server.
- Compliance expands over the span of all courses and drops the slots
between them; every compliance read selects the courses. A rolling
cadence is not anchored by an intake from a previous course.
- The card names the last course's end and the running course's number;
the detail Lifecycle group lists the courses with start, edit and
delete. The Coach compliance block, the MCP resource, the doctor
report window and table, the clinician view and FHIR (one
MedicationStatement per course with effectivePeriod and status) read
the courses.
- Backup carries courses per medication and the restore writes them and
re-projects; an older file derives the one course its window
describes. Wipe plan, encrypted columns, rotation and OpenAPI follow.
A certificate of incapacity for work (in Germany the
Arbeitsunfähigkeitsbescheinigung) is now its own document kind,
SICK_NOTE, instead of Other. The enum value rides migration 0370.
Uploads, edits, the kind filter, bulk edits, the import script and the
backup format accept it; the vault files it with a thermometer glyph
between prescriptions and referrals; the filing suggestion can propose
it and names the words a sick note carries. The extraction's own kind
field keeps its three values.
A medication can be filed under a category the person named, next to the
built-in ones (#1041). The labels live in medication_category_labels,
encrypted at rest, capped at 20 per account; a medication filed under one
carries its custom:<uuid> key in the category column.
- GET/POST /api/medications/categories (POST idempotent, 30/min per actor,
admitted at MANAGE like the cycle symptom vocabulary) and owner-only
PATCH/DELETE /api/medications/categories/{key}. DELETE moves every
medication filed under the category to OTHER in the same transaction
and answers movedCount.
- category is widened to a built-in value or custom:<uuid> on the wire,
the create and update bodies refuse a key that is not the caller's
(422 medications.category.unknown), and list, detail, the insights
medication tiles and the MCP medications resource carry categoryLabel.
A key with no owned label reads as OTHER and is counted on the event.
- The wizard's class step lists the person's categories under the
built-in rows with an inline add row; the list page gains a category
filter (session only) and a manage sheet for rename, hide and delete.
- Backup carries customMedicationCategories (decrypted labels) and the
restore writes them before the medications; wipe plan, encrypted
column registry, rotation script, sharing guards and OpenAPI follow.
The extract route accepts `Prefer: respond-async` for all three inputs
(text, stored, vision) and queues a DOCUMENT_EXTRACT run (added to
migration 0370). The run's result is the count of staged facts and the
document status; the facts themselves are read from the document.
Without the header the route answers with the document detail as
before; both paths run one shared body.
A background read can take minutes, and a review the person finishes in
the meantime must not have its approved facts replaced. Staging now
locks the document row and asks again whether it may stage, refusing
with the route's own alreadyConfirmed / alreadyPartlyConfirmed codes.
The synchronous path gets the same re-check.
The vault's stored-text extract uses the background path and shows the
shared waiting line in place of its hint while the read runs.
Every active symptom definition enters pattern discovery as an outcome
channel SYMPTOM:<id>, labelled with its decrypted name (the data label
the interpretation already prefers over the key). The daily value is the
day's highest intensity in the person's zone; days without an occurrence
count as 0 only between the first and the last logged day. At most eight,
the active-definition cap. The SYMPTOM: prefix resolves to the illness
module in the surface map, so the family leaves the matrix with the
module, and the discovery guard lists the new fetcher.
A symptom of one's own ("aura", "headache") is a definition with an
encrypted name and an icon from the cycle custom-symptom allowlist; each
occurrence carries a time, a 0-10 intensity (the pain score's scale), an
optional encrypted note and an optional link to one of the record's open
illness episodes. Both ride the illness module: every route answers
illness.disabled when it is off.
- Migration 0369 (idempotent): symptom_definitions and symptom_events,
intensity CHECK 0-10, episode link ON DELETE SET NULL.
- /api/symptoms/definitions{,/[id]} and /api/symptoms/events{,/[id]}:
eight active definitions per record (a hidden one frees its slot,
showing it again re-checks), hide or purge, events create/list/edit/
delete, a new link must name an open episode of the same record
(422 symptoms.episode.notOpen). Error group symptoms.*.
- Sharing: reads at READ, logging an occurrence at WRITE, defining and
editing at MANAGE; C1/C3/C4 audit details; frozen lists, the handler
matrix and the activity verbs updated.
- Both encrypted columns registered and rotated; both models in the wipe
plan; both carried at both backup ends with occurrences nested in their
definition, plaintext in a portable file and ciphertext in a recovery
file, and a link to an episode the file lacks dropped and named
(symptomEpisodeReference).
- OpenAPI regenerated.
The summary and suggest routes accept `Prefer: respond-async`: after the
same quick refusals they answer 202 with a run id, and the background
document AI worker runs the read under two new run kinds
(DOCUMENT_SUMMARY, DOCUMENT_SUGGEST, migration 0370). Without the header
both routes answer synchronously as before; both paths run one shared
body, so they cannot drift.
The worker asks the capability for the record and re-checks the wire for
the provider it picks before every read. A summary that is to be stored
marks the document PENDING while queued, READY when it lands, and
UNAVAILABLE when the run fails or the write does not land. A second
request attaches to a live run only when it would produce the same thing.
The vault asks for both in the background and shows the same calm
waiting line the index uses, worded so a transient read never promises
that leaving the page keeps it. The egress guard and the route inventory
now hold every run kind to its dispatcher re-check and its capability.
The clinical category lived in a side table the app created at runtime,
after migration 0004 had dropped it as orphaned. It is now the
MedicationCategoryAssignment model. Migration 0368 creates the table
IF NOT EXISTS with the runtime DDL, so an existing table converges with
its rows untouched (verified with prisma migrate diff against a database
holding the runtime table and against a fresh one).
The helpers keep their names and run on the Prisma delegate; the runtime
CREATE TABLE and the pre-transaction ensure call in the restore are gone.
The backup plan judges the table as a schema model (backed up, two-ended,
seeded in the round trip) instead of under RAW_SQL_TABLES, which is now
empty; its guard proves its matcher on a fixture string instead.
The rollup tier and sleep resolve every (type, day) to the source the person's
ladder picks before they aggregate. The dense and raw reads of
/api/measurements/series filtered on user and type only, so with a second
provider a day's value blended the two, a reading present in both showed
twice, and the configured ladder had no effect on those charts.
Both reads now take the ladder-winning source per day through the shared
canonicalMeasurementsCte helper, as the other live readers do:
- the day and hour bucket read of the dense kinds (pulse, glucose, HRV, SpO2),
and the stats strip over its rows;
- the generic raw read (weight, body fat, steps, resting heart rate, ...).
The collapse is per day for the hour grain too, as everywhere else. The helper
gains an optional single-type scope so a series scans one metric, not every row
the person has in the window.
Only a type that has a ladder is collapsed. For a type without one (glucose,
body water, bone mass) the helper breaks the tie by source name, which would
silently hide one of two real sources, so those keep every source as before.
A single-source account reads exactly what is stored.
Tests: a real-database test covering the ladder in both directions, the
per-day fallback, a reading mirrored into two sources, the day-bucket, hour-
bucket and raw reads, a type with no ladder and a single-source account. The
route's unit test file reads its fixtures through a small adapter for the new
raw statements, and the dense-read guard drops the exception for the removed
findMany. The OpenAPI description states the rule.
Refs #1108
The integration-backfill drain handed its job budget only to the
Google Health backfill. The lab-biomarker backfill now takes it too,
asks it before each analyte group and, once it says stop, throws
without touching the rest. Finished groups are linked and drop out of
the next scan, so the retry resumes where the pass stopped instead of
running on past the job's expiry beside its own retry.
The boot check decided on the first value it found under a key id, so
one damaged or foreign-sealed row could take a correctly keyed server
down with 503 on every request. The probe now samples up to five
values, one per column first. One value that opens clears the key; a
mismatch needs at least two values and none opening. A single value
that does not open is inconclusive: the server keeps serving, writes no
canary, logs a warning and probes again at the next start.
Integration setup truncates encryption_key_canaries between tests.
A server that issues Secure session cookies cannot sign anyone in on a
plain http:// page: the browser drops the cookie. The login request was
still verified, the MFA ticket minted and then claimed, and the next
attempt answered "Invalid or expired challenge".
The web client now sends an optional clientTransport on the browser
sign-in requests (password login, passkey start, both MFA completions).
When the server issues Secure cookies and the page is plain http on a
non-loopback host, the request is refused with 409
auth.session.insecure_transport before the password is checked, a
ticket is minted or claimed, a code is spent, or a passkey challenge is
issued. A request without the field is never refused. The SSO start
takes client_proto/client_host and sends such a page back to the login
page instead of to the provider.
GET /api/auth/registration-status publishes sessionCookieSecure, and the
login page shows the reason with both fixes and disables the sign-in
controls before anything is sent; on localhost it shows a note only.
SESSION_COOKIE_SECURE keeps its meaning; the header heuristic log line
stays.
Readers label every linked reading with the marker's unit, so changing
it relabelled stored numbers without converting them. PUT
/api/biomarkers/{id} now answers 422 biomarkers.unit.locked with
meta.readingCount when the unit differs and any reading (tombstoned
ones included) is linked. The marker editor says so under the unit
field.
"Read with AI" and the lab scan used to run their model calls inside the
request. At the default response time a read of three calls outlasted the
100 s a proxy in front of the server commonly allows, and the person saw a
cut connection for a read the server was still doing.
Both now run as a background run (`DocumentAiRun`, migration 0366):
- `POST /api/documents/inbound/{id}/index` with `Prefer: respond-async`
answers 202 with a run id once every quick refusal has passed. Without
the header it answers exactly as before, which the iPhone app relies on.
A second request while a read of the same document is queued or running
attaches to that run.
- `POST /api/labs/ocr/extract` always answers 202. Only the web client
calls it.
- `GET /api/ai-runs/{id}` serves the run: the same body the synchronous
route answers with, or the failure code and status it would have given.
Scoped to the caller; another account's run is a 404.
The worker (`document-ai-run`, a long queue with a one-hour expiry, one
job per run under `lockedPass`) claims the run, asks the capability again
for the record and re-checks the wire for the provider it picks, sets the
run's deadline from the person's AI response time and refuses a read that
cannot fit the job's expiry. Sync route and worker run one body per kind,
so the two paths cannot drift. Budget is reserved at enqueue and settled
once by whoever ends the run.
Input and result are sealed with labelled AES-256-GCM, registered for key
rotation; the input is dropped when the run ends. A five-minute reaper
fails a run no worker took within fifteen minutes (and hands its
reservation back), fails a run past its deadline, and deletes a finished
run after an hour. Runs are outside backups and inside the data wipe.
The web client polls through the query-key factory and says, calmly, that
the read continues in the background, and when no worker has picked it up.
Every configured key id now has a canary in encryption_key_canaries,
checked at boot. A key that cannot open its canary, or existing data
written before the canary existed, puts the process in a refusal state:
every API route answers 503 encryption.key_mismatch, /api/health reports
reason encryption_key_mismatch, the pages render one explanation, and
neither the queue producer nor the worker starts. The process stays up
so the reason is visible instead of hiding in a crash loop.
The admin Encryption page gains a "Back up your encryption key" step:
key id and fingerprint (never the key), where the key lives per
platform, a check of a pasted copy that is compared and discarded, and
a confirmation bound to the active key id and fingerprint. The overview
gets a tile and admins get a dashboard banner while it is due.
HEALTHLOG_PLATFORM picks the platform the step opens on. Adds the
TrueNAS self-hosting page and key backup sections in the docs.
The manual form sends one manual: id per opening and resends it after
an edit. The batch route now treats a MANUAL entry with a manual: id
as an overwrite, the way stats: measurements are: the stored row takes
the new values and the entry reports "updated", or "duplicate" when
nothing changed. Records keyed on the workout are removed under the
personal-records lock and re-derived. Every other id stays
first-write-wins. The status enum gains "updated".
Marc-André Bombeck