A sample the app wrote before the origin marker existed still carries the
HealthLog row id as HKExternalUUID. A hit on one of the account's rows of
the same kind now leaves it out exactly, for Withings and import mirrors as
well as manual ones; either id of a blood pressure pair covers both halves,
since the app stamps both samples of a reading with one id.
The manual-mirror lookup read one OR clause per row on every batch, even
for an account without a single MANUAL row. The types that hold MANUAL rows
are now read once per import; a batch asks nothing for any other type, and
one time window per type for those.
Cycle samples carry the marker too and are now left out like the rest, and
a sample the export repeats (a correlation's records also stand on their
own) is counted once.
The systolic and diastolic reads each took the ladder's source per day on
their own. Whenever a source held only half of a reading on a day (a deleted
diastolic, an orphaned sample, a sub-floor systolic), the two picks diverged
and the pairing joined one source's systolic to another source's diastolic.
A sub-floor systolic could also claim a day and then contribute nothing,
leaving the day empty although a lower-ranked source had a valid reading.
The source is now picked per day from the systolic rows that pass the floor,
both halves are read from it in one statement, and a pair is only made
within one source, which also covers a reading that straddles midnight.
The insights comparison snapshot took its current and baseline pulse figures
from a reading AVG, and the Coach's data inventory told the model a reading
mean of the whole pulse history. Both now read the mean of the user's local
days, each the mean of its hours' means, through the day-mean helper. Every
other type keeps its aggregate.
The feature extraction, the signals of the day, the status cards, the graded
series, the Coach snapshot, the mood correlations and the report's trend arrow
each still averaged pulse over its readings, so a dense workout hour decided
the day and a workout day the window. They now read a pulse day as the mean of
its hours' means and a window as the mean of its days, through the day-mean
helper. Counts, extremes and spreads stay over every reading; every other type
is unchanged.
The doctor report's pulse average and day points, the correlation channels,
the illness correlation, the personal baseline and the Coach's tiered context
each folded pulse readings into a day by plain mean. They now take the day as
the mean of its local hours' means and a window as the mean of its days, in
the zone each reader buckets by. The day aggregates carry the weighted sums for
an hourly-mean type, so every caller reads one day value. Every other type is
unchanged.
A blood pressure reading is a <Correlation> wrapping its systolic and diastolic
records, and the marker can sit on the correlation instead of on each record.
The import only read it from a record's own metadata, so a marked correlation
would have come back in full.
A correlation's records are now held until it closes, then all kept or all left
out together: the marker on the correlation, before or after its records, or on
a record, applies. Tests cover each, the marker not spilling onto the next
record, and a self-closing record.
The dashboard summaries, the insights aggregate and the series stats strip
still took a plain mean over pulse readings for their 7-day, 30-day, prior-month
and all-time figures, while the year-ago figure and the chart already weighed
each day once, so the year delta compared two different statistics. They now
go through the day-mean helper: each day is the mean of its hours' means and a
window the mean of its days, in the frame each reader buckets its days by.
Count, min, max, the median and the spread stay over every reading; every other
type is unchanged.
The blood-pressure branch of /api/measurements/series still read systolic and
diastolic with an unranked findMany, so with a second provider the pairs blended
both sources and a reading present in both showed twice, while the other kinds
already follow the ladder since the dense and raw reads were moved onto the
shared helper.
Each of the two types now reads the ladder-winning source per day through the
same helper. The physiological floors (systolic 40, diastolic 20) and the
five-minute pairing are unchanged.
Rows written before this change keep the reading mean: the boot backfill and
the freshness check fill only missing days, so nothing rewrites them. Migration
0373 recomputes the mean of every stored PULSE row from the readings, DAY as
the mean of its hours' means and WEEK / MONTH / YEAR as the mean of their days,
and leaves count, min, max and the sums alone. The writer stores the coarse
rows the same way, so a refold and the migration agree.
The fixture is invented, in the shape Apple's DTD documents, and says so in its
header and its file name. It shows the parser reads a file laid out like an
export; it cannot show that Apple writes a custom metadata key verbatim.
The iOS app writes manual entries into Apple Health, and mirrors Withings and
import rows into it, each stamped with dev.healthlog.app.origin = healthlog.
The server already holds those readings under their own source, so the same
sample arriving through an Apple Health export is a second copy: the series
shows it twice and the rollups count it twice.
The export import now leaves out a record carrying that marker. The marker is a
child <MetadataEntry>, read after the record's open tag, so a record is
committed at its close tag. As a fallback for manual entries mirrored before the
app set the marker, a record is also left out when a MANUAL row of the same
type and value sits within 2 s (same rule and tolerance as the sync path).
Withings and import rows are not matched by value and time: they reach Apple
Health only through the mirror, which always sets the marker.
The import summary says how many records were left out for this reason.
Fixes#1109
A watch samples pulse about every 5 to 10 minutes at rest and about every
5 seconds in a workout. The daily value was the plain mean over a day's
readings, so a workout hour of several hundred readings outweighed the other
23 hours: a day with one hard hour and 23 resting hours (60 bpm) read about
128 bpm, not 64.
A day of pulse is now the mean of its per-hour means, in the rollup DAY
bucket, the live per-day read the chart serves, and the series route's day
grain. Windows over several days weigh each day once instead of by its
reading count, so the bias does not return one level up. Reading counts,
min/max, and the regression sums still run over every reading. The chart
tooltip and the data table name the statistic, in all seven locales.
The statistic is limited to pulse; HRV, SpO2 and glucose keep the plain mean
(see src/lib/measurements/day-statistic.ts). Stored DAY rows are rewritten by
the next rollup recompute (admin recompute route), not on read.
Fixes#1107
Refs #1124. The rasterizer opened pdfjs without its standard font, CMap
and wasm directories and told it to use system fonts. The runtime image
has none, so any text pdfjs could not draw from an embedded program (a
non-embedded standard font, or an embedded one it cannot parse) came out
as nothing, and an image-only provider was sent the page background. A
workstation hides this because its own fonts stand in.
pdfjs now gets the directories from its own package, resolved at runtime
from the working directory, and Liberation Sans from that package is
registered as the canvas' sans-serif, serif and monospace faces for the
text pdfjs draws with a substitute.
A document whose every rendered page is blank now reports render-failed,
so the read records raster-failed instead of indexing a transcript of an
empty page as a success.
With every sealed column filled, the column round trip now also builds
the portable file of the same account and fails on any section that
carries a sealed column as stored ciphertext, unless the section is named
with a reason. The disaster-recovery file of the same account is the
floor, so a matcher that finds nothing cannot pass the check.
The account's own mood categories, tags and hidden set were written only
inside the restore's mood-entry branch, so a vocabulary set up before the
first entry did not come back. They are written whether or not the file
carries an entry.
A practitioner's note, a visit's reason, outcome and body site and a
vaccination's note travelled as stored ciphertext in a portable file, the
same defect the cycle and mood sections had. They now go through the
shared sealed-text helpers: readable in a portable file, sealed under the
receiving host's key on restore, ciphertext kept only where this host
opens it and otherwise reported as visitCiphertext or
vaccinationCiphertext. The practitioner address and phone ciphertext of a
disaster-recovery file gets the same check, since the key preflight now
leaves these sections to it.
The replay cell was keyed on (user, key, method, path). A client that sends
two different writes under one key got the first write's cached response for
the second, and the second was never saved. Native clients up to 1.1.0 post
the two halves of a manual blood pressure exactly that way, so the diastolic
value was silently dropped.
Each cell now stores a SHA-256 fingerprint of the request body (JSON with
recursively sorted keys, otherwise the raw bytes; scoped by method and path;
at most the first MiB is read). Lookup replays, or refuses as in flight, only
when the fingerprint matches. A different body, or a row written before the
column existed, runs the handler without claiming or caching and leaves the
existing cell for its own retries; a lost claim race re-checks the winner's
fingerprint the same way. The body itself is never stored.
Migration 0371 adds the nullable idempotency_keys.request_fingerprint column.
The OpenAPI parameter description, the client notes and the Home Assistant
guide say what a replay now requires. docs/ops gains a read-only query that
lists systolic readings left without their diastolic half, held to the
schema by an integration test.
The labels of the mood tags and mood categories an account created
travelled as the stored ciphertext in a portable file, so a restore onto
a host with another key refused the file or wrote labels nobody could
open. They now follow the cycle note: readable in a portable file, sealed
under the receiving host's key on restore, ciphertext kept only where
this host opens it and otherwise reported as moodLabelCiphertext.
The sealing helpers move to one module both sections share.
The row update (name, schedule, oneShot) committed before the course
write. When a concurrent course made the course write refuse, the client
got the refusal while the rest was already written. The schedule replace,
the row update, the pause era and the course write now commit together,
so a refusal leaves the medication unchanged. The rollup recompute runs
after the commit.
A portable file carried the day-log note, the sensitive-category envelope
and custom symptom labels as the stored ciphertext, so a restore onto a
host with another key either refused the file or wrote values nobody
could open. The portable file now carries them readable and the restore
seals them under the receiving host's key; a disaster-recovery file keeps
the ciphertext.
A file written before this still restores: each ciphertext value is
opened with this host's keys first, and one that does not open is kept
out of the row and reported in the skip summary as cycleCiphertext. The
key preflight leaves the cycle sections to that per-value check.
The probe decided on the eight oldest values alone. An install
reinstalled under a fresh key long ago holds old rows sealed under a key
that is gone and all current data under the key it runs with; it works
today, and the check would refuse it with 503 on every request.
When every oldest value fails, the newest values of the same columns are
now sampled in reverse order. One that opens makes the verdict
inconclusive (mixed): the process serves, warns and probes again at the
next start. Only old and newest values that all fail are a mismatch. A
newest-only open never seals; sealing stays with the oldest-first rule.
Every probe query now runs in its own transaction under a
transaction-local statement_timeout (2 s), so a large or locked table
ends that column with an error and the probe as incomplete instead of
holding up the boot. The newest order is the exact mirror of the oldest,
so an index on the timestamp serves both directions.
The module comment lists the three conditions that still yield a
mismatch.
At a first start there is no recorded key yet, so an operator facing a
refusal they are certain is false has no row to delete. With
ENCRYPTION_KEY_CHECK=warn (default enforce) the check still runs and logs
its full outcome, /api/health names a mismatch as a warning field without
failing, requests are served, and no key is recorded unless every key id
passes.
The variable is on the compose whitelist, in both example files, the env
manifest, the Unraid and Portainer templates and the published-env guard.
The log block, the mismatch page and the self-hosting pages offer it as the
last resort after the three real fixes, and say that it disables a safety
check.
The backfill and the restore of an older file read the creation day of
an end-only window in UTC while every reader of the window uses the
account's zone; both now use the account's zone (the default zone when
the stored one is unknown to Postgres). A window whose start lies after
its end, which the API refuses but older rows hold, becomes a one-day
course on the end day: every reader treated it as empty and ended, and
a swap would have invented expected doses. One shared rule
(courseFromWindow) serves the restore and the course writer.
apiHandler answers 503 encryption.key_mismatch before auth, but the remote
MCP endpoint and the OAuth authorize, token and register routes wrap in
withBackgroundEvent instead and never checked. An MCP write tool could seal
new rows under the wrong key, and those rows would open under it at the
next boot.
The refusal moves into one helper used by apiHandler, /mcp (same envelope)
and the OAuth routes (RFC 6749 temporarily_unavailable with the error code
as an extension member). A structural test fails when a route outside
apiHandler, or a server action, lacks the refusal in every exported handler
and is not on a short allowlist of routes that never touch the database.
The boot key check recorded whatever key was configured whenever its probe
found nothing to try. It only read string columns, so an install whose
ciphertext sat in Bytes columns (notes, documents, ECG, symptoms) read as
empty; a probe that ran out of time did the same; and rows written by a
process that served under a wrong key could open on the next boot and get
that key recorded as the right one.
The probe now reads every registered column, string and Bytes, in all four
stored layouts, and tries the oldest values first, ordered by the row's
updatedAt (else createdAt, else primary key). A key is recorded only when
the oldest value that opens is preceded by at most one that does not, or
when no column holds a single value under the key id. A time-out, an
unreadable table, oversized values or mixed results are inconclusive: the
process serves, records nothing and checks again at the next start.
When the first insert loses a race, the stored record is read back and must
open under this process's key, else the outcome is a mismatch.
A rotation that walked every column without errors removes the record of
each previous key id that no longer holds any value. The refusal log block
and the self-hosting pages explain how to reset the check for a key id.
The paragraph described the old values. It is deleted with them, its generation claim is detached the way a delete detaches it (the day's slot stays spent), and the workout's arrival is raised again so the paragraph is written from the corrected values through the ordinary path, under the same daily cap.
The history minted bands only from the latest course, so misses in an
earlier course counted in the rate and appeared nowhere in the history.
It now mints over the span of every course and keeps the slots on a
course day. The compliance ledger and the cadence missed tally drop the
gap slots before the reconstruction instead of after it, so a dose
logged between courses stays visible as an off-schedule take in both.
A medication with no course rows and no window is running since its
creation; a course added beside it projected the medication into ending,
which stopped its reminders and dropped its history from compliance. The
create now refuses that (medications.course.currentOpen; End course
writes the first course). A window the rows never recorded is
materialised as the first course before the new one is checked.
An iPhone build that predates custom categories decodes an unknown category
as OTHER and sends it back on every edit. A Bearer update sending OTHER over
a custom category now keeps the custom one; the web can still move it back.
A medication taken now and then (#1024) keeps one row and gains course
rows: calendar spans, both ends inclusive, an optional encrypted note, no
schedule or dose of their own. Courses never overlap and every course but
the latest has ended, so the latest course is always the one the
medication's startsOn/endsOn project. src/lib/medications/courses.ts is
the only writer of those two columns for a medication with courses; it
writes under a per-medication advisory lock and re-projects in the same
transaction, so reminders, the next-due line and intakeActionable keep
reading the columns and stay right.
- Migration 0368 adds medication_courses and backfills one course per
medication window (an end-only window starts on the creation day, or
on its end day when that lies earlier); a medication without a window
stays course-less. Idempotent, data-preserving.
- GET/POST /api/medications/{id}/courses and PATCH/DELETE
.../courses/{courseId} (owner-only); create seeds the first course
from a window; the medication PUT edits the latest course and refuses
a window that would break the rules (medications.course.*).
- List, detail, create and update carry courses, courseCount,
previousCourseEndedOn and canStartCourse, resolved on the server.
- Compliance expands over the span of all courses and drops the slots
between them; every compliance read selects the courses. A rolling
cadence is not anchored by an intake from a previous course.
- The card names the last course's end and the running course's number;
the detail Lifecycle group lists the courses with start, edit and
delete. The Coach compliance block, the MCP resource, the doctor
report window and table, the clinician view and FHIR (one
MedicationStatement per course with effectivePeriod and status) read
the courses.
- Backup carries courses per medication and the restore writes them and
re-projects; an older file derives the one course its window
describes. Wipe plan, encrypted columns, rotation and OpenAPI follow.
A certificate of incapacity for work (in Germany the
Arbeitsunfähigkeitsbescheinigung) is now its own document kind,
SICK_NOTE, instead of Other. The enum value rides migration 0370.
Uploads, edits, the kind filter, bulk edits, the import script and the
backup format accept it; the vault files it with a thermometer glyph
between prescriptions and referrals; the filing suggestion can propose
it and names the words a sick note carries. The extraction's own kind
field keeps its three values.
A medication can be filed under a category the person named, next to the
built-in ones (#1041). The labels live in medication_category_labels,
encrypted at rest, capped at 20 per account; a medication filed under one
carries its custom:<uuid> key in the category column.
- GET/POST /api/medications/categories (POST idempotent, 30/min per actor,
admitted at MANAGE like the cycle symptom vocabulary) and owner-only
PATCH/DELETE /api/medications/categories/{key}. DELETE moves every
medication filed under the category to OTHER in the same transaction
and answers movedCount.
- category is widened to a built-in value or custom:<uuid> on the wire,
the create and update bodies refuse a key that is not the caller's
(422 medications.category.unknown), and list, detail, the insights
medication tiles and the MCP medications resource carry categoryLabel.
A key with no owned label reads as OTHER and is counted on the event.
- The wizard's class step lists the person's categories under the
built-in rows with an inline add row; the list page gains a category
filter (session only) and a manage sheet for rename, hide and delete.
- Backup carries customMedicationCategories (decrypted labels) and the
restore writes them before the medications; wipe plan, encrypted
column registry, rotation script, sharing guards and OpenAPI follow.
The extract route accepts `Prefer: respond-async` for all three inputs
(text, stored, vision) and queues a DOCUMENT_EXTRACT run (added to
migration 0370). The run's result is the count of staged facts and the
document status; the facts themselves are read from the document.
Without the header the route answers with the document detail as
before; both paths run one shared body.
A background read can take minutes, and a review the person finishes in
the meantime must not have its approved facts replaced. Staging now
locks the document row and asks again whether it may stage, refusing
with the route's own alreadyConfirmed / alreadyPartlyConfirmed codes.
The synchronous path gets the same re-check.
The vault's stored-text extract uses the background path and shows the
shared waiting line in place of its hint while the read runs.
Every active symptom definition enters pattern discovery as an outcome
channel SYMPTOM:<id>, labelled with its decrypted name (the data label
the interpretation already prefers over the key). The daily value is the
day's highest intensity in the person's zone; days without an occurrence
count as 0 only between the first and the last logged day. At most eight,
the active-definition cap. The SYMPTOM: prefix resolves to the illness
module in the surface map, so the family leaves the matrix with the
module, and the discovery guard lists the new fetcher.
A symptom of one's own ("aura", "headache") is a definition with an
encrypted name and an icon from the cycle custom-symptom allowlist; each
occurrence carries a time, a 0-10 intensity (the pain score's scale), an
optional encrypted note and an optional link to one of the record's open
illness episodes. Both ride the illness module: every route answers
illness.disabled when it is off.
- Migration 0369 (idempotent): symptom_definitions and symptom_events,
intensity CHECK 0-10, episode link ON DELETE SET NULL.
- /api/symptoms/definitions{,/[id]} and /api/symptoms/events{,/[id]}:
eight active definitions per record (a hidden one frees its slot,
showing it again re-checks), hide or purge, events create/list/edit/
delete, a new link must name an open episode of the same record
(422 symptoms.episode.notOpen). Error group symptoms.*.
- Sharing: reads at READ, logging an occurrence at WRITE, defining and
editing at MANAGE; C1/C3/C4 audit details; frozen lists, the handler
matrix and the activity verbs updated.
- Both encrypted columns registered and rotated; both models in the wipe
plan; both carried at both backup ends with occurrences nested in their
definition, plaintext in a portable file and ciphertext in a recovery
file, and a link to an episode the file lacks dropped and named
(symptomEpisodeReference).
- OpenAPI regenerated.
Marc-André Bombeck