Migrations 0402 to 0410: AI usage levels, biomarker aliases and merges,
medication ingredients (with the supplement label run kind), life
circumstance fact kinds, AI sections and AUDIT-C, Dawarich context, the
pulse day-mean method, intraday hour buckets, the readable location drop
and five relation link tables. Every new column carries its reader or a
pending-consumer marker.
0409 refuses to run while any home, location period or stored day still
holds a readable location, and names the counts. Locations are read from
the sealed copy only; the encryption backfill no longer handles them, and
the demo seed writes sealed locations only.
The new tables are backed up and wiped, the new sealed columns are in the
rotation registry with labels of their own, the AI usage level and the
life-circumstance confirmations travel with the backup, and the wellbeing
score reads the mental-health screeners only.
DELETE /api/biomarkers/{id} used a bare requireAuth() while the PUT beside
it requires requireRecordAuth("manage", "labs"). The delete now uses the
same authority and, as a manage-level hard delete, files what it destroyed
(name, unit, reading count) through destroyedDetails.
The watchdog's first pass after a deploy records standing incidents as
announced without paging. Their end then sent a recovery note for
something the operator was never told about. Such incidents now carry
the epoch as their announcement time, which no real announcement can
have, and close without a note.
The extra-dose rule split a dose that HealthLog already recorded: a tap
at 08:00 and the same dose arriving from Health at 08:20 became two
doses and consumed stock twice, and a dose edited in Health (a new
record, a new time) became a second dose next to the first. The rule
now follows the calendar and the record: a take on another local day
is another dose; on the same day, a take logged in HealthLog converges
with the incoming record, a new record that names the slot as its
scheduled time is the slot's dose recorded again, and only an
unscheduled record more than a minute from the held one is an extra
dose.
The watchdog sent one page per account, so a provider outage or an
expired app credential paged once for every connected account. A pass
now collects its messages and sends them per provider at its end: more
than three accounts of one provider opening (or closing) an incident in
the same pass become one message naming the count, the reasons with
their earliest start and the accounts, and one recovery note. Every
incident in it counts as announced.
A failure streak reaching the alert threshold paged the operator even
when the watchdog had already announced an open incident for the same
connection. The ledger now checks for that announced incident first,
stamps the streak as announced and sends nothing; the watchdog's
recovery note still ends it.
The watchdog opened an incident for any degraded state that lasted a
day, so an Apple Health account whose phone app stopped running read as
no_new_data for good and paged. Incidents now open for action_required,
for provider errors and for the watchdog's own unexpected silence; stale
or stalled data alone never pages. The first pass after a deploy, with
no watch row yet, records what stands as already announced instead of
paging every long-broken connection at once.
A walk that reads without the edge bound passed over everything earlier
slices wrote on every slice, and once it crossed the edge the skipped
pages were counted against the walk's page ceiling. The skip now stays
counted for the whole walk, runs on the slice's clock and its own
three-minute limit, and saves the page token it stopped at (on a limit,
a rate limit or the slice budget) so the next slice re-reads that one
page and goes on. A saved place counts as progress only when it moves
forward; a token Google refuses, or one that no longer reaches the edge,
sends the walk back to the newest page as before.
Oura returns detected periods the person dismissed (type rest) and ones
they deleted in the sleep collection. They were imported as sleep, with
their lowest heart rate, HRV and breathing rate, which added an extra
session to the day. They now map to nothing, and rows an earlier sync
stored for such a period are soft-deleted on the next sync through the
shared write tail.
The guardian's read of a managed record's connections now carries the same
server-resolved health the owner's own envelope does, loaded for the record
and never for the guardian. The legacy state stays on the wire for one
transition release.
Their rings and straps upload only when the phone app is opened, so a day
without opening it is ordinary and must not page the operator. Wearables
whose cloud receives data on its own keep 24 h, Apple Health keeps 48 h,
and scales, cuffs and Strava never alert on silence.
A fixture dated near the day it was written passes until the date slides
out of a window the route counts back from the clock. Five did: a snooze
until a fixed date that becomes the past in January, a prognosis read
against the wall clock over days seeded at a fixed date, sleep rows and a
weight streak read through windows that end now, and a sync token that
would expire on a fixed date. The e2e mood heatmap mock and a timeline
URL are dated from today as well.
A guard scans the integration and e2e trees for near-now absolute dates
within reach of a window parameter, with an allowlist for files whose
windows are fixed on both ends.
1.58 million readings, paged by a generated fake of the list API that
honours the edge and lower-bound filters, with a server error, a timeout and
an hour-long rate limit injected. Pins that the backfill completes over
several slices, imports every reading exactly once without rewriting a row,
lets the hourly sync run between and beside slices, writes one log event per
slice with its requests and edges, and keeps the heap flat across the
history under the production heap ceiling.
A backfill noted progress only once a whole collection was read to the end.
A heart-rate history of years that one attempt could not finish (its budget,
a rate limit it could not wait out, a single failed request among thousands
of pages) started the next attempt from the newest page again, finished
nothing, and failed with "marker not stamped" on every attempt.
- Every spot-reading collection read page by page now records, after each
page it writes, the oldest sample it has written (backfillProgress.cursors).
The next attempt asks Google only for samples at or beyond that edge. The
edge is held back once a page's write fails, so no rows are skipped.
- The backfill runs in slices of five minutes or 500 pages. A slice that
finished a collection or moved an edge continues with the next one; only a
slice that moved nothing fails, and its error names each unfinished
collection and why the slice stopped. The slice ceiling is derived from the
collections and the page ceilings of the resumable ones.
- A rate limit Google will not lift within a minute ends the slice instead of
failing the walk, and the next slice starts five minutes later. Server
errors, timeouts and dropped connections are retried three times.
- The heart-rate stream is read last, so the light collections finish in the
first slice.
- The hourly sync waits up to eight minutes for a running slice instead of
skipping the account for as long as the import runs.
- Each slice annotates googleHealth.backfill.slice with pages, elapsed time,
stop reason, per-collection status and edges, and reports progress on the
job.progress line.
The visit form's lab picker offered the newest 200 results, the condition
picker the newest 100 conditions, the document pickers the newest thousand
documents and the document sheet's visit picker the newest 100 visits.
On a long record everything older could not be linked.
Each picker now opens on the records around the date of the thing being
linked, pages older and newer from there, and searches through the list
route itself. Records already linked always show, from the labels the
record's own read carries and from what was picked in the sheet.
The lab list takes a q filter over analyte and panel, the condition list
from/to/q/offset/sortDir, and the visit list an offset.
An hourly and a daily pass flag a continuous source that delivered on
most of the previous 14 days and has gone quiet, and open one operator
incident per connection when a connection stays degraded or needs action
for more than a day. The incident is announced once through the existing
admin alert path, adopted rather than repeated when the sync ledger
already paged, and closed with one recovery note. Scales, cuffs and
workout logs are watched for errors only, never for silence.
A HealthKit dose event binds to a slot by window band, and a weekly
schedule's band reaches a day either side of its slot. An extra dose
logged the day after a scheduled one landed on that slot and overwrote it
last-write-wins: the slot's time changed and the extra dose was stored
nowhere. An external dose record whose band slot already carries a
different take is now recorded as its own unscheduled dose.
The dose history now lets the take stored on a slot claim it before an
extra dose stored on its own instant, instead of following the order the
rows were read in, so the extra dose reads as off-schedule.
Changing the value of a converted reading cleared sourceValue and
sourceUnit but kept the printed range text, which is written in the
report's unit. With the unit gone every reader presented it as the marker
unit's window, beside numbers in another unit. The text now goes with the
conversion record (a whole-form echo of it does not bring it back); the
bounds, already converted into the marker's unit, stay.
The matcher stored the registry entry's key, so LDL and HDL markers were
recorded as total-cholesterol. The key now names what the marker is (the
catalog slug, or a registry-only key for an analyte with no seed), and the
registry resolves ldl and hdl to the cholesterol entry for the conversion as
before. A key already written in the old form is rewritten by the same fill
step that brings a stale key in line with its name, the first time a reading
needs it. The OpenAPI descriptions say what the key is and that it follows
the marker's name and unit.
The single-reading edit cleared sourceValue and sourceUnit whenever the body
carried a value, and the web edit sheet always sends one, so a date or note
edit erased the "converted from" hint. It is now cleared only when the value
differs from the stored one. The sheet also opened a converted reading at its
stored precision (100.90191999999999); it now starts from the value as the lab
surfaces show it and leaves an untouched value out of the request, so the
stored full-precision number is not replaced by its rounded display.
A converted reading's bounds move with its value, but the printed range text
stays in the unit the report used, and every reader preferred that text: the
list showed "Report range 3.9 to 5.5" beside 100.9 mg/dL, and the doctor
report and the FHIR referenceRange.text did the same. The text is still
stored verbatim. The range resolver now takes the reading's sourceUnit (as a
required field, so no reader can forget to select it) and, for a converted
reading, leaves the window's text empty so every surface states the converted
bounds in the marker's unit, and hands the printed text over separately with
its unit. The lab DTO carries it as printedReferenceText, and the note beside
a reading shows it as "as printed in mmol/L: ...".
The key is a function of the marker's name and unit, but it was only ever
filled, never replaced, so a marker renamed from Glucose to Cholesterol kept
converting readings with the glucose factor. The marker edit now re-derives
it with the same match the creation paths use (clearing it when the new name
matches nothing), and the fill step a reading runs brings a stored key back in
line with the name, so a key written before this fix is corrected the first
time it matters. The scan review derives the key the same way as the commit.
A quick answer whose request never reached the route left no trace: a
redirect in front of the app turns ntfy's POST into a GET, which Next
answered 405 before any wide event, and the worker-only and demo-mode
refusals in the proxy wrote nothing either. The proxy now answers a
non-POST on the action path itself with 405 and Allow: POST, and every
refusal it makes there writes one warn line with a reason and the path
without its token. The route's rate-limit refusal carries a reason too.
The regression test sends the action ntfy publishes, in the shape ntfy
sends it, through the real proxy and route against Postgres.
Refs #1203
A chronic ongoing condition has no end date, so counting it as rest mode
kept the account in it for good: Day N counted from an onset years back,
cadence nudges never resumed and every streak gap since the onset was
bridged. Only active episodes that are not CHRONIC_ONGOING (acute,
recurring, flare) now trigger rest mode, and Day N counts from the
earliest of those. The achievements streak freeze applies the same
filter to its own episode query.
Chronic conditions stay context: a coincident vital deviation is still
framed as explained by any active episode, and the Coach block and the
briefing still name them, with restMode false.
Refs #1216
Slice 3 of phase 3 of #1095. The scan review no longer treats every reading in
another unit as unwritable. The server tells it which analyte the matched marker
is (markerAnalyteKey on each extracted row); a row whose unit differs but can be
converted starts selected and carries a badge saying what Save will store and
what the report stated ("Saved as 100.9 mg/dL, converted from 5.6 mmol/L"). A
row that cannot be converted keeps the existing blocked badge and error. Save
still sends what the report printed; the server converts, as in slice 2.
The review's duplicate flag also recognises a re-scan of a reading that was
stored converted, by what the report printed, the same rule the commit applies.
A converted reading carries a quiet "converted from 5.6 mmol/L" on the lab list
(row and tile) and the history list (phone and table), so a number nobody typed
can always be traced back to what the report printed.
Strings in all seven locales.
Refs #1095
Slice 2 of phase 3 of #1095. A reading that stated a unit other than its
marker's was refused. When the marker's analyte is known and the registry has the
pair, the three write paths now convert it instead: POST /api/labs, the OCR
commit and the document commit.
The converted value is stored at full precision in the marker's unit, with what
the report printed in the existing source_value and source_unit columns. The
report's own reference window is in the printed unit, so it is converted with the
value; otherwise the verdict would compare a converted value with an unconverted
window. A pair the registry does not have is still refused as before (the 422
now says why), as is a marker whose analyte is unknown.
The marker's key is filled the first time a reading needs it. The same scan
committed again is still a duplicate: a converted row holds the number in the
marker's unit, so the duplicate check also matches on what the report printed.
Editing the value of a converted reading clears the conversion record, because
the number is then the person's own.
The lab result DTO and the OpenAPI schemas (labs and the OCR commit) carry
converted, sourceValue and sourceUnit.
Refs #1095
Slice 1 of phase 3 of #1095. A Biomarker row keeps only its name, so nothing
said which analyte it was. This adds the explicit alias list in the catalog the
issue settled on, including the registry-only keys (urea, blood urea nitrogen,
uric acid, total calcium, total bilirubin), and sets biomarkers.analyte_key
(from migration 0385) from it.
matchAnalyteKey equals a marker's folded name against the localised names of the
catalog seeds the registry knows and against the alias list. It matches the
whole name only, so "Calcium (ionisiert)" is not "Calcium"; a name two analytes
claim matches neither; and the marker's own unit must be one the analyte reaches,
so a marker called "Glucose" in a unit the registry cannot convert stays
unlinked.
The key is set when a marker is created (the biomarker route, the mint used by
lab writes, the legacy backfill) and, for an existing marker, by
ensureAnalyteKey the first time a reading needs it. It never overwrites a key
that is set. There is no boot job for the existing markers: the conversion that
uses the key fills it on first use, and a sweep can be added later if wanted.
The key is returned on the biomarker API and in the OpenAPI schema.
Refs #1095
The worker expands no slot past endsOn, so it never reminds for a
course that has ended, but remindersActive still read true for it. The
reminder gate now includes the course end on the person's clock, so the
published value, the worker and the admin check make the same decision.
Marc-André Bombeck