A backfill noted progress only once a whole collection was read to the end.
A heart-rate history of years that one attempt could not finish (its budget,
a rate limit it could not wait out, a single failed request among thousands
of pages) started the next attempt from the newest page again, finished
nothing, and failed with "marker not stamped" on every attempt.
- Every spot-reading collection read page by page now records, after each
page it writes, the oldest sample it has written (backfillProgress.cursors).
The next attempt asks Google only for samples at or beyond that edge. The
edge is held back once a page's write fails, so no rows are skipped.
- The backfill runs in slices of five minutes or 500 pages. A slice that
finished a collection or moved an edge continues with the next one; only a
slice that moved nothing fails, and its error names each unfinished
collection and why the slice stopped. The slice ceiling is derived from the
collections and the page ceilings of the resumable ones.
- A rate limit Google will not lift within a minute ends the slice instead of
failing the walk, and the next slice starts five minutes later. Server
errors, timeouts and dropped connections are retried three times.
- The heart-rate stream is read last, so the light collections finish in the
first slice.
- The hourly sync waits up to eight minutes for a running slice instead of
skipping the account for as long as the import runs.
- Each slice annotates googleHealth.backfill.slice with pages, elapsed time,
stop reason, per-collection status and edges, and reports progress on the
job.progress line.
The visit form's lab picker offered the newest 200 results, the condition
picker the newest 100 conditions, the document pickers the newest thousand
documents and the document sheet's visit picker the newest 100 visits.
On a long record everything older could not be linked.
Each picker now opens on the records around the date of the thing being
linked, pages older and newer from there, and searches through the list
route itself. Records already linked always show, from the labels the
record's own read carries and from what was picked in the sheet.
The lab list takes a q filter over analyte and panel, the condition list
from/to/q/offset/sortDir, and the visit list an offset.
The matcher stored the registry entry's key, so LDL and HDL markers were
recorded as total-cholesterol. The key now names what the marker is (the
catalog slug, or a registry-only key for an analyte with no seed), and the
registry resolves ldl and hdl to the cholesterol entry for the conversion as
before. A key already written in the old form is rewritten by the same fill
step that brings a stale key in line with its name, the first time a reading
needs it. The OpenAPI descriptions say what the key is and that it follows
the marker's name and unit.
A converted reading's bounds move with its value, but the printed range text
stays in the unit the report used, and every reader preferred that text: the
list showed "Report range 3.9 to 5.5" beside 100.9 mg/dL, and the doctor
report and the FHIR referenceRange.text did the same. The text is still
stored verbatim. The range resolver now takes the reading's sourceUnit (as a
required field, so no reader can forget to select it) and, for a converted
reading, leaves the window's text empty so every surface states the converted
bounds in the marker's unit, and hands the printed text over separately with
its unit. The lab DTO carries it as printedReferenceText, and the note beside
a reading shows it as "as printed in mmol/L: ...".
Oura's current authorization server enforces scopes per collection.
daily_spo2 needs spo2; vO2_max, daily_cardiovascular_age and
daily_resilience need heart_health. The authorize request now asks for
both.
A grant issued before keeps answering 401 on those four collections
while every other one succeeds. Such a 401 next to a success is now
recorded as a scope gap rather than a failure: it no longer counts
toward the reconnect threshold, it no longer forces a token refresh once
known, and the Oura card asks for one reconnect instead. The Withings
card names the readings the account's API plan withholds.
A chronic ongoing condition has no end date, so counting it as rest mode
kept the account in it for good: Day N counted from an onset years back,
cadence nudges never resumed and every streak gap since the onset was
bridged. Only active episodes that are not CHRONIC_ONGOING (acute,
recurring, flare) now trigger rest mode, and Day N counts from the
earliest of those. The achievements streak freeze applies the same
filter to its own episode query.
Chronic conditions stay context: a coincident vital deviation is still
framed as explained by any active episode, and the Coach block and the
briefing still name them, with restMode false.
Refs #1216
Slice 3 of phase 3 of #1095. The scan review no longer treats every reading in
another unit as unwritable. The server tells it which analyte the matched marker
is (markerAnalyteKey on each extracted row); a row whose unit differs but can be
converted starts selected and carries a badge saying what Save will store and
what the report stated ("Saved as 100.9 mg/dL, converted from 5.6 mmol/L"). A
row that cannot be converted keeps the existing blocked badge and error. Save
still sends what the report printed; the server converts, as in slice 2.
The review's duplicate flag also recognises a re-scan of a reading that was
stored converted, by what the report printed, the same rule the commit applies.
A converted reading carries a quiet "converted from 5.6 mmol/L" on the lab list
(row and tile) and the history list (phone and table), so a number nobody typed
can always be traced back to what the report printed.
Strings in all seven locales.
Refs #1095
Slice 2 of phase 3 of #1095. A reading that stated a unit other than its
marker's was refused. When the marker's analyte is known and the registry has the
pair, the three write paths now convert it instead: POST /api/labs, the OCR
commit and the document commit.
The converted value is stored at full precision in the marker's unit, with what
the report printed in the existing source_value and source_unit columns. The
report's own reference window is in the printed unit, so it is converted with the
value; otherwise the verdict would compare a converted value with an unconverted
window. A pair the registry does not have is still refused as before (the 422
now says why), as is a marker whose analyte is unknown.
The marker's key is filled the first time a reading needs it. The same scan
committed again is still a duplicate: a converted row holds the number in the
marker's unit, so the duplicate check also matches on what the report printed.
Editing the value of a converted reading clears the conversion record, because
the number is then the person's own.
The lab result DTO and the OpenAPI schemas (labs and the OCR commit) carry
converted, sourceValue and sourceUnit.
Refs #1095
Slice 1 of phase 3 of #1095. A Biomarker row keeps only its name, so nothing
said which analyte it was. This adds the explicit alias list in the catalog the
issue settled on, including the registry-only keys (urea, blood urea nitrogen,
uric acid, total calcium, total bilirubin), and sets biomarkers.analyte_key
(from migration 0385) from it.
matchAnalyteKey equals a marker's folded name against the localised names of the
catalog seeds the registry knows and against the alias list. It matches the
whole name only, so "Calcium (ionisiert)" is not "Calcium"; a name two analytes
claim matches neither; and the marker's own unit must be one the analyte reaches,
so a marker called "Glucose" in a unit the registry cannot convert stays
unlinked.
The key is set when a marker is created (the biomarker route, the mint used by
lab writes, the legacy backfill) and, for an existing marker, by
ensureAnalyteKey the first time a reading needs it. It never overwrites a key
that is set. There is no boot job for the existing markers: the conversion that
uses the key fills it on first use, and a sweep can be added later if wanted.
The key is returned on the biomarker API and in the OpenAPI schema.
Refs #1095
The worker expands no slot past endsOn, so it never reminds for a
course that has ended, but remindersActive still read true for it. The
reminder gate now includes the course end on the person's clock, so the
published value, the worker and the admin check make the same decision.
The token ledger keyed every day on UTC while the daily briefing and the
other per-day generations run on the account's local day, so east of UTC
the first generations after local midnight were refused against the
previous day's spent ceiling. The budget day is now the calendar day in
the account's time zone; each request still resolves it once and books
its reservation and reconcile on the same row. The medication extraction
reads relative phrases against the same local day, the operator's spend
card shows the operator's own day, and the limit copy says the budget
resets at midnight in the person's time zone.
Every raw-SQL clock read against a timestamp column without a zone now
writes NOW() AT TIME ZONE 'UTC', so passkey and login challenges, rate
limits, provider pauses, coach usage stamps, tombstone and measurement
windows no longer pass through the session time zone even where the
per-connection pin cannot apply. The regression-window bounds stay naive
UTC timestamps instead of being turned into timestamptz. A structural
guard keeps bare NOW() / CURRENT_TIMESTAMP out of raw SQL outside a short
list of timestamptz targets. The integration test sets the database
default to Europe/Berlin and drives challenge claims, session expiry, the
native sign-in handoff freshness check and the rate-limit window through
the app's client, plus the challenge claim through an unpinned client.
Self-hosting docs describe the pin for an existing or shared Postgres.
A history backfill carried one access token through walks that outlive it.
After an hour every request answered 401, which classifies as a revoked
grant, so the account was parked at error_reauth with a refresh token that
was still valid, and every later run skipped it until a manual reconnect.
Running beside the hourly sync, the backfill also pushed the account over
Google's per-minute quota, and each 429 failed its whole collection.
- Walks take a token source that renews the access token before it expires
and once more when Google refuses it mid-walk.
- A 429 is waited out (Retry-After, RetryInfo, else a doubling backoff) and
the request repeated; it is never a reauth.
- error_reauth only short-circuits a run when the token endpoint refused the
grant (needsReauth). A connection parked by a data request's 401 refreshes
on its next run and clears itself when that run completes.
- One run per account at a time: syncs, the backfill and the sleep repair
share an advisory lock. The hourly poll skips a held account, a manual
sync answers 409 sync_in_progress, the backfill waits a bounded time.
- A backfill attempt records each collection it finishes cleanly
(backfill_progress) and the next attempt resumes with the rest.
- The incremental watermark is stamped with the cycle's start, not its end.
Overlapping sleep segments that share a stage and an end share the natural
key (type, measuredAt, source, sleepStage). A re-score that moved one row
onto a sibling's slot threw P2002 and failed the sync on every run.
- A batch keeps one reading per natural key, chosen the same way each time
(longer segment, then smaller externalId).
- A move that still collides is settled by reconcileExternalMeasurement,
which retires the stale row, instead of failing the sync.
- Updates run before inserts, and the key-format rescue no longer re-keys a
row this batch is moving.
- The sleep replace-by-window protects every session's fresh rows, so two
overlapping sessions no longer tombstone and resurrect each other hourly.
Refs #1194, #1195
The dashboard and the Coach now show the fixed strips at the right edge
with the day docked beside them, and the search field in the top bar.
Adds the command palette searching for sleep and the Health Score
history page. The timeline keeps All with one row per medication, five
value lines and a selected quarter. Each in light and dark, from the
demo seed on a local instance.
Day view docked beside the dashboard with its running line, values and
the five scores, and as a phone sheet; the timeline at All with one row
per medication, five value lines and a selected quarter; the phone
chronicle; the air quality and pollen card; the readiness inventory;
and the Coach beside the docked day with the new-chat button. Each in
light and dark, from the demo seed on a local instance.
Day view docked beside the dashboard and as a phone sheet, the timeline
at All with a selected quarter, the phone chronicle, the air quality and
pollen card, and the readiness inventory, each in light and dark. Taken
from the demo seed on a local instance.
The layout settings gain a switch for the dashboard's top card (the Today
card above the tiles). Stored server-side on the layout blob as
`todayCardVisible`: missing means shown, only an explicit false hides it,
the default is omitted from storage, a save that omits the field keeps
the stored choice, and a layout reset shows the card again. Hidden, the
card leaves no skeleton or error card behind. The flag rides the
account-settings backup with the rest of the layout.
The field is deliberately not `heroVisible`, the retired opt-in hero's
flag that older stored layouts may still carry.
An end-to-end journey on its own account switches the card off and on
and walks the score histories' range tabs.
The health score had no history anywhere, readiness and the sleep score
showed a fixed sparkline, and only the stored nightly scores offered the
range tabs. Every score page now charts its course with the same
7 / 30 / 90 / All tabs, remembered per page, and each point opens its day.
- GET /api/insights/score-history?score=&days= serves the daily course of
the health score (its stored day rows), readiness (the nightly blend,
filed on its wake day) and the sleep score (each night against the
trailing window ending on it), so a point equals what the day view shows
for that day. MANAGE record read, module-gated for readiness and sleep,
days 1 to 3650, additive.
- The health score's line breaks where its recipe changed (configuration
or algorithm version); the usual range is formed from the newest side
of the last seam.
- The range tabs move into one shared component used by every chart;
recovery, stress and strain keep their measurement chart, now with the
range remembered and no overlay dropdown.
- New page /insights/health-score, linked from the score panel and from
the day view's score tile.
A medication arrived as up to four kinds of item (its own span, its
courses, its dose changes, its pauses) and was drawn item by item, so a
medication with a course read as two bars on two rows. Every item of one
medication now carries the medication's id as `group`, and the lane draws
one row per group: the courses are the stretches it was taken, each dose
change cuts them into pieces labelled with their dose, and a pause is a
gap in the same bar. The selection bar and the phone chronicle list one
entry per medication with the dose in the text.
The value series cap of six is gone. The response carries
`availableSeries`, every series the record can show, and the picker is
one compact "Values (n)" button whose menu lists them. Past the fifth
line the five data colours repeat; every line is named at its left.
Choosing a range opens a popover anchored to the "Range" segment (a sheet
on a phone) with "Apply", so the chart no longer moves; the URL keeps the
range as before. The selection bar sits on the card under a hairline
instead of a grey box, the legend explains the dashed stroke and the
hollow point, and no timeline text joins its parts with a middle dot or
calls an open span "ongoing".
The day view gains a scores section below the values: the health score as
recorded that day, the readiness the nightly job persisted for the night
that ended that morning, a device's own recovery, the sleep score of that
night and strain (computed, or the device's day strain on its 0 to 21
scale). Each tile carries the person's usual range over the 30 days before
and opens the score's own page; a score the day does not hold has no tile,
and a day without any has no section.
Every value comes from where its page reads its history: the stored health
score record, the persisted RECOVERY_SCORE and STRAIN_SCORE rows read on
the day they describe, the canonical recovery ladder, and the sleep score
computed over one bounded read of the 30 nights up to the day. The stored
score rows leave the values list of the day they are stamped on, so a
readiness no longer appears a day early among the readings.
Scores follow their own module (recovery, sleep; the health score is core)
and map to the whole record for sharing: a scoped grant sees them named as
not shared, a whole-record grant sees them. get_day sends them to the
Coach and the MCP endpoint by name and number, and the Coach exclusions
take out every score built on excluded data.
A running cycle now states its cycle day and phase ("Cycle day 12,
follicular phase"), computed the way the cycle calendar does. Only logged
cycles count, the next logged start closes a cycle, and an open cycle
stops counting past its typical length plus the grace window instead of
reading a day count nobody recorded.
New conversation leaves the conversations panel's header and floats in
the conversation itself: a round 56 px button with a plus, the floating
Coach button's face, named "New conversation". It sits in the composer's
row, so it always belongs to the chat column, never to the list or a
docked day beside it. Where the column is wide it takes the bottom right
corner beside the composer; where the composer fills the width (a phone,
the drawer, a narrow column) it rises above the composer's edge. The
drawer shows it once there is a conversation to leave, in place of the
plus in its header. An empty conversation shows nothing either way.
The docked conversations panel folds like the day: its header opens with
a fold control, and folded it keeps a 48 px edge whose button opens it
again (aria-expanded, focus moved between the two, Escape inside folds
it). The top bar's toggle remains for the sheet below 1280 px. On /coach
the day docks right of the panel. Below 1600 px only one of the two is
open: a day opening folds the list without touching the remembered
choice, and opening the list folds the day to its edge. From 1600 px both
stay open. Switching or starting a conversation keeps an open day in the
URL.
The empty conversation drops everything under the composer: the daily
seeded opener and the scope pill are gone, and with them the only reader
of GET /api/insights/coach/seeded-question and of the
coach.ambientSuggestions preference. The route, its OpenAPI entry, the
preference and its Settings switch go too.
Marc-André Bombeck