A restore clears every row of the account, tombstones included, inside one
long transaction under the account's exclusive restore lock. The nightly
purge deleted expired tombstones across accounts, one statement per batch,
without that lock: on an account under restore it waited on the restore's
row locks for as long as the restore ran, past its statement timeout, or
deadlocked with it once each side held rows the other needed.
Each batch is now deleted one account at a time, in a transaction that
first takes the restore lock in shared mode, the same order the
consolidation passes keep. An account under restore is refused at once,
kept out of the rest of the run, and purged the next night. Still one
delete per account per batch, never one per row. The job reports how many
accounts it deferred.
The integration test holds a stand-in restore on one account while the
purge runs: the purge returns without waiting, leaves that account alone
and purges the other. With the lock call removed it waits out the hold.
Refs #1031
The shared admission queue sent its jobs without an expiry, so every one
took pg-boss's fifteen minutes. Against pg-boss 12.34, a handler that runs
past its expiry is not stopped: the job is failed, its abort signal fires,
the worker and group slots are freed, and the retry of the same job starts
in the same worker beside the run it replaced. A per-minute heart-rate
history is thousands of pages, so on a slow link one import turned into up
to four copies of itself, on the one queue meant to run a single import at
a time.
- The admission job now carries a sixteen-hour expiry, sized for the
longest legitimate import and under pg-boss's 24-hour ceiling.
- Each run holds a session advisory lock on its identity (kind, provider,
account) on a connection of its own. A delivery that finds it taken does
nothing and says so in the log; the running import stamps its marker or
is offered again at the next boot. Ending the connection or the process
releases the lock.
- The Google Health backfill runs under the job's budget and abort signal.
The page walk asks before each request, stops there, and reports the run
truncated, so it throws without the completion stamp and is retried.
- The drain moves out of the registrar into its own module so it can be
driven against a real pg-boss.
The integration test pins the pg-boss behaviour and runs the lane's
handler with a two-second expiry: without the lock the import ran twice.
Refs #1023
The production audit flags a critical advisory against next 16.2.0 to 16.3.5,
fixed in 16.3.6, and newer advisories against the transitive fast-uri (ajv),
ip-address (the MCP SDK's rate limiter) and dompurify (jspdf) copies. Next is
pinned to 16.3.6 rather than the newest patch so the install stays inside the
minimum release age without new exclusions.
A restore on a freshly booted instance failed its clearing step with a
deadlock (40P01) while the boot-time step consolidation was folding the
same account. A consolidation day updates or mints the day's total and
then soft-deletes the day's per-sample rows; the restore deletes every
reading of the account in one statement. When the restore reached a
day's samples before its total, each side held what the other needed
next.
The restore already took an advisory lock per account before its first
delete. Every day transaction of the passes built on runConsolidation
(step and mean consolidation, the per-sample cumulative drain, the dense
intraday retention) now takes the same lock in shared mode as its first
statement. A restore that starts mid-day waits for that day to commit;
a day that would start while a restore runs is refused at once, and the
pass leaves that account for its next run instead of folding rows the
restore is about to replace. The passes still share the lock with each
other.
A new integration test builds the lock order that deadlocked, with a
row trigger holding the consolidation on the day's total while the
restore starts; before this change the consolidation was cancelled with
a deadlock. A guard test requires every transaction in a pass on
runConsolidation to open with the shared lock.
Refs #1031
Prisma's debug logger keeps the arguments of its last hundred calls in
memory whether or not logging is on, and the client and the Postgres
adapter both log every statement with its parameters. A restore that
wrote 1 000 readings per statement therefore kept a copy of about the
last 100 000 readings alive for the whole measurement write: about
60 MB on 1.8 million Apple Health readings, next to a V8 heap limit of
524 MB in the default 1 GB container.
The bulk insert now splits whatever it is handed into statements of at
most 100 rows, which holds about 7 MB. The Apple Health export import
goes through the same helper and gets the same bound. The write costs a
little time: 94 s instead of 76 s for 1.8 million readings on a fast
host.
Refs #1031
A type served from day buckets (pulse or glucose past 90 days, and since
the previous change a dense pulse stream on any window) is drawn one point
per day, each the day's mean at the day's start. The summary line counted
those points as measurements, and the FHIR export stamped the last day's
mean as the latest observation: a value and a time nobody measured.
The day-bucket stats now carry when their newest reading was taken
(`latestAt`). `latestReading` returns that reading when it is there, and
the summary line counts each type's readings from its stats. Raw series
are unchanged: their stats count equals their points and their last point
is the newest reading.
Past 90 days every pulse reader already folded in SQL. Inside it, three
still read every raw heart-rate row, which on a watch that records a
reading a minute is 130 000 rows per read (#1023):
- The insights page's resting-pulse proxy, used when an account has no
resting heart rate, walked the whole window to take one percentile per
day and held the event loop for over a second per view. It now folds in
Postgres (`readRestingPulseProxy`) with the same percentile, rounding,
minimum-sample rule and day anchor.
- The per-kind series sent every reading: 14.7 MB for 90 days. A pulse
window with more than 10 000 readings is now bucketed per local hour,
mean plus low/high band, with stats still over the readings.
- The doctor report drew every reading into one sparkline. A pulse window
over the same cap now takes the per-day path longer reports already use.
Sparse pulse (a cuff, hourly imports, a watch sampling every few minutes)
stays under the cap and keeps its raw shape.
A full-history backfill read every heart-rate point of an account into one
array, mapped it into a second, and wrote the lot in one call; the cycle's
deferred-rollup ledger then kept one entry per written reading. On a watch
that records a reading a minute that is over a million points, and the
backfill ran a 1 GB heap out of memory within seconds, restarting the
instance it shares with the app (#1023).
The metric walk now hands each page to its write before requesting the
next, and both the Google Health and Fitbit ledgers collapse to one key per
touched type-day as they record. The heart-rate walk's page ceiling rises
from 1000 to 10000: at one point a minute the old ceiling is under two
years, a longer history always ended truncated, and the backfill retried
the whole walk on every attempt and every boot without completing.
The clinical category a person picks for a medication lives in a side table
created at runtime, outside the schema. The backup builder and the restore
only walk schema models, so no backup ever carried it: after a restore every
medication read as "Other".
Both backup formats now carry the category inside its medication, and the
restore writes it back in the same transaction as the medication. A file
from before this change restores as before; a category this server does not
know reads as Other instead of refusing the file.
The backup plan gains a list for tables created with raw SQL, and a check
that every such table in the source either maps to a schema model or has a
verdict there, so the next one cannot drop out of backups unnoticed.
A week or month point of a cumulative metric is the average daily total,
which keeps the axis comparable with the 7, 30 and 90 day tabs. The
caption said "Weekly avg", which reads as an average week's total. Steps,
energy, distance, flights and daylight now say "Daily avg per week" or
"per month" on the caption and in the tooltip, in all seven languages;
level metrics keep the plain average.
The All range is read from the WEEK or MONTH rollup tier once a history is
long enough, from DAY rows for a short one, and from the live table on a
coverage miss. The chart folds every path into the average day of each
week or month, but a tier row carried a cumulative metric's whole total
and a level's reading-weighted mean. A month of steps therefore read about
thirty times higher for a long history than for a short one, under a
caption that said average.
A tier row now carries the average day of its bucket on every tier: a
cumulative metric's total over the days that hold a reading, a level's
mean of its daily means. The rollup fold counts the bucket's days and adds
up their means for it. Only the web chart's series reads (the chart
endpoint and the dashboard batch) use these rows.
The add and edit wizard had no row for the thyroid, digestive, skin and
sleep-aid categories, so such a medication opened on "Other" and saving any
change, even a new dose time, rewrote its category to OTHER. The wizard
now offers a row for every category the app knows, translated in all seven
languages.
An edit also sends the category and treatment class only when the category
row was changed. A stored value the wizard cannot show, such as one a newer
server or another client wrote, now survives an unrelated edit.
Antidepressants, anxiolytics, mood stabilisers and ADHD medication had no
category of their own and ended up under "Other". MENTAL_HEALTH joins the
clinical category list: the add and edit wizard offers it as its own row
with a brain icon, the card badge names it, and it is translated in all
seven languages.
The category is a text value, so no migration is needed. Clients that
read the category as a string show the new value; an app that does not
know it yet falls back to its own label for it.
The release branch alone measured 3699 KB gz against the 3700 cap; the
review fixes added 1.3 KB gz across the whole build, nearly all of it new
copy in the seven lazy locale catalogs. The four eager routes measured
the same on both builds and stay inside their ceilings. The aggregate cap
moves to 3724, the same headroom as the earlier re-measurements.
A medication whose end date had passed still showed taken and skip in the
table and on its card, because the action row only looked at whether the
medication was active. The schedule engine already stops minting doses
after the end date, so reminders, the doses card and the today list were
quiet, but the medication itself kept inviting a dose that is no longer
part of the course.
The server now resolves one rule and publishes it on the list and detail
reads: `courseStatus` (UPCOMING, CURRENT or ENDED, in calendar days on the
user's clock, both ends inclusive) and `intakeActionable` (active, intake
tracked, course includes today). The table and both card variants render
their actions from it, the take-all flow skips such a medication, the card
and table name an ended course with an "Ended" badge, and no overdue
escalation is raised for it. A course that ends today keeps its actions
for the whole local day.
The detail header's "Ended" status now follows the same field. It compared
the end date's UTC midnight with the clock and so called the whole last
course day ended. Logging a past dose from the history is unchanged.
A restore writes rows back with their original updatedAt, behind every
paired client's watermarks, and removes rows without leaving tombstones.
A client that kept its cursor would miss both on its next incremental
pull. The same holds after the person deletes all their data.
`users.sync_reset_at` (migration 0363) records when the record was last
replaced: the restore stamps it as its last write and again right after
the commit, the per-account and the admin wipe stamp it inside their
transactions. Cursors now carry the time they were issued, and
/api/sync/changes answers its existing `cursorExpired` to a cursor issued
before the reset, or one without an issue time on an account that has
one. The comparison is on the issue time, because after a restore a
freshly drained cursor still carries old watermarks.
The persisted copy of a streamed reply used to mount as a new bubble, so
what the reader had done on the streamed one (the chart or table view, the
open steps list, the evidence panel) reset when the conversation refetch
landed. A turn now keeps one key from its first streamed frame to its
persisted copy, both render in the same wrapper, and the copy keeps the
live tables instead of refetching them, so React keeps the one bubble.
The wrapper stops being a live region once the turn is settled. The
150 ms twin-hiding window this replaces is gone.
"As a table" used to drop the table's chart, so the reply offered no way
back to it. The copied table now keeps its chart and carries view: table,
which the panel opens on the table (the chart one tap away on the toggle)
and the chips read to offer "as a chart". show_result with view table
does the same. The field is optional on the result metadata; a client
that ignores it shows the chart first as before.
Marc-André Bombeck