The restore dialog asked GET /api/admin/backups/{id}/summary for the
contents of a copy, and the route read the whole stored copy to answer:
every piece opened, every measurement parsed, the rest validated. On an
account of 1.8 million readings on slow hardware that took about two
minutes. The dialog stopped waiting after 15 s and said the contents
could not be read, and its automatic retry made the server read the
whole copy a second time.
The copy passes through the process once anyway while it is written, so
the preview is worked out there and kept on the row (DataBackup.preview,
migration 0364): the summarizeBackup counts, the schema version, and
the inner key ids with the sections they sit in and one sample value
each. Every creation path goes through storeBackupBlob (the weekly and
manual pass, and an upload), so every new copy carries one. The column
is plain JSON: counts and dates only, and the samples are ciphertext
that stands for what is inside the copy, which a key rotation must not
re-seal.
The route answers from it without opening the copy. Both verdicts are
taken again on every read, from what the preview keeps: whether this
release restores the schema version, and whether this server holds the
keys the copy needs. The pieces are still counted against the row, so
a copy that lost one is refused as the restore and the download refuse
it. A preview names the copy it describes and is ignored once the copy
changes; converting a single stored value into pieces carries it over.
A copy written before this has no preview. It is read whole once, as
before, and the result is kept on the row, so it is slow only the first
time; concurrent requests for the same copy share the one read.
Only a file that fails the schema answers 422 "Backup payload failed
schema validation". Read refusals keep their codes; anything else is
the server failing and answers 500.
The dialog no longer retries the preview, waits up to 55 s, and when it
stops waiting on an older copy it says the copy is still being read and
to open the dialog again in a few minutes.
Refs #1031
Past 90 days every pulse reader already folded in SQL. Inside it, three
still read every raw heart-rate row, which on a watch that records a
reading a minute is 130 000 rows per read (#1023):
- The insights page's resting-pulse proxy, used when an account has no
resting heart rate, walked the whole window to take one percentile per
day and held the event loop for over a second per view. It now folds in
Postgres (`readRestingPulseProxy`) with the same percentile, rounding,
minimum-sample rule and day anchor.
- The per-kind series sent every reading: 14.7 MB for 90 days. A pulse
window with more than 10 000 readings is now bucketed per local hour,
mean plus low/high band, with stats still over the readings.
- The doctor report drew every reading into one sparkline. A pulse window
over the same cap now takes the per-day path longer reports already use.
Sparse pulse (a cuff, hourly imports, a watch sampling every few minutes)
stays under the cap and keeps its raw shape.
Antidepressants, anxiolytics, mood stabilisers and ADHD medication had no
category of their own and ended up under "Other". MENTAL_HEALTH joins the
clinical category list: the add and edit wizard offers it as its own row
with a brain icon, the card badge names it, and it is translated in all
seven languages.
The category is a text value, so no migration is needed. Clients that
read the category as a string show the new value; an app that does not
know it yet falls back to its own label for it.
A medication whose end date had passed still showed taken and skip in the
table and on its card, because the action row only looked at whether the
medication was active. The schedule engine already stops minting doses
after the end date, so reminders, the doses card and the today list were
quiet, but the medication itself kept inviting a dose that is no longer
part of the course.
The server now resolves one rule and publishes it on the list and detail
reads: `courseStatus` (UPCOMING, CURRENT or ENDED, in calendar days on the
user's clock, both ends inclusive) and `intakeActionable` (active, intake
tracked, course includes today). The table and both card variants render
their actions from it, the take-all flow skips such a medication, the card
and table name an ended course with an "Ended" badge, and no overdue
escalation is raised for it. A course that ends today keeps its actions
for the whole local day.
The detail header's "Ended" status now follows the same field. It compared
the end date's UTC midnight with the clock and so called the whole last
course day ended. Logging a past dose from the history is unchanged.
A restore writes rows back with their original updatedAt, behind every
paired client's watermarks, and removes rows without leaving tombstones.
A client that kept its cursor would miss both on its next incremental
pull. The same holds after the person deletes all their data.
`users.sync_reset_at` (migration 0363) records when the record was last
replaced: the restore stamps it as its last write and again right after
the commit, the per-account and the admin wipe stamp it inside their
transactions. Cursors now carry the time they were issued, and
/api/sync/changes answers its existing `cursorExpired` to a cursor issued
before the reset, or one without an issue time on an account that has
one. The comparison is on the issue time, because after a restore a
freshly drained cursor still carries old watermarks.
"As a table" used to drop the table's chart, so the reply offered no way
back to it. The copied table now keeps its chart and carries view: table,
which the panel opens on the table (the chart one tap away on the toggle)
and the chips read to offer "as a chart". show_result with view table
does the same. The field is optional on the result metadata; a client
that ignores it shows the chart first as before.
The recheck and follow-up rules now ride the tool-mode prompt only once
the conversation holds a table, or from the round after the turn produced
its first one. The clarifying-question rules are shorter and English for
every locale, like the other instructions to the model; the German copy
of them and of the recheck rule is gone. A turn without tables sends
about 1 300 characters less on every round.
The golden transcripts change only in the system prompt's hash and
length (4 509 to 3 187 characters; 4 728 to 3 280 for the German turn).
Sealing the practitioner's address left the appointment arm reading the old
readable column, which is now empty, so a booked visit's push lost the address
on booking, rescheduling and restoring a deleted visit.
The reminder no longer keeps its own copy of the address. The reminder tick
resolves it from the owning visit's practitioner when it builds the push and
decrypts it there, so an edit of the address also reaches a nudge that is
already booked. MeasurementReminder.location stays NULL on appointment rows:
booking writes none, a re-anchor clears one an earlier release wrote, and a
backup neither carries nor restores one. A checkup's own free-text location is
unchanged.
Ten scenarios, each in German and English, describe what a turn must do
with a question: which tool it calls and with which arguments, whether it
may read anything new, whether it asks back, declines a dose, offers
chips only for what the record holds, answers a chip without a provider,
offers to keep looking after a forced answer, and keeps an earlier
table's title out of the model's context. evaluateScenario grades a run
against them.
The deterministic suite drives POST /api/insights/chat with a scripted
provider; the tool loop, executor, table tool, chips, clarification
filter, method line, reply guards and stream run for real. Every check
also runs against a misbehaving model or a broken record and must fail
there.
route-followups.test.ts pins the front door for chips and answered
questions: validation, the rate limit and capability gate on a reuse
turn, the stored chip winning over anything the request adds, stale
chips and answers degrading to a plain message, and the continue line.
Up to three chips, derived on the server from what the turn read: the
other view of a table with a chart, the period before, a year earlier,
a wider window, and a metric a discovered pattern links to. A chip is
offered only when its read was done this turn and the record holds the
metric; the year-ago and wider-window chips also need the record's
first reading to reach back that far. The model may name chips in a
FOLLOWUPS block, but only to reorder ones the rules already allow; every
label is rendered from the catalog.
A tapped chip resolves against the chips stored on the conversation's
latest reply. "As a chart" and "as a table" are answered from the
stored table without a model call or a budget reservation: the rows are
copied unchanged and the message is tagged providerType "reuse". Other
chips run a normal turn with a server-written context line. A chip that
is no longer current falls back to a plain message.
A setting turns the chips off (absent means on).
The line that resolves an answered clarifying question was already
appended to the system prompt on both paths; nothing proved it for the
no-tools (local model) path. Three route-level cases now do: the local
prompt and the tool-mode prompt both carry the server-resolved line, and
a turn without an answer carries neither. Removing the line from the
local prompt fails the first case.
The golden harness gains a coachMessage.findMany mock for this; it
defaults to no rows, so no existing transcript changes and the golden
snapshots are untouched.
The result-table caption test used the regex dotAll flag, which the
TypeScript target rejects (TS1501); it matches across lines with
[\s\S] instead.
get_metric_table reads one metric as days, weeks or months through the
chart reader, cut in the user's zone and collapsed to one source per day,
with null for a period without a reading. The model reads a bounded
summary; the full table rides the turn, encrypted at rest. show_result
shows a table an earlier reply of the same conversation fetched, resolved
only against that conversation's own messages. Workouts, labs, sleep and
adherence results project into tables, earlier tables are named in the
context by metadata alone, and result:rN marks are stripped from the reply.
The evidence disclosure under a Coach answer now carries one muted
line: which sources and windows the turn read, how many readings, and
how they were aggregated ("Blood pressure, last 90 days: 142 readings,
weekly averages · Sleep, last 30 days: no readings in this window").
The server builds it from the recorded steps and tables in the request
locale; it holds catalog strings and counts only, never a reading, and
a source that turned up nothing says so.
The tool-mode prompt gains a recheck rule: when the person questions a
figure, name the source and window, read it again (or reuse the stored
table), and correct the answer if the two differ.
Each tool call now emits a running step as it starts and its final
status as it settles: found with a server-counted number of readings,
empty with the reason, or failed. The no-tools path shows one step for
the record summary. Steps carry catalog labels rendered in the request
language, closed-enum domains and windows, and integers only; a lab
analyte, a value or a model-written argument never reaches one. At most
twelve per turn. They persist on the provenance, so a reload shows the
same list.
The conversation window reaches the step so a call without a window
names the one it actually read. The no-tools transcripts gain the
summary step; a new transcript pins found, empty and failed steps ahead
of the first token.
A reply may end with a ---CLARIFY--- block (kind metric, window or
context, 512-byte cap). The server keeps only metric choices the record
holds, drops the question when fewer than two remain, takes window
choices from the presets, screens the question text like any reply, and
never offers a second question right after one. Choice labels come from
the catalog, never from the model.
An answer arrives as clarification { messageId, choiceId? }; it resolves
against the stored question on the conversation's latest message and
adds one server-written line to the next turn's context. The card sits
above the composer, announces politely and takes no focus.
A reply that is only the block, with no question before it, is now an
empty reply instead of falling back to the unstripped text.
GET /api/insights/chat/{id}/messages/{messageId}/results returns the
tables one assistant message read, narrowed to the conversation's owner
like the conversation read, with a table withheld as module_disabled when
its domain's module is off and as unavailable when it cannot be read. A
data read: no AI capability.
The published document now describes every stream frame as the
CoachStreamEvent union, including the suggestedAction frame it never
named, the new step, result, followUps and clarification frames, their
components, the provenance fields, the followUp and clarification request
fields, and the reuse provider type.
Each later piece of the dialog gets its own module, answering nothing for
now: the step builder, the table projection and chart choice, the result
ref marks, the follow-up derivation, sentinel, resolution and continue
chip, the reuse turn, the clarification parse and answer, the method line,
and the four prompt addenda. The turn pipeline calls them where they
belong, so the frames go out in the documented order (step, token,
provenance, result, cards, clarification, followUps, done) and a blocked
turn drops its tables, chips and clarification. With every piece empty
the stream and the prompt stay byte for byte as before.
The home location of the environment module was rounded to 2 decimals,
about 1 km, which places a home in a neighbourhood rather than the town the
schema comment promised. The weather archive it feeds is gridded at 9 to
25 km, so the extra precision bought nothing.
Every stored coordinate of the module now goes through the same 1-decimal
rounding (about 11 km north-south, 7 km east-west at 50° N): the home, the
travel periods, the geocoder's results, the per-day weather rows, and a
restore of a file written with finer values. Migration 0361 rounds the
existing users, travel periods and per-day rows; running it twice changes
nothing more. The answers keep their shape, plain numbers. The API
descriptions and the schema comment say 1 decimal now.
The route keeps the front door: auth, the coach capability, the body
read and schema, the rate limit, the locale and the inbound refusal
screen. Everything after it moves into one module per step:
conversation, context, chain, budget, model, reply-guards, cards,
provenance, persist, sse, errors and pipeline. The capability re-check
at the egress site stays owned by the route and is handed to the
pipeline, so the capability inventory still reads both calls there.
The stream producer now writes through a TurnEmitter created inside
createSseStream and passed to the model step, which is where live
progress frames will come from. Nothing writes to it early yet.
No behaviour change: the golden transcripts pass byte-identical, the
existing route tests pass unmodified, and every annotate, audit and
error-code name appears as often as before.
Guards follow the code. The fence call-site check reads turn/context.ts,
the German-default locale ban covers every file in turn/, and the
fenced module-graph test now asserts that turn/sse.ts and turn/errors.ts
reach neither the tool registry nor the snapshot builder. That test
named lib/ai/coach/tools.ts, which stopped existing when the registry
became a directory, so its negative check could not fail; it points at
tools/index.ts now, and a new sanity check fails if the tool route can
no longer reach the path it names.
The GPS track of an outdoor workout was stored readable, and its first and
last points are usually the person's front door. The phone number and address
in the address book of doctors and practices were readable too, while the
note beside them was already encrypted.
Migration 0361 adds workout_routes.geometry_encrypted (the geometry loses
its NOT NULL) and practitioners.phone_encrypted / location_encrypted, plus
partial indexes on the backfill's discovery predicates. A track is sealed as
one binary AES-256-GCM value bound to its purpose; the contact fields use the
same Bytes codec as the note. The practitioner's name, practice and specialty
stay readable, because the picker searches and sorts on them.
Every writer stores ciphertext only: workout batch ingest, practitioner
create and edit, and both restore formats. Readers prefer the ciphertext and
fall back to the old column for rows not yet migrated: the workout detail
(map, splits, GPX), the activity insight's climb figure, the practitioner
DTO behind the address book, visits and vaccinations. The wire shape does not
change. An edit's audit row now names a changed phone number or address
without recording either value.
free-text-encryption-backfill covers the three columns in the same pass,
finding a route through its workout. All three are registered for key
rotation (registry, CLI script, and the in-app rotation). A disaster-recovery
backup carries the contact fields as ciphertext only; a portable export
carries them readable. Workout routes are still not part of a backup, as
before. The old columns are marked for removal in a later release.
Drive POST /api/insights/chat through the real SSE stream and the real
reply guards for every branch a turn can take: tool mode with present,
drifted and missing figures, the no-tools path, inbound and replayed
refusals, the budget refusal, each provider failure class, a cancelled
turn, the outbound block, every sentinel in its valid and malformed
forms, learn-link scrubbing, the three no-provider paths, a workout
scope, the post-call defect path, and the refusals that answer before
the stream opens.
Each transcript records the frames the client receives (heartbeat
comments dropped) and the ordered side effects: persisted messages,
budget reserve and reconcile, audit rows, annotate names with their
meta keys, and the provider inputs as prompt fingerprints.
The per-account password limit refused every attempt once ten failures
had landed in fifteen minutes, counted before the password was checked.
Ten guesses from two addresses therefore locked a password-only account,
the app's password sign-in included, and repeating them kept it locked
for as long as anyone cared to.
A place the account has signed in from before is now neither counted nor
held back: a remembered browser, a phone presenting the device id a login
of this account was issued to, or an address a session or device login of
this account came from in the last thirty days. Anywhere else gets five
free failures a day and then a wait that doubles with each failure, from
thirty seconds to at most fifteen minutes, instead of a wall. The per-IP
limit is unchanged, and an unknown identifier is limited the same way on
its hash.
Under an SSO-only policy, a password sign-in to an account other than the
one a pending link names is now refused before the password is checked,
so that exception no longer tells right passwords from wrong ones.
Allowing an AI assistant hands it a standing credential to the record
that outlives the session, the same class of act as minting a token, but
the consent took a live session alone. A stolen session could connect
its own assistant and keep reading after the owner signed everything out.
The consent page now shows the Allow form only when the session signed in
or re-proved within five minutes (with the second factor on an account
that has one). Otherwise it links to a new confirm page that runs the
same re-proof dialog as the rest of the app and returns to the exact
consent request. The allow decision checks again and mints nothing
without it; deny needs no proof. The confirm page only ever returns to
the consent endpoint.
Offering access to the record took a live session alone, and an
invitation outlives the session that sent it: whoever held a stolen
session could invite their own account and keep a way into the record
after the owner signed everything out. POST /api/account/grants now takes
the recent-proof gate at every level (a sign-in or re-proof within five
minutes on the browser, with the second factor on an account that has one;
a step-up elevation on a token, which no shipped client uses here), and
the invite form opens the re-proof dialog and sends again.
Sign out everywhere now withdraws every invitation the account offered
that nobody has accepted. Accepted grants are left standing on purpose:
each is a person the owner chose, often a carer who still needs the
record, and a sign-out button must not quietly cut them off. The response
lists them as grantsKept and the sessions card shows each with a one-click
End access.
GET /api/export with type=all (the default), POST /api/export/health-record
and GET /api/fhir/Patient/$everything hand over the whole record but took a
live session alone, so a stolen browser session could still walk off with
everything the full-backup gate was meant to protect. A browser session now
needs a sign-in or re-proof within five minutes, with the second factor on
an account that has one, and the doctor-report panel opens the re-proof
dialog and retries. When a manager builds the report for a record they
manage, their own session is the one asked.
On a token the shipped app keeps working: health-record and $everything
take the token as before, including a fhir:read token, and the recent-proof
guard names both. GET /api/export?type=all, which no shipped client calls
with a token, wants a step-up elevation there. Single-type exports are
unchanged.
The API reference now also describes the enrollment rule for accounts
with a second factor and the passkey refusal it adds.
A reminder's first-due date could only be sent as a date-time, and the
web form sent the browser's local midnight, so a browser set to a zone
other than the profile's could move the day. The API now also takes a
bare calendar date and stores it as that day's midnight in the user's
zone, and the web form sends the date itself. A date-time still means the
instant it names.
On an account with a second factor, TOTP setup, security-key registration
and passkey registration no longer accept the password, a password re-proof
or a session younger than five minutes. A remembered browser skips the
second factor at sign-in and still creates a fresh session, so the password
alone could add a key that then satisfies every gate the factor protected.
Only a TOTP code, a security key, a passkey or a completed second factor
within five minutes opens enrollment there; the Bearer arm of TOTP setup and
security-key registration asks the same of its elevation. Accounts without
a second factor keep the previous rules.
The refusal names the proofs the account can give, and the enrollment
dialogs offer exactly those. A proof that verifies now gives its attempt
back to the shared re-proof budget, including the current password on a
password change.
A data wipe during the nightly upload could leave the pre-wipe copy in
the bucket: the purge the wipe started removed its request at once, and
the upload still in flight checked that request after finishing and found
nothing. A wipe's request now stays until a nightly run has walked the
account since the wipe. Until then each purge removes only the copies
dated up to the wipe's day, so the copy of what the account holds after
the wipe stays, and the admin card no longer counts the waiting request
as copies still in the bucket.
A re-import that moved a reading to another instant re-folded only the day
it moved to, so the day it left kept the reading in its rollups. And a
write chunk that failed part-way ended the request before the re-fold ran;
a retry then read the landed rows as duplicates and touched nothing, so
those days kept their pre-import rollups for good. The re-fold now covers
the old instant of a moved row and runs whether or not the writes finish.
Making `=true` grant nothing would stop a single-user self-host's local
model on a patch update. It is deprecated for one release cycle instead,
in a narrowed and safe form: it still grants a private host, but only to
the operator's own configurations (the instance-wide admin provider and
AI settings saved on an admin account), and only through the pinned
operator-approved dispatcher, so redirects stay forbidden and metadata,
link-local and unspecified addresses stay refused. A non-admin account's
own base URL needs an exact origin in AI_PRIVATE_ORIGINS.
Provider resolution passes the owning account's role to the client
(`operatorTrusted`); the save route and the connection test apply the
same rule to the account being configured. The boot summary now says the
setting is deprecated and names the replacement, and the admin AI
settings page shows the exact AI_PRIVATE_ORIGINS line for the private
origins saved on the instance (every account's Local and gateway URL and
the admin base URL, scheme, host and port only), so the operator can copy
it and drop the old setting.
The admin wipe of every account is the operator clearing their own host,
and the bucket's copies are the documented way back from a wipe pressed
by mistake, so it writes no off-host deletion request. Deleting an
account, deleting a managed profile and "Delete all data" still do.
The backup runbook no longer says the worker never deletes an object: it
lists the grant the worker needs (DeleteObject included), when objects
are deleted, the version-4 envelope, the lifecycle check and how to
rotate the off-host key. The rotation runbook no longer says backups are
covered: rotation reaches a backup's envelope, not the content a restore
writes back, and a retired key stays needed until no backup lists it.
The privacy notice beside account deletion says the same about off-host
copies.
A failed import records its failure in a plain try block, and the
extraction test that inflates past the 64 MiB floor gets the time it
needs on a loaded runner.
Marc-André Bombeck