The fixtures built their rollups with the read refresh, which folds ninety
days. Under the stricter coverage a type folded only that far takes the
live path; the fixtures now fold the whole history first, as the boot
backfill does, so the walk still reads the rollup tier.
The analytics snapshots change only in what the old coverage hid: the
summaries count all 120 weight and blood-pressure readings instead of the
91 the folded window held, with the means and the all-time in-target share
that follow from them.
The coverage probe called a type covered as soon as one DAY bucket existed.
A write hook folds the day it writes, so a type whose history was not yet
folded (an account before its backfill, or a read racing the detached
refresh) had one fresh bucket, took the bucket-composed path, and its
summary counted that one day: three weight readings read as one.
A type is now covered when its oldest DAY bucket starts no later than its
oldest live reading inside the fold window. The bucket holding a reading
starts at or before it in any time zone, so the comparison holds without
knowing the account's day boundaries. Both sides are one index descent per
type; anything not covered falls back to the live aggregate as before.
The summaries read starts ensureUserRollupsFresh detached. When the third
row's DAY bucket landed before that refresh, WEIGHT counted as covered by
its one bucket and the read reported a count of 1; a CI shard hit exactly
that.
Migration 0491 cleared the shared "profile" placeholder to null. Those
connections sync normally, but the one-account-per-record check cannot see
them, so the same Google account could still be connected to a second
record until the first one reconnected.
After a clean sync cycle, a connection without an id now reads it the way
the callback does (profile, then identity) and stores it, guarded on the id
still being null so a reconnect landing meanwhile wins. An id another
record already holds is not written, and a failed read leaves the id for
the next cycle without touching the cycle's verdict. The outcome is on the
wide event as googleHealth.accountId.heal.
The profile's resource name is `users/{user}/profile`, and the connect
stored its last segment, the same constant for every account. Since the
one-account-per-record check, a second record on an instance could not
connect or reconnect Google Health: it was refused as `already_linked`.
The connect now stores the `{user}` segment, reads `users/me/identity`
when the profile answers through the `me` alias, and stores the id as
unknown when neither names the account. A migration makes the column
nullable and clears the placeholder, so the next connect stores the
real id.
The privacy check looks for the fixture's latitude 52.52 in every stored
row, and a timestamp written at :52.52x seconds holds the same digits. It
failed once in CI with no coordinate stored. Clock times are masked first;
a coordinate in any field is still found.
Every night's replace-by-window clean-up carried every fresh segment id of
the fetch. On a first sync that reads years of sleep that is one query per
night with tens of thousands of bound values, over Postgres's limit, and
the process spent minutes building them before each one failed. The sweep
can only reach rows inside the window, so each window now protects the
fresh rows inside it, from whichever session reported them.
Refs #1195
Hiding a marker takes it out of the labs catalogue and the entry
pickers, but the timeline still offered it among the lab values.
The timeline now offers only markers that are not hidden. A hidden
marker that was chosen before keeps drawing its line and stays in the
menu as a ticked entry, named from that line, so it can be unticked,
the way the entry form keeps a hidden marker that is already selected.
Once unticked it is no longer offered.
Restored rows keep the ids they had in the file. Once an erased
account's copy had gone into one account, the same file uploaded into a
second account restored into a primary-key conflict, reported only as
the generic "the database rejected a record" failure.
For a copy put into a chosen account, a primary-key conflict now ends
the restore with backup.restore.rows_exist_elsewhere and a sentence that
says the file was most likely restored into another account already.
The transaction still rolls back, so neither account changes. The code
is in the catalogue, the console names it, and the runbook says a file
goes into one account only.
The upload creates a passwordless account before it reads the file and
deletes it again when the file is refused. The only audit row was
written after a successful store, so a refused upload whose delete also
failed left an account nobody could sign into and no record of where it
came from.
The account and its admin.user.create row (outcome "created") now
commit together, and the discard writes admin.user.delete with outcome
"deleted" or "failed", so a leftover account always has a trace. The
runbook says how to remove one by hand.
A restore read the account settings under the operator's AI endpoint
policy whenever the receiving account was an admin. For a copy put into
a chosen account, the settings were written by the erased account's
owner, not by the operator, yet an admin account could be chosen as the
target, and an ordinary target promoted later got the same trust.
Admin accounts are now refused as a target, in the check and the
picker, with backup.target.admin_account. A copy uploaded into a chosen
account is stored as RETARGETED_UPLOAD_<ms>, and its restore never reads
its settings as the operator's, whatever the account's role is by then.
The empty-account check counted every backed-up model, and signing up
and answering the setup questions always leaves a setup row, plus a
cycle profile when the cycle area is chosen. The account of someone who
has signed up again, the case the restore target is for, was therefore
always refused.
Those two models are one row per account and are deleted and re-created
from the copy by every restore, so they no longer count. A reading or
medication entered on setup's last screen still refuses the account,
because the restore would delete it. A guard holds the exempt models to
that shape, and the erased-account integration file now drives the real
setup routes before the restore.
A values list naming more than 24 lab markers answered 422. The list is a
device preference, so once a 25th marker was ticked every later visit
failed: the page rendered its error card, and the values menu that could
untick it only renders on success.
The server now keeps the first 24 markers and leaves the rest out, as it
already does for a marker the record no longer has; a malformed key still
answers 422. The page applies the same cap before it asks, and the values
menu holds further unchosen markers back at 24 with a line saying why,
while a chosen one can always be removed.
The timeline showed a lab report as an event on the axis, but the lines
under the lanes could only be measurements. A lab marker can now be named
in `values` as `lab:<biomarkerId>`, beside the measurement types and
`MOOD`, so one choice, one request and one stored list carry every line.
`GET /api/timeline` answers with `labSeries`, one entry per named marker
with its readings in the window as single points (lab values come a few a
year, so they are never averaged into buckets), each dated by the record's
local day, judged against its own window and linked to the report it was
read from when the caller may see documents. `availableLabs` lists every
marker the record has a numeric reading for, so a client offers only
those.
A reading stored in another unit than its marker's is converted through
the unit-conversion registry where the pair is known, and otherwise left
out and counted in `excluded`, never drawn on a scale it is not in. A
marker id that is not in the record yields no line instead of a 422, so a
stored choice survives a deleted marker; ids are capped at 24 a request.
Refs #1299
After an account is erased the operator still has its disaster-recovery
file, and the upload refused it because its owner was gone. The refusal
now carries backup.owner.missing, and the file can be sent again with a
target in the query: an existing account that holds no records yet,
confirmed by typing its username, or a new account created for the copy.
The file's owner is rewritten to that account as it is read, so the
stored copy, its preview, its download and its restore are that
account's, and the restore that follows is the ordinary one.
The new account has no password and no email; an admin sets its first
password with the existing reset. A file whose own account is still here
is refused with a target, and an account made for a refused file is
removed again. The refusal of an unknown owner no longer writes the
owner's id into the audit trail, since the erasure took every row that
named it.
Under a loaded full run the wrapper around pg_dump and pg_restore crashed
with an unhandled stream error, and the dump under test reported only the
tail of that stack. The wrapper now leaves the exit code to the tool and
writes the reason to stderr.
The check whether an account's full rollup backfill is queued or
running answered false on any error, so a failed read let the dense
fold delete raw samples the backfill still had to read. A missing queue
table still answers false, since a web-only deployment has no backfill;
any other failure now counts as pending and is annotated. A backfill
that failed for good is still not pending, as the comment now says.
The admin recompute for one account called the full fold without a
statement budget, so on the large accounts it is meant for each yearly
statement ran under the database's ordinary limit and timed out, as the
boot job had before it got its own. It now passes the same per-statement
budget.
The MFA status read listed oidc among the proof methods for a Bearer
caller too, although the provider round trip stamps a browser session
and nothing else, and the API contract says a token is never offered
it. The status read now leaves it out on the Bearer path.
One staged fact that would not decrypt threw out of the whole repair, so
the job failed, retried, and was enqueued again on every boot. It is now
skipped and counted as undecryptable.
The candidate query read an account zone Postgres does not know as UTC,
while the repair resolves it to the instance default and then
Europe/Berlin. A value confirmed near midnight stayed a candidate the
repair always left alone, so the set never converged. The query now
falls back the same way.
The alert added every attempt of a failed job to the streak, so one job
with two retries read as three failures. It also judged a queue by its
newest job across all accounts: on a queue that runs one job per
account, one account failing every hour beside accounts that succeed
never reached the streak, or flipped between failing and recovered.
The streak now counts jobs, and is kept per account (the job's userId,
or the whole queue for jobs without one). A queue is failing while any
account has failed three jobs in a row, and recovered only when a
failing account has itself run successfully since. A reminder needs a
failure newer than the notice before it, so a one-off job that failed
and does not run again is announced once.
The dense fold deleted Google Health and Oura samples outright, on the
reasoning that no client pulls them. The change feed serves every
source, so a client mirroring it kept those samples next to the hourly
means with nothing to tell it they were gone.
The nightly increment of a server-pulled source is tombstoned again: the
feed reports each sample and the tombstone cleanup purges them after the
retention. A backlog day, such as an account's first fold, is still
deleted outright, and the account's delta cursors expire instead, since
tombstoning millions of rows would undo the point of the fold. The
compaction-tombstone class leaves these sources out, so the feed, the
backlog purge and the restore treat their tombstones like any deletion.
Apple Health and Health Connect are unchanged.
The status prompts carry the unattended flag for background calls, the
queue list holds the lab date repair, and an account whose only way to
confirm is its identity provider is now offered that way.
The reading delete found the cuff's pulse by owner, source and identical
instant. The dense fold anchors an hour of heart rate at the local HH:30
under the samples' source, so a reading taken at half past took that hour
with it. Aggregate (stats:) rows are never part of a reading now, for the
delete and for the restore.
420 000 heart-rate readings, one every five minutes over about four
years, written through the real Google Health backfill with Google's
API faked at the safeFetch seam. The account's DAY rollups lose their
reading extremes, as rows folded before 0417 have them; the boot
discovery finds it, and the windowed full backfill refills every bucket
in the fold window with the raw readings' own extremes. The dense fold
then thins everything older than 90 days to hourly Google Health rows
(420 000 rows to about 59 000) while the DAY tier keeps its figures,
and a second full-history backfill over the same samples inserts
nothing. Without the ingest guard it re-inserts about 394 000 of them.
Each step reports its duration on stderr. On a developer machine:
backfill about 85 s, discovery 0.2 s, full backfill 4 s, fold 40 s,
second backfill 4 s.
Nothing covered the operator's position after an erasure: a backup file,
an admin session, and a server that no longer knows the account. Export
the shared backup seed, delete the account through the settings route
under a 14-day window and run the purge job past its end, then:
- scan every text and JSON column for the account's id; only a
rate-limit bucket that expires within a day may remain;
- upload the file as an admin: refused as owner_not_found, nothing
stored (the documented contract);
- recreate the account row under its id, upload and restore: every
section's count matches the export;
- point the file at a freshly created account, upload and restore: every
count matches and nothing names the erased account afterwards.
Every backup test exports and restores with the same code, so a change
that moves both ends at once passes all of them and still breaks a file
a self-hoster downloaded last month. Commit the disaster-recovery file
the 1.46.1 exporter writes for the shared backup seed, with a manifest of
its counts and SHA-256, and restore each committed file through the admin
upload and restore routes into an empty account, comparing the re-exported
counts section by section.
The folder takes one file per release (GOLDEN_BACKUP_WRITE=1 writes the
current one; the README says when). The seed's own illness-symptom
catalogue key is swapped for a shipped one so every link resolves on a
fresh instance. Ciphertext is sealed under the suite's fixed key. A
measurement tombstone that ages past retention may only lower its count.
A full rollup backfill that failed on every boot showed only on the
admin page's card of failing background jobs. From the account's side
the only symptom was empty dashboard cards, and nothing pointed the
operator at the card (#1298).
A new hourly evaluation reads the newest terminal runs of every queue
whose failure has no account-facing surface (the system audience in
subsystem-surface.ts) from pg-boss's own ledger. A failed run adds its
attempts, retries included, and the first successful run ends the
streak. Three failed attempts in a row send every admin a SYSTEM_ALERT
once, a reminder at most once a day while the queue keeps failing, and
one notice when it runs successfully again: the decision the backup
health notices use, with the same compare-and-set claim, so two workers
never send a notice twice.
The memory is one row per announced queue in system_job_alert_notices
(migration 0490). The notice names the queue, the streak and the last
error, and opens the admin status page.
The full rollup backfill refills each day's figures from the raw
readings, and the dense fold deletes those readings once it has their
hourly means. On a large account the backfill can run longer than the
ten minutes the fold's boot job waits, and a year it reached after the
fold had taken it would compute its days from hourly means: the day
count, minimum and maximum of every folded day would change.
The consolidation walk now asks a per-account deferUser hook before it
reads an account, and both entries of the dense fold (the per-account
boot job and the nightly drain) pass isRollupFullBackfillPending, which
looks for a queued or running full fold of the account in pg-boss. The
account is folded on a later run, once its rollup tier is rebuilt.
There was no way for an admin to remove someone else's account; the
users list only offered the undo for a deletion the account itself or a
Guardian had asked for.
DELETE /api/admin/users/{id} takes the account's own deletion path,
now shared as requestAccountDeletion: with a grace window the account
is locked at once (credentials revoked, record untouched) and erased by
the purge job when the window ends, and the existing undo gives it back
until then; with ACCOUNT_DELETION_GRACE_DAYS=0 it is erased in the
request. The route is cookie-only behind requireAdmin and the
recent-proof gate, takes the typed DELETE_ACCOUNT confirmation, and
refuses the caller's own account, a managed profile (its Guardians
delete it), an account already pending deletion, the last admin who
can sign in and the last Guardian of a managed profile, the last three
decided under their locks inside the transaction. Both the admin and
the account get an audit row.
The users list says who asked for a pending deletion, now including an
admin, and flags managed profiles. Each other account's row carries a
delete button whose confirmation names the purge date, or says the
erasure is immediate when the server keeps no window.
The dense hourly fold only took Apple Health and Health Connect samples,
so heart rate from the provider syncs that store one row per sample
stayed raw forever. One self-hosted account held about 25 million
Google Health pulse rows after a full-history backfill, and every
reader and fold that falls back to the raw table paid for it (#1297).
The fold now walks a map of sources to the dense types each stores
sample by sample: every dense type for Apple Health and Health Connect,
heart rate for Google Health and Oura. Their daily SpO2 values are no
intraday stream and stay as they are; Fitbit does not sync intraday
heart rate, and WHOOP, Withings and Polar write dense types only once
per recovery, night or spot check. Each source folds into its own
hourly stats rows, and the DAY rollup is still taken from the raw
samples before they go.
The fold deletes the samples it averages, and the provider syncs write
first-write-wins on the externalId, so a backfill or full sync would
have put every folded sample back. Google Health would even have
re-keyed an hourly row into a sample that fell on its anchor through
the natural-key rescue. The folded_window rule now has one hour rule
per fold source, and both syncs drop the samples of an hour the fold
already took before they write. Recent samples cost no lookup.
The boot discovery of the fold probes each account and source for the
first raw sample it would take instead of grouping every such row, and
the one-time fold repair stays on the two sources it was written for.
The provider sent PUT /api/auth/me/locale on mount and on every switch,
signed in or not. Signed out the route answers 401, the once-per-tab
marker was never set, and every signed-out page load repeated the
refused write.
The mirror moves out of the provider into LocaleServerSyncBridge, which
watches the account payload without fetching it and writes only once an
account is known. The choice made on the sign-in page therefore reaches
the account right after signing in, and the marker is keyed on the
account as well as the language, so another account signing in on the
same tab is written for too. The browser test stub no longer treats the
PUT as ambient, which is what lets its test see a stray write.
An account created through OIDC holds no password, passkey or second
factor, so the recent-proof gate in front of token minting, MCP
connections, exports and the admin reset had nothing to ask it for and
the web could only say "sign in again" (#1296).
GET /api/auth/oidc/reproof sends the browser back to the configured
provider with prompt=login and max_age=0, its own state, nonce and PKCE
verifier, and the signed-in account and session sealed into the state
cookie. The shared callback branches on that intent: it refuses unless
the browser still carries the bound session (checked before the code is
redeemed), the ID token names the account's linked issuer and subject,
and auth_time lies inside the gate's five-minute window. It then stamps
Session.reproofAt exactly as a password re-proof does, creates no
session, and returns to the page it left with ?reproof=done|failed.
recentProofMethods offers "oidc" only to an account without a second
factor whose identity is linked while a provider is configured, and the
fresh-factor gate never lists it. GET /api/auth/me/mfa publishes the
list as proofMethods, so the passkey card offers what the account can
actually give, and register-options takes { method: "oidc" } on a fresh
provider re-proof, so such an account can add a passkey as a second way
to confirm. Both re-proof dialogs offer the provider and a toast reports
the outcome on return.
The callback's missing-state branch and the passkey card's enrollment
flow move into their own functions to keep both handlers under the size
ratchet.
Systolic, diastolic and the cuff's pulse are three rows with no link
column, and the single DELETE tombstoned only the row it was given, so
deleting a reading's systolic left its diastolic and pulse behind as half
a reading. The bulk delete had the same gap.
A reading is the rows with the same owner, the same source and the
identical measuredAt: that is how the entry form, the MCP write, a
Withings measure group and a HealthKit correlation all store one, and how
the FHIR read pairs a diastolic onto its systolic. The looser five-minute
pairing of the chart reads is right for drawing a point and wrong for
taking rows away.
Deleting a systolic or diastolic now tombstones the rest of its reading in
the same transaction with the same deletedAt, each row with its own
syncVersion bump so paired clients see every tombstone. The restore route
brings back the rows deleted together with a named half (identical
deletedAt), so the undo, which names only the row that was tapped, puts
the whole reading back, from the web list and from any client that calls
it the same way. A pulse is still deleted on its own, and a row deleted at
another moment stays deleted. The response shapes do not change; the
OpenAPI descriptions say what the counts now include.
On an account with tens of millions of readings the boot full backfill
never finished. It ran four aggregates over the whole five-year window,
each held to the connection's 60 s statement limit, so every attempt
was cancelled with 57014, the reading extremes added by 0417 stayed
NULL, and the readers that need them fell back to the raw table and
timed out as well (#1298).
The full backfill now folds the window one UTC calendar year per pass,
newest first, and every statement runs under its own five-minute limit,
the shape the pulse day-mean rebuild already uses. Calendar-year cuts
write exactly the buckets one pass writes: day, month and year buckets
fall inside one window and a week across New Year is folded whole by
both. Each window commits on its own, so a run that reaches its job
budget hands the remaining years to a follow-up job instead of failing
and starting over on the retry.
The boot discovery read every live reading in the window to find
uncovered days, the same scan that kept the fold from finishing. It
now walks the days per account and type off the live index, so its
cost follows the number of days with readings, and runs under the same
limit. The full backfill code moves to its own module.
Before the confirm step dated an undated value by its document, a lab
value confirmed from a report that prints its collection date only in the
header was stored with takenAt set to the moment of confirmation. The fix
covers new confirms; the values already filed that way stay on the wrong
day until something moves them.
A one-shot maintenance job, sent at boot only while there is something to
repair, moves exactly those values to the document's report date (else its
filing date) at noon UTC, the same rule the confirm step now uses. A value
qualifies only when the chain is certain: written by the document confirm
path, named by exactly one approved observation fact, takenAt still within
five seconds of its own createdAt, no calendar date of its own in the
decrypted fact, and a document day that differs from the day the value is
filed under in the owner's zone. A value re-dated since, one really taken
on its confirmation day, or one whose document has no date is left alone.
A moved value no longer sits at its creation moment, so the pass converges
and a second run changes nothing.
The job definition lives in a new registry file beside maintenance.ts so
that file stays under the size ratchet.
A range longer than a year reads its series from the week or month
rollups, which are kept for the last five years. When the rollups held
any rows the read returned them alone, so on a record that goes back
further the All range started where the rollups start: a weight history
from 2015 charted from late 2021. The readings older than the first
rollup day are folded live at the tier's grain and lead the series; the
head ends where the first tier bucket begins, so no month is counted
twice. One indexed lookup when there are none.
Against a real Postgres and the real confirm route: an undated value takes
the report date over the filing date, a value's own date is kept, an
impossible own date counts as none, and a document without a report date
dates the value by its filing date.
The forwarder ran every tool as docker exec -i with its own standard
input piped through, also for pg_dump, which reads none. With -i and an
input already at its end the docker client may close the stream before
all output has arrived; one CI run saw the second of three nightly dumps
fail where every local run and the two runs before passed. Attach stdin
only for a file fed to pg_restore, and put the run's error into the
assertion so a failure says why.
realisedFamily probed for the closest two slots up to range.to only, so
the answer moved with the range. A Saturday + Sunday plan started the
weekend before the autumn DST change, read around its Saturday dose: the
±24 h range ends at 06:00 UTC on the Sunday, whose 08:00 slot is at 07:00
UTC after the change. Cut off, the closest slots were six days apart
(weekly); the wide re-mint saw the 25 h gap (daily). The two passes
disagreed, no band survived, and the dose was filed as unscheduled, so a
second backdated dose within a minute came back as the first one (200).
Probe the same reach on both sides of the range's middle.
Marc-André Bombeck