A /api/fhir/* handler answers its own refusals as FHIR: a bad search
parameter is a 400 OperationOutcome, the hourly fhir bucket a 429
OperationOutcome (issue code throttled), both served as
application/fhir+json, the same way the 404 on a read already was. The
contract declared no 400 and the generic JSON 429 for the 429, and no
403 for the doctorReport module gate, which every data route runs.
The searches now declare the 400, every data route the 403
module.disabled envelope and the FHIR 429 with the rate-limit headers
apiHandler attaches to it. The module gate keeps answering the JSON
envelope: it is the errorCode every client already branches on for a
disabled module, and it runs before the FHIR face does. The route tests
drop their skips and assert the 429's content type and Retry-After.
requireAuth answers 403 auth.scope.insufficient to a narrow Bearer token
on a route that names no scope it holds, and requireAdmin answers 403
auth.admin.required to a session that is not an administrator's. Both
run before any route body, and the contract only said so in the prose
of the shared 401, so a client generated from it had no typed branch
for either. The route kit accepted the scope refusal on any operation
that declared a 401 for the same reason.
The 403 is added to the assembled path table rather than to
stdResponses. That set is spread into most operations, about three
hundred of them carry a 403 of their own, and most spread the set after
it, so a shared 403 there would have overwritten every module-off and
sharing refusal in the document. withScopeRefusal adds the shared
response where an operation has none and prefixes an operation's own
403 with the gate's code, keeping the sharing sentence last. Operations
that take no credential, and the three surfaces that authenticate by a
credential of their own, are left as they were.
The kit no longer tolerates the refusal on an operation's behalf, and a
guard holds every operation behind the gate to declaring it.
The environment check pinned the database a worker talks to as the single
shared one. Each worker now runs against its own clone, so the check names
the clone and asserts it is not the migrated template the clones copy.
Vitest's --shard cut the file list into slices of equal length, which gave
the integration shards 954 s to 1 449 s. A sequencer now deals files out by
their measured duration, plus a fixed per-file cost for import and setup,
longest first to the shard with the least work, and starts the longest files
first inside a shard. Timings live in tests/durations/*.json and are
regenerated from Vitest JSON reports with scripts/test-durations.mjs; every
CI shard uploads its report.
Integration shards run three workers each, one cloned database per worker.
The unit suite runs as three projects decided per file from its source:
guards and unit share a module graph inside a worker, isolated keeps a fresh
one for every file that mocks, spies, stubs or resets. In CI it is three
duration-balanced shards next to a job for lint, format, types and the
OpenAPI drift check.
The required check names stay as they were. "Lint, Typecheck & Test" is now
the job that answers for the static checks and every unit shard, the way
"integration" already answers for its shards. The browser-install guard
recognises a direct, flags-only vitest run as a run of the unit suite.
tests/integration/query-counter.ts counts the statements the app sends
while a function runs, by wrapping Client.prototype.query on the copy of pg
the Prisma adapter itself resolves (so a second pg in node_modules cannot
leave it reading zero; a window with no statement throws). Background tasks
are settled before the window closes, so fire-and-forget writes count with
the request that started them.
query-budget-hot-routes.test.ts seeds a small account (four months of the
main types with overlapping sources, stats rows, sleep, glucose, a
medication, workouts and mood), warms each route once, evicts the user's
caches and counts the next call. Budgets, measured on this fixture:
/api/auth/me 18, dashboard summary 86, dashboard snapshot 103, pulse series
30/365/3650 days 6 each, day view 48, insights health status 60,
correlations 44, a 500-row batch ingest 31. A failure prints every
statement grouped by text.
Proof: one extra `SELECT 1` before every handler in apiHandler failed all
ten budgets by exactly one (18 → 19, 86 → 87, … 31 → 32), which also shows
the counts are deterministic; with the probe removed all ten pass.
scripts/bench/seed-large-account.ts (moved from scripts/, extended) writes
one synthetic account of ten years: 1,265,561 measurement rows across the
main types and five sources (Apple Health dense spot rows, Withings blood
pressure and scale, a weekly manual weight, three years of Oura nightly
values, 90 days of Nightscout CGM), `stats:` and per-sample cumulative rows,
nightly sleep stages, 3,020 workouts, 1,095 mood entries and three
medications with 3,285 intake events. Everything goes in through set-based
INSERT … SELECT generate_series; values and timestamps are functions of the
row index and `--anchor`, so a run is reproducible. The rollup tier is
folded with recomputeUserRollups, as the boot backfill does, and the tables
are analysed. About 80 s on the Mac mini.
scripts/bench/run.ts boots the production build once with every role (so
the queues exist), then measures against a web-only process under the
production heap ceiling: p50, p95 and max latency and SQL statements per
request (pg_stat_statements, read once nothing is in flight and the count
holds still) for /api/auth/me, the dashboard summary and snapshot, the pulse
series over 30, 365 and 3650 days, the day view, insights health status and
correlations, each cold (caches evicted by a one-row manual-sync ingest
before every request) and warm, plus a 5,000-row batch ingest as ten
requests of 500. It writes JSON and a markdown table.
docs/contributing/benchmarks.md describes the setup against a throwaway
Postgres in Docker with the production pg_stat_statements settings.
tests/quarantine/quarantine.json is the only way to take a flaky unit or
integration test out of a run. An entry names the file, the full test name,
the issue, the reason, the day it was added and the day it expires, at most
14 days later. Both setup files skip a listed test while its entry is live,
reported as skipped with its issue rather than retried into green. Past its
expiry date the test runs again, and quarantine-guard.test.ts fails until the
entry is removed; it also fails on an entry naming a file or test that does
not exist. The list starts empty.
fast-check properties with one shared configuration (tests/property/): a
fixed seed on every pull request, a fresh one and more cases at night through
FC_SEED and FC_NUM_RUNS, and a printed seed with the shrunk counterexample on
failure.
- Time zones: wall clocks agree with Intl in twenty awkward zones, a wall
clock maps back to an instant showing it, a skipped hour resolves forward
on the same day, every instant lies inside the window of the day it is
filed under, and day keys shift and count as inverses.
- Statistics: every median in the tree says the same number, medians and
spreads obey order, shift and scale, a least-squares fit recovers an exact
line and composes from stored sums exactly, and the segmented regression
recovers a noiseless interrupted line.
- Cadence and compliance: weekly and daily schedules expect exactly what a
day-by-day oracle expects, in any zone and on transition days, and the
rate stays within bounds and never falls when a missed dose is taken.
- Encryption: every codec opens what it sealed under the key it names,
rotation moves the key id and nothing else, and a tampered value never
opens to anything but the sealed bytes.
- Backups: randomly generated accounts come back from a disaster-recovery
restore with an identical canonical dump of every generated table.
Phase config, cadence, side effects, inventory, intake purge, bulk delete and
the import status run through the real handlers against the database. The
advisory-lock ordering the mocked tests read off call order is shown with a
real concurrent transaction holding the inventory lock, cache eviction through
the next list read with a control that the cached list would otherwise stay
stale, and soft deletes on the tombstone and sync counter they leave.
The channel settings routes, their test-message routes, the reminder
thresholds and the workout detail, delete and batch-scope tests run through
the real handlers against the database. Only the far ends are faked (the
relay, the ntfy server, the Telegram Bot API, SMTP), and the tests assert what
those received. The private-origin source guard stays a unit test and now
points at the two files that carry its runtime proof.
The effective tag read, hide and show, custom tags and groups, and the tag
layout are tested through the real handlers on stored rows instead of a
mocked Prisma client: the migration-seeded catalogue served after a truncate,
owner scoping that leaves another account's tag byte-identical, the caps,
sealed labels, soft delete against purge, a deleted group re-homing its tags,
and the layout's optimistic token from 200 through 409 to a successful retry.
The Strava writer had eight mocked test files and no test against a
database. strava-sync.test.ts runs the real sync behind a fake Strava API at
the safeFetch boundary: field mapping, the after= watermark, idempotent
re-sync on the (user, source, external id) key, in-place overwrites that keep
what only the detail read carries, a cursor that never moves backwards or
past a row that failed to write, the paged backfill with its resume point
and requeue, token refresh with both rotated tokens stored sealed, and the
integration ledger after transient and permanent failures. The sync, the
disconnect and the credential routes are covered through the kit.
The mocked cases that asserted call counts and mock ordering are removed;
the pure retry-after parsing stays a unit test.
Fourteen route tests for the /api/auth/me preference family replaced Prisma
with a mock and asserted the calls it received. Their claims are now shown
through the real handler on the stored row and the response: validation
refusals with their codes, normalisation and defaults on read, optimistic
lock conflicts against a real second writer, rate limits driven to their
first refusal in the Postgres limiter, audit rows read back, cache entries
evicted, and the jobs a change queues observed on a real pg-boss queue.
Assertions that were only about the shape of a Prisma call are replaced by
the behaviour they stood for, and two that described a state only a mock
can reach (a missing column) by the stored value that state resolves to.
DELETE /api/auth/me/sessions/{id} takes the session's public handle, an HMAC
of the row id, and compared that handle with the caller's row id to decide
whether the caller signed out the session it was using. The two never match,
so every revoke was audited and annotated with self: false, including the
ones where a person ended their own session. The handle is compared with the
handle of the caller's session now.
The mocked route tests for the session list and revoke, the device list and
revoke, the security-activity feed and the FHIR surface move onto the kit.
Each claim is now shown on the stored rows and on the next request: a
revoked session, device login or token is refused when it is presented again,
another account's session or device answers 404 and stays signed in, the
current-session and current-device flags follow the credential that anchors
them, and no row id, full IP or push token reaches a response.
The FHIR read set, covered against a database only by the scope check until
now, is driven with seeded readings and medications: every resource carries
the seeded values, another account's rows never appear, a fhir:read token
reaches every data route and an unrelated narrow token is refused, the module
gate closes all eleven data routes, and $everything serves the same entries
as the document builder with references that resolve inside the response.
The module-gate inventory now names the new file as its behavioural proof.
One table holds an entry per write route, and every User column the backup
classifies as a setting appears in exactly one entry; a new setting column
fails the inventory case until it has one. For each entry the file sets a
non-default value through its write route, reads it back through its read
route and through /api/auth/me where that publishes it, and checks the stored
column moved off what a fresh account holds.
The backup leg runs through the routes too: the portable full backup a
person downloads, the operator's upload, and the restore into an account
recreated under the same id. Every setting must read back identically on the
restored account, through the same routes and the account payload, and every
stored column must come back with its value.
tests/kit/ holds typed fixture factories and callRoute(handler, options).
The factories write through the application's Prisma client, so a fixture has
the shape production stores and the compiler checks it; sessions and Bearer
tokens are minted by createSession and issueApiToken, so a fixture cannot
drift into a shape authentication no longer accepts.
callRoute builds a real NextRequest, fills the next/headers stand-in from the
auth fixture and runs the exported handler, apiHandler included. Every call is
also a contract check: the status must be declared for that path and method in
the OpenAPI registry, a JSON body must parse with the registered schema, and a
key the schema does not declare fails the call. The scope gate's generic 403,
documented on the shared 401, is the one answer accepted without its own
entry. kit-contract.test.ts proves each refusal can fire.
The token list and revoke move onto the kit. The mocked unit test asserted
the list's where clause and proved the API switch is not consulted by reading
api-handler.ts for a string; the kit test authenticates a Bearer token while
the switch is off, revokes it, and shows it refused on the next request.
Global setup migrates one template database and reads what a fresh install
holds: its schemas, its tables and the rows the migrations seed. Each Vitest
worker clones the template on its first file with CREATE DATABASE ... TEMPLATE
and keeps the clone for the rest of the run, so files run in parallel without
seeing each other's rows, advisory locks or queue schema. Schemas created at
runtime (pg-boss, migration probes) are dropped when the next file starts.
The truncate list is the catalogue now, every table in public except the
migration bookkeeping, instead of a hand-kept list of 42 of 166 tables. The
seeded catalogues are put back after each truncate in foreign-key order, so a
test starts from what a fresh install has. The settings singleton is the one
seeded table a test starts without, declared with its reason.
next/headers and the boot compatibility probe are mocked once in the setup
file. Two lock-wait probes now look at their own database only, since other
workers' backends share pg_stat_activity.
The free-text backfill no longer has an insights cache step since the
readable cached-text column was dropped, so its summary carries no
insightsCachesMigrated count. The expected summaries follow.
GPS tracks are kept only after the account turns storeWorkoutRoutes on.
The two route-storing cases opt in first, and a new case proves that an
account which has not opted in keeps the workout, answers it as
inserted, and stores no track.
The two provenance reads take required query parameters and had no
entry, so the run could not call them. Give them a day and a range on
the steps, seed a second steps source for the last week so the pick and
the comparison have two sources to explain, and save a labs list order
so the module list layout read answers with a row.
The restore admits activity goals key by key against the goals schema,
so the generic { roundTrip } JSON fill was refused as an unknown goal
rather than carried. Fill the column with real goals and pin the
per-goal admission (valid goals kept, unknown keys and out-of-bounds
goals named) in the unit suite.
The medication CSV import created the medication and then set its
category in a second write. A crash between the two left a medication
without a category, and a retry skipped it as a duplicate, so it never
got one. Both writes now share one transaction.
Every accepted row was its own insert, so a 20 000-row file meant
20 000 round trips and an import cut off part way left an arbitrary
prefix behind. Each 200-row chunk is now one createMany: it lands whole
or not at all, and uploading the same file again skips the written
chunks as duplicates. The importers report each committed write, and
the audit row is written in a finally block, marked partial with the
rows written when the import did not finish.
The rows were parsed in the query's timezone while the duplicate keys
used the account's, so two draws on separate days of the file's zone
could collapse into one, or one day split in two.
The health-status read scanned five vitals over 90 days for a dated,
sustained level shift. It now scans every vital with a stable daily level
(adding RMSSD, diastolic pressure, pulse, body fat, respiratory rate,
oxygen saturation, walking heart rate and VO2 max) over 180 days, reads one
value per day per metric, leaves a metric unread when its module is off,
and reports how many days stand on each side of the break. The Coach keeps
its five-metric, 90-day scan unchanged.
Two streamed endpoints read a spreadsheet record by record with a
column mapping from the query: separator, decimal mark, date order and
zone are stated, never guessed per row. Lab rows resolve to markers the
way scanned reports do, convert units where the conversion is known and
keep what the file said; medication rows map name, dose, daily times,
as-needed and a dated window, checked against the create schema, with
reminders off unless asked for. A dry run writes nothing, a re-upload
only finds duplicates, and every refused row comes back with its line
and reason.
A read-only view per metric and UTC day: every source with readings, its
day value, its place in the person's source priority, and which source
the charts use with the reason. A second card lines two sources up over
30, 90 or 365 days. The pick is explained from the rank the canonical
SQL returns, so the page cannot disagree with the charts. Linked from
the metric pages and from Settings > Source priority.
GET /api/fhir/Observation keeps its snapshot answer when no date is
sent. A date range (FHIR prefixes eq/gt/ge/lt/le, UTC when no offset)
switches it to the time series: every reading in the range plus
workouts, newest first, paged in the database through one UNION id
query and a count, so a page costs what it holds. Series ids are the
row's own (m-<id>, w-<id>) and read back through /Observation/{id}.
The series replays the same saved report selection as the snapshot,
pairs systolic and diastolic recorded at one instant into the
blood-pressure panel, leaves sleep stage segments out, and reads an
hourly heart-rate bucket as a mean over its hour. A code parameter
narrows either answer; a malformed date is a 400 OperationOutcome.
A workout is an activity Observation after the Physical Activity IG:
LOINC 73985-4 with the sport as its value and duration (55411-3) and
energy (41981-2) as components. RMSSD gets its own registry coding,
text-only because LOINC has no RMSSD term, so it never borrows the SDNN
code. The capability statement lists the two new parameters.
Both routes resolved through bare requireAuth, which refuses a session
acting for another record, so the list view controls and the preview
switch failed in exactly the context they are used in. They are the
person's own presentation and now resolve through requireActorAuth,
admitted on the actor-surface list with their reasons.
The layout PUT and the preview PATCH read and write the stored blob
under a row lock in one transaction, so two saves at once no longer
start from the same value and drop each other's change.
Posts one batch for two accounts with the same stored rows, once as the
route runs and once with the set-based insert forced onto its fallback,
and asserts identical per-entry outcomes and stored rows across fresh
samples, a stored duplicate, a tombstone, a stats: overwrite, an
external id twice in the batch and a natural-key collision.
0440 adds its columns with ADD COLUMN IF NOT EXISTS like 0424 and
0432; a structural test holds every migration from 0422 on to that
form. 0430 clears insights_cached_at where no sealed payload is left,
so the nightly pregenerate refills an account whose only copy was the
readable column instead of keeping an empty card fresh.
The deploy runbook notes that 0430 drops a users column: an old process
fails on its next full user read, so stop before starting rather than
running old and new side by side.
The discovery matrix gains the families people log but could not test:
water, caffeine and up to four varying micronutrients from the nutrient
log, alcohol days from the mood tag, workout minutes (one per session) and
the day's training impulse, exercise, stand and mindful minutes, State of
Mind as an outcome, and the overnight room climate against that night's
sleep only. Every channel runs through the same seasonal adjustment,
effective-n p-value, Benjamini-Hochberg control and effect-size floor, is
read only while its module is on, and drops out below the paired-day floor.
State of Mind, mindful minutes and alcohol days are statistics for the
person: the Coach, the period narrative and the assessment cards drop every
pair that touches them, after the shared scan so the remaining pairs keep
the statistics the correlations page shows. A guard walks the import
closure of the analysis modules and refuses any path to a provider.
A week or month of steps, active minutes, workouts by sport and training
load, each day placed against the person's own spread over the 84 days
before the period and each period against the median of the twelve weeks
or six months before it. Optional weekly and monthly goals live on the
account (activity_goals_json), travel with the backup and are cleared by
a data wipe. The calendar sits on the workouts page; the steps page links
to it. Computed server-side without AI.
A guardian finishing a connect for a managed profile could attach their
own provider account to the profile. Every OAuth callback (Withings,
WHOOP, Fitbit, Google Health, Strava, Polar, Oura) now refuses a
provider account another record already holds with reason
already_linked, shown as its own message on the integrations page.
WHOOP's owner_conflict tag becomes already_linked as well.
Oura keeps the account id of a grant (personal_info, migration 0441) so
the check has something to compare; existing connections fill it at
their next connect.
A disconnect leaves Withings webhook subscriptions and the Strava app
authorisation in place while another record holds the same account,
and the Withings webhook routes an account held twice to the record
that linked it first instead of whichever row a lookup returned.
The provider app copied from a guardian onto a managed profile is now
marked (migration 0442) and removed again on disconnect or when that
guardian no longer holds a live MANAGE grant: revoke, renounce,
removal, handover, account deletion and data wipe. Each removal is
audited as integration.client_app.removed.
Migration 0434 turns the route opt-in on for every account that already
holds a stored workout track, so its tracks keep arriving after the
upgrade; only accounts without one start off.
Workouts stored before this release with a track but no elevation get
their metres climbed written into Workout.elevationM by a one-time
background pass. It runs detached at worker boot, reads one track at a
time with a pause between batches of 100, and keeps its position in
app_settings, so a restart resumes and a finished pass ends at once.
The "starting" health reason is bounded: after five minutes the probe
says plain degraded and the process logs one line, and the time the
background role took to come up is logged either way. The compose
healthcheck's ten-minute start period already keeps those probes from
counting; its comment now says so.
POST /api/measurements/batch ran one interactive reconciler transaction
per fresh sample: a savepoint, three advisory locks, two candidate reads
and the insert, about ten round trips. A 500-entry upload of new samples
was 500 of those, the 2 to 15 second outliers. Fresh samples now land
through INSERT ... ON CONFLICT DO NOTHING under the same per-account
advisory lock. Both measurement identities are full unique indexes, so
the statement inserts exactly what the reconciler would have inserted,
and every skipped row still goes through the reconciler for its own
verdict. Only rows whose external and natural keys each occur once in
the batch are buffered, and the buffer is flushed before any other row
is reconciled, so the order of effects stays the sequential one. The
contract and every per-entry status are unchanged; a golden test runs
each batch through both paths against the same table.
/api/health took seconds after a start because register() awaited the
whole pg-boss worker start (queues, schedules, boot discovery) before
the server took a request. The worker, or a web process's producer, now
starts in the background after the blocking boot checks; the probe
answers at once with reason "starting" until it is up, and gives the
database two seconds before calling it not answering.
Every Oura collection, workouts and heart rate included, is now read
back to the start of the account's history by a backfill that runs
through the shared integration lane in five-minute slices. Each
collection walks from the day the backfill began toward the past one
window at a time (ninety days, thirty for sleep, seven for heart rate)
and the progress note on the user row records after every window where
it got to, so a slice cut short by its budget, the lane or a rate limit
resumes there. A 429 ends the slice and the next one waits for the
Retry-After Oura named. Activity and sleep are walked first to learn
where history starts; the rest stop a week before it.
Heart rate streams a page at a time and keeps only the days it touched,
so memory stays flat however long the history is. A collection the
grant does not cover is a scope note, not a failure; the first slice of
the next chain asks again, and a connect clears the completion stamp so
a newly granted scope is read without walking anything twice.
The two columns arrive empty, so every existing connection is offered
the backfill at the next boot without reconnecting. Before anything
else the backfill retires the WALKING_RUNNING_DISTANCE rows earlier
syncs wrote from the equivalent walking distance and refolds their days;
an account that kept such rows after disconnecting is offered too.
The binding UI rulebook (card anatomy, spacing scale, text colour,
tokens, primitive mapping, enforcement) lived outside the repository,
so a contributor could follow a code comment to a file they could not
open. It now lives at docs/contributing/ui-standards.md, every comment
that cited it points there, and the docs index and ui-guidelines.md
say which document wins when they disagree.
Every write has gone to insights_cached_text_encrypted since 0411, and
the boot-time free-text backfill sealed the rows written before it.
Migration 0430 drops insights_cached_text and the backfill index that
served it. A host that skipped the intervening releases loses only a
cache of model output; the next warm writes a fresh, sealed payload.
The cipher module now reads and writes the sealed column only, the
backfill job loses its insights step, and consent withdrawal, the wipe
plan, the backup plan and the session projection follow the schema.
Backups never carried the column, and restore ignores unknown account
settings; a round-trip test now proves an old backup naming the field
still restores cleanly. The access guard pins exact counts for where
the sealed name may appear. Fixtures, the demo seed and the demo
migration script seed the sealed column instead of plaintext.
Marc-André Bombeck