A `DataBackup` row that fails to decrypt answered 500 on the restore, the
download and the preview. It is not a server fault: either the key that wrote
the copy is no longer in `ENCRYPTION_KEYS` — a rotation that dropped the legacy
entry before the script reported zero rows, the case the rotation runbook warns
about — or the stored bytes are not the ones that were written. Every other
bad-input arm on those routes already answers 4xx, and the 500 also paged the
error reporter as if the process had broken.
All three now answer 422 with `meta.errorCode = backup.payload.undecryptable`
and one shared sentence, so the three routes cannot drift into three wordings
for one condition. The download's second 500 stays: a payload this instance
wrote and cannot parse really is a fault here.
The two routes that open a stored copy move from the unpublished list into the
contract, because the refusal is now a promise an operator can look up rather
than whatever the handler happened to answer. The restore publishes its
`Idempotency-Key` header with them.
A disaster-recovery snapshot carries the singleton `app_settings` row —
registration, the MFA requirement, the default locale and timezone, module
availability, the notification and AI configuration, the document size cap and
quota — and the restore wrote it back as a side effect of restoring one
account. An operator putting a single record back on Wednesday reconfigured the
whole host from Monday's file, silently, with nothing on screen saying so.
Rebuilding a host from a snapshot is a real case, so the write stays: it is now
`restoreInstanceSettings: true` on the request and a checkbox beside the typed
gate, off unless the operator ticks it. Restoring an account and reconfiguring
the installation are two decisions, so they are two answers.
The 422 confirmation sentence and the dialog copy drop the instance-wide claim;
the opt-in carries it instead, in all seven catalogues. Integration coverage
runs both arms: without the flag the settings row keeps its own values, with it
the file's values land.
One journey across the whole path: an account writes readings in three
domains and files a PDF, downloads its own record as the passphrase-sealed
archive, an admin snapshots it from the backups console, and the snapshot is
restored over an account that has since lost a reading and a dose and gained
one the snapshot never saw.
The assertions address what the documentation promises rather than what the
implementation happens to do. The archive is checked byte by byte against the
HLX1 header in src/lib/export/passphrase-archive.ts — magic, version, KDF id
and the Argon2 cost the file carries — because "a file arrived" is equally
true of plaintext JSON under a new extension. The restored reading and dose
are checked by their ORIGINAL ids, because a restore that re-created them
under fresh ones would have written a different record that reads the same.
The reading created after the snapshot is asserted GONE: the runbook says the
restore replaces the account's data tables, and replacing is not merging.
Two refusing controls. A stored copy with one flipped ciphertext byte is
refused with the route's decrypt failure and leaves the record untouched — the
envelope is AES-256-GCM and its tag covers every byte, so the flip is caught
before the transaction opens. A delegate holding whole-record READ inside the
account gets the not-covered panel instead of the export card, never reaches
the console, and is refused 403 by the list, the manual run and the restore,
the last of them naming the real snapshot so the refusal cannot be mistaken
for a missing row.
Waiting is on the job's own evidence: the console's endpoint is polled until
this account's stored copy is newer than the click, so nothing in the file
guesses at a queue. The journey has its own account, its own delegate and its
own cookie jars; only the reset between repetitions and the deliberate
corruption reach Postgres directly, because a browser cannot express either.
The backups console gains data-testid hooks and a data-backup-id on each row
so the spec addresses controls rather than translated labels.
The catch-all angle-bracket pattern read as an HTML sanitiser to static analysis. The annotation Apple writes is a number, so the pattern now matches exactly that, and a unit with anything else between the brackets stays unknown instead of being silently trimmed into a lookup.
E2. `loadOffhostConfig` parsed BACKUP_RETENTION_DAYS, clamped it to a
minimum of one day and put it on the config object, where no caller read
it. Neither annotation fits: it is not deliberately internal, because
there is no internal use, and no consumer is pending. The design forbids
one outright — the worker's grant excludes DeleteObject on purpose, so
nothing here can act on a retention window without giving a compromised
worker the ability to wipe the history. A clamp that guards nothing only
read as an enforcer the worker is not.
So the field and the clamp go, and the reason sits where the parse was.
The variable keeps its place in the two example files and on the compose
whitelist: it is the number the operator sets the bucket's own lifecycle
rule to. The docs table said "defaults to 30", which implied the app had
an opinion; it now says the app does not read it.
E1. Two halves of one correction. `.env.example` still said backup objects
older than BACKUP_RETENTION_DAYS are pruned, which no code does, so a
self-hoster reading that file alone still expected the worker to keep the
bucket tidy. It now carries the same sentence as the production example.
And that sentence overreached in the other direction: "the worker never
deletes an object" is not true of `_healthcheck/<ts>.bin`, the 1-byte key
the Test-connection button writes and then deletes. The delete swallows
its own failure, so an operator who granted exactly the three documented
actions leaves one orphan per press and is never told. Both files now say
"a BACKUP object", and name the one key and what happens without
DeleteObject on that prefix.
Low, review 2. An account with one out-of-range candidate is refused
whole, rows and audit row included, but the dry run counted its rows into
"N row(s) would be repaired" — so the number an operator read before
--apply was larger than the number --apply could ever produce. The refused
accounts are now counted and reported separately, and the per-account line
says its rows are not in the total.
The refusal itself told the operator to check the Health app "before
forcing anything". There is no --force and never was. It now names the
only route that exists: re-run with the matching --unit if the archive
really was in another one.
U1. The new `unit` description on the batch entry said `value` is read in
the unit HealthLog stores the identifier in. Sixteen mappings disagree:
`convertToDbUnit` runs on every entry, and a waist circumference arrives
in metres to be stored in centimetres, an oxygen saturation and a body fat
arrive as a fraction to be stored as a percent. A client generated against
that sentence sends 98 and stores 9800.
The description now says what happens: `value` is read in Apple's own
HKUnit for the identifier, HealthLog applies its storage scaling
afterwards, and `unit` is recorded for audit rather than validated
against. Two worked examples, because the percent-shaped ones are the pair
that bites. YAML regenerated.
Low, review 2. The self-test walked every leg but never once asked whether
the redactor actually holds. The excerpt path is the only thing between a
token in a failure body and the run log, so it gets its own case: a body
carrying this project's own key prefix must come back with the token
replaced.
Low, review 2. The delete leg asserted HTTP 200 and printed "account left
clean". An instance that answers 200 and keeps the row would have printed
the same line, which is the failure mode the journey exists to catch.
The leg now reads the id back and requires a 404. The self-test carries a
mock that answers 200 without deleting; with the read-back removed that
case goes green, which is the proof the check can fail.
S2. The legs ran in one loop that returned on the first red one, so a
failed read back never reached the delete and the probe account kept the
row. Worse, the id was assigned only after the write's await returned: a
POST that timed out once the server had already committed left a row the
script could not even name.
Legs 3 to 5 now sit in a try/finally. The finally sweeps whatever the
delete leg did not, and when there is no id it asks the instance for the
row by a marker the write puts in `notes`, unique per run and known before
the request goes out. The sweep never throws — the red leg is the verdict,
and it says out loud what it could not remove.
--self-test asserts the mock holds no rows after every case that got as
far as the write, so a red read that skipped the delete is caught.
S1. The journey signs in with a real account and POSTs that account's
password to whatever BASE_URL names, and the workflow took BASE_URL as
free text. Anyone with repo write, or a leaked token, could dispatch the
run at a collector of their own and keep a working credential; secret
masking guards the log, never the egress.
The host now has to be on a short const allowlist in the script, checked
before anything is sent, with exit 2 and a named refusal otherwise. The
workflow input is a choice between the two hosted instances, so the form
cannot offer a third; the script's own check still stands behind it for a
dispatch that goes round the form.
--self-test covers the refusal, the prefix lookalike a startsWith check
would have waved through included.
The client's notification service extension replaces the medication name
on the lock screen, and iOS routes a payload through an extension only
when the alert carries `mutable-content`. The dispatcher has set the flag
on every alert it builds since the category work for the v0.5.4 client,
but the only assertion on it sat inside a category test, where dropping
it would have failed nothing and the name would have quietly reappeared
on the lock screen of a device that asked for it to be hidden.
Two cases now say it out loud: the medication reminder carries the flag,
and so does another event type, because that is the shipped behaviour and
narrowing it would take a capability away from the client rather than
tidy something up.
The native client replays its offline outbox with the same
Idempotency-Key on every attempt, and `POST /api/medications`, the
side-effect log and the inventory register ran the handler again on the
replay: after a lost success response the account was left with a
duplicate medication, a duplicate symptom entry or a second pen in the
supply count. The dedupe on `(externalSource, externalId)` only ever
covered a MIRRORED create; a manually entered medication carries
neither field and had nothing to collapse a retry onto.
All three wrap in `withIdempotency`, the composition the batch ingest
already ships, and the three operations publish the header so a client
generated against the contract can tell that a retry is safe. Zod
parsing and `requireRecordAuth` are untouched.
An integration test drives each route twice against real Postgres and
asserts what a status check cannot: one row in the table, and a replayed
201 carrying `X-Idempotent-Replay: true`.
Post-publish verify proves the published image boots. Nothing proved that
on a running instance a person can sign in, write a reading and read it
back — which is exactly the shape of the defects this line keeps
rediscovering: a value written and never read.
`scripts/synthetic-journey.mjs` walks that journey in plain Node 22 with no
dependencies, so it runs from a workstation, a runner or the deploy host.
It warms `/`, `/api/health`, `/api/version` and the measurements list first
and discards those timings (the first hit on each route after a deploy is a
cold start), then asserts five legs in order: the served version equals the
expected one, the password flow signs in through a cookie jar, a weight
reading is written with an Idempotency-Key, the same row reads back with
its value, unit and source, and the reversible delete leaves the account
clean. One line per leg with the elapsed time; the first failure ends the
run non-zero with the status and a body excerpt that has every token shape
redacted out of it.
The workflow is `workflow_dispatch` only, with `base_url` and
`expected_version` as inputs and the credentials read from repository
secrets. It is deliberately not wired into the deploy path: the target
instance and the secrets are the maintainer's call.
Mutation proof, against a production build served from
`.next/standalone/server.js` on a throwaway Postgres.
Green:
warm 35 ms (discarded)
ok version 2 ms serves 1.38.13
ok sign in 78 ms session cookie held
ok write 64 ms wrote cmtup9lwu0007zhovn0mxajdz
ok read back 8 ms 77.7 kg (MANUAL)
ok delete 27 ms account left clean
synthetic journey green
Read leg broken deliberately — the same server behind a throwaway forwarder
that empties the `GET /api/measurements` list and changes nothing else:
ok version 4 ms serves 1.38.13
ok sign in 38 ms session cookie held
ok write 40 ms wrote cmtupa4ck000gzhovwnij64fz
FAILED read back 8 ms read back: the list came back without the reading
status 200
body {"data":{"measurements":[],"meta":{"total":0}},"error":null}
synthetic journey RED on "read back"
Version leg broken the same way, by asking the instance for the previous
tag:
FAILED version 3 ms version: instance serves "1.38.13", expected "v1.38.12"
synthetic journey RED on "version"
`--self-test` carries that proof in the repository: it runs the journey
against an in-process instance nine times, once with nothing broken and
once per broken leg, and fails if a broken leg still comes back green. The
workflow runs it before it touches the real instance.
Gates: typecheck, `node --check`, lint (the three known warnings, no
errors), prettier, the workflow and CI-determinism guards.
The glucose display unit had thirty-odd readers and, until recently, no
control to set it. Nothing in the browser suite walks the path a person
actually takes: capture a reading in mg/dL, switch the account to mmol/L,
capture a second one, and read both back — each converted, none of them
reinterpreted — in the list and on the dashboard strip.
The journey covers that end to end, plus the boundary: a value past the
plausible ceiling is refused by the server, the capture keeps what was typed,
and the list is exactly what it was.
One reading arrives with no meal-time context, the way a meter export or a
device sync delivers one. It reads back correctly in the list; the tile it
should paint does not exist yet (#943), so the tile assertion is skipped by
issue number rather than softened into something that passes.
Three supporting changes:
- The list rows carry what they hold — type and resolved display unit on the
row, a slot on the value cell — so the journey pins "this reading now reads
in mmol/L" structurally instead of matching translated copy that also paints
through a different subtree on the card list. The meal-time options carry
their enum for the same reason.
- The sharing-marker weight rows were seeded by handing a Date to a
`timestamp without time zone` column, which stamps them in the host's zone.
On a machine east of UTC that put them hours in the future, where every fold
whose window ends "now" skips them, and the type stayed uncovered — which
keeps the snapshot's thick phase cold and the whole tile strip short. Seed
the instant as UTC, the shape the app's own writes use.
- The file mutates one shared account and reads global verdicts off it, so it
runs in a single project alongside the visits and immunization journeys.
Upload a synthetic export.zip on the import card, wait out the queued
job through the card's own status poll, and read the imported readings
back off the measurements list with the units the parser resolved.
Weight, the blood-pressure pair and the step count are asserted; the
walking-distance case is written and skipped, naming issue #944 — a
record written in km lands as metres today, and the assertion is kept
at full strength so the fix has something to turn green.
The refusing control uploads a valid ZIP with no export.xml: the card
surfaces the extractor's own refusal and the account's Apple Health row
count does not move.
Measurement rows now carry data-measurement-type / -value / -unit
beside the rendered cell, on both the desktop table and the mobile
list, so an assertion binds to the reading rather than to whatever the
locale formatter made of it. The import card's error line gets a test
id for the same reason.
The runbook said the worker prunes objects older than BACKUP_RETENTION_DAYS. It never calls DeleteObject; its grant is PutObject, GetObject and AbortMultipartUpload, on purpose, so a compromised worker cannot wipe the history. The sentence now says what expires objects and what the variable is for.
BACKUP_S3_REGION and BACKUP_RETENTION_DAYS are both on the compose whitelist and
both read by the backup job, but the example file listed neither — so an
operator following it had no way to learn that the region defaults to auto, or
that the retention number is what the bucket's own lifecycle rule has to be set
to, since the worker never deletes an object itself.
The settings profile form painted the server's refusal under the input
and left it there until the next save, so a value the person had already
corrected still read as refused. The onboarding baseline step drops that
sentence while they type; the account card now does the same.
Only the edited field's slot goes: the other fields the same save
refused were not touched, so their sentences still stand. The guard
derives both sets from the form itself, so a new rejection slot without
a clear fails.
Review finding L3.
The guard read `* 1000` anywhere in the two mapping files as a
re-implemented kilometre factor. Both files also turn seconds into
milliseconds, so an unrelated conversion would have failed it for the
wrong reason and the next person would have deleted the rule.
A rescale now counts only in a length context: a distance identifier on
the same line or within the five above it, which covers the body of a
distance-named helper. The offending line is named in the failure. The
matcher's limit is written next to it — a length conversion in a helper
named nothing like a distance, more than five lines below its signature,
still slips through.
Checked both ways against the real file: a millisecond conversion passes,
a `km * 1000` helper fails.
Review finding L4.
The rollups and the cached status insights are recomputed after the
transaction, so a throw there left the rows repaired and every chart on
the pre-repair numbers — and no later run would notice, because the
audit row makes the account skip from then on.
The tail is wrapped now. The audit row is written with
`rollupsRefreshed: false` and only flipped once the recompute returned,
so a re-run can tell "repaired, rollups pending" from "done" and says so.
A run that could not finish the tail names the account, prints the
backfill to re-run — `scripts/backfill-rollups.ts --user <userId>` — and
exits non-zero.
Review finding M2.
The route was exempt from the published contract as a browser handoff. It
is not one: Nightscout has no OAuth step and no authorise page, and the
route is an authenticated JSON POST that Zod-parses a URL and an optional
token, probes the instance and encrypts the pair onto the row. It was the
only POST among eight `connect` exemptions, and its three siblings —
status, test, disconnect — have been published all along, so a client
could read every part of the Nightscout flow except the one that starts
it.
Register it from `nightscoutConnectSchema` so the request shape stays
single-source, document the 400 / 413 / 422 / 429 arms the handler
actually answers with, and drop the exemption. Removing the registration
now fails the coverage guard by name.
An in-place multiplier with no reverse operation was documented with only
a dry run in front of it. Point at the backup runbook where the write
happens.
Review finding M1.
The provenance criterion proves where a row came from, not which build
imported it. An account re-imported on the fixed build carries the same
`EXPORT_XML_SOURCE_MAX` stamp and the same `stats:` external id with the
right numbers and no audit row, so it is selected exactly like a broken
one — and its rest days, under 200 m because the phone stayed at home,
survive the plausibility check after another thousandfold multiply.
So one candidate row that would leave the range now refuses the whole
account: nothing written, no audit row, the offending rows and a printed
reason instead. The idempotency guard only ever held against the
script's own prior run; this holds against the case it could not see.
The runbook says the same in words — a re-imported account must not be
repaired, the re-import already did it — and says what the dry run is
for: it is where the decision is made, not a formality in front of one
already taken.
Integration coverage for the re-imported account: correct rows, an
`--apply` run, nothing changes and no audit row is left behind.
Review finding H1.
The record-unit conversion is now opt-in per caller and only the
`export.xml` archive path opts in. There the attribute is authoritative:
Apple stamps every quantity `<Record>` with the account's own display
unit, so `km` means kilometres and the reading is meaningless without it.
`POST /api/measurements/batch` keeps its documented contract — `unit` is
captured for audit and never read. Nothing in the tree pins the native
client's unit strings for the seventeen convertible identifiers, and a
client that reads a sample in metres while stamping the person's display
unit into the field would have had every walking distance multiplied by a
thousand on ingest. Reading the field there is a wire change and needs
the client's strings pinned first.
The mapping-table audit and the shared conversion module stay. The batch
path's identity behaviour is written down where it is decided and pinned
twice: every convertible identifier round-trips a sibling unit unchanged
by default, and a walking-distance entry sent as `km` is stored as it
arrived. The OpenAPI description says what the server actually does.
Review finding H2.
The audit had moved off the pull_request trigger entirely, which left a window in which a tag could be cut from a merge the audit had not seen. It runs on pull requests again as an advisory job (continue-on-error), stays blocking on the push run for main and on the daily schedule, and a tag is only ever cut from a green push run.
The onboarding step keeps the preference and the adapter where the
save happens and renders the control from its inputs file, so the
three conditions belong to the surface, not to one file. The control
check is word-bounded now: a wrapper named after it passed a substring
check while rendering something else.
A partial save is a whole-class trap: any 200 read as a clean save
throws the person's correction away. The guard enumerates every
surface writing to either profile route and requires each to call the
per-field describer, or to prove its body carries a single field, for
which the partial arm is unreachable.
It shares the route and had the same gap: the banner named the first
refused field and nothing marked the input that held the value. Each
of the seven submitted fields now carries the reason in its own slot,
through the same component the onboarding step uses.
The onboarding baseline step flashed a toast naming one refused field
and walked on to the done screen, stamping the account as set up over
a value the server never stored. The value the person typed was gone
with the step.
A refused field now writes its reason into that field's own error
slot, marks the control invalid, and holds the step until the person
corrects it. Every refused field is named, not only the first, and the
reason comes from the validator code rather than the validator's own
prose, in all seven shipped languages.
The step's profile fieldset moves to `baseline-fields.tsx` unchanged.
The markup can now be rendered against a server answer on its own,
which is where the next commit puts a test.
The defect was invisible from the outside because every behavioural test in
the suite seeded a tagged reading. Two checks close that:
A structural guard that reads the dashboard surfaces and fails when a tile
derives presence from a list of enum buckets with no null / unknown arm and
no written reason, following one hop through a named constant. It carries
its own positive and negative controls, so a refactor that silently stops
matching anything fails here rather than going quiet.
An integration test that seeds twenty readings with a genuinely NULL context
against a real Postgres and reads the tile back through the same resolver
the page renders with: eligible, latest value intact, and a tagged reading
still filed under its own context.
Refs #943
Marc-André Bombeck