A spec that arms `page.route(...)` in a `beforeEach` and never drops the
handler races its own teardown. The context closes, an async handler is
still awaiting, its next call rejects with "Test ended.", and Playwright
attributes that to the test that had just passed. With `retries: 2` plus
`failOnFlakyTests`, which spec bites is a matter of timing.
`e2e/setup/test.ts` re-exports Playwright's `test` extended with an auto
fixture that unroutes every page and the context after each test, and
clears the page event listeners a spec subscribes to; `expect` rides
along so a spec needs one import. Auto means a spec author cannot forget
it and does not have to know it exists. Every spec now takes `test` from
there, and `e2e-route-cleanup-guard.test.ts` keeps the import boundary
closed with an allowlist that carries its reason.
The per-file `afterEach` on the cycle spec goes away — leaving both would
read as not trusting the fixture.
Not covered: a page inside a context a spec builds itself. Only
`share-documents.spec.ts` does that, and it owns those contexts.
A HealthKit blood-glucose sample carries no context either — the mapping has
no field for one — so the batch ingest was building the same row the old
CHECK refused. Fold that source into the case that pins the contextless
shape.
Refs #640
The example CSV is the file people copy. It showed a glucose row with a
FASTING context and nothing else, which is the shape a meter produces, not
the shape a continuous sensor produces. It now carries both.
Refs #640
Same shape as the CSV and JSON cards: reaching the done state rendered the
success tick whether or not a single row was upserted. The archive's count
picks the tone now, and an archive that wrote nothing says so.
The clinical-record count stays out of that decision. Skipping
electrocardiograms and lab documents is a documented, deliberate exclusion,
not a refused row, so it must not read as a warning.
Refs #640
A CSV import of a continuous-sensor export wrote nothing and rendered a
green success state. Three faults compounded, and the reporter took all
three apart before filing.
The parser refused every BLOOD_GLUCOSE row with a blank glucoseContext.
Behind it, the CHECK constraint from migration 0021 read both ways at
once: a glucose row MUST carry a context, every other type MUST NOT. The
second half is right and stays. The first was written when the only
glucose writer was the entry form, where a person picks fasting or
post-meal at the moment of the reading. Every machine writer that
arrived since has no such answer per sample, so the Nightscout pull, the
MCP log tool, the Telegram capture and both bulk importers were all
building rows the database then refused.
A missing context now stores as NULL, never as a fifth enum member.
Every per-context reader selects by equality against the four members,
so the reading stays out of the doctor report's fasting rows, the
matching FHIR observations and the per-context targets, while the chart,
the all-time summary and the clinical panel keep it in full. Hand entry
still asks for a context, because at that moment you know it.
The import cards now read the counts rather than the status code.
Nothing written is a failure, a mixed file is a warning, and the refused
rows are grouped by reason with a count instead of fifty identical
lines, with the per-line detail one disclosure away. Both cards render
through one outcome table, so a tick cannot reach a result that wrote
nothing.
Refs #640
`medication_intake_events.idempotency_key` carried a GLOBAL unique from
0001_init. The value is client-supplied, so two people running the same
client can mint the same key — and on a global unique the second one is
simply refused by the constraint. Their dose is not written and nothing
tells them so. The bulk drain made it worse: its replay probe read the
key with no user scope, resolved the first user's row, and reported the
second user's entry as a duplicate carrying a stranger's row id.
Migration 0273 replaces the global unique with `(user_id,
idempotency_key)`, the grain every other client-supplied identity on the
table already uses. Both unscoped reads in the bulk drain now key on the
composite; the other four reads were already narrowed by user.
On existing data this is a relaxation, so nothing needs reconciling: the
old constraint is strictly stronger than the new one, and no pair of rows
can violate an index the previous one already forbade. The migration
still carries a reconciliation step for a database whose global index was
dropped by hand at some point. It never deletes or rewrites a dose — it
clears the dedup HINT on the later row of a colliding pair and keeps
every intake event, timestamp, attribution and inventory stamp intact.
The new index is created before the old one is dropped, so a failure
aborts with the old protection still in place.
A structural guard freezes the read side: no `medicationIntakeEvent`
query may filter on the key without narrowing by `userId`.
The dedup key on the medication intake surfaces is not only `externalId`.
`MedicationIntakeEvent.idempotency_key` is a `@unique` column persisted
with the dose row forever, with no TTL and no sweep, matched by plain
equality on the per-medication POST, the bulk drain and the external
ingest bridge. That makes it an identity, not a replay token, and it
carries the same failure mode: a key that rotates per client launch never
matches its own earlier post, so every retry logs the dose again. The
blank case is sharper still — the column is unique table-wide, so an
empty key resolves to whichever unrelated row stored it first. Refuse
both shapes at the schema on the single-entry paths and per entry on the
bulk drain.
The MCP write path hashes a host-supplied key into the `externalId` half
of its dedup key, so the shape is laundered before anything downstream
could see it and the check has to run on the input. That closes a live
one: every tool handler falls back to `""` when the host omits the
argument, and `""` digests to a constant, so each keyless write after the
first reported `already_logged` and was silently dropped.
The HTTP `Idempotency-Key` header stays out, and now says why in the
guard: `IdempotencyKey.expiresAt` bounds it and the cleanup job sweeps
it, so it is a windowed replay token and rotation is its design.
Extend the structural guard with a provenance rule for identities that
never pass a Zod request schema — a spreadsheet column, an MCP tool
argument. Inside the import and MCP trees, a module handling an identity
must apply the floor itself or import a module that does. That is a real
chain rather than a list of known offenders: a new importer that mints an
`externalId` from file input and imports nothing checked fails it, with
no allowlist to escape into.
Harden the structural guard so it reads the file body with its imports
stripped: grepping the raw source cannot tell a file that APPLIES the
check from one that merely imports it, so deleting the call while
leaving the import slipped straight through.
Add the partial-success case on a second batch envelope — the mood bulk
ingest echoes the offending entry back with its index, status and
reason while the good rows still upsert.
An external id is half of every ingest dedup key — (userId, source,
externalId), (userId, externalSource, externalId), (userId, type,
source, externalId). Re-posting the same triple is idempotent and
returns the existing row, which holds only while the id is stable
across client launches. A client that serialised an opaque platform
object with its default object description sent a per-allocation
memory address instead, so no sweep ever matched its own earlier row
and each one minted a fresh record: a live instance grew 23 phantom
medications in a day, none of which could collect a dose, because the
dose side resolved through the same rotating string.
Add one shared classifier for identifiers that provably cannot be an
identity — an angle-bracketed object description carrying a hex
pointer, a bare 0x address, and a blank value — and wire it at every
surface that accepts a client-supplied externalId.
Single-entry creates refuse through the schema, so the client gets the
standard 422 multi-issue envelope with a message naming the shape it
sent and what to send instead. Batch and bulk surfaces classify per
entry and report the offending row as skipped with reason
unstable_external_id, leaving the rest of the payload to land: one bad
row must not stop a sync. Every refusal carries pinned wide-event meta
(count, shape, surface) and never the id itself.
A structural guard freezes both routes into the floor: a request-side
externalId declaration either applies the field check or carries a
marker naming the file that checks it per entry, and the guard verifies
that file really does. The backup-restore payload is the one exemption,
with its reason recorded — it replays ids the server itself wrote.
Whether a column has a reader anywhere in the tree is a whole-program property,
and grep is the only cheap approximation. Grep is unsound both ways here:
generic names match unrelated identifiers and read as consumed, while
consumption through a select-less read plus whole-row serialization reads as
unconsumed. A gate built on that would drift towards green and stop meaning
anything, which is the failure this whole line of work exists to prevent. So it
stays a review tool with a human reading the output, run from the deep-audit
step and never in CI. The script says so itself and prints the false-positive
sources under every run.
It also carries the convention that makes the tool useful: a change that adds a
column, payload field, or preference contains its consumer, or the schema
comment carries `@internal` for something deliberately never surfaced or
`@pending-consumer(#issue)` for one still coming. The second marker is
re-listed on every run, so a follow-up that never happened surfaces at the next
release audit instead of never. Recorded in the contributor notes alongside the
two questions that are exact enough to be gates.
Every finding behind this work is a two-ended change whose ends shipped in
different releases with nothing holding them together, and every gate the repo
runs proves one end: typecheck proves the export exists, knip proves it is
imported, the OpenAPI check proves the schema matches. None of them notice when
the second end never arrives.
Two questions here can be answered exactly, so they become tests.
The account endpoint rides every app boot and carries the user's preferences.
Its field names are read out of the response literal and each one must be read
by a client module — property access or destructuring, comments stripped,
because a doc comment claiming a consumer is what the remembered practice name
had for its whole dead life. That field would have failed this on the day it
shipped. One field is allowlisted today with its reason written out: the
insurer institution number has no web surface at all, which the entry keeps
visible rather than papering over.
The measurement backup select is now a named constant, compared field by field
against the model in the schema. Both sides are literals, so the comparison is
exact — no name heuristics, nothing that can drift towards accepting. A column
added to the model and forgotten by the backup fails at the commit that adds
it, which is how the aggregation provenance went missing. Scoped to the one
model; extend it as each further model becomes backup-relevant.
The reverse-coverage guard asserts that every key in the bundle is reachable
from a call site. It had stopped doing that for most of the bundle. A template
whose only static anchor was a namespace root registered that root as a subtree
base, so a single per-provider OAuth key marked every key under `settings.*`
reachable, and the same shape did it for `measurementReminders.*`. A cache-key
string that happens to start with a namespace word had quietly done it for
`insights.*` too. The guard stayed green throughout, which is the part worth
noticing: a pair guard degrades into a one-end guard without any signal.
Extract the matcher into a helper with its own unit tests, and change what it
accepts. A template carrying both a literal prefix and a literal suffix now
records the pair and requires both ends to match, so the OAuth template covers
the provider keys and nothing else. A template with no suffix whose anchor is a
bare namespace no longer whitelists anything — it fails the guard by name, with
the two honest remedies in the message. The same door is used by the template,
assignment and concatenation arms, so none of them can reopen it. Template
bodies are read by scanning forward from a known opening backtick rather than
by pairing backticks across the tree, which was drifting and dropping whole
files out of the scan.
Three call sites build their keys whole instead of interpolating onto a
namespace root: the two measurement-reminder due helpers, the mental-health
option labels, and the OAuth error copy, which now names its per-provider base.
Two audit-log action names that merely look like keys carry a written
exemption; the API error catalog is a runtime lookup and is allowlisted with
its reason.
With the matcher honest, 149 keys had no call site left. They belong to an
explainer popover, a hypnogram, a passkey table and an insights hero that were
all rebuilt or removed, and they are deleted from all six locales. The
explainer guard stops pinning its title leaves, which no surface has rendered
since the popover went.
The disaster-recovery backup selected every Measurement scalar except the
aggregation fields, so a restore brought every `stats:` aggregate back with no
provenance and the Apple Health authority ladder came back flat: an export.xml
source-day estimate could then overwrite a native HealthKit statistic.
`aggregationProvenance` now rides the select, the disaster-recovery
serialization, the payload schema and the restore mapping, following
`externalSourceVersion` exactly — including staying out of the reduced
portable shape, which carries the human-readable subset only. The schema field
is optional, so every backup written before this parses unchanged.
Restoring one of those older backups no longer recreates the ambiguity
migration 0263 repaired: an Apple Health row under a `stats:` identity with no
provenance comes back as LEGACY_UNKNOWN, the sentinel the cumulative drain
already writes in that situation, so a native sync or archive reprocess can
repair it. The gate is scoped as 0263 scoped its backfill — other providers
mint `stats:` identities for their cumulative dailies and have never carried
provenance, and ordinary point rows keep NULL.
Migration 0263 added three columns beside the Apple Health `stats:*`
authority ladder. Only `aggregation_provenance` is ever read back: the
external-measurement reconciler selects it and refuses a lower-authority write
against a higher-authority canonical row, and the cumulative drain reads it
too. That column stays.
`aggregation_contributor_count` and `aggregation_selected_source_hash` are
written by three call sites and read by none. No select, no raw SQL, no export,
no backup, no API contract, no repair path. The import-time selection keeps its
deterministic tiebreak, which compares an in-memory map key computed per flush
rather than a value read back from the table.
Both are plain nullable scalars with no index, unique, check or foreign key, so
the drop is a catalog-only operation that touches no measured value.
The doctor-report contract has always described the practice / clinic name as
a stored preference, and `/api/auth/me` has returned
`User.lastReportPracticeName` since it shipped, but nothing ever wrote the
column. Every report asked for the clinic name again.
The export route now persists the sanitised name after the requested artefact
has actually been produced, so a generation that throws cannot overwrite the
remembered value with a half-typed one. Clearing the field persists null, which
keeps the cover line cleared next time. The export panel seeds its input from
the auth payload during render, gated per user id, so a late resolve never
clobbers a field the user is already typing into, and a finished export
invalidates the cached payload so a remount reads the new value.
Interface housekeeping. The overview sections drop their hand-rolled card
frames for a shared shell with one inner rhythm, failed reads on dense metric
pages get a named row primitive that puts the message in the content tier, and
the last off-scale spacing and text sizes go back onto the scale.
The tag chip and the sub-twelve-pixel text step are primitives now, so neither
can be hand-copied at a slightly different value again.
The stage-composition chart card was the last insights tile with a
hand-rolled title row. It now uses the canonical header primitive; the
subtitle and the window toggles keep the shape they had, since the toggles
have to drop to their own row on a phone.
Also sweeps the last off-scale spacing: the two dense tiles inside a
recommendation move onto the sanctioned inner-tile step, and the not-found
button onto the scale. The banned twenty-pixel step is now absent from the
tree. The chart body is untouched.
Seven overview sections had hand-rolled the same card literal, and the copies
had already drifted on the one thing the literal did not pin: the space
between blocks. Three used 12 px, two 8 px, one 16 px, one under the other in
the same column.
`InsightSectionCard` composes `Card`, so the frame comes from the card
contract, and fixes the inner rhythm at 12 px. The daily briefing moves onto
it as well, and the lab-changes list keeps its edge-to-edge rows through the
documented flush variant.
Visible change on the overview column: two bodies open by 4 px, one tightens
by 4 px, and all of them pick up the shadow their card neighbours already
had. Chart bodies are untouched.
The HealthKit metric sub-pages and the vitals dashboard had each hand-rolled
the same failed-read row, byte for byte. The tall centred error card is the
right shape for a list surface but dominates a dense metric strip, so the row
stays — as a named primitive rather than a third copy.
`QueryErrorRow` carries the shipped shape with one correction: the failure
message renders in the foreground tier. An alert is the content of a failed
surface, not meta about it, and both copies had it muted. The retry glyph
matches its tall sibling. Both converted surfaces pass their existing slot
names through, so their hooks are unchanged.
Three integration cards carried the same hand-copied chip literal in the
card-header accessory slot, each free to drift on the next touch; the size
itself was an arbitrary value repeated at three slightly different spellings
across the tree.
- Mint `--text-2xs` (0.6875rem with a matched line-height) as the one
sanctioned step below `text-xs`.
- Mint `TagChip` and adopt it on the Fitbit, Google Health, and Nightscout
cards. It stays distinct from `Badge`, which carries status meaning.
- Sweep the arbitrary micro sizes: the stepper label and the two dose-ledger
outline badges move onto the token; the muted target caption and the muted
ledger meta row move up to the `text-xs` muted floor.
No visible change beyond the two muted rows gaining a fraction of a pixel.
Arm the response waiter before navigating, so a response that lands during
the initial load is still observed instead of timing out on an event that
already happened, and poll for the returning refetch rather than sleeping a
fixed span that a loaded machine can outrun.
Mood has been a first-class module here for a long time, with entries,
structured tags, rated factors and notes. The bridge to the external moodLog
service had nothing left to add, and its Settings card was withdrawn in
a56e6fe61 on the promise of a dedicated removal. This delivers it.
Removed: the hourly `moodlog-sync` queue, cron and handler binding; the
inbound webhook and its shared-secret handling; the reverse push and its four
call sites; the pull-sync module; the per-user credential routes and the
instance-wide toggle; the entry on the `/api/integrations/status` envelope,
the `IntegrationKey` union, the cadence map and the display-name table; the
boot-time secret rotation pass; 22 message keys per locale.
Migration 0271 drops the five `users` credential columns and
`app_settings.mood_log_global`, clears the ledger row for the provider, and
deletes the retired pg-boss schedule so no cron keeps enqueuing work no
handler drains.
Mood data is untouched. `mood_entries` is not named anywhere in the migration
and cannot be reached from it: the dropped columns carry no index, constraint
or foreign key, and `mood_entries.user_id` references `users(id)`, not any of
them. Rows the bridge imported keep `source = 'MOODLOG'`, and every read path
- list filter, source badge, exports, backups - still resolves that value.
`src/lib/validations/moodlog.ts` held the native mood-entry schemas alongside
the bridge's own; it is now `validations/mood.ts` and carries only the former.
The load-bearing one is the panel-coverage test: render the panel with an
envelope carrying every integration key and require each configured one to
surface as a card or as a fallback row. Drop the fallback row and it goes red,
which is the whole point — the previous shape of this bug was a provider on
the wire that no screen rendered, and no card-shaped test could have caught it.
Alongside it: the ladder table with the month-silent fixture that resolves to
stalled, the 24-hour and 7-day boundaries asserted against literals rather than
the constants they check, the per-metric stale flag firing only under a healthy
provider, and a structural guard that the verdict-to-pill mapping is defined in
exactly one module.
The card suites migrate with the semantics: a transient failure now expects the
warning pill everywhere, and the Apple Health suite trades its bespoke badge
states for pill assertions. The e2e mocks gain the verdict shape, the
nightscout entry, and an assertion that the card-less row is on the page.
Three `pillStateFor` implementations disagreed with each other, so the same
ledger state painted red on one card and amber on the next, on one page. All
three are gone; the client projects the server verdict once.
Red now means "your action fixes this" and nothing else. A transient upstream
failure carries the warning tone: telling a user to reconnect against a 503
sends them clicking at something they cannot repair, and the detail belongs in
the inline error line anyway. Three new pill states carry the rest — no recent
data, sync stopped, waiting for first data.
Per-metric freshness has been computed on every status call since v1.27.30 and
rendered nowhere. Every connected card now opens a disclosure listing each data
type the connection has delivered and how long ago, with a quiet one flagged.
Collapsed by default so a 25-type card keeps its shape, but the collapsed
summary already names the count, or the signal hides behind a click. A type a
provider never delivered simply is not listed.
The panel iterates the envelope and renders a plain status row for every
configured entry no card claims. That is the structural half: a connection can
no longer be running, failing and absent from the page at once.
Apple Health drops its bespoke badge for the shared pill, gains the divider and
body inset every sibling carries, and says something when a week has passed
with nothing arriving. Nightscout reads the envelope instead of its own status
round-trip. The OAuth card uses the shared error primitive, all nine cards
carry the divider testid, and the four original connectors gain the "view your
data" link the newer ones already had.
The ledger records what the last attempt did. It has no way to say what is
no longer being attempted: a row frozen at `error_transient` for a month
reads exactly like one retrying every hour, because nothing anywhere read
the age of `lastAttemptAt`.
`sync-verdict.ts` derives the missing reading as a pure read-side
projection: eight verdicts down one ladder, no writer, no cron, nothing that
can desynchronise from the row it reads. `stalled` is the new one, and it
deliberately outranks `failing` — an integration nobody is calling is not
failing, it is not running.
Every entry on `/api/integrations/status` carries the verdict, computed
inline from data the route already holds. That placement is the point: a
card-level verdict misses a card-less provider by construction, which is how
a dead sync stayed invisible. Nightscout joins the envelope so its card can
stop fetching its own status.
The freshness module gains the workout leg (Strava writes no measurements at
all) and a source-parameterised core so the Apple Health route can reuse it.
Its per-metric `stale` flag fires only while the provider itself reads
healthy: that combination is the dead-pipe signature, and when the whole
connection has stopped every metric ages together anyway.
MCP reports the verdict beside `connected`, which only ever meant "not
explicitly disconnected".
Sweep every feet + inches pair the fields can offer and every whole-inch
height across the canonical range, asserting the value comes back
unchanged and lands inside the server's window. Assert the metric branch
with toBe so a re-derivation of a stored height cannot slip through.
Render both entry surfaces under each preference: the metric case keeps
the single centimetres box with the canonical guardrails, the imperial
case shows the feet + inches pair seeded from the stored value. Extend
the structural guard so a surface that drops the adapter, hardcodes a
centimetre guardrail, or hardcodes a unit label trips a tripwire.
Height is a profile column, not a MeasurementType, so the display
transform that carries the metric/imperial preference across every
reading surface never reached it. Add the adapter that closes the gap.
With imperial selected both entry surfaces render a feet + inches pair
instead of a centimetres box; with metric selected the field is what it
was. The adapter converts to canonical centimetres at the entry
boundary, so storage, the wire, the Zod bound, and every consumer of
the value stay unchanged.
Imperial entry quantises to whole inches, which map to centimetres
without a remainder, so save-then-reopen returns the typed pair. The
offered feet window rounds inward against the canonical 50-300 window
so a height the fields allow is never refused on save.
The unit select shares the Settings card with the height field, so a
flip re-expresses the in-flight draft through the new branch rather
than stranding an empty field.
Stable per-segment sleep keys plus the replace-by-window sweep, a one-shot
repair over the past year, Retry-After handling, and the unchanged-value write
skip. Migration 0270 records which connections the repair has been through.
Mirrors the Google sleep suites name for name: the mapper keeps its externalIds
across a re-score that re-classifies and renumbers segments, the sweep stays
bounded to the night's window and never touches the fresh keep-set, a null
window issues no delete at all, and a cleanup failure never fails the sync.
The end-to-end fixture runs one night stored under the old keys through
`syncUserSleep` and pins the ordering the heal depends on — sweep first, then
the rescue re-keying the tombstoned rows in place, no parallel insert — plus a
second pass that writes nothing at all.
Also covers the repair's verdict gates (bounded 365-day window, marker stamped
only on a clean run, watermark untouched, dead token and reauth park returning
without a stamp), the unchanged-value skip including the re-classified segment
it must not swallow, and both Retry-After paths.
Fitbit re-scores a night after the fact, and the per-segment externalId carried
a positional index plus the stage label. Both moved on a re-score: renumbered
segments minted fresh keys, the previous scoring's rows stayed live because
nothing swept them, and the night total counted both copies from then on.
Key the segment on the stable session anchor plus its own start instant
(`<logId>:sleep:<segment-start>`), and add the replace-by-window sweep every
other segment-writing provider already has. The two ship together with the
natural-key rescue that has been sitting in the Fitbit upsert unused: the sweep
tombstones the old-keyed rows first, so the rescue re-keys those very rows in
place instead of inserting a parallel copy or losing the insert to
`skipDuplicates` against the natural unique index.
A one-shot repair queue re-reads the bounded sleep history once per connection
so no user has to do anything; it stamps `sleepRepairedAt` (migration 0270) only
on a clean run, never touches the sync watermark, and reports the rows it
removed. Nights older than the 365-day backfill horizon keep their old keys and
whatever duplicates they already hold — they can no longer gain new ones.
Alongside: `fitbitGet` now honours a `Retry-After` inside a 120 s cap by waiting
once and retrying once, and fails fast above it so a long hold cannot starve the
rest of the poll cohort or outlive a job's expiration. The measurement upsert
skips a write whose payload is unchanged, which takes the hourly overlap off the
delta feed for every Fitbit resource.
Marc-André Bombeck