The 1500-line list route carried the day-grouped list, the sleep list
and its SQL night page, each with its own copy of the window-total
paging. They now live in src/lib/measurements/list/ with one paging
helper, and the route dispatches.
The sleep list gives every page night a row. A night whose
reconstruction held no main sleep (a day of naps alone) used to be
dropped while meta.total still counted it, which left short pages; its
row now carries 0 minutes and its naps. The SQL night cut splices the
asleep stages from ASLEEP_STAGES and folds rows into hour buckets
before the window functions, which cannot move a session boundary and
sorts a handful of rows per night instead of every stage. A drill-down
reads its night the way the night view does.
groupBy=day refuses valueMin / valueMax with 422: a value range filters
single readings, and the rows promise whole days.
An integration test runs the SQL page and reconstructSleepSessions over
one set of awkward shapes and holds them to the same nights, and the
list, the night view and the series to one answer per day. The unit
test of the day-grouped branch now pins only the branch; what the rows
hold is pinned against Postgres.
The SQL page labels a sleep session by the last row of any writer; the
reconstruction labels it by its winning writer. A watch stopping before
midnight beside a phone whose in-bed span ran past it gave two days,
and matching the two by label dropped the night from the list while
the total still counted it. Each reconstructed session is now filed
under the page night whose span holds it and takes that night's label,
which the drill-down also asks by.
A sleep drill-down scans 36 hours either side of its day instead of the
whole history. Both day branches and the sleep list resolve the zone
through the same stored-zone resolver, so a drill-down always cuts the
days its list row summed. The list description states the real page
bounds: a day-sum page takes `limit` as asked, a sleep page caps at
1000 nights.
Refs #1245
The day-grouped list read the newest `limit` raw rows (the web asked
for 5000), folded them into days and returned every day at once: one
page of thousands of rows, and every day older than the 5000th raw row
missing without a word. The sleep list did the same at a year.
Both now pick the page in Postgres. The day mode selects the page's
local days with the distinct-day total from `COUNT(*) OVER ()` in the
same query, then folds only those days into per-source groups in SQL
and lets the source ladder choose among the groups, so no raw row
reaches the process. The sleep mode cuts sessions with the same gap
rule as the reconstruction, pages wake days the same way and reads only
the page's stage rows; the 365-day cut and `meta.windowFrom` are gone.
A sleep page holds at most 1000 nights, echoed in `meta.limit`.
The web list pages both modes at 25 like the per-sample list. The
`partial` day marker is dropped from the contract; no day is cut any
more.
Refs #1245
A sync from a watch or an integration now waits the instance's sync gap
(AppSettings.aiSyncRegenGapHours, else AI_SYNC_REGEN_GAP_HOURS, else 6 h)
before it may regenerate a status card again; a hand entry keeps the hour.
Every caller of afterMeasurementMutation and invalidateStatusInsightsForTypes
names its source, so a new ingest path cannot skip the choice.
Refs #1217
Migration 0401 turns healthkit_type_syncs.type into text so the ledger can
hold MEDICATION_DOSE, and adds last_background_at (seeded from rows whose
latest arrival was a background one). POST /api/medications/intake/bulk
takes syncTrigger and records APPLE_HEALTH dose arrivals; the Apple Health
status read serves a MEDICATION_DOSE freshness entry and lastBackgroundAt
on every entry with arrival facts.
Every raw-SQL clock read against a timestamp column without a zone now
writes NOW() AT TIME ZONE 'UTC', so passkey and login challenges, rate
limits, provider pauses, coach usage stamps, tombstone and measurement
windows no longer pass through the session time zone even where the
per-connection pin cannot apply. The regression-window bounds stay naive
UTC timestamps instead of being turned into timestamptz. A structural
guard keeps bare NOW() / CURRENT_TIMESTAMP out of raw SQL outside a short
list of timestamptz targets. The integration test sets the database
default to Europe/Berlin and drives challenge claims, session expiry, the
native sign-in handoff freshness check and the rate-limit window through
the app's client, plus the challenge claim through an unpinned client.
Self-hosting docs describe the pin for an existing or shared Postgres.
- Publish timezone, glucoseUnit and a features block on the account
payload; glucoseUnit reads as the closed pair the contract names, and
features.trackIntake tells a separately released client that
medications take the record-only switch.
- Refuse a new dose for a record-only medication with
medication.intake.notTracked on every dose-writing route; the bulk
route reports it per entry as intake_not_tracked, and correcting an
already resolved event stays open.
- Answer passkey sign-in failures by name instead of a 500:
passkey.challenge.expired, passkey.unknown, passkey.response.invalid
and passkey.verification.failed, each with an audit row.
- Carry one account-wide adherence figure in meta.aggregate on the
batched compliance read, weighted by expected doses, and accept
?days=30|60|90|180|365 for a report window.
- Accept ?days=30|90|180|365 on the mood insights read so stability
covers the period shown, and name the window it covers.
- Keep only the reminder that announces a dose off APNs when the phone
owns medication reminders; follow-ups for a dose still open are
pushed again, marked by a followUp flag the reminder cron sets.
- Document which batch validation issues belong to an entry and pin it
with a test that splits and resends a batch; publish rejectedFields
items as ValidationIssue.
The nightly folds soft-deleted every raw Apple Health sample they folded
into an hourly or daily stats row and kept it for 75 days. Nothing ever
read those rows as deletions; they only held a re-upload of the sample
off, and they were most of the measurements table and its indexes.
Ingest now refuses such a re-upload directly: a raw sample of a dense or
mean type, old enough to have been folded, whose hour or day a live
stats row covers, is a duplicate with reason folded_window. The batch
route and the Apple Health export import apply the same rule. With that
in place the dense retention and the daily-mean consolidation delete the
raw rows outright, and a new compaction-tombstone-purge job removes the
backlog: 5,000 rows a batch, one account at a time under the restore
lock, a short pause between batches, resumable because the predicate is
idempotent, queued at every boot and continuing itself while it makes
progress. A person's own deletions are never in its predicate. The sync
feed leaves compaction tombstones out and a restore does not write them
back; all three classify through the same function.
The batch route answers an exact re-sent sample without a transaction:
one read over the batch's sample ids, live or deleted, decides what the
reconciler would have decided, at the same point in the loop, so every
status, reason, counter and the cross-source merge stay as they were. A
batch of duplicates only no longer queues PR detection, a reminder sweep
or the detection audit row. Caches are evicted only for a foreground or
manual sync or a batch that wrote a hand-entered reading, and marked
stale otherwise; the targets cache follows the same split.
Also:
- POST /api/admin/maintenance/measurements queues VACUUM (ANALYZE) and a
REINDEX INDEX CONCURRENTLY per index, largest first, on a connection of
its own; it refuses while the purge runs, drops invalid leftover copies
and stops at the first failed rebuild. Runbook in docs/ops.
- The pure volume queues keep their finished jobs for 96 hours instead
of seven days, never less than the failure window plus a day.
- work_mem is set per app connection from DATABASE_WORK_MEM (16MB), also
when the statement timeout is disabled.
- The PR-detection fallback tick re-scans only accounts whose readings
or workouts changed since the previous tick.
- Type discovery for the status refill walks the live-rows index as a
loose index scan instead of grouping every live row.
- The host metrics chart polls every five minutes, like its sampler.
- The batch route records per-type HealthKit arrivals.
Land every file the v1.42 work shares before any of it is built, so the
parts built side by side never edit the same file.
Schema and migrations 0379-0386: the HEALTH_CONNECT measurement source,
ImportJob.kind, the air-quality columns and the per-user switch on the
environment tables, sealed environment locations with nullable plaintext,
managed-profile handover links, the per-type HealthKit arrival ledger, lab
unit-conversion provenance, and user-defined vaccines. Every new column is
annotated for the column-reader sweep.
HEALTH_CONNECT reaches every exhaustive list: the source labels, the
read enum (not the write allowlist), the source-priority ladder cap, now
the number of sources rather than a fixed eight, and the write-path smoke
test. RMSSD gets its own priority class and a 300 ms ceiling. The batch
syncTrigger gains `manual`, with its label on the Apple Health card.
Queues for the Health Connect import, the compaction-tombstone purge and
the measurement-table maintenance are registered with stub handlers;
host-metric sampling moves to every five minutes. Environment and work_mem
variables reach the compose whitelist, the production example and the
manifest. The proxy admits the handover link and redirects it at the edge,
and the Health Connect upload bypasses the proxy body limit.
The new routes answer 501 behind their final authentication and are
published in the contract; backup, wipe and key rotation classify the new
models and columns; the seven message bundles carry the base copy. Guards
that list release-pending keys and columns fail once an entry is wired, so
those lists empty themselves before the tag.
Also re-anchor a login-throttle fixture to the wall clock, where its
expiry is compared, and re-base the aggregate bundle cap on a measured
12 KB of new locale strings.
The systolic and diastolic reads each took the ladder's source per day on
their own. Whenever a source held only half of a reading on a day (a deleted
diastolic, an orphaned sample, a sub-floor systolic), the two picks diverged
and the pairing joined one source's systolic to another source's diastolic.
A sub-floor systolic could also claim a day and then contribute nothing,
leaving the day empty although a lower-ranked source had a valid reading.
The source is now picked per day from the systolic rows that pass the floor,
both halves are read from it in one statement, and a pair is only made
within one source, which also covers a reading that straddles midnight.
The dashboard summaries, the insights aggregate and the series stats strip
still took a plain mean over pulse readings for their 7-day, 30-day, prior-month
and all-time figures, while the year-ago figure and the chart already weighed
each day once, so the year delta compared two different statistics. They now
go through the day-mean helper: each day is the mean of its hours' means and a
window the mean of its days, in the frame each reader buckets its days by.
Count, min, max, the median and the spread stay over every reading; every other
type is unchanged.
The blood-pressure branch of /api/measurements/series still read systolic and
diastolic with an unranked findMany, so with a second provider the pairs blended
both sources and a reading present in both showed twice, while the other kinds
already follow the ladder since the dense and raw reads were moved onto the
shared helper.
Each of the two types now reads the ladder-winning source per day through the
same helper. The physiological floors (systolic 40, diastolic 20) and the
five-minute pairing are unchanged.
A watch samples pulse about every 5 to 10 minutes at rest and about every
5 seconds in a workout. The daily value was the plain mean over a day's
readings, so a workout hour of several hundred readings outweighed the other
23 hours: a day with one hard hour and 23 resting hours (60 bpm) read about
128 bpm, not 64.
A day of pulse is now the mean of its per-hour means, in the rollup DAY
bucket, the live per-day read the chart serves, and the series route's day
grain. Windows over several days weigh each day once instead of by its
reading count, so the bias does not return one level up. Reading counts,
min/max, and the regression sums still run over every reading. The chart
tooltip and the data table name the statistic, in all seven locales.
The statistic is limited to pulse; HRV, SpO2 and glucose keep the plain mean
(see src/lib/measurements/day-statistic.ts). Stored DAY rows are rewritten by
the next rollup recompute (admin recompute route), not on read.
Fixes#1107
The rollup tier and sleep resolve every (type, day) to the source the person's
ladder picks before they aggregate. The dense and raw reads of
/api/measurements/series filtered on user and type only, so with a second
provider a day's value blended the two, a reading present in both showed
twice, and the configured ladder had no effect on those charts.
Both reads now take the ladder-winning source per day through the shared
canonicalMeasurementsCte helper, as the other live readers do:
- the day and hour bucket read of the dense kinds (pulse, glucose, HRV, SpO2),
and the stats strip over its rows;
- the generic raw read (weight, body fat, steps, resting heart rate, ...).
The collapse is per day for the hour grain too, as everywhere else. The helper
gains an optional single-type scope so a series scans one metric, not every row
the person has in the window.
Only a type that has a ladder is collapsed. For a type without one (glucose,
body water, bone mass) the helper breaks the tie by source name, which would
silently hide one of two real sources, so those keep every source as before.
A single-source account reads exactly what is stored.
Tests: a real-database test covering the ladder in both directions, the
per-day fallback, a reading mirrored into two sources, the day-bucket, hour-
bucket and raw reads, a type with no ladder and a single-source account. The
route's unit test file reads its fixtures through a small adapter for the new
raw statements, and the dense-read guard drops the exception for the removed
findMany. The OpenAPI description states the rule.
Refs #1108
The sleep list capped its read to the last year only when no `from`
was sent. An explicit early `from` walked every per-stage row back to
that date, on a multi-source account several rows per night for years.
The read window is now at most a year ending at `to` (or now). `from`
narrows it but never widens it, older nights are reached by moving
`to` back, and `meta.windowFrom` names the instant a cut window
started, so a client can tell a cut from an empty history.
The dense rule in /api/measurements/series covered pulse and glucose
only. Heart-rate variability and blood oxygen read every raw row for
any window up to 3650 days, and a watch or ring that writes them every
minute makes that the same unbounded read pulse had.
Both kinds now fold per local day beyond 90 days and per local hour
inside 90 days once the window holds more than 10 000 readings. A
bucket keeps the point shape of its kind: only pulse carries
valueMin / valueMax. Statistics stay computed over every reading, and
a sparse series inside 90 days is unchanged. The OpenAPI description
names the four kinds.
A sensor reading every five minutes puts about 26 000 glucose readings in
a 90-day window, and the per-kind series sent every one of them raw. The
hourly bucketing that already applies to a dense heart-rate stream now
applies to glucose too, with the same 10 000-reading cap: the hour's mean
converted once to the user's unit, statistics over every raw reading.
A fingerstick meter stays far under the cap and keeps its raw points.
The doctor report keeps glucose raw inside 90 days. Its clinical panel is
computed from the raw readings of every source, while the day path works
from canonical-source day buckets, so the two do not give the same figures
when a meter and a sensor overlap.
Past 90 days every pulse reader already folded in SQL. Inside it, three
still read every raw heart-rate row, which on a watch that records a
reading a minute is 130 000 rows per read (#1023):
- The insights page's resting-pulse proxy, used when an account has no
resting heart rate, walked the whole window to take one percentile per
day and held the event loop for over a second per view. It now folds in
Postgres (`readRestingPulseProxy`) with the same percentile, rounding,
minimum-sample rule and day anchor.
- The per-kind series sent every reading: 14.7 MB for 90 days. A pulse
window with more than 10 000 readings is now bucketed per local hour,
mean plus low/high band, with stats still over the readings.
- The doctor report drew every reading into one sparkline. A pulse window
over the same cap now takes the per-day path longer reports already use.
Sparse pulse (a cuff, hourly imports, a watch sampling every few minutes)
stays under the cap and keeps its raw shape.
The status assessments folded their recent days, weeks, months and years on
Berlin days whatever the user's zone, the mood and cycle crosstabs keyed
their metric days in Berlin, the resting-pulse proxy defaulted to Berlin
days, the briefing's opener rotated at Berlin midnight, and every
assessment prompt labelled today's date as Berlin's. They now use the
user's zone. Rollup months and years are labelled by their UTC start,
which is how they are cut; read in a zone west of UTC the label named the
month before.
"Europe/Berlin" was written out in some eighty places. DEFAULT_TIMEZONE in
lib/tz/format is now the one definition: the second copy in mood/date-key,
DISPLAY_TIMEZONE, the Berlin-only day key helpers and the per-route copies
are gone. An in-repo lint rule rejects the literal outside lib/tz and
tests; four areas another change owns are listed as pending.
Local day bounds had three implementations. The day-key one read the zone
offset at UTC midnight, which on a transition day in zones far from UTC
(Pacific/Auckland, Australia/Sydney) is the offset after the change, so the
day started an hour off. The instant one converged on local 00:00, which in
zones that skip midnight (America/Santiago, America/Havana) does not exist,
and returned 23:00 of the previous day.
startOfLocalDayKey and localDayWindow in lib/tz/local-day are now the only
implementation: the first instant that reads the day, tried under every
offset in force around it. A skipped wall clock resolves to the moment the
clocks jump, the day-keyed copies in the measurement drains, the compliance
rollups and the dashboard batch window are gone, and the noon and mid-hour
anchors use the same converge.
A bridge such as Tasker or Home Assistant could only send sleep stages
through the batch route, in HealthKit's terms: the category identifier,
a stage codepoint from 0 to 5, and a value it had to work out in
minutes. It can now post one segment to `POST /api/measurements` with
`type: "SLEEP_DURATION"`, a `sleepStage` of `IN_BED`, `ASLEEP`, `AWAKE`,
`CORE`, `DEEP` or `REM`, and either `startDate` + `endDate` or
`measuredAt` (the end) + `value` (minutes).
The row is the one the batch route writes for the same segment, and the
dedupe is the batch route's: the opposite-source same-reading check,
then one reconcile over both unique identities. Without an `externalId`
the server derives one from the stage and both instants, so a re-post
answers 200 `duplicate`, a corrected start answers 200 `updated`, and a
new segment answers 201 `inserted`. The narrow `measurements:write`
token is accepted and still writes only MANUAL rows. Every other body
on this route behaves as before.
Requested in #972.
The validation 422 from `POST /api/measurements/batch` now carries
`meta.errorCode: "measurement.batch.invalid"`, beside the existing
`too_large` and `source_not_permitted` codes, so a client can file the
rows as permanently refused instead of holding its anchor on a guess.
A `value_out_of_range` entry now also carries `convertedValue`, the
reading after conversion into the stored unit, and `range`
(`min`, `max`, `unit`), the band it was checked against. Both appear
only on that skip. A fraction sent where a percent was expected is now
visible in the response instead of looking like a sensor glitch.
Status codes, the envelope and every existing field are unchanged.
Under a `measurements:write` credential the row's source is now decided by
the server on both routes, and the body may not name one at all.
The 422 widens from "names APPLE_HEALTH" to "names any source". It was
narrow only because the forced value was MANUAL, so a body saying MANUAL
was never actually overridden. Now it would be — and overriding a client's
explicit assertion quietly is the thing both routes' comments rule out,
while honouring it would defeat the change: a bridge that kept sending
MANUAL would stay indistinguishable from a typed reading. Status and
errorCode are unchanged, and the documented Home Assistant payload sends no
source at all.
The single-POST predicate keeps reading the raw body: the create schema
defaults `source` to MANUAL, so after parsing "said MANUAL" and "said
nothing" are the same value. Both of that route's create sites resolve the
source, the array arm included — that is the combined BP + Pulse path a
bridge uses, and missing it would leave exactly those rows unbadged.
Two consequences worth naming. Bridge rows no longer join the
MANUAL/APPLE_HEALTH cross-source merge, so they can't be swallowed by a
rule written for the iOS adopt-on-pair case. And a bridge posting at the
same instant as a hand entry now lands both rows instead of a 409, which is
honest and does not double-count: the rollup collapse is per
(type, day, source) and MANUAL outranks the unlisted EXTERNAL.
Adds the `EXTERNAL_SOURCE` leaf beside `scopes.ts` and mirrors the value in
`measurementSourceEnum` so the read shapes and the sync decoder see it. It
stays out of `WRITABLE_MEASUREMENT_SOURCES`: a source a client could assert
would prove nothing about where a reading came from, and that constant is
published to clients as `ingest.writeAllowlist`.
The value-edit gate in `PUT /api/measurements/[id]` needs the other question
answered, though — whether the number is the user's own to fix — so it now
reads a separate `USER_CORRECTABLE_MEASUREMENT_SOURCES`. The 409 that
protects a Withings row exists because the value is the provider's reading;
the hardware behind an ingest token is the user's own scale, and locking
those rows would contradict what the settings card and the Home Assistant
guide promise. That is the only site reading the wider set.
The bulk-delete, restore and by-external-ids suites mocked
`invalidateStatusInsightsForTypes` on the comprehensive-generate barrel,
while the shared post-mutation tail imports it from status-invalidation.
The mock was never called, the real re-warm ran the capability loader
against a stub database, and the "no invalidate" assertion could not
fail. The mock now sits on the imported module; the positive cases
assert the re-warm with the distinct types, the zero-row cases assert
its absence.
Switching a chart from 7 to 30 days moved steps, active energy,
flights and distance totals onto neighbouring days. The 7-day range
buckets raw rows by the user's calendar day; the 30-day range read the
DAY rollup tier, whose buckets are UTC days, and the chart labelled each
bucket's UTC-midnight start in the user's zone. East of UTC a sample
shortly after local midnight counted on the previous day; west of UTC
every bucket read one day early.
The daily series (the chart read and the batched dashboard read) and the
aggregate=daily|weekly|monthly list now fold live in the user's zone and
return the instant of local midnight, so every range and every summed
metric shares one day boundary. The fold runs in Postgres and returns
one row per day; about 0.1 s for 90 days of per-minute samples. Windows
longer than the DAY cap still step up to the coarser rollup tiers.
Refs #1026
The eleven single-record writes answered their shared 429 with prose and
nothing else, while the envelope's own description tells a client to
branch on `meta.errorCode`. On `POST /api/medications/[id]/intake` and
`POST /api/cycle/day-logs` a second, narrower bucket can answer 429 as
well, so the presence of the rate-limit headers was the only thing
telling the two refusals apart.
All eleven now carry `record_write.rate_limited`, the catalogue lists
it, the published 429 names it, and the contract test fails if a site
stops emitting it or the sentence stops publishing it.
The same description now states what a client could not predict from
"300 requests per 60 seconds": refused requests count too, because the
check runs before the body is read.
The batch endpoints have been capped at 60 calls a minute since they
were written. The per-record siblings a naive or older client loops
over — the measurement create, the mood entry, both medication-intake
writes, a lab result, a custom-metric entry, a cycle day-log, a
vaccination, an allergy, a biomarker and an encounter — had no ceiling
at all. That is backwards from the risk: the batch endpoint is the one
a well-behaved client uses, and the unlimited path is the one a broken
loop finds.
All eleven share one generous per-account bucket, `record-write`, at
300 requests a minute. Five writes a second sustained for a full minute
is beyond any hand-driven session and any sync burst, and every one of
these routes is already wrapped in `withIdempotency`, so an honest
retry storm is deduplicated before it is counted — the bucket exists to
cut off the pathological loop, not to shape traffic. It keys on the
ACTING account, the precedent the delegable medication routes set, so a
delegate burns their own allowance rather than locking the record's
owner out of their own log.
The eleven operations publish the bucket name and the cap instead of
the generic refusal, on the shared 429 that already declares
`Retry-After` and the rate-limit triple. A test pulls the key and the
window out of the query the limiter runs and holds the published
sentence to them, so the paragraph cannot drift away from the bucket it
describes.
Fifty-nine 422s raised off a Zod `safeParse` answered with one opaque
sentence and nothing a client could map onto a field, while the other
166 sent `details.issues` — the multi-issue envelope that exists so a
caller with three bad fields fixes all three in one round-trip. Twelve
of the fifty-nine put a dotted machine token in `error` where the
sentence belongs, so `error` was sometimes text to show a person and
sometimes a token that must not be shown, with nothing in the payload
to tell them apart.
Every one of them now answers through `apiValidationError`, which keeps
the existing `error` string and the existing `meta` byte for byte and
adds the issue list beside them. The dotted tokens keep their place in
`error` for the clients that already match on them and are published in
`meta.errorCode` as well, which is the field a machine code belongs in
and the one a new client should read.
A guard reads the route tree per refusal site rather than per file —
per file is what let this grow, since a route that used the helper once
looked compliant to a `grep -l` sweep — and asserts it still finds the
sites it judges, so a matcher that stops matching fails instead of
reporting a clean zero.
The comment on the measurements list justified the tiebreaker with an Apple
Health import stamping long runs with one instant, and said the unique index
forbids two live measurements of the same type at that instant. The index is
`(userId, type, measuredAt, source, sleepStage)`, so the named example is the
one case it does block — while ties survive across `type` (the `stats:` daily
rollups), across `source` (two writers landing together) and across
`sleepStage`. Same conclusion, honest premise.
The per-day drill-down on the same route still sorted on `measuredAt` alone.
It slices in memory rather than with `skip`, so nothing drifts between pages,
but two identical requests were free to render a tied block in different
orders, and a list that reshuffles on refresh reads as data changing.
Three of the tiebreakered routes had nothing pinning the order: custom-metric
entries, the per-medication intake history (both sort arms, so the NULLS LAST
pin cannot be lost to the tiebreaker) and the mental-health assessment
history. Each now asserts the `orderBy` the way the illness day-log test does.
`/api/measurements` and `/api/labs` sort on one non-unique column and
page with `skip`/`take`. When the sort key ties, Postgres does not
promise the same order for the query that fetches page N and the one
that fetches page N+1, so a client paging its own history can receive a
row twice and never receive another — silently, on both sides.
Ties are not hypothetical here. The partial unique index forbids two
live measurements of the same type at one instant, but the tie that
occurs in production is the cross-type one: a daily `stats:` rollup
posts steps, distance, flights and active energy at the identical
timestamp, and an unfiltered history list pages straight through the
block. A lab panel drawn in one sitting shares `takenAt` across every
analyte in it.
Appending `{ id: <same direction> }` to the `orderBy` makes the order
total. Same fix on the five siblings that page the same way — medication
intake history, illness day-logs, custom-metric entries, reminder
history, mental-health assessments — because the defect is the shape,
not the two routes it was found on. No wire change; `/api/sync/changes`
already ordered this way.
The fixtures seed ids in an order unrelated to insertion order, so the
assertion on order inside the tied block can only hold if the secondary
key is doing the work: with the tiebreaker removed the new file fails on
all three cases.
A refused client had nothing to back off against. `rateLimitHeaders()`
emitted `X-RateLimit-Remaining` and an ISO `X-RateLimit-Reset`, no
`Retry-After` and no cap, and only about 70 of the 199 route files that
consult a limiter attached even those two. The rest answered with a bare
prose envelope, including every endpoint an offline replay queue hits:
the four batch ingests, `/api/sync/changes` and `/api/auth/refresh`. A
first-run Apple Health import trips the 60-per-minute batch ceiling
reliably, and the only strategy left to the client was a blind backoff.
Threading headers through 129 handlers is 129 edits the next new route
forgets, so the verdict is carried instead: `checkRateLimit` records it
in a request-scoped cell and `apiHandler` dresses any 429 that leaves
without headers. A route that already built its own is left alone —
`Retry-After` is the presence test, and it comes from the same builder.
`Retry-After` is whole seconds rounding UP (RFC 9110 §10.2.3), floored
at zero. `X-RateLimit-Limit` is new and needs the cap, so the limiter
result carries it. `X-RateLimit-Reset` keeps its ISO form: it is the
wrong convention for the name, but it is on the wire today and changing
it is a decision of its own.
The cell lives in its own leaf module because 177 test files replace
`@/lib/rate-limit` wholesale with a mock factory, and `apiHandler` must
not depend on anything those factories stand in for.
A `z.object()` without `.strict()` strips an unknown key and parses
clean, so a caller that misspells a field name gets a 200 and loses the
value with nothing to tell it. The worst of them was the Coach
question-dismiss endpoint, where an omitted `question` means "dismiss
all": a typo in that key wiped every pending question and answered as
though one had been dismissed.
Twenty-four inline schemas across twenty-two endpoints are strict now —
the settings toggles, the restore and bulk-delete endpoints, message
feedback, the admin user update, the onboarding acknowledgements, the
provider chain, the chart overlay preferences, the medication extractor
and the two client-side reporters.
The strip stays where it carries weight. Sync ingest, device
registration and the layout PUTs are built out of fields that shipped as
additive and optional, on the promise that a server which does not know
one ignores it; an app running ahead of the install it talks to is
routine here rather than exceptional, and refusing would turn a dropped
field into a refused batch — a year of history lost to one row, a
dashboard that will not save, a phone that quietly stops registering for
push. A read filter discards nothing when a stray query parameter rides
along, the OAuth endpoints are required by RFC 6749 and RFC 7591 to
ignore parameters they do not recognise, and an import file comes from
another version or another tool by definition. Each is registered with
its reason in a new inventory test, so a schema cannot join them without
someone making the call. That test parses the tree rather than grepping
it: fourteen of these schemas are written `z\n .object({…})` and a
`z.object(` sweep never saw them.
`sanitiseZodIssues` renders `unrecognized_keys` instead of passing it
through. Zod leaves `path` empty and puts the names in `issue.keys`, so
the sanitised issue named nothing at all and the caller was told only
that its body had failed. The names now ride a bounded `keys` field and
the message is rebuilt from them, capped at ten keys of 64 characters
and restricted to identifier characters, so a caller-supplied key cannot
echo back verbatim through the error path or into the audit ledger. The
names survive `stripValuesFromMessage` because a field name is
structure, not user-typed content.
That file stubs @/lib/api-handler wholesale, so it lists the exports it
knew about when it was written. /api/measurements now reaches for
isScopedCredential to decide source attribution and whether the sync
checkpoint may move, the mock does not provide it, and every test in the
file dies on import rather than on an assertion.
`false` is the honest stub: those cases resolve as a cookie session,
which is exactly what unscoped means, so the route takes its ordinary
path through both rules.
Checked the rest of the tree rather than fixing only the file that went
red — this is the only suite mocking that module for a route which
reaches the new export. The batch route's tests mock at a different
seam and are unaffected.
Not verified here: this environment cannot generate the Prisma client,
so that file is one of the suites which fail to import for an unrelated
reason and the fix could not be run.
Since v1.30.17 there has been no self-service path to a token that can
write a measurement. That release made the Bearer scope default
fail-closed, which closed a real privilege-escalation hole, and removed
the generic mint at POST /api/tokens along with it. The generic mint had
been the only way to get a token that could reach /api/measurements, so
a local sensor pushed through Home Assistant or Node-RED lost its door
while the OAuth integrations kept theirs.
Add a narrow scope accepted by exactly two routes, both writes, both on
the holder's own record.
The scope constant is a leaf module beside the surface that consumes it,
matching FHIR_READ_SCOPE and the MCP scopes. requireRecordAuth grows an
options object rather than a sibling function, because the sharing
surface guard keys every leg on that one identifier and a second name
would mean two copies of the delegable declaration. AuthContext carries
the admitting scope as a single nullable string rather than the token's
permission array, so nothing downstream can re-derive an authorisation
decision from it.
A scoped credential is refused a record selector before any grant is
read. Placing the refusal above the sharing check rather than beside it
means no future widening of what a grant permits can reach one of these
tokens.
Rows are attributed MANUAL. An entry that explicitly names APPLE_HEALTH
is refused 422 rather than relabelled: that source is half the
(userId, type, source, externalId) dedup key the iOS client writes into,
it participates in the cross-source merge, and it gates the HealthKit
sync fields. For the same reason a scoped write no longer advances
User.lastSyncedAt, which the phone reads to size its undelivered window.
Also closes a narrower gap found while building this. The idempotency
wrapper resolved a replayed delegated response against a live grant
without considering which credential was presenting it, so a cached
response about a shared record could be returned to a token the handler
itself would refuse. No write was reachable that way; the response body
was.
Guards: T3 gains the mint site, T4's vocabulary gains the scope and
learns to read the requireRecordAuth form (without which this change
would open a way to widen a route that the closed vocabulary could not
see), and a new T5 freezes the set of delegable routes naming a scope.
The sharing surface guard's AST matcher reads only the first two
arguments, so it needed prose but no code.
The six measurement routes carried eight copy-pasted rollup-recompute +
status-invalidate blocks — the documented two-ended failure class
waiting for the next surface to remember one leg and forget the other.
All of them now call afterMeasurementMutation (the helper the deletion
fix introduced), and the two write surfaces that bypassed the hook
entirely join it: the nightly Recovery/Stress/Strain score rows and the
PHQ-9/GAD-7 assessment projection now recompute their (type, day)
bucket on the tick that writes them.
Every interactive path, every import, MCP and Telegram checked an incoming
measurement against VALUE_RANGES before storing it. The provider sync writers
did not, so a decode slip or a glitched sensor landed a row the application
itself calls impossible, and every mean, median and personal band derived from
it inherited the impossibility. Filtering the read side hid the row. It kept
landing.
The gate is one module reading the domain the app already declares, applied at
two kinds of seam. reconcileExternalMeasurement is the shared identity writer
for Polar, Oura, WHOOP, the Withings sleep leg, the Apple export importer and
the batch endpoint, so gating there means a provider added later inherits the
refusal rather than having to remember a check it never knew about; the new
rejected_range verdict carries no row, which is what makes the compiler force
every caller to handle it. The five writers that hold Prisma themselves gate at
their entry, before the existence probe decides insert versus overwrite, so an
impossible reading can no longer replace a good stored value.
An out-of-band value is dropped. Not clamped, because a clamp invents a reading
nobody took. Not stored behind a flag, because a quarantined row is still a row
and every consumer would have to learn the flag. Not fatal to the batch, because
one bad sample must not strand the run's other readings behind a held watermark.
Absence reads as absence.
A silent drop, though, is indistinguishable from a provider that sent nothing,
so every drop is tallied on the wide event under measurement.range_rejected with
the metric, the provider and the edge it broke. Counts only: the reading itself
is health data and never enters the log line, and the direction plus the
metric's own declared bound is what a diagnosis actually needs. The tally
accumulates rather than replacing, because a provider gates once per resource
leg and the reconciler once per row. One warning per provider lifts the event to
warn so a drop does not hide inside a green line.
A structural guard enumerates all twenty measurement writers with the reason
each is allowed to write, so a new one has to answer the question before it
ships. scripts/diagnose-implausible-measurements.ts reports the rows already
stored, aggregated per metric and source with no value beyond the bucket's own
extremes. It repairs nothing: whether a stored impossible row is a decode slip
or a band that is too narrow is not a call a script gets to make.
The batch path had no unique-violation handling at all, so one duplicate
anywhere in the transaction answered 500 — indistinguishable from a
server fault, and what every client retries against. The single path
narrowed with instanceof, the form prisma-errors.ts exists to replace.
A grant has meant one thing since it shipped: the whole record, to read or to
add to. This lays the words for the two shapes the rest of the sharing work
needs — part of a record, and management of one — while changing what no
existing grant does.
`ShareDomain` is a closed set of eight sections in `src/lib/sharing/scope.ts`,
clustered from the delegable read surfaces rather than invented, plus the
distinguished `record` for the routes that read across them. It never converts
to a `ModuleKey` and never imports one: two masks, two questions, no shared
spelling, because the day they share a name is the day somebody makes one
imply the other, and a module the owner enabled is not a section the owner
shared.
`account_grants.scope_json` is NULL on every row that exists, and NULL is the
entire record — the answer those rows were consented as, so no backfill and no
re-consent. A non-NULL value is read through one fail-closed normaliser: a
blob it cannot parse opens nothing, including a section the blob happened to
name. That is the deliberate inverse of the module preference normaliser,
which fails open on purpose; both do the safe thing with a value nobody can
read, and the safe thing differs because one answers an authorization question
and the other a presentation question.
`MANAGE` joins the level enum, ordered above WRITE so a write grant can never
satisfy it, and carries no scope at all: management of part of a record
promises a boundary that cannot survive an edit, so the invite path refuses to
write one.
`requireRecordAuth` takes the section as a second required argument. The
requirement is the lever — a delegable route without a classification does not
typecheck, so the set of classified routes cannot fall behind the set of
delegable ones. All fifty-seven modules carry theirs. The structural guard
freezes each against the production vocabulary, fails when a module's own arms
disagree, and pins a non-zero count of resolved sections so a matcher that
stopped reading the argument fails instead of agreeing with everything.
`requireGuardianAuth` is exported and reaches nothing yet. It gates on the
managed-profile marker rather than on the grant, so an adult who granted
management of their own record cannot arrive at that record's settings by any
argument about what MANAGE means: widening MANAGE would widen the record
resolver and would not touch this one. Its frozen list lands empty, with the
declaration site as the positive control that keeps "nothing reaches it" a
measurement rather than a renamed symbol.
`users.managed_profile_at` says since when a record is one somebody else runs.
A timestamp and not a boolean, for a reason the schema has to keep possible:
clearing it is the handover, and the grants and the activity trail span it
untouched.
Marc-André Bombeck