Switching a chart from 7 to 30 days moved steps, active energy,
flights and distance totals onto neighbouring days. The 7-day range
buckets raw rows by the user's calendar day; the 30-day range read the
DAY rollup tier, whose buckets are UTC days, and the chart labelled each
bucket's UTC-midnight start in the user's zone. East of UTC a sample
shortly after local midnight counted on the previous day; west of UTC
every bucket read one day early.
The daily series (the chart read and the batched dashboard read) and the
aggregate=daily|weekly|monthly list now fold live in the user's zone and
return the instant of local midnight, so every range and every summed
metric shares one day boundary. The fold runs in Postgres and returns
one row per day; about 0.1 s for 90 days of per-minute samples. Windows
longer than the DAY cap still step up to the coarser rollup tiers.
Refs #1026
The eleven single-record writes answered their shared 429 with prose and
nothing else, while the envelope's own description tells a client to
branch on `meta.errorCode`. On `POST /api/medications/[id]/intake` and
`POST /api/cycle/day-logs` a second, narrower bucket can answer 429 as
well, so the presence of the rate-limit headers was the only thing
telling the two refusals apart.
All eleven now carry `record_write.rate_limited`, the catalogue lists
it, the published 429 names it, and the contract test fails if a site
stops emitting it or the sentence stops publishing it.
The same description now states what a client could not predict from
"300 requests per 60 seconds": refused requests count too, because the
check runs before the body is read.
The batch endpoints have been capped at 60 calls a minute since they
were written. The per-record siblings a naive or older client loops
over — the measurement create, the mood entry, both medication-intake
writes, a lab result, a custom-metric entry, a cycle day-log, a
vaccination, an allergy, a biomarker and an encounter — had no ceiling
at all. That is backwards from the risk: the batch endpoint is the one
a well-behaved client uses, and the unlimited path is the one a broken
loop finds.
All eleven share one generous per-account bucket, `record-write`, at
300 requests a minute. Five writes a second sustained for a full minute
is beyond any hand-driven session and any sync burst, and every one of
these routes is already wrapped in `withIdempotency`, so an honest
retry storm is deduplicated before it is counted — the bucket exists to
cut off the pathological loop, not to shape traffic. It keys on the
ACTING account, the precedent the delegable medication routes set, so a
delegate burns their own allowance rather than locking the record's
owner out of their own log.
The eleven operations publish the bucket name and the cap instead of
the generic refusal, on the shared 429 that already declares
`Retry-After` and the rate-limit triple. A test pulls the key and the
window out of the query the limiter runs and holds the published
sentence to them, so the paragraph cannot drift away from the bucket it
describes.
Fifty-nine 422s raised off a Zod `safeParse` answered with one opaque
sentence and nothing a client could map onto a field, while the other
166 sent `details.issues` — the multi-issue envelope that exists so a
caller with three bad fields fixes all three in one round-trip. Twelve
of the fifty-nine put a dotted machine token in `error` where the
sentence belongs, so `error` was sometimes text to show a person and
sometimes a token that must not be shown, with nothing in the payload
to tell them apart.
Every one of them now answers through `apiValidationError`, which keeps
the existing `error` string and the existing `meta` byte for byte and
adds the issue list beside them. The dotted tokens keep their place in
`error` for the clients that already match on them and are published in
`meta.errorCode` as well, which is the field a machine code belongs in
and the one a new client should read.
A guard reads the route tree per refusal site rather than per file —
per file is what let this grow, since a route that used the helper once
looked compliant to a `grep -l` sweep — and asserts it still finds the
sites it judges, so a matcher that stops matching fails instead of
reporting a clean zero.
The comment on the measurements list justified the tiebreaker with an Apple
Health import stamping long runs with one instant, and said the unique index
forbids two live measurements of the same type at that instant. The index is
`(userId, type, measuredAt, source, sleepStage)`, so the named example is the
one case it does block — while ties survive across `type` (the `stats:` daily
rollups), across `source` (two writers landing together) and across
`sleepStage`. Same conclusion, honest premise.
The per-day drill-down on the same route still sorted on `measuredAt` alone.
It slices in memory rather than with `skip`, so nothing drifts between pages,
but two identical requests were free to render a tied block in different
orders, and a list that reshuffles on refresh reads as data changing.
Three of the tiebreakered routes had nothing pinning the order: custom-metric
entries, the per-medication intake history (both sort arms, so the NULLS LAST
pin cannot be lost to the tiebreaker) and the mental-health assessment
history. Each now asserts the `orderBy` the way the illness day-log test does.
`/api/measurements` and `/api/labs` sort on one non-unique column and
page with `skip`/`take`. When the sort key ties, Postgres does not
promise the same order for the query that fetches page N and the one
that fetches page N+1, so a client paging its own history can receive a
row twice and never receive another — silently, on both sides.
Ties are not hypothetical here. The partial unique index forbids two
live measurements of the same type at one instant, but the tie that
occurs in production is the cross-type one: a daily `stats:` rollup
posts steps, distance, flights and active energy at the identical
timestamp, and an unfiltered history list pages straight through the
block. A lab panel drawn in one sitting shares `takenAt` across every
analyte in it.
Appending `{ id: <same direction> }` to the `orderBy` makes the order
total. Same fix on the five siblings that page the same way — medication
intake history, illness day-logs, custom-metric entries, reminder
history, mental-health assessments — because the defect is the shape,
not the two routes it was found on. No wire change; `/api/sync/changes`
already ordered this way.
The fixtures seed ids in an order unrelated to insertion order, so the
assertion on order inside the tied block can only hold if the secondary
key is doing the work: with the tiebreaker removed the new file fails on
all three cases.
A refused client had nothing to back off against. `rateLimitHeaders()`
emitted `X-RateLimit-Remaining` and an ISO `X-RateLimit-Reset`, no
`Retry-After` and no cap, and only about 70 of the 199 route files that
consult a limiter attached even those two. The rest answered with a bare
prose envelope, including every endpoint an offline replay queue hits:
the four batch ingests, `/api/sync/changes` and `/api/auth/refresh`. A
first-run Apple Health import trips the 60-per-minute batch ceiling
reliably, and the only strategy left to the client was a blind backoff.
Threading headers through 129 handlers is 129 edits the next new route
forgets, so the verdict is carried instead: `checkRateLimit` records it
in a request-scoped cell and `apiHandler` dresses any 429 that leaves
without headers. A route that already built its own is left alone —
`Retry-After` is the presence test, and it comes from the same builder.
`Retry-After` is whole seconds rounding UP (RFC 9110 §10.2.3), floored
at zero. `X-RateLimit-Limit` is new and needs the cap, so the limiter
result carries it. `X-RateLimit-Reset` keeps its ISO form: it is the
wrong convention for the name, but it is on the wire today and changing
it is a decision of its own.
The cell lives in its own leaf module because 177 test files replace
`@/lib/rate-limit` wholesale with a mock factory, and `apiHandler` must
not depend on anything those factories stand in for.
A `z.object()` without `.strict()` strips an unknown key and parses
clean, so a caller that misspells a field name gets a 200 and loses the
value with nothing to tell it. The worst of them was the Coach
question-dismiss endpoint, where an omitted `question` means "dismiss
all": a typo in that key wiped every pending question and answered as
though one had been dismissed.
Twenty-four inline schemas across twenty-two endpoints are strict now —
the settings toggles, the restore and bulk-delete endpoints, message
feedback, the admin user update, the onboarding acknowledgements, the
provider chain, the chart overlay preferences, the medication extractor
and the two client-side reporters.
The strip stays where it carries weight. Sync ingest, device
registration and the layout PUTs are built out of fields that shipped as
additive and optional, on the promise that a server which does not know
one ignores it; an app running ahead of the install it talks to is
routine here rather than exceptional, and refusing would turn a dropped
field into a refused batch — a year of history lost to one row, a
dashboard that will not save, a phone that quietly stops registering for
push. A read filter discards nothing when a stray query parameter rides
along, the OAuth endpoints are required by RFC 6749 and RFC 7591 to
ignore parameters they do not recognise, and an import file comes from
another version or another tool by definition. Each is registered with
its reason in a new inventory test, so a schema cannot join them without
someone making the call. That test parses the tree rather than grepping
it: fourteen of these schemas are written `z\n .object({…})` and a
`z.object(` sweep never saw them.
`sanitiseZodIssues` renders `unrecognized_keys` instead of passing it
through. Zod leaves `path` empty and puts the names in `issue.keys`, so
the sanitised issue named nothing at all and the caller was told only
that its body had failed. The names now ride a bounded `keys` field and
the message is rebuilt from them, capped at ten keys of 64 characters
and restricted to identifier characters, so a caller-supplied key cannot
echo back verbatim through the error path or into the audit ledger. The
names survive `stripValuesFromMessage` because a field name is
structure, not user-typed content.
That file stubs @/lib/api-handler wholesale, so it lists the exports it
knew about when it was written. /api/measurements now reaches for
isScopedCredential to decide source attribution and whether the sync
checkpoint may move, the mock does not provide it, and every test in the
file dies on import rather than on an assertion.
`false` is the honest stub: those cases resolve as a cookie session,
which is exactly what unscoped means, so the route takes its ordinary
path through both rules.
Checked the rest of the tree rather than fixing only the file that went
red — this is the only suite mocking that module for a route which
reaches the new export. The batch route's tests mock at a different
seam and are unaffected.
Not verified here: this environment cannot generate the Prisma client,
so that file is one of the suites which fail to import for an unrelated
reason and the fix could not be run.
Since v1.30.17 there has been no self-service path to a token that can
write a measurement. That release made the Bearer scope default
fail-closed, which closed a real privilege-escalation hole, and removed
the generic mint at POST /api/tokens along with it. The generic mint had
been the only way to get a token that could reach /api/measurements, so
a local sensor pushed through Home Assistant or Node-RED lost its door
while the OAuth integrations kept theirs.
Add a narrow scope accepted by exactly two routes, both writes, both on
the holder's own record.
The scope constant is a leaf module beside the surface that consumes it,
matching FHIR_READ_SCOPE and the MCP scopes. requireRecordAuth grows an
options object rather than a sibling function, because the sharing
surface guard keys every leg on that one identifier and a second name
would mean two copies of the delegable declaration. AuthContext carries
the admitting scope as a single nullable string rather than the token's
permission array, so nothing downstream can re-derive an authorisation
decision from it.
A scoped credential is refused a record selector before any grant is
read. Placing the refusal above the sharing check rather than beside it
means no future widening of what a grant permits can reach one of these
tokens.
Rows are attributed MANUAL. An entry that explicitly names APPLE_HEALTH
is refused 422 rather than relabelled: that source is half the
(userId, type, source, externalId) dedup key the iOS client writes into,
it participates in the cross-source merge, and it gates the HealthKit
sync fields. For the same reason a scoped write no longer advances
User.lastSyncedAt, which the phone reads to size its undelivered window.
Also closes a narrower gap found while building this. The idempotency
wrapper resolved a replayed delegated response against a live grant
without considering which credential was presenting it, so a cached
response about a shared record could be returned to a token the handler
itself would refuse. No write was reachable that way; the response body
was.
Guards: T3 gains the mint site, T4's vocabulary gains the scope and
learns to read the requireRecordAuth form (without which this change
would open a way to widen a route that the closed vocabulary could not
see), and a new T5 freezes the set of delegable routes naming a scope.
The sharing surface guard's AST matcher reads only the first two
arguments, so it needed prose but no code.
The six measurement routes carried eight copy-pasted rollup-recompute +
status-invalidate blocks — the documented two-ended failure class
waiting for the next surface to remember one leg and forget the other.
All of them now call afterMeasurementMutation (the helper the deletion
fix introduced), and the two write surfaces that bypassed the hook
entirely join it: the nightly Recovery/Stress/Strain score rows and the
PHQ-9/GAD-7 assessment projection now recompute their (type, day)
bucket on the tick that writes them.
Every interactive path, every import, MCP and Telegram checked an incoming
measurement against VALUE_RANGES before storing it. The provider sync writers
did not, so a decode slip or a glitched sensor landed a row the application
itself calls impossible, and every mean, median and personal band derived from
it inherited the impossibility. Filtering the read side hid the row. It kept
landing.
The gate is one module reading the domain the app already declares, applied at
two kinds of seam. reconcileExternalMeasurement is the shared identity writer
for Polar, Oura, WHOOP, the Withings sleep leg, the Apple export importer and
the batch endpoint, so gating there means a provider added later inherits the
refusal rather than having to remember a check it never knew about; the new
rejected_range verdict carries no row, which is what makes the compiler force
every caller to handle it. The five writers that hold Prisma themselves gate at
their entry, before the existence probe decides insert versus overwrite, so an
impossible reading can no longer replace a good stored value.
An out-of-band value is dropped. Not clamped, because a clamp invents a reading
nobody took. Not stored behind a flag, because a quarantined row is still a row
and every consumer would have to learn the flag. Not fatal to the batch, because
one bad sample must not strand the run's other readings behind a held watermark.
Absence reads as absence.
A silent drop, though, is indistinguishable from a provider that sent nothing,
so every drop is tallied on the wide event under measurement.range_rejected with
the metric, the provider and the edge it broke. Counts only: the reading itself
is health data and never enters the log line, and the direction plus the
metric's own declared bound is what a diagnosis actually needs. The tally
accumulates rather than replacing, because a provider gates once per resource
leg and the reconciler once per row. One warning per provider lifts the event to
warn so a drop does not hide inside a green line.
A structural guard enumerates all twenty measurement writers with the reason
each is allowed to write, so a new one has to answer the question before it
ships. scripts/diagnose-implausible-measurements.ts reports the rows already
stored, aggregated per metric and source with no value beyond the bucket's own
extremes. It repairs nothing: whether a stored impossible row is a decode slip
or a band that is too narrow is not a call a script gets to make.
The batch path had no unique-violation handling at all, so one duplicate
anywhere in the transaction answered 500 — indistinguishable from a
server fault, and what every client retries against. The single path
narrowed with instanceof, the form prisma-errors.ts exists to replace.
A grant has meant one thing since it shipped: the whole record, to read or to
add to. This lays the words for the two shapes the rest of the sharing work
needs — part of a record, and management of one — while changing what no
existing grant does.
`ShareDomain` is a closed set of eight sections in `src/lib/sharing/scope.ts`,
clustered from the delegable read surfaces rather than invented, plus the
distinguished `record` for the routes that read across them. It never converts
to a `ModuleKey` and never imports one: two masks, two questions, no shared
spelling, because the day they share a name is the day somebody makes one
imply the other, and a module the owner enabled is not a section the owner
shared.
`account_grants.scope_json` is NULL on every row that exists, and NULL is the
entire record — the answer those rows were consented as, so no backfill and no
re-consent. A non-NULL value is read through one fail-closed normaliser: a
blob it cannot parse opens nothing, including a section the blob happened to
name. That is the deliberate inverse of the module preference normaliser,
which fails open on purpose; both do the safe thing with a value nobody can
read, and the safe thing differs because one answers an authorization question
and the other a presentation question.
`MANAGE` joins the level enum, ordered above WRITE so a write grant can never
satisfy it, and carries no scope at all: management of part of a record
promises a boundary that cannot survive an edit, so the invite path refuses to
write one.
`requireRecordAuth` takes the section as a second required argument. The
requirement is the lever — a delegable route without a classification does not
typecheck, so the set of classified routes cannot fall behind the set of
delegable ones. All fifty-seven modules carry theirs. The structural guard
freezes each against the production vocabulary, fails when a module's own arms
disagree, and pins a non-zero count of resolved sections so a matcher that
stopped reading the argument fails instead of agreeing with everything.
`requireGuardianAuth` is exported and reaches nothing yet. It gates on the
managed-profile marker rather than on the grant, so an adult who granted
management of their own record cannot arrive at that record's settings by any
argument about what MANAGE means: widening MANAGE would widen the record
resolver and would not touch this one. Its frozen list lands empty, with the
declaration site as the positive control that keeps "nothing reaches it" a
measurement rather than a renamed symbol.
`users.managed_profile_at` says since when a record is one somebody else runs.
A timestamp and not a boolean, for a reason the schema has to keep possible:
clearing it is the handover, and the grants and the activity trail span it
untouched.
Both POST arms of /api/measurements resolve the record rather than the
caller. Every `userId: user.id` in the handler already meant "the record",
so the bodies are untouched: the safety-floor check, the reminder
satisfaction, the rollup recompute and the cache eviction are statements
about the record and stay addressed to it.
The two validation breadcrumbs move from a bare client create to
auditLog(). That helper is the only writer that stamps the acting account,
and a delegate's malformed payload filed without it reads as the owner's
own typing.
Six routes move from `requireAuth()` to `requireRecordAuth("read")`: the
measurement list, the single reading, the two series reads, and the two
reminder reads. Each is admitted in the delegable classification with its
five answers written; each enters the frozen list with the argument for why
a delegate reading through it cannot extend their own reach.
Only the GET arms move. The POST, PUT and DELETE arms in the same files keep
`requireAuth()`, which refuses outright while a switch is active, so a file
entering the list admits a read and nothing else.
The two validation breadcrumbs on the list and series reads now write through
`auditLog()` rather than a bare `prisma.auditLog.create`. That helper is the
only writer that stamps `actorUserId`, and without the stamp a delegate's
malformed query lands on the owner's record looking like the owner's own.
Integration coverage drives the shipped GET exports against real Postgres:
the owner's rows reach a delegate holding a live grant, the delegate's own
rows do not, a caller without a grant is refused `sharing.access.denied`, the
request after a revocation is refused with no re-login, and a caller who
never switched sees exactly what they saw before.
A GET must not have side effects — api-handler.ts says so next to
READ_HTTP_METHODS, where an MCP-audience token is admitted on the safe
methods precisely because they are assumed to write nothing. Three routes
broke that: the sync handshake bumped User.lastSyncedAt, the achievements
read inserted unlock rows, and the cycle calendar fired a discarded promise
at the forecast cache so a failed write left no trace anywhere.
Each write moves to the place that causes it.
The sync checkpoint is now stamped by the batch ingest endpoints for
measurements, mood and intakes. The handshake reports it. That also fixes
the watermark: the checkpoint used to advance past a window the client had
not drained, and a client trusting it would skip that window. It can only
trail a real sync now.
The unlock rows move to a nightly achievement-unlock-sweep. The badge grid
never needed them — unlocked comes from the live metrics and the completion
date is derived — so the payload is unchanged; the rows pin that date and
carry it into the backup, and the sweep reaches accounts that never open
the page.
The cycle forecast moves to a nightly cycle-prediction-refresh. The
CyclePrediction row is not a cache of a read: it is the index the reminder
cron scans, so which reminders an account received depended on whether
somebody had opened the calendar. persistPredictionCache no longer
swallows its errors either — the job reports them.
The batch route has accepted a `syncTrigger` since v1.32.8 and put it on the
wide event, where no human reads it. Half of what was promised got built: the
value is written and appears in three places in the codebase, none of which is
a screen anyone looks at.
Persist it on the account, alongside the instant of the most recent batch that
arrived without the app being opened, and publish both from
`GET /api/integrations/healthkit`. The card now names how the last sync arrived
and when data last came in on its own. `healthKitLastSyncedAt` alone reads
identically whether the phone delivers by itself or only while someone has the
app open, which is why "is background delivery working" could not be settled
from either side.
A batch that declares no trigger writes null rather than leaving the previous
one standing — the field describes this sync, and an older client's silence is
not evidence about it. A foreground batch leaves the background timestamp
alone; overwriting it would erase the only answer the card has.
Also here: WHOOP, Fitbit and Google Health join the workout-freshness table.
All three write workouts, and without an entry their cards could report every
measurement type fresh while the workout leg had been silent for weeks. And the
status pill drops a generic `aria-label` that replaced its own accessible name,
so a screen reader announced the same words for a healthy connection as for a
broken one.
`failing_leg` was one slot, so two legs of the same provider failing in
overlapping windows overwrote each other. The row remembered whichever failed
last, and that leg recovering then cleared an error the other one was still
causing — the ledger forgot a live failure and painted the card green over it.
The slot becomes a set. A failure adds its leg, a success removes its own, and
the error state, the per-kind buckets and the streak anchor clear only once the
set is empty. The array is ordered oldest failure first, so the last element is
the leg that wrote the message `last_error` holds; when that leg recovers while
others are still failing, the message is cleared rather than left describing a
leg that is now healthy.
An unattributed failure adds nothing and removes nothing. A single-leg provider
therefore behaves exactly as before, and a multi-leg provider no longer loses
what its other legs already reported.
The column never reached a client — it is absent from the status serialiser and
from the OpenAPI contract — so it is replaced rather than shimmed.
An external id is half of every ingest dedup key — (userId, source,
externalId), (userId, externalSource, externalId), (userId, type,
source, externalId). Re-posting the same triple is idempotent and
returns the existing row, which holds only while the id is stable
across client launches. A client that serialised an opaque platform
object with its default object description sent a per-allocation
memory address instead, so no sweep ever matched its own earlier row
and each one minted a fresh record: a live instance grew 23 phantom
medications in a day, none of which could collect a dose, because the
dose side resolved through the same rotating string.
Add one shared classifier for identifiers that provably cannot be an
identity — an angle-bracketed object description carrying a hex
pointer, a bare 0x address, and a blank value — and wire it at every
surface that accepts a client-supplied externalId.
Single-entry creates refuse through the schema, so the client gets the
standard 422 multi-issue envelope with a message naming the shape it
sent and what to send instead. Batch and bulk surfaces classify per
entry and report the offending row as skipped with reason
unstable_external_id, leaving the rest of the payload to land: one bad
row must not stop a sync. Every refusal carries pinned wide-event meta
(count, shape, surface) and never the id itself.
A structural guard freezes both routes into the floor: a request-side
externalId declaration either applies the field check or carries a
marker naming the file that checks it per entry, and the guard verifies
that file really does. The backup-restore payload is the one exemption,
with its reason recorded — it replays ids the server itself wrote.
Migration 0263 added three columns beside the Apple Health `stats:*`
authority ladder. Only `aggregation_provenance` is ever read back: the
external-measurement reconciler selects it and refuses a lower-authority write
against a higher-authority canonical row, and the cumulative drain reads it
too. That column stays.
`aggregation_contributor_count` and `aggregation_selected_source_hash` are
written by three call sites and read by none. No select, no raw SQL, no export,
no backup, no API contract, no repair path. The import-time selection keeps its
deterministic tiebreak, which compares an in-memory map key computed per flush
rather than a value read back from the table.
Both are plain nullable scalars with no index, unique, check or foreign key, so
the drop is a catalog-only operation that touches no measured value.
Route-level tests that run the real apiHandler + requireAuth so the
cookie-vs-Bearer derivation is exercised end to end, not papered over by a
unit mock:
- a cookie POST records WEB, a Bearer POST records API, and a body that
forges `source` is ignored;
- the dose-history ledger selects and returns `source` per row, null on a
legacy row;
- a batch carrying `syncTrigger:"background"` rides the ingest annotation,
a batch without it annotates null, an invalid value is 422, and the field
leaves the per-row insert/update outcomes byte-identical.
Root-cause the intake source attribution and give the HealthKit batch a
diagnostic sync-trigger, one attribution surface for both.
AuthContext now carries a read-only authMethod ("cookie" | "bearer"),
derived from the same branch that sets the wide event's auth_method. The
medication-intake route maps it to the stored IntakeSource: a Bearer
(native) call records API, a cookie (browser) call records WEB. The other
producer values are untouched — a converge onto a pending reminder row
updates through applyCanonicalSlotWrite, which never rewrites source, so
REMINDER / IMPORT / APPLE_HEALTH provenance is preserved. The value is
never client-asserted, the same posture as userId being narrowed from auth.
The dose-history ledger now selects and returns source per intake row so a
client can label how each dose was recorded. The endpoint gains its first
OpenAPI entry (path plus response schema) so the field lands in the
iOS-consumed contract.
The measurements batch accepts an optional per-request syncTrigger
(foreground | background | push). It is diagnostic-only: recorded on the
measurement.batch.ingest wide event and nothing else — never persisted,
never part of the dedup key, never an input to attribution or how a sample
is stored. Sending it or omitting it produces byte-identical row outcomes.
Regenerated docs/api/openapi.yaml.
Merge the eight reviewed audit fixes that were cut a week ago and never
landed. Main has moved on since, so each fix was re-checked against the
current tree and each of the five conflicts resolved by understanding
both sides.
Kept, still open on main:
- Proxy early exits now carry the transport security header subset via a
shared helper. Reconciled with main's /api/health worker-only carve-out.
- Batch measurement ingest bounds the entry instant so a future-dated row
can no longer win "latest reading" or poison the rollup buckets.
- Web Push dials through the pinned safeFetch dispatcher with
requirePublicHost, closing the last user-controlled-host egress.
- Mood bulk upsert and its tag/factor links run in one transaction, so a
rejected rating rolls the mood row back instead of leaving a tagless row.
- Full backup carries NutrientIntakeDay with its unit and source. Folded
into main's portable-export / disaster-recovery split.
- Live daily-series fallback restores the min/max spread band the tiered
read already emitted.
- Coach stream aborts its in-flight reader on unmount and resets send state
on a conversation switch.
- Manual water writes share the day-bound helper, nutrient add clamps after
the increment, and the thiamin cap drops to a real supplement ceiling.
- Compliance point window widens symmetrically instead of wrapping overnight.
- Oura, Polar and Fitbit token helpers classify an empty or non-JSON 2xx
body as a handled transient error.
- WebAuthn relying-party fallback to localhost is gated to development.
- Chart-token strip list is derived from the allowlist and pinned by a test.
- Regression tests pin the snapshot-cache tenant key, the GlitchTip
path-secret redaction and the Withings credentials delete.
Superseded by main, tests re-pointed at the current shape:
- The intake CSV import was rewritten on main to run through a background
job with a single-active-job admission gate. The future-date bound is
re-applied at request validation and the per-user limiter is kept for
parity with the bulk twin; the old synchronous-write test is replaced.
Related to #585. Investigated the intraday pulse chart's read path
end to end: the ZIP-import raw-sample fold, the live-sync uploaded
10-min bucket fold, their merge, and the chart's render logic. All of
it treats the two ingestion shapes consistently — the chart has no
shape awareness at all, it paints whatever bucket series it receives.
No read-side or display-side defect explains a systematic density
difference between the two ingestion paths.
What was missing is visibility into the write and read sides that
would let a future report actually be diagnosed instead of
re-investigated from scratch. The batch endpoint's wide-event trace
never broke a skip down by reason, and never distinguished an
HR-bucket write from any other entry in the same request. The
intraday route's own trace surfaced only the final bucket count, with
no visibility into which shape (raw sample rows vs uploaded bucket
rows vs folded hourly rows) a day's data actually came from. Both
traces now carry that breakdown, so the next report of "the chart
looks sparser after a given date" is answerable by grep instead of by
re-deriving the whole read path again.
`classifyIntakeTiming` treated `windowStart === windowEnd` as an overnight
wrap, inflating a degenerate point window into a ~30h on-time band — a dose
taken many hours late still classified `on_time` and fed a perfect-compliance
streak. Widen the point window symmetrically instead, matching the treatment
`resolveLegacyWindowBounds` already applies.
`/api/measurements/batch` never applied the entry-instant plausibility bound
its single-entry twin has carried since v1.17 W1b, so a future-dated row landed
and permanently won 'latest reading' while poisoning the rollup buckets and the
canonical picker. Skip per entry, like the range guard. The past floor stays at
1900 — a historical backfill is a legitimate shape here.
The intake CSV import had neither an upper date bound nor a rate limiter, while
running up to 1000 sequential find + create + consume iterations per call. Add
both, on the bulk twin's bucket shape and budget.
Every ingest path in the product can now emit a typed, salience-classified
event into pg-boss, consumed by one worker that claims a durable per-day
reaction marker and fans out to the reaction surfaces.
The recency classifier runs at the seam, before anything is enqueued, so a
mass backfill — a multi-year Apple Health export, a provider re-sync, a
30-day catch-up — emits nothing at all. Non-salient writes are annotated and
dropped.
The worker makes no provider call by construction: it imports no provider
module, writes only markers, and enqueues nothing that is not budget-gated
downstream. Business refusals return a status rather than throwing, so a
refusal is never retried against a ceiling that will not move until the local
day rolls over.
Seams: sleep (one hook, inherited by all seven transports), workouts (the
batch route, the four provider syncs, the export importer), weight and blood
pressure (both measurement routes), labs (manual, OCR commit, document fact
commit).
The provider workout syncs could not previously tell a created row from an
updated one, so a re-sync of a recent session would have re-fired on every
poll; they now recover the distinction from the row the upsert already
returns.
Reaction lines are AES-256-GCM at rest, following the InsightNarrative
precedent, and the column is registered for key rotation. Markers are pruned
after 14 days by a daily maintenance job.
Each case fails on the pre-fix code: the rollup tests on an unaligned
recompute window, the batch tests on the older stats: snapshot winning,
and the resting test on a series that stops at the fold horizon.
Two entries carrying the same stats: key in one batch both miss the
existing-row probe, so the overwrite branch never runs, both land in the
bulk insert, and skipDuplicates keeps whichever the unique index saw
first. The older snapshot of a per-day cumulative total or of a filling
ten-minute heart-rate bucket therefore won and the newer, larger figure
was dropped silently — both entries reported inserted, the counters
still summed, and the client checkpointed past both.
stats: rows carry overwrite semantics by contract, so within one batch
the last entry for a key is the authoritative one. Resolve it up front
and report the superseded earlier entries as duplicate with a
superseded_in_batch reason, a status the client already checkpoints
past.
Kill the above-the-fold first-load flash on `/`: the hero digest and the
batched chart series still fired client-side after JS download + hydrate,
so the hero and charts painted empty and filled in seconds later (LCP
~5.5 s mobile-throttled while the snapshot already dehydrated).
Extend the existing `page.tsx` snapshot prefetch to also seed the daily
digest and the chart series batch into the same dehydrated TanStack cache,
under keys byte-identical to the client's:
- Extract the series-batch route core into `readSeriesBatch(...)` so the
RSC and the route share one code path (no HTTP-to-self); the route thins
to auth + Zod + the shared reader, behaviour unchanged.
- New client-safe pure module `batch-chart-types.ts`: `deriveBatchChartTypes`
(the single implementation of the visible-with-data type-set derivation,
order + resting-HR-vs-pulse pick preserved) and `computeBatchWindow`
(profile-tz end-of-day minus the coverage window). Server and client run
the same function over the same snapshot payload, and the window is
computed once server-side and threaded to the client as a prop, so the
`chartSeriesBatch` key matches on both sides by construction.
- Render the hero data-first so the SSR HTML carries the populated hero
(the LCP element) — hydration-safe because the digest is dehydrated and
never restored from the IndexedDB persist allowlist; bound that invariant
with comments on both the hero gate and the allowlist.
- Warm the shared chart-runtime chunk during hydration.
Both prefetches honour the `DASHBOARD_SSR_PREFETCH=false` escape and fail
soft to the client-fetch path on any error; a ~400 ms soft budget bounds
the cold-day digest cost. No contract change, no visual change.
Two data-integrity gaps found in review of the v1.30.7 contract:
- The bucket externalId guard accepted a non-canonical timestamp that V8
silently rolls over (T24:00, 04-31, non-leap 02-29). Such a row would key
under a literal string the canonical client never emits, so the two would
never collapse on the overwrite key, and it would plot on the wrong local
day. Require canonical round-trip (toISOString() === suffix) in both the
guard and the bucket-start parser.
- The per-bucket min/max bypassed the plausibility-range guard that value
passes; now that they drive the DAY min/max band and the intraday envelope,
an out-of-range or mis-ordered spread poisoned the aggregate. Persist the
spread only when both extremes are in range and ordered around value;
otherwise drop to null and keep the average.
Refine the go-forward aggregated HR wire contract (iOS #34) from the never-
adopted hourly bucket to a 10-minute UTC bucket, so one contract serves both
the measurements-list de-explosion and the intraday-pulse 10-min curve.
- Batch route accepts stats:HKQuantityTypeIdentifierHeartRate:<10-min ISO-Z>
with avg + min/max, overwrite semantics, malformed-suffix rejection; the
validator is identifier-generic (allowlist) so respiratoryRate/SpO2 can join
later without a grammar change.
- The intraday reader classifies a day's PULSE rows three ways by externalId
shape (raw / uploaded 10-min bucket / server-folded hourly), builds the
10-min curve directly from uploaded buckets with the min/max envelope and
tension detection enabled, and overlays buckets over raw on the one
cutover-straddling day.
- DAY rollup min/max read COALESCE(value_min, value) so a bucket-native day's
band reflects the true intra-bucket extremes.
No schema change (value_min/value_max exist since v1.19.2). Reconciliation is
go-forward cutover; the nightly retention fold handles the raw backlog.
The raw-row read backing the day-collapsed list is desc + take(limit).
On a dense account whose in-window row count crosses the cap, the
oldest returned row sits mid-day — every earlier sample on that same
calendar day was cut off, so its bucket's SUM silently understates the
real day total with no marker.
Flags exactly that one bucket partial: true when the cap was actually
hit, so a page that came in under the cap (the whole window covered)
carries no flag on any bucket.
Best-effort background work was swallowed with `.catch(() => {})` — a
persistent failure (a broken reminder enqueue, a lost re-auth transition)
left no annotate() or logger trail, against the project's own
'annotate() on every interesting path' rule.
Add `fireAndForget(promise, { action })`: it annotates the rejection into
the current wide event under a per-action meta key (so it rides the
central redaction at emit time — the raw message never hits the console)
and warns the action label only for failures that settle after the event
flushed. Control flow is unchanged; the promise stays fire-and-forget.
Adopt it at the meaningful sites: reminder-satisfy enqueues (measurement
+ labs writes and the five integration sync workers), the post-write
safety-floor dispatch, and the consent audit-log write. The Withings
re-auth mark stays awaited (its persisted state feeds the same response)
but now breadcrumbs its failure instead of swallowing it.
Four data-correctness fixes in the same class as the workouts-list
ordering bug (05f80ad37):
- GET /api/measurements groupBy=day: the raw-row scan ordered oldest
first and sliced to `limit` before bucketing, so a long history never
surfaced its recent days. Reads newest-first now. The per-day sum
also ran over every source unfiltered — an Apple Health + Withings
write for the same day doubled the total. Routes through
pickCanonicalSourceRows first, matching the rollup tile's collapse.
- PR detection's cumulative-day picker summed every row on a calendar
day regardless of source (false "record" days) and bucketed on the
UTC calendar day instead of the user's timezone. Rewritten to
source-collapse before summing and bucket on the user's IANA zone,
reading through the worker's injected Prisma client so the existing
dependency-injection contract for tests stays intact.
- The correlation-discovery matrix (the insights route, the per-metric
assessment card, and the Coach's get_correlations tool — three
independent copies of the same fetch) averaged cumulative types
(steps, daylight) and sleep across mismatched grains: per-sample
chunk means blended with drained daily totals, and sleep averaged
per-stage segment durations instead of summing a night's total.
Hoisted the grain resolution into one shared
buildMeasurementDailySeries helper (source-collapsed day-sum for
cumulative types, per-night reconstruction for sleep, mean
unchanged for spot vitals) so all three surfaces agree.
Regression tests cover newest-first ordering, cross-source collapse
matching the rollup, PR no-double-count, and the shared grain helper.
The nightly insight pre-pass runs ~04:30, before last night's sleep has
synced, so the morning digest and health score reference the day without
the current sleep. Close the gap the event-driven way instead of an
on-mount generation.
A day's digest is now provisional until the sleep-dependent output has
been regenerated with last night's sleep, and final after. The lifecycle
is driven by a real marker (User.morningDigestRefreshedOn, the user-local
date the refresh completed), not a heuristic: it flips the instant the
refresh runs, ahead of the snapshot cache that feeds the sleep last-seen
fallback. A day with no sleep device stays honestly provisional and keeps
the sleep-pending note rather than showing a stale final score.
When a completed sleep segment for last night lands from any of the three
sleep transports — Withings sync-sleep, the WHOOP sync, the Apple-Health
measurement batch — a debounced morning-digest-refresh job is enqueued.
"Last night" is judged in the user's profile timezone, so a full-history
re-sync never re-triggers for an old night. The queue singletonKey plus
the marker collapse the many samples of one night, across all sources,
into one refresh per user per local morning; the job re-runs the same
comprehensive pipeline the nightly cron uses (forced, so the content-hash
gate turns a no-change regen into a cheap no-op) and stamps the marker. A
hard generation failure leaves the day provisional and hands recovery to
the existing 45-minute briefing-retry machinery.
Platform-wide hardening: the failure classes found live this week hunted down
across every integration.
Fitbit carried the same tombstone wedge fixed for Google — probe now matches
soft-deleted rows and the update resurrects them; the comment describing a
partial unique index that never existed is corrected. Sleep segment ids are
stable across re-scoring for Withings (session+segment-start), WHOOP/Polar
(one segment per stage, no positional index) and Oura (run-start keyed), each
with a session-scoped sweep that soft-deletes rows a re-score orphaned — so
legacy and affected nights heal as they are re-read.
One platform rule for deleted rows: source-owned readings resurrect on
re-import (Withings, WHOOP, Oura, Polar, Nightscout, mood webhook, CSV
re-imports, Apple stats day-totals); Apple sample-grain deletions stick per
the paired-client contract, and the mood bulk route's tombstone match is now
a true no-op reported as duplicate. Import counts reconcile against actual
createMany counts.
Google lifecycle: a dead token registers on the cycle's hard-fail ledger, so
an expired connection parks instead of stamping success hourly; backfill and
sleep-repair markers stamp only after a clean pass (pg-boss retries are now
real); failed measurement writes hold the watermark; no-op updates are
skipped; mapping.md tells the current truth.
Scale: doctor-report reads only consumed columns and enabled sections;
exports and backups keyset-paginate; achievements tally via SQL buckets;
series charts day-bucket dense kinds beyond 90 days; BP pairing is a linear
two-pointer; distinct-type scans run server-side; the graded-series cold
fallback is bounded; the fitbit defer tracker loops instead of spreading.
PUT /api/measurements/[id] was the one write surface that skipped the
per-type plausibility bands — POST, batch, CSV import, Apple export,
Telegram and MCP all enforce VALUE_RANGES — so an implausible edited
value flowed unchecked into rollups, the health score, the BP gates
and the Coach snapshot. The route now validates the new value against
the row's own type after the lookup and returns the standard
multi-issue 422 envelope on violation.
Value edits on server-owned sources (anything outside
WRITABLE_MEASUREMENT_SOURCES: connector, import, computed rows) are
refused with 409 and the stable errorCode
measurement.update.server_owned_source — editing a provider-attributed
number forges a reading the server never received. Timestamp and note
edits stay allowed, and the edit sheet shows a localised message for
the new code.
The schema's generic min(0).max(500000) bound is dropped: the per-type
band is stricter where it matters, and min(0) blocked legitimate edits
of the signed types (ANS_CHARGE, BODY_TEMPERATURE_DEVIATION). Route
tests cover out-of-range 422, negative signed-type accept, the
server-owned 409, a timestamp-only edit on a server-owned row, and the
happy path. The PUT endpoint is not part of the published OpenAPI
surface — the generated contract is unchanged.
DELETE /api/measurements/by-external-ids now only tombstones rows the iOS
client minted through its HealthKit ingestion path (source APPLE_HEALTH).
An externalId the app never minted — an integration- or manually-sourced
row that happens to collide on externalUUID — falls outside the predicate
and is a no-op rather than a delete. Defence-in-depth so a foreign
HealthKit sample with a colliding UUID cannot be tombstoned by the
periodic reconciliation. The delete stays user-scoped as before.
Add a route test proving a foreign-origin externalId is not deleted, and
regenerate the OpenAPI spec for the narrowed contract.
Marc-André Bombeck