A soft-deleted measurement never reached the rollup tier: the WHOOP
delete webhook tombstoned rows without recomputing the dirtied buckets,
and the freshness watermark probe filtered tombstones out, so the
deletion's updatedAt bump was invisible and the ghost aggregate read as
fresh forever.
Extract the post-write tail every measurement surface repeats into
afterMeasurementMutation (rollup recompute + status-insight
invalidation, both best-effort), call it from the WHOOP delete webhook
with the (type, measuredAt) pre-image collected before the updateMany,
and drop the deletedAt filter from the watermark probe so a tombstone
counts as staleness.
The unit battery derives rollup fixtures in JS; this pins the same
parity through the pipe itself. Seed raw measurements, read the tiered
fetch before the fold (coverage miss, raw path baseline), fold with the
real recomputeUserRollups DAY pass, read again, and assert the two
daily series are equivalent — including a dual-source weight day whose
per-source rows must compose to the all-source mean. A drifted writer
projection, coverage-probe SQL, or bucket compose surfaces here.
Both paths read identical fixtures — per-(day, source) DAY rollup rows
derived from the same raw rows the fallback reads — and must produce
equivalent daily series: the plain daily-mean case, the multi-source
day (the all-source weighted mean, never a ladder-collapsed single
source), and the pre-sum_value mean-times-count compose. A far-from-UTC
profile timezone must force the raw path so day keys stay profile-tz
keyed; the near-UTC midnight band's single-day attribution shift is
pinned explicitly so it stays a documented decision rather than drift.
Coverage misses and empty in-window bucket reads fall back per channel;
sleep and the cumulative types never enter the rollup query.
Every assertion family was broken on purpose and observed failing with
the site named before restoring green; the record sits in the test's
doc comment. The route test mocks the new coverage probe and bucket
read as empty so every channel rides the raw path unchanged there.
The discovery scan read every raw measurement in the 180-day window for
all nine measurement channels and bucketed them into daily means in JS,
paying tens of thousands of rows on every cache miss (a CGM glucose
channel alone can exceed the read cap). The DAY rollup tier already
stores those per-day aggregates, so eligible spot channels now read one
pre-aggregated row per day per source and compose the all-source
count-weighted mean — the exact reduction toDailyMeans applies.
Read-swap replaces the raw read per channel, with fallback-on-miss:
structural ineligibility (SLEEP_DURATION's per-night reconstruction,
the cumulative types' device-type collapse axis the rollup rows do not
carry), a far-from-UTC profile timezone (the v1.4.38 W-A near-UTC
guard), a coverage-probe miss, or an empty in-window bucket read all
route that channel to the existing raw path. The SWR cache cell stays
in front; the swap only lowers the cost of a cache miss.
The prefetch-caching guard's frozen route set and the session-surface
guard's direct-derivation list both name every RSC prefetch wrapper;
/checkups and /mood join them with the same reasoning as their five
predecessors — getUnswitchedSession only, skipped under a switch.
placeholderData keeps the previous range painted while a health-chart
range switch fetches, paired with its truth condition: the container
dims and goes inert while isPlaceholderData is true, so the old range
cannot masquerade as the new one. The mood chart needs no pairing — its
range is a client-side slice over one static-key query and never drops
to a skeleton.
/checkups and /mood gain the dashboard-template RSC prefetch: the
primary list read moves into a shared lib read (route + wrapper consume
one query, so filter/order/wire shape cannot drift), the wrapper honours
the DASHBOARD_SSR_PREFETCH kill-switch, resolves identity through
getUnswitchedSession, JSON-round-trips the dehydrated state, and fails
soft to the bare client on every error path. Page-prefetch unit tests
pin the seeded keys and the fail-soft arms; the SSR hydration spec
gains an assertion per page.
The achievement-unlock poll drops to five minutes (in-app writes already
invalidate the cell; the poll only covers out-of-band ingest), and the
plan-proposal poll gains the advisor-style bounded attempt ceiling so an
open thread stops polling after its budget.
GET /api/insights/correlations was the most expensive uncached read in
the app: a 180-day window, ~10 parallel series reads, Pearson + FDR
in-process, plus two syncAcceptedPatterns DB writes — all paid on every
mount. The built body now reads through the analytics bucket's SWR cell
(60 s fresh / 1 h stale, keyed userId|correlations|locale), and the
pattern-sync writes live inside the builder, so a cache hit performs no
write on the read path. Eviction truth comes free: measurement, mood,
medication, illness and custom-metric writes already sweep the user's
prefix in that bucket, and a pattern dismiss sweeps it through
invalidateUserCorrelationPatterns.
The route test pins the contract: a second GET inside the fresh window
re-runs neither the measurement window read nor the pattern sync, and
serves a byte-equal body.
Ten client mutation sites forgot a consumer: a coach-accepted checkup
stayed invisible for up to five minutes, a threshold or target edit left
the dashboard bands stale, a score-config save marked the daily reads
stale but never refetched them, a booked visit missed the Today rail, a
source-priority reorder kept every chart on the old canonical picks, an
adopted self-context note never reached the Settings read. Each site now
invalidates what its write changed and forces the unmounted snapshot +
digest reads to refetch through refetchInactiveDailyReads.
New illnessDependentKeys bundle carries the daily reads (Rest Mode
frames both), dailyDigest joins encounterDependentKeys, and the
chart-data prefix gains a factory accessor for whole-cache flushes.
The daily-reads guard learns the cycle / encounter / vaccination /
illness bundle families and a second matcher for direct snapshot/digest
invalidateQueries without a forced refetch, each with a frozen reasoned
allowlist. Both matchers were proven live by reverting a fix and
watching the guard go red (see the test doc comment).
Integration syncs (Fitbit, Oura, Polar, WHOOP, Strava, Nightscout) now
mark the per-user analytics cells stale and sweep the workouts cache at
their sync tails — imported rows used to stay invisible to the cached
dashboard, analytics and workout readers until the TTL lapsed. The WHOOP
per-resource webhook/cron driver and the Polar workouts leg carry the
same call, matching the posture Google Health and the workout batch
route already had.
Illness, cycle and encounter write routes hard-evict the analytics
bucket through the new invalidateUserHealthContext helper: Rest Mode,
the cycle context and the Today-rail appointment feed all read from
cached cells under that bucket, and an interactive write must be true on
the very next read.
The settling state's copy, the index-outcome plain-words line across
six locale catalogs and the reindex-count wiring take the whole-build
client sum to 3429 KB gzip, one kilobyte over the cap set at v1.37.4.
Every per-route ceiling stays met, so nothing a person downloads on a
given route grew beyond its ceiling; the aggregate cap moves to 3452
with the same margin as previous re-measurements.
The residual scan from the report is a CCITT Group 4 single-image PDF,
the codec class pdfjs rasterization is known to fail on. Chain checks:
a pdfjs open failure or throw already lands as render-failed with the
error text annotated; the zero-pages shape now annotates too instead
of returning silently. An end-to-end test drives the REAL rasterizer
through prepareVisionInput and pins the rasterFailed mapping; the test
header documents why a tiny in-repo CCITT fixture is deliberately not
checked in (probed against this pdfjs version, it renders the page as
blank instead of throwing, so the fixture cannot deterministically
fail and would be flaky across upgrades). The detail view's
raster-failed line now names scanned fax compression formats as a
known cause, in all six locales.
Refs #776
Same guarded transition, expressed as the render-phase state
adjustment the hooks lint rule asks for: a resolved settle clears the
state in the same committed render that carries the outcome, so the
falling-edge toast still sees both flips at once and the block runs at
most once per settle.
Two contract tests enumerate every branch of the summary job and the
index tree and assert each ends in a terminal state write: the summary
job's opt-out branch heals a phantom PENDING back to NONE, every
failure branch lands UNAVAILABLE or WITHHELD, success lands READY; the
index tree records an attempt outcome on every terminal branch,
success included. A third block pins the enqueue side: no PENDING is
claimed when the job's only branch would be the opt-out no-op, and
none before a job id exists. Honest scope stated in the test header:
these pin the two known machines, not a generic detector.
Refs #776, #788
A content-index attempt that failed (raster failure, a scan with no
text layer, a provider error) recorded its reason only in the server's
observability stream. In the vault, hasContentIndex false looked
identical for a document nobody ever tried and one that failed three
times, and the detail sheet had nothing to say about either.
Apply the summaryState pattern to the index. Two additive columns on
InboundDocument (migration 0333) record when the last attempt finished
and why it produced no index, as a closed reason set: local-empty,
local-unsupported, decrypt-error, raster-failed, provider-error,
pdf-needs-anthropic, empty-transcription. The writer sits at the choke
point the auto job and the corpus backfill share, and in the manual
index route; a success clears the reason. The provider path's specific
failure now survives the local fallback, since "no text" is less
actionable than "the PDF would not render". rasterizePdf names its
failure class so a build that cannot rasterize at all is distinguished
from a PDF that would not render. New guard in both index paths: an
empty provider transcription never becomes a successful empty index.
The detail sheet renders the reason in plain words (muted, six
locales). The DTO and OpenAPI carry the two fields; the disaster
recovery backup and restore carry them verbatim like their summary
siblings.
Refs #776
The force regenerate aborts its request at 45 s while the server's
inline generation keeps running and usually writes the fresh briefing
moments later. The old handling fired an error toast at the abort and
re-read the old cache once; with a one hour staleTime and focus
refetch off, nothing ever converged, so the page claimed failure while
holding a finished assessment at arm's length.
A timeout now enters a settling state: the hook remembers the cachedAt
baseline it was showing and polls the read path on the existing
bounded cadence (25 s, up to 10 attempts) until it can prove an
outcome. Cache advanced past the baseline: the fresh payload lands and
the success toast fires. A failure marker newer than the baseline, or
the attempt cap: an honest error. While settling, the strip's button
stays busy and says the assessment is still being prepared; no toast
fires until the outcome is proven. Both poll reasons live in the
nextAdvisorPollInterval seam, the verdict in resolveAdvisorSettle, the
toast mapping in regenerateToastKind; all three are pure and pinned by
unit tests. The contract is documented at both ends (hook and generate
route). New copy in all six locales.
Refs #786
The index-all trigger returned Boolean(jobId) while the client typed
the field as a number, so the toast read "Indexing true document(s)".
The route now answers with the count of live documents still lacking a
content index, computed from the same live-scoped counts the usage
gauge reads, and an honest 0 when no job was created. OpenAPI schema
updated to the integer shape; the hook's type carries a contract note
so the wire and the toast cannot drift apart again. Route test pins
the number, a component test pins that the toast renders the count and
never the word "true".
Refs #776
Since the record fence, vault previews load through object URLs: the
detail sheet frames a decrypted PDF from a blob: URL and thumbnails
render from blob: images. The app CSP carried no frame-src, so the PDF
iframe fell through to default-src 'self' and was blocked outright,
while img-src 'self' data: silently degraded every thumbnail to its
fallback icon.
Add frame-src 'self' blob: and extend img-src with blob: in both the
dev and prod policies. This admits exactly the app's own documents: a
blob: URL is mintable only by same-origin script, script execution
stays nonce-bound in production, and object-src 'none', frame-ancestors
'none' and default-src 'self' are untouched. The vault serve route
keeps its stricter document CSP. A unit test pins the directives so a
future CSP edit cannot silently regress the previews.
Refs #787
The upload path enqueued a summary job and stamped PENDING without
consulting the owner's auto-read opt-in; the job's opt-out branch then
deliberately left the state alone, so a document uploaded with the
opt-in off said "summary is being generated" forever. Two writers,
contradictory assumptions.
Both ends of the contract now agree. enqueueDocumentSummary checks the
opt-in first and refuses to mint a job (or a PENDING) it knows the job
cannot resolve; the job keeps its own check for a pref flipped between
enqueue and run and heals a stranded PENDING back to NONE, scoped so
READY / WITHHELD / UNAVAILABLE stay untouched. Migration 0333 drains
the rows already orphaned. The view needs nothing: NONE already
renders the manual summary action.
Refs #788
Sister file of the HR-curve integration fixture fixed on the release
branch: the same fixed session day aged past the 90-day dense-intraday
horizon today. This suite stayed green because its pulse read is mocked
empty either way, which is worse — the fallback branch its comments
describe was silently no longer reachable, and the timezone assertion
pinned a literal next-day key. The session day is relative now and the
day key is computed from it.
The workout session sat on a fixed calendar day, and the pulse-window
fallback only reconstructs a curve inside the 90-day dense-intraday
retention horizon. On day 91 the fixture aged out and all three curve
assertions went red on every branch, ours and main alike. The session
day is relative to the test run now, ten days back, which stays inside
the horizon forever.
The medication quick-add mounts from the bottom-nav capture picker on
every page, not only on the dashboard, so its bare dependent-key bundle
invalidation (default active refetch) left the unmounted Today digest
and dashboard snapshot stale until the next poll — back on the home
screen the dose still read as due for up to two minutes. Route the
submit through invalidateMedicationReads, the one blessed entry point
that pairs the bundle with the forced inactive refetch, and drop the
surface's allowlist entry in the refetch guard: its dashboard-only
rationale stopped being true when the capture picker started mounting
this form globally. The other capture-picker forms (measurement, mood)
already pair the refetch.
A weekly or rolling medication with the default dose window opens its
availability ~25 h ahead of the anchor (one early day plus the grace
minutes), so the rail announced tomorrow's slot as today's pending
business — on Friday the Saturday dose appeared as due. Thread the
profile-timezone end of today from the digest IO seam into the builder
and keep a not-yet-overdue candidate off the rail unless it falls due
before that instant. Overdue candidates still always surface, and the
early-take affordance on the medication card is untouched.
The extracted export.xml lands in the shared system tmpdir under a
name built from the clock and Math.random, which is predictable enough
for a local pre-creation or symlink attack to aim at. The name comes
from crypto.randomBytes now. Flagged by code scanning on the import
path.
The auto-read spec pinned the old collapse rule: with auto-read ON the
sheet always dropped the AI block. Since the read action stays visible
for a document that has no content index yet, the collapse assertion
now runs against the indexed content fixture, and a second test pins
the recovery path: an unread document keeps its AI section and manual
read under auto-read. Refs #777.
The facts fold already skips the empty assistant row a cancelled turn
leaves behind; the plan fold read it as a blank turn and spent one of
its recent-turn slots on nothing. Same skip, same reason. Also state
the accepted one-slot slack a late rate-limit refund can leave in a
freshly opened window. Refs #781.
The rolling-summary and fact-extraction workers read message bodies
directly, so an empty cancelled marker would ride their prompts as a
blank assistant line. Skip empty bodies in both folds. Refs #781
The free-text self-context note and the free-text allergy line are
personal medical context, so they join conditions, allergies and family
history in Settings -> Anamnese. The note is its own card (same AI-surface
gate as the inclusion card); the allergy line sits directly under the
structured AllergyManager in the same card, labelled as a free-text
supplement so the two inputs never read as duplicates. The account
section drops the old panel. Store, API and query keys are unchanged;
the scoped Clear (note only, allergies untouched) carries over with its
tests. Orphaned strings removed, new labels in all six languages.
Refs #159
An aborted stream now leaves a trace on both ends. The unmount cleanup
records the interrupted conversation (sessionStorage, one-shot) before
the abort fires, and invalidates the cached thread + rail so the return
visit refetches the advanced server state. The /coach page consumes the
record on its next mount: a known id reopens that thread, a first-turn
abort resolves via most-recent; every explicit entry intent (?c=, ?doc=,
?workout=, ?scope=, ?ask=) wins over the resume. The persisted cancelled
marker renders as a muted interrupted note with an Ask-again action that
resubmits the question through the normal send path and its gates.
Localised in all six languages. Refs #781
Navigating away mid-reply aborts the SSE fetch; the provider call tears
down, but the conversation was left with the user message dangling and
no trace of the interruption. Persist an empty assistant row tagged
providerType "cancelled" (the refusal/nudge sentinel channel) when the
request signal is aborted, refund stays intact, and the abort is
annotated as insights.coach.cancelled instead of a provider failure or
a GlitchTip defect. Cancelled markers are filtered out of the provider
transcript on later turns. Refs #781
Generate a 700k-record export.xml (~200 MB decompressed, 500k
cumulative step records each carrying a fresh HKDevice runtime address,
200k interleaved spot rows) streamed into a temp stored-method ZIP —
never checked in, never held in memory whole — and push it through the
real extractExportXml + streamParseExportXml pipeline against a
discarding Prisma stand-in.
Live heap is measured after forced collections at every progress tick;
the asserted growth bound (48 MiB) is a quarter of the decompressed
volume. Both historical failure modes blow past it: reverting the
device-address normalization retains ~61 MiB of fragmented source
buckets, and unbounding the batch buffers retains ~76 MiB and starves
the progress ticks. The fixed pipeline measures under 1 MiB of live
growth. The suite also pins semantics alongside memory: every record
read, every spot row flushed, and exactly one cumulative estimate row
per day rather than one per device address. Refs #775.
A user whose import died to a worker restart saw the literal string
interrupted_by_restart in the import card — the reconcile writes it as
a machine-readable code, and the card echoed it raw. Classify the two
coded reasons (interrupted_by_restart, the insufficient_memory prefix
from the memory preflight) onto translated, actionable copy across all
six locales; the copy tells the user their data is safe and to upload
the export again, which the re-enabled card already supports. Honest
free-text reasons keep passing through verbatim. Refs #775.
Before the multi-minute unpack starts, compare the archive's declared
uncompressed export.xml size against the V8 heap limit. The streaming
parse keeps resident state span-bounded, so the model is generous —
a fixed 192 MiB base plus 1/64 of the XML size; the 8 GiB extraction
cap needs ~320 MiB, which any default heap clears. The check only
refuses the truly impossible (an operator-pinned tiny heap against a
huge export) and does so with an actionable failureReason naming
NODE_OPTIONS=--max-old-space-size, instead of an OOM-killed worker
and a job stuck in a running state (#775).
Document the memory model, the insufficient_memory refusal, and the
interrupted-import reconcile in the self-hosting scaling guide.
The export.zip extractor buffered the whole archive through
readFileSync — hundreds of MB resident for the entire unpack phase on
a real multi-year export. The central directory is now located through
a bounded tail read on a file handle, read as one size-capped slice,
and the export.xml member streams through a byte-range read stream
into the inflate pipeline, so peak extraction memory no longer scales
with archive size.
The cumulative fold hashed the raw device attribute into its per-day
source buckets. Apple serialises that attribute from the live HKDevice
object including its runtime address (<<HKDevice: 0x283a08640>, ...),
which is not stable across the export enumeration — the map fragmented
toward one bucket per record and an 11-million-record export grew it
past the default 1 GB Node heap, killing the worker mid-parse (#775).
Stripping the volatile address before hashing restores the documented
invariant: parser state bounded by actual source cardinality.
Also carry split multi-byte UTF-8 sequences across read-chunk
boundaries with a StringDecoder — a bare per-chunk toString decayed a
split umlaut in a source name to U+FFFD.
GET/PATCH /api/auth/me/diabetes carried no test; the web toggle on the
Targets page and the iOS client both ride it. Cover the auth gate, the
resolved-state echo both ways, the field-by-field write, the audit row,
the 422 shape rejections, and the 429 path.
POST /api/consent/ai/web now accepts an optional { intent } body. Without
one (the AI-settings mount heal) a standing revocation still blocks the
mint; intent "affirmative" is the user's own consent act and may
supersede an earlier revocation, exactly as a first grant would. The
consent card offers the explicit grant control whenever consent does not
stand, and the audit row records which intent minted the receipt.
The document auto-read opt-in already declared "affirmative" on enable;
the settings-mount heal posts no body. Both stay as they are.
The server chain existed end to end (extract stages PENDING facts, per-fact
edit clears the low-confidence gate, confirm is the only write into the
structured stores) but the document detail offered no control over any of
it: it counted observation facts and linked to Labs. A user whose automatic
staging run was skipped had to re-upload the same PDF through Labs > Scan a
report.
- The detail sheet gains a staged-facts block with honest states: pending
facts open an inline review (approve/reject per fact; a low-confidence
fact is checked and saved first via the existing per-fact edit); a
lab-filed, already-read document with nothing staged offers "Extract lab
values" over its stored text; otherwise nothing renders.
- The extract route gains { mode: "stored" }: the same text-structuring
pass the automatic worker runs, over the document's own content index.
No re-upload, no second read of the original; full gauntlet including
the shared document-AI bucket with charge-at-dispatch honesty; 422
notIndexed when no stored text exists.
- A manual "Read with AI" / "Read again" now continues into the same
lab auto-staging the background worker performs; the response reports
labFactsStaged and the toast names the values waiting for review.
- Unchecked values are rejected and the copy says so; an unverified
low-confidence fact receives no decision and stays pending rather than
being silently discarded. Confirmation remains the only write into
Labs, conditions, and medications; approved facts carry the
committedRecordId link back to the source document (existing model, no
schema change).
- The "lab values from this document" link now counts APPROVED facts
only — it claims values that actually exist in Labs.
- Copy in all six locales; OpenAPI regenerated for the new extract mode
and the index response field.
Marc-André Bombeck