An account of 1.25 million measurements could not export its data or be
restored from any backup (#1031). Measured in a 1 GB container:
- The full export and the passphrase export built the whole payload and
its JSON string in memory; the app ran out of heap and restarted.
- The weekly backup's JSON is 662 MB, longer than any string V8 can hold,
so `unpackBackupBlob` threw on every read. Restore, download and the
restore preview all answered "could not be decrypted".
- Upload accepted 10 MB, a fraction of a large export.
Every reader and writer of a backup file now streams it:
- `scanBackupJson` reads a backup's bytes once, keeps every section but
the named bulk arrays, and hands their elements out one at a time.
`readStreamedBackup` uses it twice: a first pass that validates each
measurement against the element schema and keeps the rest of the file,
and a second pass that hands the measurements to the writer in batches.
Stored copies and off-host objects are authenticated whole on their
compressed ciphertext before any plaintext is read; only the
decompression is streamed.
- Restore writes the measurements with one `INSERT … FROM unnest` per
batch. Prisma caches a query plan per `createMany` row count, and one
1 000-row plan kept about 145 MB alive, which alone put the restore past
the heap limit. The transaction budget grows with the measurement count.
The awards and ECG sections are handed the ids they reference that were
written, not every written id.
- The export routes write the file into the response as it is produced.
The passphrase export spools its ciphertext to a temporary file because
the HLX1 format puts the tag first; the format is unchanged, so existing
archives open as before.
- Upload accepts the raw body, JSON or gzip, up to the 512 MB request
ceiling, checks it as it arrives and stores it in the same pass; the admin
page compresses the file in the browser before sending it. Multipart stays
for small files.
- Download streams the stored JSON; the preview counts measurements from
the stream. The monthly restore drill and `scripts/restore-backup.ts`
read the off-host object the same way; the script can write `.json.gz`.
On the seeded account in a 1 GB container: export 25 s at 389 MB peak,
passphrase export 25 s, both uploads 19 s, restore of the weekly backup
94 s at 554 MB peak, and export, upload and restore round-trip to exactly
the same rows. The encrypted-column guard now also recognises the raw SQL
statement that writes `DataBackup.data`.
The cycle delete now runs the shared removal, which finds live starts only.
A second delete of the same cycle read nothing to remove; the route now says
so explicitly and answers 204 before the transaction, and the tombstone the
first delete left stays as it was.
A start logged within one period before an existing start folds that start
into itself and soft-deletes it, since the two are one period entered out of
order. Nothing recorded which start was folded, so taking the earlier start
back could not bring the later one back: a mis-tap on the 5th with a real
start on the 10th, then deleted, left the 10th gone and the cycle before it
reopened.
Migration 0345 adds menstrual_cycles.absorbed_into_id. The folded row names
the start that absorbed it; removing that start restores it, re-derives its
own end and hands its days back to it. The column is needed because a
soft-deleted row alone cannot tell a folded start from one the person
deleted, and the audit trail is not state a removal can read. Deleting a
cycle directly now runs the same shared removal as deleting its day-log.
The runtime accepts "+05:30" as a zone id, so isValidTimezone let it through,
but Postgres reads a bare offset in AT TIME ZONE with the POSIX sign (east
of UTC is negative), and the day-grain SQL aggregates would cut every day on
the wrong side of UTC.
isValidTimezone now accepts named zones only: whatever the runtime resolves
to a bare offset is refused, while named zones, their aliases, UTC and the
Etc/GMT family stay accepted. Every write that already validated through it
refuses offsets now, and the screener and mood linked-context request schemas
validate their zone too. A value stored before this change can still sit in
the database, so the SQL call sites that read the stored zone directly (the
dashboard summary, achievements, the doctor report, the tiered AI context)
fall back rather than splice it, as the resolver and the day aggregates
already did.
For a cycle-tracking account the strip painted Strain first and swapped
it for the cycle ring once the calendar read landed, moving the row
under the reader. The ring's decision now reports while it is pending,
and the strip keeps its skeleton until then.
Migration 0343 moved the operator's Coach answer from module availability
into the Coach switch and folded every live row. Restoring the instance
settings from an older file wrote the dead coach key back into module
availability and left the Coach switch on, so a Coach the operator had off
came back.
The restore now applies the same fold: coach false in the file's module
availability switches the Coach off, and the key is dropped either way.
Reminders are the person's own record, like the stored facts,
conversations and plans. The list and delete routes no longer ask the
Coach module; creating, confirming, resolving or rescheduling one stays
Coach use and is refused by POST and PATCH as before.
Settings -> Coach now always mounts the reminders card (shown with Hide
Coach on only when reminders exist), and Settings -> AI lists them through
the stored-memory block while Settings -> Coach is out of reach. With the
Coach unavailable the card is read-only: every reminder can be read and
deleted, while keep / done / dismiss are not offered.
Plans are the person's own record, like the stored facts and
conversations. The list and delete routes no longer ask the Coach
module, and the plans page turns read-only instead of redirecting to
Insights: every plan can be read and removed, while confirming or moving
one stays Coach use and is refused by the PATCH route as before.
The scores strip reserved a cell for the cycle ring, and dropped Strain to
make room, whenever the cycle module was on. Without an active cycle the
ring renders nothing, so the row showed a gap where Strain used to be. The
overview now asks the same calendar read whether there is a dial today.
Reading a document, scanning a lab report and turning a medication
description into a schedule follow documentAi, labsOcr and
medicationExtract. Unavailable, the action is not offered. With only the
document-reading consent missing, the place that reads asks for it and
grants exactly that (ai_extraction), nothing wider.
Refusals are read by meta.errorCode: a 403 is no longer always called a
missing consent, and a spent daily budget no longer shows the rate-limit
sentence.
The daily briefing, the hero's briefing paragraph, the trend sentences and
the status notes follow the briefing and statusText capabilities: hidden
while unavailable, never an error card or a per-card "connect a provider"
state. The advisor and status reads only fire while their capability is
available; a 403 settles as an outcome instead of throwing.
A failed comprehensive read no longer replaces the overview, and every
overview section is fenced so a section that throws becomes one error row.
The period review always renders its composed narrative. Cycle, breathing
and lab-change sections go through the surface map. One calm setup hint
appears on the overview while no provider is configured and one could be.
Signals, rhythm events and the mood patterns card say a failed read with a
retry, the score anatomy no longer calls a failed read insufficient data,
the catalog marks only the rows whose read failed, and workouts are not
probed while the module is off.
The floating button, the drawer, the launch pills, the nav entry and the
Coach pages read the coach capability from /api/auth/me instead of
combining the operator switch with Hide Coach on their own. A missing
provider or consent now hides the Coach too, instead of opening one that
can only answer with an error.
Stored conversations stay reachable: while the Coach is unavailable the
conversation list turns read-only, opens a transcript in place and keeps
delete. Refusals before the stream opens are read by meta.errorCode and
each code has its own copy.
The bulk-delete, restore and by-external-ids suites mocked
`invalidateStatusInsightsForTypes` on the comprehensive-generate barrel,
while the shared post-mutation tail imports it from status-invalidation.
The mock was never called, the real re-warm ran the capability loader
against a stub database, and the "no invalidate" assertion could not
fail. The mock now sits on the imported module; the positive cases
assert the re-warm with the distinct types, the zero-row cases assert
its absence.
Eight files mocked `@/lib/feature-flags` with `getAssistantFlags` alone.
The capability loader reads `loadAssistantSwitches`, so wherever it ran
it threw and every capability resolved to `check_failed`. A shared
factory now points both readers at the function the test drives.
Only the config-change chain reached the loader. Its fixture means AI
switched off by the operator; it now supplies the loader's other inputs
and asserts `operator_disabled` on the digest, so a starved loader turns
it red instead of passing on the failure path.
Nine places assigned window.location.href directly: OAuth connects and
sign-in hand-offs that land on API routes answering with a redirect, and
the reload after account deletion. The newer Next lint rule reads each as
an in-app page change. They now go through one helper that says why a
document load is the right tool there.
Serving stored text and starting model work are separate questions. A
delegate may not start model work on somebody else's record, but reads
the owner's stored briefing, period narrative, status notes, workout
notes and Coach unread signal exactly when the owner would: those
readers now resolve the record's own capability state, not the
viewer's. Generation, warms and enqueues still ask as the viewer, so a
delegate's read of a stale narrative queues nothing. A hidden
model-written narrative is no longer overwritten by a deterministic
warm while AI is unavailable.
The sharing front door reads the digest's briefing lead again, and an
integration case pins both halves.
Refs #975
Brings in the jobs, stored-text and consent package and the extraction
routes. Conflicts resolved:
- The digest keeps its capability-driven reaction line and check-in,
now through the shared `aiCapabilityForRecord`; the local copy of it
is removed.
- The route inventories classify the extraction routes (documents,
labs OCR, medication text, fenced chat and attachments) as actions;
a capability call counts as the gate for the module inventory.
- With every caller moved, `requireAssistantSurface` and
`AssistantDisabledError` are deleted. The `assistant.disabled.*`
codes stay on the capability refusal, and the guard now asserts the
retired gate is gone.
- `aiCapabilityToServe` answers whether stored model text may be shown
from the record's own state, apart from who is reading.
Refs #975
Each family of ids must name a surface that exists (nav hrefs, sub-page
directories, section ids, widget ids, derived ids, Settings slugs,
correlation channels). Sub-pages, trend slots and correlation channels
must follow the owner of the types they plot, joined against the
dashboard's summary owners, and a pinned table covers the owners no
join can derive. The replaced per-surface keys and the redirecting page
guard are searched for in source, and the derived route's gate is
compared with the map entry by entry.
Broken on purpose before committing: dropping the sleep sub-page entry,
the mood trend entry or a nav entry, drifting a derived owner, and
reintroducing a per-destination key each fail by name.
The correlation scan still paired mood, sleep, glucose, adherence,
symptoms and weather with everything else after their modules were
switched off. The discovery matrix now drops every channel whose module
is off, read through the surface map, and the correlations route passes
the record's module map, skips the lab draws with labs off, and keys its
cache on the switched-off set.
Settings sections and Layout groups no longer declare their own module
keys; they ask the surface map. The Insights overview arrange list marks
the breathing, labs changes and cycle blocks unavailable when their
modules are off, as the overview itself would not show them. The rows
stay in the list so a save keeps each block's place.
Navigation destinations, the bottom bar's fixed slots and the tour stops
no longer carry their own module keys; each asks the surface map by its
href. Insights has no owner there, so switching AI analysis off leaves
the Insights entry, slot and tour stop in place.
The shell's page gate now resolves Insights sub-pages by slug as well
as top-level pages, so /insights/mood, /insights/medications and the
recovery metric pages show the module notice when their module is off,
worded from the access reason, with the Settings link only for the
record's own switch. The per-page redirect guard, the achievements
redirect and the nutrients page's own notice are gone; nothing
redirects any more.
Whether a vaccination is still current was known only on the checkups
page. The vaccination log now shows it on each antigen's heading:
the booster the person planned when they logged a dose, marked as
overdue, due soon (within 30 days) or scheduled, tinted only in the
first two cases.
The state is resolved on the server from the person's own confirmed
booster reminder and published on the list response as renewals, so
the web page and the native client read the same answer. Nothing is
worked out from age, history or the catalogue: an antigen without a
planned booster shows no state. Renewals are null for a grant that does
not reach the measurements section the reminders live in.
Refs #1005
The dashboard widget and summary maps are now views of the surface map.
The legacy dashboard feed painted the stored layout as saved, so the
medications card stayed with the module off; the mask the snapshot
applies now runs on that feed too, in the browser, from the same map.
The widgets route keeps returning the stored layout unmasked, because
Settings saves that value back.
The Today digest is the day's data and no longer waits on the AI
analysis switch, on the server prefetch or in the browser.
A document could show its visits read-only and its vaccinations not at
all, so a childhood record covering a dozen doses had to be attached
from a dozen dose forms. The document's detail sheet now lists the
visits and vaccinations it is filed against and, for someone who may
manage the vault, links and unlinks them in one place, with the records
dated near the document on top. Both ends write the same link tables
through the link service, so the dose's form and the document's sheet
cannot disagree about what is filed where.
The document detail response carries the linked doses, named by
catalogue slug or the person's own wording. It is null rather than
empty for a grant that opens the vault but not the health background,
so a shared page never claims to record no doses. The update route
takes a replace-set of vaccination ids, up to 100 so one record can
cover a whole primary schedule; ownership is re-checked, a foreign id
answers 404 and the change lands in the audit log beside the other link
fields. Linking from the dose suggestion now refreshes the sheet.
Refs #1024, #1005
The status family (the five metric cards, medication compliance, the
generic metric status and the biomarker assessment) asked the retired
switch and the `insights` module and refused with 403. The note is model
text and the card around it is data, so they are mixed reads now: while
the `statusText` capability is available the note is served and warmed
as before, with `ai` beside it; otherwise the answer is 200 with the
note null, `preparing: false`, `hasProvider` as provider presence only
and `ai` naming the reason, and neither the cache nor the warm queue is
touched. The generic metric status still reports `insufficient` from the
data. The mood card keeps its mood module gate.
Kept apart so it can be held back while the shipped iPhone app's
rendering of a 200 without a note is confirmed.
Refs #975
The status chokepoint, the comprehensive briefing, reaction lines and
AI-composed nudges now ask `aiEgressRefusal` about exactly the chain they
are about to send to, instead of each pairing the operator-key check
with a surface receipt lookup of its own. The consent kinds come from the
capability table, so the wire and the published capability answer from
the same rule.
The document summary, its catch-up pass and the index backfill stop on a
closed switch or module before anything is picked, and leave "no
provider" and "consent needed" to the document pick, which re-checks the
wire for the provider it chose. A closed switch still only heals PENDING
to NONE; a withheld pick records UNAVAILABLE. The per-pick document
consent helper is removed, since nothing calls it any more.
The auto-read toggle mints the shared `AI_EXTRACTION_CONSENT_KIND`, and
the web grant helper goes back to minting the master grant only.
Refs #975
Brings in the extraction routes and the capability re-check at the wire
(`aiEgressRefusal`), so the jobs and stored-text work can share it.
Conflicts in the document routes, the fenced chat, the auto-read toggle
and its contract text resolve to the extraction branch: those files no
longer call the per-pick document consent helper.
The weight tile's arrow, 7-day delta and comparison caption were wired
as "up is bad", so a person below their own target who was gaining saw
every step toward it coloured as a setback. Movement is now judged
against the target they entered: below the band a gain is progress,
above it a loss is, and inside it holding steady is, with a move either
way shown neutral rather than orange. Without a stored target the tile
reads exactly as before; the height-derived band is not treated as a
target because nobody chose it.
The direction is resolved once on the server and published on the
dashboard snapshot as tiles.weightTrend, so the web tile and the native
client colour the same reading the same way. The personal-record
direction reads the same placement helper. The feature payload behind
the briefing, the overview analysis and the Coach now carries the
target band and which way is progress, so a narrative no longer calls a
gain a regression when gaining is the goal. The insights vitals grid
drops weight from its fixed "up is bad" set.
Refs #1006
Data never depends on AI. The Insights data reads no longer ask an AI
switch or the `insights` module (now only the AI analysis opt-out): ECG
list, detail and upload, rhythm events, derived scores, rule cards,
correlations and patterns, the baseline strip, health status, intraday
pulse, and the Today digest with its dismiss. Breathing screening moves
to the sleep module and lab changes to the labs module. Stored Coach
facts, conversations, message ratings and the seen stamp stay readable
and erasable while the Coach is unavailable. Cards now say
`provider: "rules"`.
Reads that mix data with model text answer 200, null the model text
and publish an `ai` state from the capability resolver, without warming
anything: comprehensive (`ai.briefing`, `ai.statusText`, `hasProvider`
from the presence probe), the derived assessment override, the period
narrative, nudge status, the seeded opener, about-me questions, the
workout paragraph, the pregenerate warm (`queued: false`), the Today
digest (lead, reaction line, check-in) and the dashboard briefing,
which is now applied per read after the snapshot cache. Hiding the
Coach no longer hides the briefing.
`insights/generate` GET and POST ask the `briefing` capability and
`insights/chat` POST the `coach` capability, with the typed refusal
envelope; a missing provider keeps the chat's `coach.provider.none`
frame and adds `reason`. The /coach page, the MCP ECG and intraday
tools and `aiAvailable` on the provider route follow the same answer.
An inventory of AI and data routes replaces the Coach gate inventory:
every route that asks a capability names it, and no data route asks.
Refs #975
Every AI job now asks its capability for the user before it builds a
snapshot, and skips with a `.skipped` annotation naming the reason: the
nightly pregenerate (briefing and statusText, each half on its own), the
status batch and the per-metric crons, the on-demand status worker, the
narrative cron, Coach nudges and their AI composer, the Coach memory
refresh, reaction lines, workout paragraphs, and the document summary,
catch-up and index jobs. Seven of these read no assistant switch at all
before, so the master switch now stops them too.
The cron candidate queries drop people who switched AI analysis off in
SQL, and no longer read `disableCoach` outside the Coach: hiding the
Coach hides the Coach. Coach nudges use the one presence probe, so a
central Codex account is no longer skipped; a missing consent still gets
the deterministic template. The arrival spine asks only the
provider-free half (switches and modules) before it enqueues, keeping
its module graph free of provider code.
Document reading needs an extraction receipt (ai_extraction or ai_full)
for any external provider; the auto-read toggle is the trigger, no longer
a consent shortcut, so a revoked receipt wins over a toggle left on. Lab
report scans get the same rule for a whole chain.
Two structural guards: every file that calls a provider is named with
the reason its call is admitted, and the chokepoints re-check a
capability; every AI job references its capability gate. An integration
matrix runs each worker under the refusing states with a provider spy.
Refs #975
A new receipt kind, ai_extraction, covers reading documents, lab report
scans and typed medication text with an external model. Switching the
document auto-read on now mints ai_extraction instead of ai_full, so it
no longer grants the Coach or the analysis as a side effect. Existing
ai_full receipts keep covering everything.
DELETE /api/consent/ai/latest now runs through withdrawConsent: the
revoke and, when it leaves no receipt covering the analysis, the
deletion of the text a model wrote under it commit in one transaction.
That is the status notes, the cached briefing, model-written period
narratives, reaction lines and workout paragraphs. Coach conversations,
facts and plans and document summaries stay. The purge is audit-logged
as consent.ai.purge with counts only.
Refs #975
Turning document auto-read on now mints an ai_extraction receipt instead
of ai_full, so it covers reading documents, lab reports and medication
text and no longer opens the Coach or the AI analysis. Receipts minted as
ai_full by earlier releases stay valid. The catch-up over stored
documents is only scheduled while documentAi is open for the record.
The AI connection test is an AI call, so it is refused with
assistant.disabled.enabled while the operator's master switch is off, and
fails closed with ai.unavailable when the switches cannot be read (403,
since the route never answers 5xx). A missing provider carries
ai.provider.none and a spent daily budget ai.budget.exceeded.
Reading a stored document (suggest, summary, extract, index, chat,
reindex), scanning a lab report and turning medication text into a
schedule now answer under documentAi, labsOcr and medicationExtract. The
operator's master switch and the new reading-documents switch stop the
work and the egress; a module off for the record, a delegate inside
somebody else's record, and a missing receipt refuse with the capability
envelope. Existing no-provider codes keep their status.
The document pick and the labs pick re-check the capability for the
provider they picked, immediately before anything is sent, so a job
enqueued before a switch flipped finds no provider and falls back to the
provider-free text layer. Consent follows the document rule for all
three: any provider that leaves the machine needs an ai_extraction or
ai_full receipt, answered for the provider actually picked.
The fenced Coach chat and attachments answer under coach and documentAi.
Uploads are always stored and indexed; the background summary is only
enqueued while documentAi is open. The capability probes report the
capability beside the provider answer and keep a read offered when only
the receipt is missing. medications/extract keeps its 503 for no provider
and adds ai.provider.none, and its budget refusal gains the
coach.budget.exceeded code.
Several chart mounts (the medications page, the coach chat, the
recovery section) pass no zone, and the chart then fell back to a fixed
Europe/Berlin, so those charts cut and labelled days in Berlin for
every user whatever their profile said.
The chart now falls back to the signed-in account's zone, and only
before that has loaded to the shared DEFAULT_TIMEZONE the server's own
resolver ends on. /api/auth/me reports the zone the server resolves:
the stored one, or the instance default when the stored one is
unusable, so the client cuts days where the server does.
Refs #1026
The dose-history response reported the slot window: `from` clamped
to the medication's creation, `to` the request's end. Recorded doses
are now read from the requested start and, for a dose recorded ahead
of its slot, up to that slot, so the response reported a narrower
window than the rows it carried.
`from` and `to` now report the window recorded doses were read over.
Both fields keep their shape; the OpenAPI descriptions say what they
mean.
Refs #1028
The log sheet took its "Day N" header and its "period ended" button
from the verdict, which is the answer about today. A January date opened
in September read "Day 2", and a back-dated period could only be closed
on dates where today happened to be a bleeding day, so the button came
and went as unrelated months were filled in.
The calendar read now carries, per date, the cycle day of the logged
cycle that date belongs to (null before the first start, after today, or
past the verdict's ceiling for an open cycle) and whether a one-tap
period end can land there. The sheet reads both off the grid day for the
date it opened on.
Saving a new entry with nothing picked used to write an empty row: the
request succeeded, the sheet closed and the calendar had nothing to
draw, which read as a save that did nothing. Save now needs something to
store and says what, and an emptied existing entry still saves. The
sheet also stops writing until the opened date's own entry has loaded,
because until then it held the previous date's row id and a quick Save
patched the wrong day.
On the server, a start recorded a few days after an earlier one is the
same period entered in the other order: opening the earlier start now
folds the later one into it instead of leaving a one-day cycle in front
of it, and a bleed too far before the next start to be its first day and
too close to be a cycle of its own no longer infers a start. Opening or
removing a start re-attributes the day-logs whose cycle changed.
The Coach gate inventory lists the correlations route with the routes
no assistant switch owns. The seeded-question test gives the switch
read a settings row, since a read that fails no longer opens every
switch.
A single account syncing heart rate from a watch took the worker down
within half an hour. The status cards' graded series read the last 91
days of raw rows, bounded in time but not in rows, so a stream sampled
every minute or faster materialised six or seven figures of objects on
every refresh and the read ran into the statement timeout.
Add readDayAggregates, which folds one type into per-day sum, count, min
and max in Postgres (optionally split into age segments and filtered to
the metric's plausibility domain), and move the readers that folded raw
rows into days in memory onto it: the graded series (recent, weekly and
the cold-tier fallback, for every type), the vitals day-mean fallback
behind the personal band, the tiered context's recent days and the plan
review. Recent, weekly and monthly buckets are identical to the raw
fold; the cold-tier yearly slope now runs over day means.
On 91 days of per-minute pulse the graded series now allocates 1.6 MB
where it allocated 62 MB, while the raw read alone holds 22 MB.
A structural guard freezes the remaining untaken measurement reads in
the worker and insight directories: each must name a sparse literal
type or carry a listed reason.
Refs #1023
Switching a chart from 7 to 30 days moved steps, active energy,
flights and distance totals onto neighbouring days. The 7-day range
buckets raw rows by the user's calendar day; the 30-day range read the
DAY rollup tier, whose buckets are UTC days, and the chart labelled each
bucket's UTC-midnight start in the user's zone. East of UTC a sample
shortly after local midnight counted on the previous day; west of UTC
every bucket read one day early.
The daily series (the chart read and the batched dashboard read) and the
aggregate=daily|weekly|monthly list now fold live in the user's zone and
return the instant of local midnight, so every range and every summed
metric shares one day boundary. The fold runs in Postgres and returns
one row per day; about 0.1 s for 90 days of per-minute samples. Windows
longer than the DAY cap still step up to the coarser rollup tiers.
Refs #1026
Marc-André Bombeck