The fold leftovers were told from a person's deletions by syncVersion = 1,
but raw samples are raised in place before a fold takes them (the collision
merge, a changed Health Connect value). The repair then recomputed windows
without those samples and could overwrite a correct mean, also in windows
one run had folded whole.
A tombstone of fold age now counts as a leftover when it carries
syncVersion = 1 or shares its deletion instant with such a tombstone of the
same day; anything else of that age is ambiguous and leaves its window as
stored. Only a window taken in two parts (fold runs at least an hour apart,
or a run and later live samples) is recomputed.
Once the repair has been through an account, the purge may remove class-A
leftovers inside the horizon, so the folds' re-fold of a stored day or hour
leaves it as stored, and the purge takes the fold lock per account. The
fold-lock transactions get an explicit timeout and maxWait, since the lock
wait counts against Prisma's five-second default.
The note for records whose findings predate the seasonally adjusted
engine compared against a fixed calendar date, so findings an instance
kept writing under the earlier engine until its deploy did not count.
The first discovery sync of the new engine now stamps
AppSettings.correlationSeasonalEngineSince before it writes; without a
stamp every stored discovery pattern counts as an earlier finding.
A restore from a portable export reset each pattern's createdAt to the
restore day and dropped the note for good. createdAt now rides every
export, and a file without it falls back to lastComputedAt.
A background job fetches air quality, pollen and UV for every earlier
local day with a reading, mood entry, dose, symptom or workout: after the
update (boot and nightly discovery), when the home is set and when air
quality is switched on. A day with a weather row gets its air quality; a
day without one gets both, at the covering location period or else at the
home, also before the home was set. Days the source does not reach (before
2013 in Europe, before August 2022 elsewhere) stay empty and are not
counted.
Pending days are grouped by coarse location and cut into ranges of up to
90 days, joining days at most 14 apart, newest first. A run holds itself to
300 of the account's 400 daily calls and half of each instance window,
stores its progress on the user (migration 0391) and sends a follow-up while
it makes progress; a budget stop waits for the next discovery. The
Location & weather card shows the progress.
The air-quality client now asks pollen and dust from 2021 and skips the
request outside Europe before CAMS global starts.
The fold repair and the regular folds rewrote an existing daily or hourly
stats: mean from the samples still in the table. Fold tombstones are purged
75 days after they were written, so for an older window those samples can
be a fragment: a re-upload after an anchor reset, the rows of one later
fold run. A mean over them replaced a stored mean that had been taken from
the whole day, and on old days it moved a lot.
A stored mean is now recomputed only for a day that starts at or after the
tombstone horizon (foldConstituentHorizon): every fold tombstone of such a
day was written after dayStart + the fold threshold and is still inside the
retention, so the live samples plus the fold leftovers are the whole day.
That is the last 75 days for daily means and, since dense days fold 90 days
late, the 75 days before the raw window for hourly means. A window also
needs at least one fold leftover; without one the rows in hand are not
known to be all of it.
Beyond the horizon:
- The repair leaves the day as it is and counts it per type as
windowsSkippedBeyondHorizon.
- The daily-mean consolidation and the dense hourly fold leave a day or
hour that already has a live stats: row exactly as it is, the stored
mean and its live samples alike (daysLeftAsStored, hoursLeftAsStored).
Up to v1.41 the fold overwrote such a row with the mean of the live rows
alone, and the first v1.42 cut with the mean of those plus whatever
tombstones were left; both are a mean of part of a day. Dropping the
live rows instead would lose samples that may not be in the stored
mean, so they stay as synced. They are few, the folded_window guard
refuses new ones, and the nightly pass walks over them without writing.
A day or hour without a stored mean still folds as before.
Inside the horizon, a pass that recomputes a stored mean soft-deletes the
live samples it took in as fold leftovers instead of deleting them, so a
second pass computes the same mean from the same samples. Before, a fold
that deleted them left the repair a fragment, and the repair then rewrote
the mean from the tombstones alone.
The repair, the mean consolidation and the dense fold now take a per-account
advisory lock after the restore lock in every day transaction, and the
repair reads the day's samples inside it, so the two never compute the
same window from two views of its samples or write the same row at once.
The repair also stops at the fold boundary: a day the fold has not
finished is the fold's to complete.
The lab unit conversion moves to the next release, so its note string
leaves all seven bundles and the i18n pending list. The three columns
that hold the conversion provenance stay (migration 0385, nothing writes
them yet); the backup now carries them both ways, so the round-trip
pending list is empty too and the conversion needs no backup change when
it lands.
A location period whose sealed location this host cannot open was
dropped from the resolver, so its days fell through to the home and the
next refresh overwrote the stored travel days with the weather at home.
Those days are now skipped and the run logs an error signal.
A restore now opens every sealed location once. One that does not open
is kept sealed, so the refresh keeps skipping its days, and is named in
the restore report under a new catalogue.
The daily-mean consolidation and the dense hourly fold cut at
`now - threshold` instead of a local day boundary. A day was folded in two
runs and the second run overwrote the stored mean with the mean of the
later part alone: every mean-type day, and the boundary hour of every
heart-rate, HRV and SpO2 day, plus the derived resting figure and the DAY
rollup of such a day. The Health Connect importer split its raw window at
the same unaligned instant.
- Fold up to the start of the account's local day the threshold ends in
(`foldBoundary`), for every consolidation pass and the Health Connect
importer's raw/hourly split.
- A window that already has a live `stats:` row is refolded from every
sample of it, live rows plus the fold's own tombstones; with none left
the stored mean stays.
- `folded_window` refuses a raw sample only when its local day lies before
the aligned boundary and a live `stats:` row of its source covers it,
also within reach of the sample after a timezone change.
- A deletion inside a day the fold had not finished is class B.
- Add a one-time, resumable fold repair per account that recomputes each
hourly and daily mean from its remaining samples and writes only values
that differ, recorded in `measurement_fold_repairs` (migration 0389).
- The compaction-tombstone purge waits for the repair per account, and the
boot sends no purge run before one account is repaired.
- The restore keys live `stats:` ids by source, and writes compaction
tombstones back from a file older than the account's repair, resetting
the repair.
The wrench beside Upload opens Settings, Appearance, Documents, the way
the illness journal offers its display options: preview cards or a
compact list, and months stacked as blocks or flowing across the full
width with the month name inline above the document that opens it. The
choice is stored per account in `users.documents_layout_json` behind
GET/PUT /api/documents/inbound/layout (preserve-when-absent, module
gated, the read delegable like the rest of the vault), carried in the
account-settings backup and cleared by the data wipe.
Deleting from the vault now always asks first. The bulk bar's Delete and
a focused card's Delete key both go through one confirmation on the page,
and the detail sheet's Delete opens its own; before, all three deleted on
the spot with only an undo toast behind them.
The selection bar is two tight lines: the count with an X at its right
edge, then one action row with Delete alone at the right in the
destructive style. Below 1024 px the verbs are icon-only at the 44 px tap
floor, so the row stays one line at 360 px. The bar carries the
selection-bar slot the Coach launcher already steps aside for.
Land every file the day view, the timeline and life events share before
any of them is built, so the server, the day view and the timeline can be
built side by side without editing the same file (#613).
Schema and migration 0387: LifeEvent with a short category list, calendar
start and end dates, a day/month/year precision, encrypted title and note,
a soft-delete tombstone and an index for the per-user live list. Every new
column is annotated for the column-reader sweep. Life events travel in both
backup purposes with their two ends in one file, are wiped with the
account, and are registered for key rotation.
src/lib/day/contract.ts carries the constants and wire types for
GET /api/day/{date}, /api/day/index, /api/timeline, /api/timeline/readiness
and the life-event routes, free of Zod so the navigation chrome can import
it; the schemas beside it feed the routes and the published contract. The
Coach and MCP tool name get_day is reserved there, and life events are
listed as the one section no model ever sees.
The opt-in timeline module, its navigation entry, the day-section and
timeline-lane surfaces, the day and timeline query keys (evicted by every
record data bundle), and stub routes that answer 501 behind their
admissions. The four cross-section reads declare the whole record for now;
the life-event list is a profile read, and life events join the profile
sharing domain and its consent copy in all seven locales. The day,
timeline and lifeEvents message namespaces carry their base copy as
release-pending keys.
The aggregate bundle cap moves to 4400: the branch already measured 4365
against 4335, and the new copy adds 10 KB.
A dose can now name a vaccine the catalogue does not list. The person
defines it once (a name, the diseases it protects against from the
catalogue's antigen list, the series length and the booster interval) and
logs doses against it; such a dose counts into each listed antigen's
series, settles a booster reminder keyed to one of them and offers a
booster, the way a catalogue combination does.
One resolver (resolve-vaccine-entry.ts) now answers for every consumer
that asked the catalogue directly: the series, the satisfy matcher, the
booster mint, the dose DTO, the doctor report, document links, the
upload suggestion, MCP search and the client's info sentences and
booster offer. The catalogue wins when a dose carries both.
The custom routes resolve the record through requireRecordAuth (list at
READ, create at WRITE with the actor-keyed write bucket, edit and remove
at MANAGE with the C4 overwrite audit) and join the sharing guard, the
MANAGE matrix and the activity feed. Removing a definition tombstones it
and lets go of its doses, each keeping a name. Definitions travel in the
backup and restore before the doses that name them; the dose form picks
them, adds one without leaving the form, and the page lists them for
editing. With the last contract stub implemented, the 501 helper and the
not_implemented code leave the tree.
Fetch the Open-Meteo air-quality feed (CAMS, domains=auto) with the weather:
17 hourly variables folded into local days by fixed rules (means and maxima
only with 18 of 24 hours, ozone as the highest 8-hour running mean, UV only
with the midday hours), a value the feed did not cover stored as null, never
as zero. Requests are cut into 90-day chunks, newest first, and chunks before
mid 2022 ask for the eight pollutant and index variables only. An air-quality
failure never costs the weather row; the day stays unfetched and the nightly
gap fill (up to 90 days per account, newest first) asks for it again.
Every Open-Meteo request asks an instance-wide budget kept in Postgres first
(500 a minute, 4000 an hour, 8000 a day, weighted by the hosted pricing
rule). A refused request is not sent and the run ends as done with
budget_blocked. OPENMETEO_AIR_QUALITY_URL and ENVIRONMENT_AIR_QUALITY_DISABLED
are honoured; the account switch lives on PATCH /api/environment/preferences.
The home, each dated location period and each stored day keep their coarse
location sealed as one value under healthlog/environment-location/v1. The
writers write the sealed value only, the readers open it and fall back to the
readable columns for rows the free-text encryption backfill has not reached,
and the backfill now seals all three and clears the readable columns. The
backup carries the location readable in a portable file and sealed in a
disaster-recovery file; a restore seals either under the receiving key.
Three correlation channels join the environment family (PM2.5, the ozone
8-hour high, the day's highest pollen count), read through the averaged
same-day window and left empty while air quality is off. The Coach and MCP
gain get_environment: module-gated, clamped to the lookback limit, no
coordinate or place name anywhere, with provenance and the attribution
lines. The settings section gets the switch, the coverage and the newest
day's values; the dashboard shows a quiet chip on a day with high pollen,
a warm night or very poor air. Copernicus and Open-Meteo are credited next
to the values.
The nightly folds soft-deleted every raw Apple Health sample they folded
into an hourly or daily stats row and kept it for 75 days. Nothing ever
read those rows as deletions; they only held a re-upload of the sample
off, and they were most of the measurements table and its indexes.
Ingest now refuses such a re-upload directly: a raw sample of a dense or
mean type, old enough to have been folded, whose hour or day a live
stats row covers, is a duplicate with reason folded_window. The batch
route and the Apple Health export import apply the same rule. With that
in place the dense retention and the daily-mean consolidation delete the
raw rows outright, and a new compaction-tombstone-purge job removes the
backlog: 5,000 rows a batch, one account at a time under the restore
lock, a short pause between batches, resumable because the predicate is
idempotent, queued at every boot and continuing itself while it makes
progress. A person's own deletions are never in its predicate. The sync
feed leaves compaction tombstones out and a restore does not write them
back; all three classify through the same function.
The batch route answers an exact re-sent sample without a transaction:
one read over the batch's sample ids, live or deleted, decides what the
reconciler would have decided, at the same point in the loop, so every
status, reason, counter and the cross-source merge stay as they were. A
batch of duplicates only no longer queues PR detection, a reminder sweep
or the detection audit row. Caches are evicted only for a foreground or
manual sync or a batch that wrote a hand-entered reading, and marked
stale otherwise; the targets cache follows the same split.
Also:
- POST /api/admin/maintenance/measurements queues VACUUM (ANALYZE) and a
REINDEX INDEX CONCURRENTLY per index, largest first, on a connection of
its own; it refuses while the purge runs, drops invalid leftover copies
and stops at the first failed rebuild. Runbook in docs/ops.
- The pure volume queues keep their finished jobs for 96 hours instead
of seven days, never less than the failure window plus a day.
- work_mem is set per app connection from DATABASE_WORK_MEM (16MB), also
when the statement timeout is disabled.
- The PR-detection fallback tick re-scans only accounts whose readings
or workouts changed since the previous tick.
- Type discovery for the status refill walks the live-rows index as a
loose index scan instead of grouping every live row.
- The host metrics chart polls every five minutes, like its sampler.
- The batch route records per-type HealthKit arrivals.
Land every file the v1.42 work shares before any of it is built, so the
parts built side by side never edit the same file.
Schema and migrations 0379-0386: the HEALTH_CONNECT measurement source,
ImportJob.kind, the air-quality columns and the per-user switch on the
environment tables, sealed environment locations with nullable plaintext,
managed-profile handover links, the per-type HealthKit arrival ledger, lab
unit-conversion provenance, and user-defined vaccines. Every new column is
annotated for the column-reader sweep.
HEALTH_CONNECT reaches every exhaustive list: the source labels, the
read enum (not the write allowlist), the source-priority ladder cap, now
the number of sources rather than a fixed eight, and the write-path smoke
test. RMSSD gets its own priority class and a 300 ms ceiling. The batch
syncTrigger gains `manual`, with its label on the Apple Health card.
Queues for the Health Connect import, the compaction-tombstone purge and
the measurement-table maintenance are registered with stub handlers;
host-metric sampling moves to every five minutes. Environment and work_mem
variables reach the compose whitelist, the production example and the
manifest. The proxy admits the handover link and redirects it at the edge,
and the Health Connect upload bypasses the proxy body limit.
The new routes answer 501 behind their final authentication and are
published in the contract; backup, wipe and key rotation classify the new
models and columns; the seven message bundles carry the base copy. Guards
that list release-pending keys and columns fail once an entry is wired, so
those lists empty themselves before the tag.
Also re-anchor a login-throttle fixture to the wall clock, where its
expiry is compared, and re-base the aggregate bundle cap on a measured
12 KB of new locale strings.
GET /api/auth/me gains coachReasoning: the level a turn runs at, the
person's own pick, the highest level the settings may offer, whether
reasoning can run at all, whether the lowest option is a real off, and
the layer that decided. Resolved on the server so the web settings and
the native client read the same answer.
The web reads it through useCoachReasoning. The Coach preferences route
names the stored level on its wide event, and a unit test round-trips
the level through the account-settings backup section.
Add the shared contract the reasoning release builds on, with no change
to how a turn runs today.
- Schema: operator reasoning controls on app_settings (0376), the
encrypted trail text on coach_messages (0377), and source,
last_used_at and source_message_id on coach_facts (0378).
- Provider contract: CompletionParams.reasoning and onReasoning,
CompletionResult.reasoning and providerState, AiMessage.providerState.
- Reasoning levels, thinking budgets and the background-job table in
src/lib/ai/reasoning/levels.ts; coachPrefsJson.reasoning, absent
meaning medium, salvaged field by field like every other pref.
- Stream: activity, memoryNote and planProposal frames, done.stop and
done.withheldResults, result.interim, a compare chart kind, eight
table refs, three more clarification kinds with an assumed choice,
the change_assumption chip, and assumptions, activity, stop,
memoryNote and planProposal on the provenance. Zod mirror, type
parity and the native-client key rule cover every new frame.
- Chat request: memoryDecision and planDecision, parsed and not yet
acted on.
- Stub contracts for the memory functions and the activity recorder,
inert until their implementations land.
- Backup, restore and rotation carry the new columns; a fact's message
reference is resolved against the restored messages like its
conversation reference.
- Message keys for the trail, memory, plans, assumptions, replies,
interim tables, comparison charts and reasoning settings in all seven
locales, each reachable from a catalog and checked both ways.
Taking the weekly preview from the writer dropped the schema check the
old scanner ran at the end, so a copy that violates the schema was
described as restorable. The collector now checks each section in its
JSON form as the writer shows it, an array section and a bulk table one
element at a time, plus the schema's one whole-document rule (a
canonical copy names every measurement by id). A copy that fails gets
no stored preview, and the preview route reads the copy itself, as it
did before. Nothing is kept beyond one element.
Measured on the preview memory test under --max-old-space-size=524:
held after the producer 3 MB with and without the check, raw peak
heapUsed 293 MB with it (335 MB without). The test now loads the
backup schema before its baseline: a module loaded once per process,
not memory that grows with the record.
backupPayloadSchema is built when validations/backup.ts is evaluated and
holds about 12 MB of heap. The restore job, the stored-backup readers and
the export writer import that module, and they sit on the boot path, so
every server process built the schema at start whether or not it ever
touched a backup.
The schema version, the compatibility check and the per-section summary
move to validations/backup-summary.ts, which never builds the schema. The
call sites that validate a file (upload, restore, the summary and download
routes, and the streamed reader's element check) load backup.ts with a
dynamic import at that moment; everything else takes only types. The
OpenAPI registry keeps its static import, since only the spec generator
loads it.
After warm-up on a production build under --max-old-space-size=524, the
used heap drops by about 29 MB, and the schema no longer appears in a heap
snapshot. A structural test refuses a new static value import of the
module outside the registry.
A document whose preview render fails for good (a PDF whose first page
renders blank) was queued by the boot-time backfill, refused, and queued
again at the next boot, without end.
The thumbnail job now keeps the failure with the document
(document_thumbnail_failures, migration 0374): the reason, the attempts
since the last success and when the newest failed. The backfill, both the
boot discovery and the per-account pass, leaves a document out after one
deterministic refusal (raster-failed, empty-render, pixel-cap) or three of
any other kind, until the document changes. A direct enqueue still renders
it, and a preview that is made clears the record.
A weekly or nightly backup job whose process died (a restart, a
container out of memory) stayed active until its two- or four-hour
expiry, and its retry walked the accounts in the same order: the account
that killed it has the oldest copy, so it led again, and every account
after it went without a copy for as long as that record stayed too
large.
Both passes now send with a five-minute pg-boss heartbeat, so a dead run
fails within minutes and its retry picks it up. Each pass records when it
started and finished every account (backup_pass_attempts, migration
0373); an account whose last attempt never finished goes behind all the
others. The backups page shows when the run in progress started and names
an account a run died under, for the weekly and the off-host card, and a
heartbeat timeout reads as what it means.
Since v1.39.6 the preview scanner parsed the JSON on its way into the
pieces, kept every section but the three bulk tables, and validated the
whole document at the end: a second copy of the record beside the
writer's own, inside the weekly job. On an account with a document vault
that is most of the record.
The writer now shows each section and each bulk row to an observer just
before writing it. The preview collects key uses from that and takes its
counts from the writer's own FullBackupCounts; nothing is parsed or kept.
An upload hands over the summary and key uses it has already validated.
The round trip pins the writer's counts and key uses to a full read of
the copy on an account with a row in every model, and a memory test holds
what the pass keeps once the producer is done.
Measured on 70,000 measurements, 3,000 mood entries and 40 one-megabyte
documents under --max-old-space-size=524: held after the producer
56 MB before, 3 MB after; peak heapUsed 398 MB before, 334 MB after.
A text column written before key ids existed holds the bare base64 of
nonce, tag and ciphertext. The key check read such a value as an encoded
Bytes column, decoded it once too often, and accused the v1 key of not
opening a copy it opens. The instance's stored AI key has exactly that
shape and is the shortest v1 value in every disaster-recovery copy, so it
was picked as the probe and refused every restore of an account with v1
values, even with the instance settings left out of the verdict.
A legacy value now reads as a string unless its decoded head is base64
text. Samples are kept per section and only the sections the verdict
counts are probed; a key is unreadable when none of its probes opens.
Binary values open with the label their column is sealed under, taken
from the encrypted-column registry, and are not probed when no label is
known. A preview stored by an earlier release is still read, and its
sample's layout is derived again from the value.
Reasoning models on Ollama and similar servers think by default and can
spend the whole max_tokens budget before the visible answer starts, so
the call comes back empty or times out. The Local and OpenAI-compatible
chain entries now carry an optional reasoning setting: Default sends
nothing, Off sends reasoning_effort "none", Low / Medium / High send
their own value. It is stored on the entry in aiProviderChain, so a
chain can mix a thinking model with one that does not know the field.
- Every exported resolver binds the entry's value onto the client, the
connection test included, keyed on the instance type so the operator
provider, OpenAI, Anthropic and Codex never carry it.
- PATCH /api/user/ai-provider accepts localReasoningEffort and
compatReasoningEffort and GET reports them. A provider absent from
the chain holds its value on a disabled entry; restating Default on a
chain that was never customised writes nothing.
- PUT /api/insights/provider-chain accepts reasoningEffort on local and
openai-compatible entries, refuses it on any other, and keeps the
stored value when the body leaves it out. Its schema is now shared
with the OpenAPI registry.
- A Default request body is byte-identical to the one sent before.
Refs #1126
The account's own mood categories, tags and hidden set were written only
inside the restore's mood-entry branch, so a vocabulary set up before the
first entry did not come back. They are written whether or not the file
carries an entry.
A practitioner's note, a visit's reason, outcome and body site and a
vaccination's note travelled as stored ciphertext in a portable file, the
same defect the cycle and mood sections had. They now go through the
shared sealed-text helpers: readable in a portable file, sealed under the
receiving host's key on restore, ciphertext kept only where this host
opens it and otherwise reported as visitCiphertext or
vaccinationCiphertext. The practitioner address and phone ciphertext of a
disaster-recovery file gets the same check, since the key preflight now
leaves these sections to it.
The labels of the mood tags and mood categories an account created
travelled as the stored ciphertext in a portable file, so a restore onto
a host with another key refused the file or wrote labels nobody could
open. They now follow the cycle note: readable in a portable file, sealed
under the receiving host's key on restore, ciphertext kept only where
this host opens it and otherwise reported as moodLabelCiphertext.
The sealing helpers move to one module both sections share.
A portable file carried the day-log note, the sensitive-category envelope
and custom symptom labels as the stored ciphertext, so a restore onto a
host with another key either refused the file or wrote values nobody
could open. The portable file now carries them readable and the restore
seals them under the receiving host's key; a disaster-recovery file keeps
the ciphertext.
A file written before this still restores: each ciphertext value is
opened with this host's keys first, and one that does not open is kept
out of the row and reported in the skip summary as cycleCiphertext. The
key preflight leaves the cycle sections to that per-value check.
The backfill and the restore of an older file read the creation day of
an end-only window in UTC while every reader of the window uses the
account's zone; both now use the account's zone (the default zone when
the stored one is unknown to Postgres). A window whose start lies after
its end, which the API refuses but older rows hold, becomes a one-day
course on the end day: every reader treated it as empty and ended, and
a swap would have invented expected doses. One shared rule
(courseFromWindow) serves the restore and the course writer.
A medication taken now and then (#1024) keeps one row and gains course
rows: calendar spans, both ends inclusive, an optional encrypted note, no
schedule or dose of their own. Courses never overlap and every course but
the latest has ended, so the latest course is always the one the
medication's startsOn/endsOn project. src/lib/medications/courses.ts is
the only writer of those two columns for a medication with courses; it
writes under a per-medication advisory lock and re-projects in the same
transaction, so reminders, the next-due line and intakeActionable keep
reading the columns and stay right.
- Migration 0368 adds medication_courses and backfills one course per
medication window (an end-only window starts on the creation day, or
on its end day when that lies earlier); a medication without a window
stays course-less. Idempotent, data-preserving.
- GET/POST /api/medications/{id}/courses and PATCH/DELETE
.../courses/{courseId} (owner-only); create seeds the first course
from a window; the medication PUT edits the latest course and refuses
a window that would break the rules (medications.course.*).
- List, detail, create and update carry courses, courseCount,
previousCourseEndedOn and canStartCourse, resolved on the server.
- Compliance expands over the span of all courses and drops the slots
between them; every compliance read selects the courses. A rolling
cadence is not anchored by an intake from a previous course.
- The card names the last course's end and the running course's number;
the detail Lifecycle group lists the courses with start, edit and
delete. The Coach compliance block, the MCP resource, the doctor
report window and table, the clinician view and FHIR (one
MedicationStatement per course with effectivePeriod and status) read
the courses.
- Backup carries courses per medication and the restore writes them and
re-projects; an older file derives the one course its window
describes. Wipe plan, encrypted columns, rotation and OpenAPI follow.
A medication can be filed under a category the person named, next to the
built-in ones (#1041). The labels live in medication_category_labels,
encrypted at rest, capped at 20 per account; a medication filed under one
carries its custom:<uuid> key in the category column.
- GET/POST /api/medications/categories (POST idempotent, 30/min per actor,
admitted at MANAGE like the cycle symptom vocabulary) and owner-only
PATCH/DELETE /api/medications/categories/{key}. DELETE moves every
medication filed under the category to OTHER in the same transaction
and answers movedCount.
- category is widened to a built-in value or custom:<uuid> on the wire,
the create and update bodies refuse a key that is not the caller's
(422 medications.category.unknown), and list, detail, the insights
medication tiles and the MCP medications resource carry categoryLabel.
A key with no owned label reads as OTHER and is counted on the event.
- The wizard's class step lists the person's categories under the
built-in rows with an inline add row; the list page gains a category
filter (session only) and a manage sheet for rename, hide and delete.
- Backup carries customMedicationCategories (decrypted labels) and the
restore writes them before the medications; wipe plan, encrypted
column registry, rotation script, sharing guards and OpenAPI follow.
A symptom of one's own ("aura", "headache") is a definition with an
encrypted name and an icon from the cycle custom-symptom allowlist; each
occurrence carries a time, a 0-10 intensity (the pain score's scale), an
optional encrypted note and an optional link to one of the record's open
illness episodes. Both ride the illness module: every route answers
illness.disabled when it is off.
- Migration 0369 (idempotent): symptom_definitions and symptom_events,
intensity CHECK 0-10, episode link ON DELETE SET NULL.
- /api/symptoms/definitions{,/[id]} and /api/symptoms/events{,/[id]}:
eight active definitions per record (a hidden one frees its slot,
showing it again re-checks), hide or purge, events create/list/edit/
delete, a new link must name an open episode of the same record
(422 symptoms.episode.notOpen). Error group symptoms.*.
- Sharing: reads at READ, logging an occurrence at WRITE, defining and
editing at MANAGE; C1/C3/C4 audit details; frozen lists, the handler
matrix and the activity verbs updated.
- Both encrypted columns registered and rotated; both models in the wipe
plan; both carried at both backup ends with occurrences nested in their
definition, plaintext in a portable file and ciphertext in a recovery
file, and a link to an episode the file lacks dropped and named
(symptomEpisodeReference).
- OpenAPI regenerated.
The clinical category lived in a side table the app created at runtime,
after migration 0004 had dropped it as orphaned. It is now the
MedicationCategoryAssignment model. Migration 0368 creates the table
IF NOT EXISTS with the runtime DDL, so an existing table converges with
its rows untouched (verified with prisma migrate diff against a database
holding the runtime table and against a fresh one).
The helpers keep their names and run on the Prisma delegate; the runtime
CREATE TABLE and the pre-transaction ensure call in the restore are gone.
The backup plan judges the table as a schema model (backed up, two-ended,
seeded in the round trip) instead of under RAW_SQL_TABLES, which is now
empty; its guard proves its matcher on a fixture string instead.
"Read with AI" and the lab scan used to run their model calls inside the
request. At the default response time a read of three calls outlasted the
100 s a proxy in front of the server commonly allows, and the person saw a
cut connection for a read the server was still doing.
Both now run as a background run (`DocumentAiRun`, migration 0366):
- `POST /api/documents/inbound/{id}/index` with `Prefer: respond-async`
answers 202 with a run id once every quick refusal has passed. Without
the header it answers exactly as before, which the iPhone app relies on.
A second request while a read of the same document is queued or running
attaches to that run.
- `POST /api/labs/ocr/extract` always answers 202. Only the web client
calls it.
- `GET /api/ai-runs/{id}` serves the run: the same body the synchronous
route answers with, or the failure code and status it would have given.
Scoped to the caller; another account's run is a 404.
The worker (`document-ai-run`, a long queue with a one-hour expiry, one
job per run under `lockedPass`) claims the run, asks the capability again
for the record and re-checks the wire for the provider it picks, sets the
run's deadline from the person's AI response time and refuses a read that
cannot fit the job's expiry. Sync route and worker run one body per kind,
so the two paths cannot drift. Budget is reserved at enqueue and settled
once by whoever ends the run.
Input and result are sealed with labelled AES-256-GCM, registered for key
rotation; the input is dropped when the run ends. A five-minute reaper
fails a run no worker took within fifteen minutes (and hands its
reservation back), fails a run past its deadline, and deletes a finished
run after an hour. Runs are outside backups and inside the data wipe.
The web client polls through the query-key factory and says, calmly, that
the read continues in the background, and when no worker has picked it up.
The insight claim row is now the daily-cap ledger. Its workout_id is
nullable with ON DELETE SET NULL (migration 0367), so deleting a workout
no longer frees the day's slot. The cap counts claims that reached the
provider plus live leases, and drops the separate count of insight rows.
A daily workout-insight-claim-cleanup queue removes claim rows untouched
for seven days.
Fill every User column on the source account, export through the streaming
writer, recreate the account with credentials of its own, restore through the
route, and compare: each setting column must come back with the value it left
with, and every identity, credential and bookkeeping column must still hold
the target's value. The section in the file is checked to carry nothing but
setting columns.
Both backup purposes now carry an accountSettings section. The restore writes
it to the account being restored inside the same transaction, names every
refused value under the new accountSetting skip catalogue, and re-warms the
stored texts after the transaction when a unit preference changed, the way the
unit settings routes do. A file without the section leaves the account row as
it is, which is how every older backup restores. The instance settings
restore stays its own opt-in.
Every column of the account row now has a backup verdict in
USER_COLUMN_BACKUP_CLASS: SETTING, CREDENTIAL, IDENTITY or OPERATIONAL. A
schema-derived guard fails on an unclassified column, and the setting columns
each get a codec in account-settings-backup.ts, so a column classified as a
setting without a way to travel does not compile.
The restore end checks each value the way the settings route that writes it
does: AI endpoints against this host's private-origin policy, languages, units
and time zones against the known vocabularies, threshold bands against their
bounds, the avatar against the upload's sniff and limits. A refused value is
left out and named; credentials, identity and the host's own cursors are never
admitted.
A column-level round trip over every two-ended model found four values
that left with the export and came back empty or re-stamped:
- MenstrualCycle.absorbedIntoId: the disaster-recovery file carried the
folded, soft-deleted start but not the start that absorbed it, so after
a restore removing the absorbing start could no longer bring the folded
one back. Carried in the disaster-recovery arm and written back only
when the file carries the start it names.
- Workout.externalSourceVersion: the source's version marker, stored so
the sync can replay it unchanged, was in neither end. Carried beside
metadata in the disaster-recovery arm.
- NutrientIntakeDay.externalSourceVersion and createdAt: neither was
selected, so a restored day lost its source marker and its first-arrival
time. Carried in both arms.
Files written before this change parse as before; every new field is
optional on the way in.
`instant.toISOString().slice(0, 10)` and parsing a key back as UTC
midnight are the idioms behind every day-boundary fix this year: the
evening west of UTC and the small hours east of it land on the
neighbouring day. A new rule, healthlog/no-utc-day-key, refuses them in
src/ outside src/lib/tz and tests, and refuses a disable directive for
itself that is not next-line or gives no reason.
The time zone module now names every legitimate form: userDayKey for a
person's day, dateOnlyKey / dateOnlyAtNoonUtc / dayKeyAsUtcMidnight for
date-only values and day labels, daysBetweenDateKeys / weekdayOfDateKey /
isCalendarDateKey / statedDateKey for key arithmetic and parsing.
Of the 133 sites the rule found, 84 are gone. Most moved onto a named
helper with no change in behaviour; these changed behaviour:
- cycle calendar, insights, Coach snapshot and prediction refresh start
their lookbacks at the first instant of the user's day
- a model-transcribed date like "June 10, 2026" no longer lands a day
early in a process east of UTC (documents and lab scans)
- a custom report range covers exactly the picked days in the profile
zone instead of starting the evening before west of UTC
- date-of-birth pickers cap at the user's today, not the UTC one
- an allergy onset date is stored at noon UTC like other date-only values
- the report's cycle summary, the Coach's usual BP range and the intake
projector's rollup recompute use the user's day
- vaccination dates render as dates instead of the previous evening
49 remain, each with its reason on the line above: 39 UTC by design (15
file name stamps, rollup and object keys, the UTC-day score engines) and
10 marked baseline. A guard test requires a classified reason on every
directive and holds the baseline to a ceiling that only goes down.
The restore dialog asked GET /api/admin/backups/{id}/summary for the
contents of a copy, and the route read the whole stored copy to answer:
every piece opened, every measurement parsed, the rest validated. On an
account of 1.8 million readings on slow hardware that took about two
minutes. The dialog stopped waiting after 15 s and said the contents
could not be read, and its automatic retry made the server read the
whole copy a second time.
The copy passes through the process once anyway while it is written, so
the preview is worked out there and kept on the row (DataBackup.preview,
migration 0364): the summarizeBackup counts, the schema version, and
the inner key ids with the sections they sit in and one sample value
each. Every creation path goes through storeBackupBlob (the weekly and
manual pass, and an upload), so every new copy carries one. The column
is plain JSON: counts and dates only, and the samples are ciphertext
that stands for what is inside the copy, which a key rotation must not
re-seal.
The route answers from it without opening the copy. Both verdicts are
taken again on every read, from what the preview keeps: whether this
release restores the schema version, and whether this server holds the
keys the copy needs. The pieces are still counted against the row, so
a copy that lost one is refused as the restore and the download refuse
it. A preview names the copy it describes and is ignored once the copy
changes; converting a single stored value into pieces carries it over.
A copy written before this has no preview. It is read whole once, as
before, and the result is kept on the row, so it is slow only the first
time; concurrent requests for the same copy share the one read.
Only a file that fails the schema answers 422 "Backup payload failed
schema validation". Read refusals keep their codes; anything else is
the server failing and answers 500.
The dialog no longer retries the preview, waits up to 55 s, and when it
stops waiting on an older copy it says the copy is still being read and
to open the dialog again in a few minutes.
Refs #1031
A restore on a freshly booted instance failed its clearing step with a
deadlock (40P01) while the boot-time step consolidation was folding the
same account. A consolidation day updates or mints the day's total and
then soft-deletes the day's per-sample rows; the restore deletes every
reading of the account in one statement. When the restore reached a
day's samples before its total, each side held what the other needed
next.
The restore already took an advisory lock per account before its first
delete. Every day transaction of the passes built on runConsolidation
(step and mean consolidation, the per-sample cumulative drain, the dense
intraday retention) now takes the same lock in shared mode as its first
statement. A restore that starts mid-day waits for that day to commit;
a day that would start while a restore runs is refused at once, and the
pass leaves that account for its next run instead of folding rows the
restore is about to replace. The passes still share the lock with each
other.
A new integration test builds the lock order that deadlocked, with a
row trigger holding the consolidation on the day's total while the
restore starts; before this change the consolidation was cancelled with
a deadlock. A guard test requires every transaction in a pass on
runConsolidation to open with the shared lock.
Refs #1031
Prisma's debug logger keeps the arguments of its last hundred calls in
memory whether or not logging is on, and the client and the Postgres
adapter both log every statement with its parameters. A restore that
wrote 1 000 readings per statement therefore kept a copy of about the
last 100 000 readings alive for the whole measurement write: about
60 MB on 1.8 million Apple Health readings, next to a V8 heap limit of
524 MB in the default 1 GB container.
The bulk insert now splits whatever it is handed into statements of at
most 100 rows, which holds about 7 MB. The Apple Health export import
goes through the same helper and gets the same bound. The write costs a
little time: 94 s instead of 76 s for 1.8 million readings on a fast
host.
Refs #1031
Marc-André Bombeck