The timeline lane, the day view and the life-events service read every
LifeEvent column; the column-reader sweep counts all six as consumed once
the markers are gone.
Twenty-one foreign keys with ON DELETE CASCADE or SET NULL had no index
leading with their column, so deleting a parent scanned the child table
once per row; medication_intake_events.medication_id was the one an
account delete paid for. Seventeen small-table indexes and the measurements
(user_id, type, measured_at) index repeated the leading columns of another
index and only cost writes.
Migration 0415 adds and drops the small-table ones in one DO block under a
bounded lock wait; 0416 drops the measurements index concurrently, in a
file of its own. An integration test holds the schema to both rules from
the catalog, each with a counter-check, and compares the plans of the
user-and-type query shapes on a seeded table with and without the dropped
index. The lock-timeout guard now also requires a CONCURRENTLY statement
to stand alone in its file.
medication_compliance_rollups lost its last reader when compliance moved
to the schedule-anchored computation, yet every dose write still upserted
it, a boot backfill refolded it, and the wipe and backup plans carried it.
The per-dose hook keeps the one derived row that still hangs off a dose
day, the supplement nutrients, under dose-day-refresh. The pending-row
mints (today projection, reminder) no longer call it, since an untaken
dose adds no nutrients. The import job finishes without its per-day
recompute phase; its progress keeps touchedDays and rollupProcessed on
the wire, inert and documented as deprecated. The backfill queue is
retired on boot and migration 0414 drops the table under a bounded lock
wait.
migrate deploy runs a migration file statement by statement, so the
top-level SET LOCAL lock_timeout in 0412 did nothing: the migration
waited as long as the lock was held, with every write to measurements
queued behind it, and a file that failed half-way stayed half-applied.
0412 now takes its locks inside one DO block with a statement-scoped
timeout, which rolls back whole, after an idempotent CREATE OR REPLACE.
The entrypoint runs migrate deploy through scripts/migrate-deploy.mjs.
A 55P03 failure of a migration marked -- @retry-on-lock-timeout is
marked rolled back and retried after 15/30/60/120/240 s, read from
Prisma's report or, after P3009, from the stored row. Every other
failure stays fatal. A guard holds migrations to the shape that makes
the marker true, and 0411/0413 add their columns with IF NOT EXISTS so
a manual re-run is safe too.
CREATE TRIGGER waits for an autovacuum that is preventing wraparound, and
every write to the table queues behind the waiting request, the running
release's included. SET LOCAL lock_timeout fails the migration after ten
seconds instead, and the file rolls back whole.
One run reads at most forty pages of a hundred activities. A longer history
ended as incomplete with no note of how far it got, so every boot read the
same newest pages again, spent up to 190 of the instance's 200 requests per
fifteen minutes on them, and never reached the older activities.
The run now records the start of the oldest activity it read and sends the
next run for after the rate window; that run asks Strava for the activities
before it. A run that gets no further records nothing and sends nothing. The
place is cleared when the backfill completes.
users.insights_cached_text held the model's briefing, readings and
recommendations about the account in the clear. Migration 0411 adds
insights_cached_text_encrypted (AES-256-GCM, binary layout, purpose-bound
label) and a partial discovery index; the free-text encryption backfill
seals the existing rows one at a time, fail-closed, carrying updatedAt and
guarded on the value it read. Every reader, writer and clear goes through
cached-text-cipher.ts, and a guard keeps the column names there. The
column joins the encrypted-column registry and the key-rotation script,
stays out of backups as operational, and is reset by the data wipe.
`updated_at` sits in the sync-feed index, so every update that moved it
wrote a new row version on another page and an entry in every index.
Provider re-syncs rewrite many unchanged rows, which grew the table to
several times its live size and sent each unchanged row down the sync
feed again. Migration 0412 adds a row trigger that keeps `updated_at` and
`sync_version` when an update changes nothing else, so such an update is
heap-only and the feed stays silent; it sets `fillfactor = 85` and a 5 %
autovacuum threshold. The maintenance queue gains a nightly check that
vacuums when dead rows pass a tenth of the live ones and warns when the
heap is more than three times what its live rows need, which only a
`VACUUM FULL` in a window gives back. The admin card keeps showing the
operator's runs only.
A resumable collection counted the pages it had written across every
backfill attempt and stopped at its ceiling (10,000 for heart rate). A
watch that records heart rate every few seconds reaches that count two to
three years back, and since it never reset, every later slice stopped at
the same place, failed as stuck, and the backfill was never stamped
complete.
A walk is now stopped only when it stops moving: 20 pages in a row that
reach no further back than the pages before them end it. A history
backfill reads back at most GOOGLE_HEALTH_BACKFILL_HORIZON_YEARS (20 when
unset). A data type that ends either way counts as finished, and where and
why it ended is kept in backfillProgress past the completion stamp. An
oldest-first list is read whole, as before. The slice chain's ceiling is
derived from sixty days of back-to-back slices instead of the old page
count. An account left at the old ceiling continues from its edge with
the next slice.
Refs #1220
Connect, status, places, settings, test, sync and disconnect replace their
stubs. A nightly job (03:20) and the connect backfill read the timeline,
visits and residency endpoints through safeFetch and store one row of daily
values per local day: distances, places visited, minutes away from the home
place and the sealed country with its travel-day verdict. A private instance
needs its exact origin in DAWARICH_PRIVATE_ORIGINS.
The connection joins the shared ledger and the connection health model,
watched for errors only. The day carries a movement block, the model
projection drops the travel country, the two movement channels join the
correlation matrix under the environment module, travel days become trip
suggestions, and the days travel in the environment backup section.
Three levels (frugal, standard, full) with an operator ceiling and default
and a per-account choice within them, resolved in the capability layer for
the record and published on GET /api/auth/me as ai.usage: the level in
effect, the stored choice, the bounds, why (user, default, admin_cap), the
three options with whether each is allowed and an estimate of tokens per
day scaled from the record's last seven full days of coach_usage, and the
measured mean. A delegate sees the level, never the owner's spend.
PUT /api/auth/me/ai-usage stores the account's own choice or null for the
default, refusing a level above the ceiling with 422 ai.usage.aboveCeiling
and meta.maxLevel. The new supplementLabel capability needs standard or
above; below it the capability is unavailable with the new usage_level
reason, refused with 403 ai.usage.levelTooLow and meta.requiredLevel.
Refs #1217
POST /api/biomarkers/{id}/merge folds markers into one in a transaction,
converting readings where the registry knows the units and refusing the
whole merge where it does not; a dry run answers the same result and
writes nothing. Aliases appear on the marker and are edited through the
marker's PUT. Scan and document review rows name the matched marker or
offer up to three, and a chosen marker is remembered as an alias. The
doctor report, FHIR and MCP group readings by marker. Aliases and the
merge ledger travel through backup and restore.
Migrations 0402 to 0410: AI usage levels, biomarker aliases and merges,
medication ingredients (with the supplement label run kind), life
circumstance fact kinds, AI sections and AUDIT-C, Dawarich context, the
pulse day-mean method, intraday hour buckets, the readable location drop
and five relation link tables. Every new column carries its reader or a
pending-consumer marker.
0409 refuses to run while any home, location period or stored day still
holds a readable location, and names the counts. Locations are read from
the sealed copy only; the encryption backfill no longer handles them, and
the demo seed writes sealed locations only.
The new tables are backed up and wiped, the new sealed columns are in the
rotation registry with labels of their own, the AI usage level and the
life-circumstance confirmations travel with the backup, and the wellbeing
score reads the mental-health screeners only.
Migration 0401 turns healthkit_type_syncs.type into text so the ledger can
hold MEDICATION_DOSE, and adds last_background_at (seeded from rows whose
latest arrival was a background one). POST /api/medications/intake/bulk
takes syncTrigger and records APPLE_HEALTH dose arrivals; the Apple Health
status read serves a MEDICATION_DOSE freshness entry and lastBackgroundAt
on every entry with arrival facts.
A backfill noted progress only once a whole collection was read to the end.
A heart-rate history of years that one attempt could not finish (its budget,
a rate limit it could not wait out, a single failed request among thousands
of pages) started the next attempt from the newest page again, finished
nothing, and failed with "marker not stamped" on every attempt.
- Every spot-reading collection read page by page now records, after each
page it writes, the oldest sample it has written (backfillProgress.cursors).
The next attempt asks Google only for samples at or beyond that edge. The
edge is held back once a page's write fails, so no rows are skipped.
- The backfill runs in slices of five minutes or 500 pages. A slice that
finished a collection or moved an edge continues with the next one; only a
slice that moved nothing fails, and its error names each unfinished
collection and why the slice stopped. The slice ceiling is derived from the
collections and the page ceilings of the resumable ones.
- A rate limit Google will not lift within a minute ends the slice instead of
failing the walk, and the next slice starts five minutes later. Server
errors, timeouts and dropped connections are retried three times.
- The heart-rate stream is read last, so the light collections finish in the
first slice.
- The hourly sync waits up to eight minutes for a running slice instead of
skipping the account for as long as the import runs.
- Each slice annotates googleHealth.backfill.slice with pages, elapsed time,
stop reason, per-collection status and edges, and reports progress on the
job.progress line.
Fold the ledger state, liveness verdict, plan and scope gaps and the
provider reauth flag into ok / limited / degraded / action_required with a
reason code, the one action, the unavailable types and redacted
diagnostics. Keep the last ten sync attempts per connection for those
diagnostics, publish the state on the integrations envelope, every
per-provider status route and the Apple Health config read, and page the
operator once per failure streak instead of once a day.
The single-reading edit cleared sourceValue and sourceUnit whenever the body
carried a value, and the web edit sheet always sends one, so a date or note
edit erased the "converted from" hint. It is now cleared only when the value
differs from the stored one. The sheet also opened a converted reading at its
stored precision (100.90191999999999); it now starts from the value as the lab
surfaces show it and leaves an untouched value out of the request, so the
stored full-precision number is not replaced by its rounded display.
Withings split its public API into plans on 2026-10-12. A request naming
one meastype outside the free plan is refused whole with status 277, so
every measure sync stopped importing anything, and the 200..299 reauth
range parked the connection at error_reauth.
The measure, sleep-summary and ECG legs now drop the documented advanced
types on a 277, split the rest until the refused keys are isolated
(bounded), remember the refusal per connection and re-probe weekly. 277
is classified persistent and never enters the ledger; connections parked
over it are lifted by the fallback tick, and the held watermark lets the
next run read the whole gap.
Every raw-SQL clock read against a timestamp column without a zone now
writes NOW() AT TIME ZONE 'UTC', so passkey and login challenges, rate
limits, provider pauses, coach usage stamps, tombstone and measurement
windows no longer pass through the session time zone even where the
per-connection pin cannot apply. The regression-window bounds stay naive
UTC timestamps instead of being turned into timestamptz. A structural
guard keeps bare NOW() / CURRENT_TIMESTAMP out of raw SQL outside a short
list of timestamptz targets. The integration test sets the database
default to Europe/Berlin and drives challenge claims, session expiry, the
native sign-in handoff freshness check and the rate-limit window through
the app's client, plus the challenge claim through an unpinned client.
Self-hosting docs describe the pin for an existing or shared Postgres.
A history backfill carried one access token through walks that outlive it.
After an hour every request answered 401, which classifies as a revoked
grant, so the account was parked at error_reauth with a refresh token that
was still valid, and every later run skipped it until a manual reconnect.
Running beside the hourly sync, the backfill also pushed the account over
Google's per-minute quota, and each 429 failed its whole collection.
- Walks take a token source that renews the access token before it expires
and once more when Google refuses it mid-walk.
- A 429 is waited out (Retry-After, RetryInfo, else a doubling backoff) and
the request repeated; it is never a reauth.
- error_reauth only short-circuits a run when the token endpoint refused the
grant (needsReauth). A connection parked by a data request's 401 refreshes
on its next run and clears itself when that run completes.
- One run per account at a time: syncs, the backfill and the sleep repair
share an advisory lock. The hourly poll skips a held account, a manual
sync answers 409 sync_in_progress, the backfill waits a bounded time.
- A backfill attempt records each collection it finishes cleanly
(backfill_progress) and the next attempt resumes with the rest.
- The incremental watermark is stamped with the cycle's start, not its end.
Overlapping sleep segments that share a stage and an end share the natural
key (type, measuredAt, source, sleepStage). A re-score that moved one row
onto a sibling's slot threw P2002 and failed the sync on every run.
- A batch keeps one reading per natural key, chosen the same way each time
(longer segment, then smaller externalId).
- A move that still collides is settled by reconcileExternalMeasurement,
which retires the stale row, instead of failing the sync.
- Updates run before inserts, and the key-format rescue no longer re-keys a
row this batch is moving.
- The sleep replace-by-window protects every session's fresh rows, so two
overlapping sessions no longer tombstone and resurrect each other hourly.
Refs #1194, #1195
The note for records whose findings predate the seasonally adjusted
engine compared against a fixed calendar date, so findings an instance
kept writing under the earlier engine until its deploy did not count.
The first discovery sync of the new engine now stamps
AppSettings.correlationSeasonalEngineSince before it writes; without a
stamp every stored discovery pattern counts as an earlier finding.
A restore from a portable export reset each pattern's createdAt to the
restore day and dropped the note for good. createdAt now rides every
export, and a file without it falls back to lastComputedAt.
A background job fetches air quality, pollen and UV for every earlier
local day with a reading, mood entry, dose, symptom or workout: after the
update (boot and nightly discovery), when the home is set and when air
quality is switched on. A day with a weather row gets its air quality; a
day without one gets both, at the covering location period or else at the
home, also before the home was set. Days the source does not reach (before
2013 in Europe, before August 2022 elsewhere) stay empty and are not
counted.
Pending days are grouped by coarse location and cut into ranges of up to
90 days, joining days at most 14 apart, newest first. A run holds itself to
300 of the account's 400 daily calls and half of each instance window,
stores its progress on the user (migration 0391) and sends a follow-up while
it makes progress; a budget stop waits for the next discovery. The
Location & weather card shows the progress.
The air-quality client now asks pollen and dust from 2021 and skips the
request outside Europe before CAMS global starts.
created_by_id cascaded, so a Guardian who deleted their own account took
every handover they had issued with them, including a claimed one whose
decision the new owner had not made yet. The claim had already applied
the proposal, possibly MANAGE for a co-Guardian, and the owner lost the
one place that asks them to confirm it.
The column is nullable now and cleared on delete (migration 0390). An
unclaimed link without a creator is refused like any link whose creator
lost access. A decision that names a Guardian whose account is gone is
refused with 422 instead of failing on a grant for a missing account.
The daily-mean consolidation and the dense hourly fold cut at
`now - threshold` instead of a local day boundary. A day was folded in two
runs and the second run overwrote the stored mean with the mean of the
later part alone: every mean-type day, and the boundary hour of every
heart-rate, HRV and SpO2 day, plus the derived resting figure and the DAY
rollup of such a day. The Health Connect importer split its raw window at
the same unaligned instant.
- Fold up to the start of the account's local day the threshold ends in
(`foldBoundary`), for every consolidation pass and the Health Connect
importer's raw/hourly split.
- A window that already has a live `stats:` row is refolded from every
sample of it, live rows plus the fold's own tombstones; with none left
the stored mean stays.
- `folded_window` refuses a raw sample only when its local day lies before
the aligned boundary and a live `stats:` row of its source covers it,
also within reach of the sample after a timezone change.
- A deletion inside a day the fold had not finished is class B.
- Add a one-time, resumable fold repair per account that recomputes each
hourly and daily mean from its remaining samples and writes only values
that differ, recorded in `measurement_fold_repairs` (migration 0389).
- The compaction-tombstone purge waits for the repair per account, and the
boot sends no purge run before one account is repaired.
- The restore keys live `stats:` ids by source, and writes compaction
tombstones back from a file older than the account's repair, resetting
the repair.
The wrench beside Upload opens Settings, Appearance, Documents, the way
the illness journal offers its display options: preview cards or a
compact list, and months stacked as blocks or flowing across the full
width with the month name inline above the document that opens it. The
choice is stored per account in `users.documents_layout_json` behind
GET/PUT /api/documents/inbound/layout (preserve-when-absent, module
gated, the read delegable like the rest of the vault), carried in the
account-settings backup and cleared by the data wipe.
Deleting from the vault now always asks first. The bulk bar's Delete and
a focused card's Delete key both go through one confirmation on the page,
and the detail sheet's Delete opens its own; before, all three deleted on
the spot with only an undo toast behind them.
The selection bar is two tight lines: the count with an X at its right
edge, then one action row with Delete alone at the right in the
destructive style. Below 1024 px the verbs are icon-only at the 44 px tap
floor, so the row stays one line at 360 px. The bar carries the
selection-bar slot the Coach launcher already steps aside for.
Land every file the day view, the timeline and life events share before
any of them is built, so the server, the day view and the timeline can be
built side by side without editing the same file (#613).
Schema and migration 0387: LifeEvent with a short category list, calendar
start and end dates, a day/month/year precision, encrypted title and note,
a soft-delete tombstone and an index for the per-user live list. Every new
column is annotated for the column-reader sweep. Life events travel in both
backup purposes with their two ends in one file, are wiped with the
account, and are registered for key rotation.
src/lib/day/contract.ts carries the constants and wire types for
GET /api/day/{date}, /api/day/index, /api/timeline, /api/timeline/readiness
and the life-event routes, free of Zod so the navigation chrome can import
it; the schemas beside it feed the routes and the published contract. The
Coach and MCP tool name get_day is reserved there, and life events are
listed as the one section no model ever sees.
The opt-in timeline module, its navigation entry, the day-section and
timeline-lane surfaces, the day and timeline query keys (evicted by every
record data bundle), and stub routes that answer 501 behind their
admissions. The four cross-section reads declare the whole record for now;
the life-event list is a profile read, and life events join the profile
sharing domain and its consent copy in all seven locales. The day,
timeline and lifeEvents message namespaces carry their base copy as
release-pending keys.
The aggregate bundle cap moves to 4400: the branch already measured 4365
against 4335, and the new copy adds 10 KB.
A dose can now name a vaccine the catalogue does not list. The person
defines it once (a name, the diseases it protects against from the
catalogue's antigen list, the series length and the booster interval) and
logs doses against it; such a dose counts into each listed antigen's
series, settles a booster reminder keyed to one of them and offers a
booster, the way a catalogue combination does.
One resolver (resolve-vaccine-entry.ts) now answers for every consumer
that asked the catalogue directly: the series, the satisfy matcher, the
booster mint, the dose DTO, the doctor report, document links, the
upload suggestion, MCP search and the client's info sentences and
booster offer. The catalogue wins when a dose carries both.
The custom routes resolve the record through requireRecordAuth (list at
READ, create at WRITE with the actor-keyed write bucket, edit and remove
at MANAGE with the C4 overwrite audit) and join the sharing guard, the
MANAGE matrix and the activity feed. Removing a definition tombstones it
and lets go of its doses, each keeping a name. Definitions travel in the
backup and restore before the doses that name them; the dose form picks
them, adds one without leaving the form, and the page lists them for
editing. With the last contract stub implemented, the 501 helper and the
not_implemented code leave the tree.
Marc-André Bombeck