A batch that carried a type with duplicates only moved its
lastReceivedAt, lastTrigger and lastBackgroundAt, so a type whose live
delivery had stalled kept reading as freshly arrived for as long as a
replay, a retry or an older client re-sent its old samples.
The arrival ledger now moves those stamps only for a type the batch
brought at least one new or changed row for. An unchanged stats:
re-post and an hour posted again with its stored value count as
duplicates here; the wire keeps answering updated for them. A
duplicates-only type stamps the new last_seen_duplicate_at column on
its existing row and nothing else. The intake bulk route's dose entry
follows the same rule through the shared recorder. lastSyncedAt and
healthKitLastSyncedAt are unchanged.
GET /api/integrations/healthkit serves the stamp additively as
metricFreshness[].lastSeenDuplicateAt, and the Apple Health card names a
re-send that came after the last arrival.
Refs #1259
The switch was stored and shown, but nothing reads the topics flag
yet, so the page promised a change it could not make. The section,
its nav entry, its loader and its copy are gone, so its palette
entry goes with them; /settings/preview is no longer a route. The
stored flag, /api/settings/preview-features and its backup round-trip
stay, and the column's pending-consumer note says why.
0440 adds its columns with ADD COLUMN IF NOT EXISTS like 0424 and
0432; a structural test holds every migration from 0422 on to that
form. 0430 clears insights_cached_at where no sealed payload is left,
so the nightly pregenerate refills an account whose only copy was the
readable column instead of keeping an empty card fresh.
The deploy runbook notes that 0430 drops a users column: an old process
fails on its next full user read, so stop before starting rather than
running old and new side by side.
A week or month of steps, active minutes, workouts by sport and training
load, each day placed against the person's own spread over the 84 days
before the period and each period against the median of the twelve weeks
or six months before it. Optional weekly and monthly goals live on the
account (activity_goals_json), travel with the backup and are cleared by
a data wipe. The calendar sits on the workouts page; the steps page links
to it. Computed server-side without AI.
A guardian finishing a connect for a managed profile could attach their
own provider account to the profile. Every OAuth callback (Withings,
WHOOP, Fitbit, Google Health, Strava, Polar, Oura) now refuses a
provider account another record already holds with reason
already_linked, shown as its own message on the integrations page.
WHOOP's owner_conflict tag becomes already_linked as well.
Oura keeps the account id of a grant (personal_info, migration 0441) so
the check has something to compare; existing connections fill it at
their next connect.
A disconnect leaves Withings webhook subscriptions and the Strava app
authorisation in place while another record holds the same account,
and the Withings webhook routes an account held twice to the record
that linked it first instead of whichever row a lookup returned.
The provider app copied from a guardian onto a managed profile is now
marked (migration 0442) and removed again on disconnect or when that
guardian no longer holds a live MANAGE grant: revoke, renounce,
removal, handover, account deletion and data wipe. Each removal is
audited as integration.client_app.removed.
Migration 0434 turns the route opt-in on for every account that already
holds a stored workout track, so its tracks keep arriving after the
upgrade; only accounts without one start off.
Workouts stored before this release with a track but no elevation get
their metres climbed written into Workout.elevationM by a one-time
background pass. It runs detached at worker boot, reads one track at a
time with a pause between batches of 100, and keeps its position in
app_settings, so a restart resumes and a finished pass ends at once.
The "starting" health reason is bounded: after five minutes the probe
says plain degraded and the process logs one line, and the time the
background role took to come up is logged either way. The compose
healthcheck's ten-minute start period already keeps those probes from
counting; its comment now says so.
Room climate from a home automation system gets three types of its own:
INDOOR_CO2 (ppm), INDOOR_TEMPERATURE (celsius, shown in Fahrenheit for
imperial readers) and INDOOR_HUMIDITY (percent). They arrive through the
existing ingest token (measurements:write, source EXTERNAL), sit in the
environment cluster beside daylight, and the daily rollup carries their
mean, low and high like any spot reading. They never mix with the body
temperatures or with the outdoor environment channels.
The Home Assistant page documents the three types, a rest_command that
sends a room as one request, a single-room automation, and an
importable blueprint (docs/integrations/blueprints) that skips sensors
reporting unknown or non-numeric states, converts Fahrenheit, and
stamps readings to the minute so a retried push is refused rather than
written twice.
Six Apple Health signals leave the deferred list and ride
POST /api/measurements/batch and the export.zip import end to end:
EXERCISE_MINUTES, STAND_MINUTES and STAND_HOURS (the exercise and stand
rings), MINDFUL_MINUTES, STATE_OF_MIND_VALENCE (iOS 18 State of Mind,
-1..1) and AFIB_BURDEN (AFib History, percent of the week). The ring
types and mindful minutes are cumulative, so the daily rollup is their
sum; AFib burden is scaled strictly from Apple's fraction and a value
above 1 is refused rather than guessed at. The archive import takes
mindful minutes from the session's dates and skips idle stand hours.
Every registry carries them: units, ranges, labels in all locales,
categories, chart tokens, report catalogue (mindful minutes and the
valence fenced as sensitive), signal registry with FHIR codings, and
MCP exposure for the three ring types only.
Workout GPS tracks are now an opt-in, off by default
(PATCH /api/auth/me/workout-routes, a switch on Data & Privacy). While
it is off, the workout batch route and the export import keep every
workout and drop its track with the same per-entry status. With it on,
the import reads the referenced GPX files from the archive one bounded
member at a time, thins each to the point cap and stores it sealed.
The climb a track carries is written to Workout.elevationM at write
time, so the workout insight no longer opens the track, and the AI
egress guard now freezes the set of files that read route geometry.
Every Oura collection, workouts and heart rate included, is now read
back to the start of the account's history by a backfill that runs
through the shared integration lane in five-minute slices. Each
collection walks from the day the backfill began toward the past one
window at a time (ninety days, thirty for sleep, seven for heart rate)
and the progress note on the user row records after every window where
it got to, so a slice cut short by its budget, the lane or a rate limit
resumes there. A 429 ends the slice and the next one waits for the
Retry-After Oura named. Activity and sleep are walked first to learn
where history starts; the rest stop a week before it.
Heart rate streams a page at a time and keeps only the days it touched,
so memory stays flat however long the history is. A collection the
grant does not cover is a scope note, not a failure; the first slice of
the next chain asks again, and a connect clears the completion stamp so
a newly granted scope is read without walking anything twice.
The two columns arrive empty, so every existing connection is offered
the backfill at the next boot without reconnecting. Before anything
else the backfill retires the WALKING_RUNNING_DISTANCE rows earlier
syncs wrote from the equivalent walking distance and refolds their days;
an account that kept such rows after disconnecting is offered too.
Every write has gone to insights_cached_text_encrypted since 0411, and
the boot-time free-text backfill sealed the rows written before it.
Migration 0430 drops insights_cached_text and the backfill index that
served it. A host that skipped the intervening releases loses only a
cache of model output; the next warm writes a fresh, sealed payload.
The cipher module now reads and writes the sealed column only, the
backfill job loses its insights step, and consent withdrawal, the wipe
plan, the backup plan and the session projection follow the schema.
Backups never carried the column, and restore ignores unknown account
settings; a round-trip test now proves an old backup naming the field
still restores cleanly. The access guard pins exact counts for where
the sealed name may appear. Fixtures, the demo seed and the demo
migration script seed the sealed column instead of plaintext.
Add per-user columns for the labs, illness and checkups list presentation
and for preview-feature opt-ins, each with a GET plus a preserve-when-absent
write under /api/settings. The list hook now reads and writes the server
value and carries a browser-stored order up once when the server holds
none, so an existing arrangement follows the person to every device instead
of resetting.
Migration 0420 looked for an RMSSD twin by externalId only. A Fitbit or
Google Health SDNN row with no externalId, or another one, on the same
night as an RMSSD row would have broken the measured-at unique key and
aborted the block at boot. The twin check now matches either identity,
a row left behind by a live twin is tombstoned, and an RMSSD tombstone
that would make a live row look twinned is removed so the row moves.
The backup restore applies the same rule and no longer counts a
tombstone as a twin.
One failing per-night stage read threw out of the loop, so the nights
already read and the summary vitals were never written. A failed read
now leaves the others to be written, books the leg failed afterwards
and marks the night to be read first next run. A night that answers
without a stage three times is no longer read on every run; a full sync
still reads it. The per-night record lives on the connection (migration
0421) and is reset on reconnect.
Fitbit's `dailyRmssd` and Google Health's daily HRV average are both
RMSSD, but the mappers wrote them into HEART_RATE_VARIABILITY, the SDNN
series. The `hrv` ladder ranks both above Apple Health, so an account
with an Apple Watch and a Fitbit had a series that switched estimator
from day to day. The mappers now write HRV_RMSSD, and the `hrvRmssd`
ladder carries both sources where they sat on the SDNN one.
Migration 0420 moves the rows already written (tombstones included,
with a sync-version bump so the change reaches the phone), drops their
SDNN rollup buckets and the RMSSD buckets of every account whose rows
moved, so the boot-time rollup discovery folds those accounts again,
and deletes the SDNN personal records those rows had set. One DO
block, so the move and its cleanup land together.
The signed-state callbacks (Oura, Polar, Strava) claimed the state before
the session, code and issuer checks. A denied consent comes back without a
code, so it spent the claim on a `nocode` error page, and allowing on the
same authorize URL afterwards never exchanged the valid code: the browser
was sent to the stored error page instead. Every check that refuses a
callback without touching the code now runs first; the claim still sits
strictly before the exchange.
Expired claims were swept only when a callback completed, so a quiet
instance kept its last rows. The daily OAuth-state cleanup at 03:20 now
sweeps them as well.
Resilience sits behind Oura's `stress` scope, not `heart_health`, so a
connection that granted every scope HealthLog asked for still answered 401
on `daily_resilience`. `src/lib/oura/scope.ts` now maps every collection the
sync reads to the scope the live API enforces and is the one source for the
authorize request, the per-collection permission detection, the reconnect
hint and the settings card. The request asks for `stress` as well.
Oura names a missing scope in its 401 body ("Token is not authorized access
stress scope."), while a dead token reads differently. A collection refusal
now carries that scope: it spends no token refresh, never counts toward the
reconnect threshold and marks only the collections it names. A 403 (a lapsed
membership) is a persistent failure rather than a reconnect. The callback
also records what the token response says was granted, so an unticked
permission shows on the card before the first sync.
A browser can deliver an OAuth callback twice with the same code. Both
requests exchanged it, and Oura answers a reused code by revoking the token
the first exchange issued. Every connect callback (Oura, Polar, Strava,
WHOOP, Withings, Fitbit, Google Health) now claims its state in
`oauth_callback_claims` before the exchange: one request exchanges, the
duplicate waits for it and redirects to the same page without touching any
token. The ledger flows claim inside the transaction that consumes their
state row. A failed exchange settles with its error page, and a new connect
mints a new state.
Oura token refreshes are serialised per user with an advisory lock, so
overlapping syncs no longer spend the same single-use refresh token, and the
write stays conditional so a reconnect's new grant is never overwritten by a
refresh of the old one.
An audit row whose address no database can place stayed without a
location, so every hourly pass looked it up again for its whole 30-day
window, and finding it walked every audit row of those 30 days through
the created_at index.
Migration 0418 adds audit_logs.geo_attempted_at and a partial index on
the rows still missing a location or carrier. Every row a pass looks at
is stamped, and a row tried less than seven days ago is not picked
again, which still lets a range a later GeoLite2 release learns resolve.
The candidate read goes through the partial index, which holds only the
backlog.
readAllTimeExtremes read every raw reading of weight, blood pressure and
pulse twice on each call, a sequential pass over the measurements table
that made it the largest read on the database. The DAY tier already
holds the same per-source day fold except for the extremes of the
readings themselves: min_value and max_value widen to a row's own
intra-hour range.
Migration 0417 adds reading_min_value and reading_max_value to
measurement_rollups. The writer fills them from the same aggregate, and
the boot re-fold refills existing rows through the arm that already
watches sum_xy. The reader folds the tier for whole UTC days inside the
fold window and the raw rows for the rest: older days, the partial day
a lookback floor falls in, and any type whose tier rows lack the new
columns. The raw arm reads narrow columns once instead of twice.
An integration test compares the result with the previous query on a
seeded history with two sources per day, hourly pulse statistics with
their own range, readings older than the window, midnight edges and
deletions, and shows the tier is the path that answers.
A marked migration's DO block holds the locks it takes until it commits,
so an app transaction taking two of its tables in the other order can
deadlock with it. The block then rolled back whole, exactly as on a lock
timeout, so the boot retries 40P01 for marked migrations as it does 55P03.
Corrects 0415's note that the drops hold their lock for an instant.
0416 ran DROP INDEX CONCURRENTLY at boot, which waits without bound for
every transaction on the table; a pg_dump or a backfill outlasted the
start period and the next boot stopped at P3009. The migration now tries
a plain drop with a two-second lock wait and tolerates the timeout, and
the nightly measurement maintenance drops what is left concurrently under
a bounded wait, retrying the next night. A fresh or shadow database takes
the lock at once, so the history still ends where the schema does. The
guard refuses a concurrent drop in a migration and ties every retired
index to a tolerant one.
The timeline lane, the day view and the life-events service read every
LifeEvent column; the column-reader sweep counts all six as consumed once
the markers are gone.
Twenty-one foreign keys with ON DELETE CASCADE or SET NULL had no index
leading with their column, so deleting a parent scanned the child table
once per row; medication_intake_events.medication_id was the one an
account delete paid for. Seventeen small-table indexes and the measurements
(user_id, type, measured_at) index repeated the leading columns of another
index and only cost writes.
Migration 0415 adds and drops the small-table ones in one DO block under a
bounded lock wait; 0416 drops the measurements index concurrently, in a
file of its own. An integration test holds the schema to both rules from
the catalog, each with a counter-check, and compares the plans of the
user-and-type query shapes on a seeded table with and without the dropped
index. The lock-timeout guard now also requires a CONCURRENTLY statement
to stand alone in its file.
medication_compliance_rollups lost its last reader when compliance moved
to the schedule-anchored computation, yet every dose write still upserted
it, a boot backfill refolded it, and the wipe and backup plans carried it.
The per-dose hook keeps the one derived row that still hangs off a dose
day, the supplement nutrients, under dose-day-refresh. The pending-row
mints (today projection, reminder) no longer call it, since an untaken
dose adds no nutrients. The import job finishes without its per-day
recompute phase; its progress keeps touchedDays and rollupProcessed on
the wire, inert and documented as deprecated. The backfill queue is
retired on boot and migration 0414 drops the table under a bounded lock
wait.
migrate deploy runs a migration file statement by statement, so the
top-level SET LOCAL lock_timeout in 0412 did nothing: the migration
waited as long as the lock was held, with every write to measurements
queued behind it, and a file that failed half-way stayed half-applied.
0412 now takes its locks inside one DO block with a statement-scoped
timeout, which rolls back whole, after an idempotent CREATE OR REPLACE.
The entrypoint runs migrate deploy through scripts/migrate-deploy.mjs.
A 55P03 failure of a migration marked -- @retry-on-lock-timeout is
marked rolled back and retried after 15/30/60/120/240 s, read from
Prisma's report or, after P3009, from the stored row. Every other
failure stays fatal. A guard holds migrations to the shape that makes
the marker true, and 0411/0413 add their columns with IF NOT EXISTS so
a manual re-run is safe too.
CREATE TRIGGER waits for an autovacuum that is preventing wraparound, and
every write to the table queues behind the waiting request, the running
release's included. SET LOCAL lock_timeout fails the migration after ten
seconds instead, and the file rolls back whole.
One run reads at most forty pages of a hundred activities. A longer history
ended as incomplete with no note of how far it got, so every boot read the
same newest pages again, spent up to 190 of the instance's 200 requests per
fifteen minutes on them, and never reached the older activities.
The run now records the start of the oldest activity it read and sends the
next run for after the rate window; that run asks Strava for the activities
before it. A run that gets no further records nothing and sends nothing. The
place is cleared when the backfill completes.
users.insights_cached_text held the model's briefing, readings and
recommendations about the account in the clear. Migration 0411 adds
insights_cached_text_encrypted (AES-256-GCM, binary layout, purpose-bound
label) and a partial discovery index; the free-text encryption backfill
seals the existing rows one at a time, fail-closed, carrying updatedAt and
guarded on the value it read. Every reader, writer and clear goes through
cached-text-cipher.ts, and a guard keeps the column names there. The
column joins the encrypted-column registry and the key-rotation script,
stays out of backups as operational, and is reset by the data wipe.
`updated_at` sits in the sync-feed index, so every update that moved it
wrote a new row version on another page and an entry in every index.
Provider re-syncs rewrite many unchanged rows, which grew the table to
several times its live size and sent each unchanged row down the sync
feed again. Migration 0412 adds a row trigger that keeps `updated_at` and
`sync_version` when an update changes nothing else, so such an update is
heap-only and the feed stays silent; it sets `fillfactor = 85` and a 5 %
autovacuum threshold. The maintenance queue gains a nightly check that
vacuums when dead rows pass a tenth of the live ones and warns when the
heap is more than three times what its live rows need, which only a
`VACUUM FULL` in a window gives back. The admin card keeps showing the
operator's runs only.
A resumable collection counted the pages it had written across every
backfill attempt and stopped at its ceiling (10,000 for heart rate). A
watch that records heart rate every few seconds reaches that count two to
three years back, and since it never reset, every later slice stopped at
the same place, failed as stuck, and the backfill was never stamped
complete.
A walk is now stopped only when it stops moving: 20 pages in a row that
reach no further back than the pages before them end it. A history
backfill reads back at most GOOGLE_HEALTH_BACKFILL_HORIZON_YEARS (20 when
unset). A data type that ends either way counts as finished, and where and
why it ended is kept in backfillProgress past the completion stamp. An
oldest-first list is read whole, as before. The slice chain's ceiling is
derived from sixty days of back-to-back slices instead of the old page
count. An account left at the old ceiling continues from its edge with
the next slice.
Refs #1220
Connect, status, places, settings, test, sync and disconnect replace their
stubs. A nightly job (03:20) and the connect backfill read the timeline,
visits and residency endpoints through safeFetch and store one row of daily
values per local day: distances, places visited, minutes away from the home
place and the sealed country with its travel-day verdict. A private instance
needs its exact origin in DAWARICH_PRIVATE_ORIGINS.
The connection joins the shared ledger and the connection health model,
watched for errors only. The day carries a movement block, the model
projection drops the travel country, the two movement channels join the
correlation matrix under the environment module, travel days become trip
suggestions, and the days travel in the environment backup section.
Three levels (frugal, standard, full) with an operator ceiling and default
and a per-account choice within them, resolved in the capability layer for
the record and published on GET /api/auth/me as ai.usage: the level in
effect, the stored choice, the bounds, why (user, default, admin_cap), the
three options with whether each is allowed and an estimate of tokens per
day scaled from the record's last seven full days of coach_usage, and the
measured mean. A delegate sees the level, never the owner's spend.
PUT /api/auth/me/ai-usage stores the account's own choice or null for the
default, refusing a level above the ceiling with 422 ai.usage.aboveCeiling
and meta.maxLevel. The new supplementLabel capability needs standard or
above; below it the capability is unavailable with the new usage_level
reason, refused with 403 ai.usage.levelTooLow and meta.requiredLevel.
Refs #1217
POST /api/biomarkers/{id}/merge folds markers into one in a transaction,
converting readings where the registry knows the units and refusing the
whole merge where it does not; a dry run answers the same result and
writes nothing. Aliases appear on the marker and are edited through the
marker's PUT. Scan and document review rows name the matched marker or
offer up to three, and a chosen marker is remembered as an alias. The
doctor report, FHIR and MCP group readings by marker. Aliases and the
merge ledger travel through backup and restore.
Migrations 0402 to 0410: AI usage levels, biomarker aliases and merges,
medication ingredients (with the supplement label run kind), life
circumstance fact kinds, AI sections and AUDIT-C, Dawarich context, the
pulse day-mean method, intraday hour buckets, the readable location drop
and five relation link tables. Every new column carries its reader or a
pending-consumer marker.
0409 refuses to run while any home, location period or stored day still
holds a readable location, and names the counts. Locations are read from
the sealed copy only; the encryption backfill no longer handles them, and
the demo seed writes sealed locations only.
The new tables are backed up and wiped, the new sealed columns are in the
rotation registry with labels of their own, the AI usage level and the
life-circumstance confirmations travel with the backup, and the wellbeing
score reads the mental-health screeners only.
Migration 0401 turns healthkit_type_syncs.type into text so the ledger can
hold MEDICATION_DOSE, and adds last_background_at (seeded from rows whose
latest arrival was a background one). POST /api/medications/intake/bulk
takes syncTrigger and records APPLE_HEALTH dose arrivals; the Apple Health
status read serves a MEDICATION_DOSE freshness entry and lastBackgroundAt
on every entry with arrival facts.
Marc-André Bombeck