The rebuild after apply now writes its state beside the dry run
(queued, running, done with counts, failed); a run left queued or
running without a job reads as failed. The card shows it and offers a
new rebuild after a failure, which the route accepts once the method is
applied, without a new dry run.
Connect, status, places, settings, test, sync and disconnect replace their
stubs. A nightly job (03:20) and the connect backfill read the timeline,
visits and residency endpoints through safeFetch and store one row of daily
values per local day: distances, places visited, minutes away from the home
place and the sealed country with its travel-day verdict. A private instance
needs its exact origin in DAWARICH_PRIVATE_ORIGINS.
The connection joins the shared ledger and the connection health model,
watched for errors only. The day carries a movement block, the model
projection drops the travel country, the two movement channels join the
correlation matrix under the environment module, travel days become trip
suggestions, and the days travel in the environment backup section.
A sync from a watch or an integration now waits the instance's sync gap
(AppSettings.aiSyncRegenGapHours, else AI_SYNC_REGEN_GAP_HOURS, else 6 h)
before it may regenerate a status card again; a hand entry keeps the hour.
Every caller of afterMeasurementMutation and invalidateStatusInsightsForTypes
names its source, so a new ingest path cannot skip the choice.
Refs #1217
Migrations 0402 to 0410: AI usage levels, biomarker aliases and merges,
medication ingredients (with the supplement label run kind), life
circumstance fact kinds, AI sections and AUDIT-C, Dawarich context, the
pulse day-mean method, intraday hour buckets, the readable location drop
and five relation link tables. Every new column carries its reader or a
pending-consumer marker.
0409 refuses to run while any home, location period or stored day still
holds a readable location, and names the counts. Locations are read from
the sealed copy only; the encryption backfill no longer handles them, and
the demo seed writes sealed locations only.
The new tables are backed up and wiped, the new sealed columns are in the
rotation registry with labels of their own, the AI usage level and the
life-circumstance confirmations travel with the backup, and the wellbeing
score reads the mental-health screeners only.
An account whose backfill slice held it made the hourly job wait up to
eight minutes, and the sync after the wait ran as long as it needed:
past the fifteen-minute expiry pg-boss failed the job and retried it
beside the run still going. The pass now spends the job's budget: the
lock wait is capped at what is left, the sync's walks stop when the
budget is spent and leave the rest of the window to the next hour (the
cycle is not stamped), and an account reached after that is skipped.
A stop on the budget is not counted as a failure to retry.
The watchdog opened an incident for any degraded state that lasted a
day, so an Apple Health account whose phone app stopped running read as
no_new_data for good and paged. Incidents now open for action_required,
for provider errors and for the watchdog's own unexpected silence; stale
or stalled data alone never pages. The first pass after a deploy, with
no watch row yet, records what stands as already announced instead of
paging every long-broken connection at once.
A walk that reads without the edge bound passed over everything earlier
slices wrote on every slice, and once it crossed the edge the skipped
pages were counted against the walk's page ceiling. The skip now stays
counted for the whole walk, runs on the slice's clock and its own
three-minute limit, and saves the page token it stopped at (on a limit,
a rate limit or the slice budget) so the next slice re-reads that one
page and goes on. A saved place counts as progress only when it moves
forward; a token Google refuses, or one that no longer reaches the edge,
sends the walk back to the newest page as before.
A backfill noted progress only once a whole collection was read to the end.
A heart-rate history of years that one attempt could not finish (its budget,
a rate limit it could not wait out, a single failed request among thousands
of pages) started the next attempt from the newest page again, finished
nothing, and failed with "marker not stamped" on every attempt.
- Every spot-reading collection read page by page now records, after each
page it writes, the oldest sample it has written (backfillProgress.cursors).
The next attempt asks Google only for samples at or beyond that edge. The
edge is held back once a page's write fails, so no rows are skipped.
- The backfill runs in slices of five minutes or 500 pages. A slice that
finished a collection or moved an edge continues with the next one; only a
slice that moved nothing fails, and its error names each unfinished
collection and why the slice stopped. The slice ceiling is derived from the
collections and the page ceilings of the resumable ones.
- A rate limit Google will not lift within a minute ends the slice instead of
failing the walk, and the next slice starts five minutes later. Server
errors, timeouts and dropped connections are retried three times.
- The heart-rate stream is read last, so the light collections finish in the
first slice.
- The hourly sync waits up to eight minutes for a running slice instead of
skipping the account for as long as the import runs.
- Each slice annotates googleHealth.backfill.slice with pages, elapsed time,
stop reason, per-collection status and edges, and reports progress on the
job.progress line.
An hourly and a daily pass flag a continuous source that delivered on
most of the previous 14 days and has gone quiet, and open one operator
incident per connection when a connection stays degraded or needs action
for more than a day. The incident is announced once through the existing
admin alert path, adopted rather than repeated when the sync ledger
already paged, and closed with one recovery note. Scales, cuffs and
workout logs are watched for errors only, never for silence.
The admission drain ran each full-history import with no wide event in
scope, so every external call, warning and annotation the provider walk
recorded was dropped and a failing backfill left only its error message.
Each admitted run now opens a job.integration_backfill event with its kind,
identity, job id and slice, emitted when the run settles. The Health Connect
import worker gets the same treatment.
The external call list on one event is bounded so a long walk cannot hold
thousands of entries: failed and annotated calls keep the larger share,
the rest is counted in external_calls_omitted.
A chronic ongoing condition has no end date, so counting it as rest mode
kept the account in it for good: Day N counted from an onset years back,
cadence nudges never resumed and every streak gap since the onset was
bridged. Only active episodes that are not CHRONIC_ONGOING (acute,
recurring, flare) now trigger rest mode, and Day N counts from the
earliest of those. The achievements streak freeze applies the same
filter to its own episode query.
Chronic conditions stay context: a coincident vital deviation is still
framed as explained by any active episode, and the Coach block and the
briefing still name them, with restMode false.
Refs #1216
Withings split its public API into plans on 2026-10-12. A request naming
one meastype outside the free plan is refused whole with status 277, so
every measure sync stopped importing anything, and the 200..299 reauth
range parked the connection at error_reauth.
The measure, sleep-summary and ECG legs now drop the documented advanced
types on a 277, split the rest until the refused keys are isolated
(bounded), remember the refusal per connection and re-probe weekly. 277
is classified persistent and never enters the ledger; connections parked
over it are lifted by the fallback tick, and the held watermark lets the
next run read the whole gap.
Slice 1 of phase 3 of #1095. A Biomarker row keeps only its name, so nothing
said which analyte it was. This adds the explicit alias list in the catalog the
issue settled on, including the registry-only keys (urea, blood urea nitrogen,
uric acid, total calcium, total bilirubin), and sets biomarkers.analyte_key
(from migration 0385) from it.
matchAnalyteKey equals a marker's folded name against the localised names of the
catalog seeds the registry knows and against the alias list. It matches the
whole name only, so "Calcium (ionisiert)" is not "Calcium"; a name two analytes
claim matches neither; and the marker's own unit must be one the analyte reaches,
so a marker called "Glucose" in a unit the registry cannot convert stays
unlinked.
The key is set when a marker is created (the biomarker route, the mint used by
lab writes, the legacy backfill) and, for an existing marker, by
ensureAnalyteKey the first time a reading needs it. It never overwrites a key
that is set. There is no boot job for the existing markers: the conversion that
uses the key fills it on first use, and a sweep can be added later if wanted.
The key is returned on the biomarker API and in the OpenAPI schema.
Refs #1095
The shared eligibility filter hid TRACKED_INTAKE_WHERE from the
structural guard that checks every schedule and intake reader honours
the switch. Each reader now spreads it itself.
A row whose follow-up threw, or whose stored time zone the runtime
refuses, aborted the pass for every other person. Each medication now
runs on its own: a failure is named in a warning, counted, and the next
one runs; an unusable zone falls back to the default as in other jobs.
The worker expands no slot past endsOn, so it never reminds for a
course that has ended, but remindersActive still read true for it. The
reminder gate now includes the course end on the person's clock, so the
published value, the worker and the admin check make the same decision.
The auto-take pass left a slot alone when any take fell inside its band,
so the morning dose pinned and taken at 19:00 also hid the 20:00 dose.
Only a take attributed to no other expected slot counts as nearby now.
A rolling cadence yields only lastIntakeAt + N, so a slot the auto-take
pass never records (before the switch-on floor, past the catch-up
window, inside a pause, skipped or removed by the person) held it still
for good. Step the cadence forward by N past every such slot, keeping
its rhythm, and feed the stepped anchor to the card, dashboard and
schedule-tool next due so they show the slot the pass will record.
The token ledger keyed every day on UTC while the daily briefing and the
other per-day generations run on the account's local day, so east of UTC
the first generations after local midnight were refused against the
previous day's spent ceiling. The budget day is now the calendar day in
the account's time zone; each request still resolves it once and books
its reservation and reconcile on the same row. The medication extraction
reads relative phrases against the same local day, the operator's spend
card shows the operator's own day, and the limit copy says the budget
resets at midnight in the person's time zone.
Since the backfill resumes per collection, a slice that ends before the
whole history is in threw "backfill incomplete" so pg-boss retried it,
which logged job.run.failed at error and grew the queue's failure count
although nothing had failed. A slice that finished at least one more
collection now queues the next slice on the admission queue under its
own singleton key and completes. Only a slice that finished nothing new
still throws, so a backfill that is actually stuck keeps its error. The
import lock keys on the import identity without the slice number, and a
slice that finds the import already stamped walks nothing.
Every raw-SQL clock read against a timestamp column without a zone now
writes NOW() AT TIME ZONE 'UTC', so passkey and login challenges, rate
limits, provider pauses, coach usage stamps, tombstone and measurement
windows no longer pass through the session time zone even where the
per-connection pin cannot apply. The regression-window bounds stay naive
UTC timestamps instead of being turned into timestamptz. A structural
guard keeps bare NOW() / CURRENT_TIMESTAMP out of raw SQL outside a short
list of timestamptz targets. The integration test sets the database
default to Europe/Berlin and drives challenge claims, session expiry, the
native sign-in handoff freshness check and the rate-limit window through
the app's client, plus the challenge claim through an unpinned client.
Self-hosting docs describe the pin for an existing or shared Postgres.
Stored times are UTC in timestamp columns without a zone, and Postgres
reads now(), CURRENT_TIMESTAMP and offset-free timestamptz parameters
through the session time zone. On a database whose default is not UTC
every expiry check ran off by the offset. The Prisma pool, pg-boss, the
lock and the maintenance connections now send TimeZone=UTC as a startup
option, or SET it on connect when the startup options are disabled for
a pooler. A boot check reads SHOW TimeZone and warns on anything else.
A history backfill carried one access token through walks that outlive it.
After an hour every request answered 401, which classifies as a revoked
grant, so the account was parked at error_reauth with a refresh token that
was still valid, and every later run skipped it until a manual reconnect.
Running beside the hourly sync, the backfill also pushed the account over
Google's per-minute quota, and each 429 failed its whole collection.
- Walks take a token source that renews the access token before it expires
and once more when Google refuses it mid-walk.
- A 429 is waited out (Retry-After, RetryInfo, else a doubling backoff) and
the request repeated; it is never a reauth.
- error_reauth only short-circuits a run when the token endpoint refused the
grant (needsReauth). A connection parked by a data request's 401 refreshes
on its next run and clears itself when that run completes.
- One run per account at a time: syncs, the backfill and the sleep repair
share an advisory lock. The hourly poll skips a held account, a manual
sync answers 409 sync_in_progress, the backfill waits a bounded time.
- A backfill attempt records each collection it finishes cleanly
(backfill_progress) and the next attempt resumes with the rest.
- The incremental watermark is stamped with the cycle's start, not its end.
Overlapping sleep segments that share a stage and an end share the natural
key (type, measuredAt, source, sleepStage). A re-score that moved one row
onto a sibling's slot threw P2002 and failed the sync on every run.
- A batch keeps one reading per natural key, chosen the same way each time
(longer segment, then smaller externalId).
- A move that still collides is settled by reconcileExternalMeasurement,
which retires the stale row, instead of failing the sync.
- Updates run before inserts, and the key-format rescue no longer re-keys a
row this batch is moving.
- The sleep replace-by-window protects every session's fresh rows, so two
overlapping sessions no longer tombstone and resurrect each other hourly.
Refs #1194, #1195
Next 16.4.0, pg-boss 12.37.1, the AWS SDK 3.1146, @napi-rs/canvas 1.0.10,
@node-rs/argon2 2.2.2, @simplewebauthn/server 14.0.3, TanStack Query and
Virtual, lucide-react, mmdb-lib, nodemailer, otpauth, pg, radix-ui,
react-day-picker, react-hook-form, tailwind-merge and yaml.
pg-boss moves its schema from 42 to 45 (three additive migrations, nullable
columns and one partial index, no table rewrite); the worker applies them on
start as before. The retention test pins the installed version, so it moves
to 12.37.1 after re-checking every source line it asserts.
zod stays on 4.4.3. 4.5 requires seconds in `z.iso.datetime()` and 4.6
rewrites nullable unions in the generated JSON Schema into `type` arrays,
which reshapes the OpenAPI document and the MCP tool schemas. Dependabot
skips zod 4.5 and later until that move is made on purpose.
The fold leftovers were told from a person's deletions by syncVersion = 1,
but raw samples are raised in place before a fold takes them (the collision
merge, a changed Health Connect value). The repair then recomputed windows
without those samples and could overwrite a correct mean, also in windows
one run had folded whole.
A tombstone of fold age now counts as a leftover when it carries
syncVersion = 1 or shares its deletion instant with such a tombstone of the
same day; anything else of that age is ambiguous and leaves its window as
stored. Only a window taken in two parts (fold runs at least an hour apart,
or a run and later live samples) is recomputed.
Once the repair has been through an account, the purge may remove class-A
leftovers inside the horizon, so the folds' re-fold of a stored day or hour
leaves it as stored, and the purge takes the fold lock per account. The
fold-lock transactions get an explicit timeout and maxWait, since the lock
wait counts against Prisma's five-second default.