A batch that carried a type with duplicates only moved its
lastReceivedAt, lastTrigger and lastBackgroundAt, so a type whose live
delivery had stalled kept reading as freshly arrived for as long as a
replay, a retry or an older client re-sent its old samples.
The arrival ledger now moves those stamps only for a type the batch
brought at least one new or changed row for. An unchanged stats:
re-post and an hour posted again with its stored value count as
duplicates here; the wire keeps answering updated for them. A
duplicates-only type stamps the new last_seen_duplicate_at column on
its existing row and nothing else. The intake bulk route's dose entry
follows the same rule through the shared recorder. lastSyncedAt and
healthKitLastSyncedAt are unchanged.
GET /api/integrations/healthkit serves the stamp additively as
metricFreshness[].lastSeenDuplicateAt, and the Apple Health card names a
re-send that came after the last arrival.
Refs #1259
The profile section in docs/contributing/benchmarks.md says how to run
scripts/bench/boot-profile.ts, what the profile of the v1.44 runtime
found (the worker graph as one 753 ms block, a live heap of about
165 MB of which the query compiler holds 50 and the catalogs 33) and what
changed. scripts/bench/results/v1.45.0-*.md hold the before and after
numbers per area, from back-to-back runs on the same machine. The size
baselines take the lowered numbers.
The rollup parity case writes over half a million readings and folds them
into DAY buckets: about 35 s on a CI runner alone. Run beside two other heavy
files on the same Postgres it took 1.7 times as long and crossed the 60 s
default. Nothing in the case waits on a lock: its advisory locks are scoped
to the worker's own database, and the measurement with the database capped
at two CPUs (14.6 s alone, 25.3 s beside the sharing matrix and the restore
memory test) shows plain contention. The case gets a 150 s budget, twice the
contended estimate. HEALTHLOG_INTEGRATION_PG_CPUS caps the database
container so a CI runner's contention can be reproduced locally.
POST /api/admin/backups/upload stores a backup file as a new copy, the
copy whose id the published restore takes, and it refuses a file for
reasons an operator has to be able to look up: a key this server does
not hold (backup.key.missing, with the key ids), an owner account that
does not exist here, a newer schema, a copy too large to store
(backup.upload.too_large). It was exempted from the contract as console
plumbing while its two siblings were published for exactly that kind
of refusal.
It is registered with the three bodies it reads (raw JSON, raw gzip, a
small multipart form) and every status it answers; the summary is the
three identity fields plus one integer count per class, so a class the
backup format gains later does not break a decoder. The round-trip
test calls it through the contract now.
The six /api/auth/me scalar writes (unit preference, glucose unit,
automatic document reading, local lab OCR, workout routes, the central
Codex switch) read their body through safeJson with a 1 KB cap, so each
answers 415 for a body that is not JSON by content type and 400 for one
that does not parse, beside the 413 they already declared. Their
descriptions said so and the responses did not. All three now come from
one block.
The modules and disable-coach route tests had been skipping the
contract check over a 400 and a 409 the registry already declares; the
skips are gone.
A /api/fhir/* handler answers its own refusals as FHIR: a bad search
parameter is a 400 OperationOutcome, the hourly fhir bucket a 429
OperationOutcome (issue code throttled), both served as
application/fhir+json, the same way the 404 on a read already was. The
contract declared no 400 and the generic JSON 429 for the 429, and no
403 for the doctorReport module gate, which every data route runs.
The searches now declare the 400, every data route the 403
module.disabled envelope and the FHIR 429 with the rate-limit headers
apiHandler attaches to it. The module gate keeps answering the JSON
envelope: it is the errorCode every client already branches on for a
disabled module, and it runs before the FHIR face does. The route tests
drop their skips and assert the 429's content type and Retry-After.
requireAuth answers 403 auth.scope.insufficient to a narrow Bearer token
on a route that names no scope it holds, and requireAdmin answers 403
auth.admin.required to a session that is not an administrator's. Both
run before any route body, and the contract only said so in the prose
of the shared 401, so a client generated from it had no typed branch
for either. The route kit accepted the scope refusal on any operation
that declared a 401 for the same reason.
The 403 is added to the assembled path table rather than to
stdResponses. That set is spread into most operations, about three
hundred of them carry a 403 of their own, and most spread the set after
it, so a shared 403 there would have overwritten every module-off and
sharing refusal in the document. withScopeRefusal adds the shared
response where an operation has none and prefixes an operation's own
403 with the gate's code, keeping the sharing sentence last. Operations
that take no credential, and the three surfaces that authenticate by a
credential of their own, are left as they were.
The kit no longer tolerates the refusal on an operation's behalf, and a
guard holds every operation behind the gate to declaring it.
docs/contributing/testing.md says which suite a test belongs in and why,
how to write a route test with the kit and what its contract check proves,
how the settings round trip and the integration database work, how the
shards are balanced and the timings regenerated, when to write a property
test, how the mutation floors are kept, and the flaky-test policy with the
quarantine format. Linked from the docs index.
scripts/bench/seed-large-account.ts (moved from scripts/, extended) writes
one synthetic account of ten years: 1,265,561 measurement rows across the
main types and five sources (Apple Health dense spot rows, Withings blood
pressure and scale, a weekly manual weight, three years of Oura nightly
values, 90 days of Nightscout CGM), `stats:` and per-sample cumulative rows,
nightly sleep stages, 3,020 workouts, 1,095 mood entries and three
medications with 3,285 intake events. Everything goes in through set-based
INSERT … SELECT generate_series; values and timestamps are functions of the
row index and `--anchor`, so a run is reproducible. The rollup tier is
folded with recomputeUserRollups, as the boot backfill does, and the tables
are analysed. About 80 s on the Mac mini.
scripts/bench/run.ts boots the production build once with every role (so
the queues exist), then measures against a web-only process under the
production heap ceiling: p50, p95 and max latency and SQL statements per
request (pg_stat_statements, read once nothing is in flight and the count
holds still) for /api/auth/me, the dashboard summary and snapshot, the pulse
series over 30, 365 and 3650 days, the day view, insights health status and
correlations, each cold (caches evicted by a one-row manual-sync ingest
before every request) and warm, plus a 5,000-row batch ingest as ten
requests of 500. It writes JSON and a markdown table.
docs/contributing/benchmarks.md describes the setup against a throwaway
Postgres in Docker with the production pg_stat_statements settings.
The beyond-horizon line said the range held "for more than {years}
years, the span your readings cover", but the horizon is capped at three
years, so a ten-year history was told its readings spanned three. The
trend now says whether the cap or the span set the horizon
(`horizonCapped`), and the card words each case on its own in every
locale.
The 100000 ceiling refused offsets a correctly paging client reaches on
its own: an Observation series with a heart-rate history runs well past
a hundred thousand rows, and the next link the server handed out led
straight into a 400. The 500 the ceiling was added for came from an
offset such as 1e20 that no longer floors to an exact integer and that
the database cannot read as an OFFSET. The refusal now sits at the
largest exact integer, and a next link never carries an offset the
server would refuse.
The strain run writes no impulse row on a day without a scorable workout
or active energy, so a rest day for someone without daily active energy
was never rated: fitness and fatigue held for months instead of
decaying, and the figures named the last stored day while the coverage
count ended there too.
A day without a stored impulse is now classified from the underlying
data: a workout on the day leaves it unrated, steps, heart rate or
active energy without a workout make it rest, and no data at all leaves
it unrated. The series runs to the day the strain run scores, so a
stretch without data after the last rated day shows in the coverage
count, and the card names the last rated day when it lies further back
than the day before.
An offset beyond what the database can skip reached Prisma unchecked
and answered 500. Every search route now refuses an _offset above
100000 with an invalid OperationOutcome.
The lab, medication and measurement CSV routes ignored parameters they
did not know, so a misspelt dryrun=1 fell back to a real import and
wrote the file the caller meant only to check. Their query schemas are
strict now: an unknown parameter answers 422 before anything is read
or a rate bucket is spent.
The automatic Newey-West rule gives a lag of 3 on 30 to 270 daily
values, which leaves most of a physiological series' carry-over out of
the intervals. The lag is now at least 7. Reference values are
recomputed with statsmodels at the new default, and the lag-3 values
stay pinned with an explicit lag. A fit whose residual autocorrelation
exceeds 0.5 is flagged, and the card, the clinician view and the PDF
say the ranges may still be too narrow.
A day without a stored training impulse was read as a rest day, so a
watch left off or a nightly run that had not happened yet pulled
fitness and fatigue toward zero. Such a day is now not rated: the
averages hold through it, the series marks it, and the card says how
many of the last 42 days were rated. A scored day of 0 stays rest. The
figures describe the last stored day rather than always yesterday.
Today is shown as in_progress instead of being placed against whole
days, and a running period is compared over its completed days only:
the total through yesterday against the usual rate times the days
completed. The period total and goal progress still count today. The
comparison gains completedDays and completedValue.
A marker whose newest reading is more than a year old is now reported
as of that reading (projection "stale"), with no exit date and no
already-outside verdict. The projection horizon is capped at the span
the readings cover, so a line is never carried further ahead than the
data reaches back. The detail read keeps the newest 2000 readings
rather than the oldest.
The health-status read scanned five vitals over 90 days for a dated,
sustained level shift. It now scans every vital with a stable daily level
(adding RMSSD, diastolic pressure, pulse, body fat, respiratory rate,
oxygen saturation, walking heart rate and VO2 max) over 180 days, reads one
value per day per metric, leaves a metric unread when its module is off,
and reports how many days stand on each side of the break. The Coach keeps
its five-metric, 90-day scan unchanged.
Two streamed endpoints read a spreadsheet record by record with a
column mapping from the query: separator, decimal mark, date order and
zone are stated, never guessed per row. Lab rows resolve to markers the
way scanned reports do, convert units where the conversion is known and
keep what the file said; medication rows map name, dose, daily times,
as-needed and a dated window, checked against the create schema, with
reminders off unless asked for. A dry run writes nothing, a re-upload
only finds duplicates, and every refused row comes back with its line
and reason.
The first page of the workout list now carries the fitness-fatigue model
over the day-total heart-rate impulse the nightly strain run already
stores: fitness as the 42-day and fatigue as the 7-day exponentially
weighted average, form as their difference. A day without a stored impulse
counts as rest. It needs 28 days and three training days, flags the first
six weeks as warming up, and reads in the person's own units only. The
workouts page shows the three figures above the list.
/api/health gives the database two seconds and answers 503 past that.
With three retries, a pool saturated for a minute after the start period
could mark the container unhealthy and take it out of the proxy's
routing. The compose healthcheck and the image's HEALTHCHECK now allow
five misses (two and a half minutes) and end wget after four seconds,
inside Docker's five, so a stalled request is one clean miss. A test
holds both to these bounds and to each other.
The UI standards drop working shorthand, state the spacing scale once in
section 2 (section 13 had listed an older scale without 2.5 and 3.5) and
use no em dashes. The v1.3 to v1.4 migration note no longer claims that
/settings#anchor URLs redirect to their section: the fragment never
reaches the server. A test holds both documents to this.
The room-climate guide said both to create one automation per room and
to track only one; it now recommends the bedroom and says why a second
room cannot work (no room name, same-minute pushes collide on 409). The
blueprint offers 5, 10, 15, 20, 30 or 60 minutes, so the push lands on
the stated minutes, and it and the guide's examples use the triggers,
conditions and actions keys of Home Assistant 2024.10 and later. A test
holds all three.
The assessments list now carries, per PHQ-9, GAD-7 and WHO-5, the latest
total against the previous and the first administration, classified
against the change studies treat as meaningful: 5 points on the PHQ-9,
4 on the GAD-7, 10 on the WHO-5 percentage scale. A smaller change reads
as within the threshold rather than better or worse. The instrument
detail states each change and whether it reaches that size, in neutral
words. The read takes totals and dates only, never an answer.
A read-only view per metric and UTC day: every source with readings, its
day value, its place in the person's source priority, and which source
the charts use with the reason. A second card lines two sources up over
30, 90 or 365 days. The pick is explained from the rank the canonical
SQL returns, so the page cannot disagree with the charts. Linked from
the metric pages and from Settings > Source priority.
GET /api/fhir/Observation keeps its snapshot answer when no date is
sent. A date range (FHIR prefixes eq/gt/ge/lt/le, UTC when no offset)
switches it to the time series: every reading in the range plus
workouts, newest first, paged in the database through one UNION id
query and a count, so a page costs what it holds. Series ids are the
row's own (m-<id>, w-<id>) and read back through /Observation/{id}.
The series replays the same saved report selection as the snapshot,
pairs systolic and diastolic recorded at one instant into the
blood-pressure panel, leaves sleep stage segments out, and reads an
hourly heart-rate bucket as a mean over its hour. A code parameter
narrows either answer; a malformed date is a 400 OperationOutcome.
A workout is an activity Observation after the Physical Activity IG:
LOINC 73985-4 with the sport as its value and duration (55411-3) and
energy (41981-2) as components. RMSSD gets its own registry coding,
text-only because LOINC has no RMSSD term, so it never borrows the SDNN
code. The capability statement lists the two new parameters.
The standard report template now carries sleep duration, heart-rate
variability as two labelled rows (SDNN and RMSSD), daily steps, a
per-sport workout summary and the change around each medication start.
Each still rides its module, and the template is still only applied by
the button that names it; saved selections and share links are
untouched.
Two structured leaves join the catalogue: WORKOUTS (activity group,
workouts module) and MEDICATION_EFFECT (medications group). The workout
block reads narrow rows through the canonical-workout picker and sums
them per sport. The effect block reuses the efficacy loader with the
report's end as its horizon, carries a target only when that target's
own measurement leaf is admitted, and is capped at eight medications.
In the PDF the wearable series chart at half width, two to a row, in
the units the table prints; the workout and effect blocks are compact
tables with short notes. The clinician view renders both blocks from
the same payload. RMSSD joins the dense day-bucket path so long windows
stay aggregated.
The biomarker detail read now carries a trend: the least-squares line
through every numeric reading of the marker (all stored in its unit), with
the slope's 95 % interval and p-value. It needs three readings over at
least a year. The line is placed against the marker's band, and an exit
date is given only when the interval excludes zero, the line is inside the
band today and reaches a bound within three years. The detail page words
it as a straight-line projection, not a forecast.
The efficacy tab keeps its before/after comparison and gains, per metric
target, the level change and the change in weekly trend at the start,
each with a 95 % interval. The fit covers up to 90 days before and 180
after the start and ends before the first dose change. After the start,
days are left out when fewer than 80 % of the doses of the preceding 14
days were taken (cadence-aware, from the dose ledger) or when the
medication was paused or between courses. Other medications that
started, stopped, changed dose or paused inside the window are listed.
Below 14 days with readings across 21 days on either side the view says
what is missing instead of showing numbers.
The efficacy response gains a nullable segmented block per target.
0440 adds its columns with ADD COLUMN IF NOT EXISTS like 0424 and
0432; a structural test holds every migration from 0422 on to that
form. 0430 clears insights_cached_at where no sealed payload is left,
so the nightly pregenerate refills an account whose only copy was the
readable column instead of keeping an empty card fresh.
The deploy runbook notes that 0430 drops a users column: an old process
fails on its next full user read, so stop before starting rather than
running old and new side by side.
A week or month of steps, active minutes, workouts by sport and training
load, each day placed against the person's own spread over the 84 days
before the period and each period against the median of the twelve weeks
or six months before it. Optional weekly and monthly goals live on the
account (activity_goals_json), travel with the backup and are cleared by
a data wipe. The calendar sits on the workouts page; the steps page links
to it. Computed server-side without AI.
POST /api/measurements/batch ran one interactive reconciler transaction
per fresh sample: a savepoint, three advisory locks, two candidate reads
and the insert, about ten round trips. A 500-entry upload of new samples
was 500 of those, the 2 to 15 second outliers. Fresh samples now land
through INSERT ... ON CONFLICT DO NOTHING under the same per-account
advisory lock. Both measurement identities are full unique indexes, so
the statement inserts exactly what the reconciler would have inserted,
and every skipped row still goes through the reconciler for its own
verdict. Only rows whose external and natural keys each occur once in
the batch are buffered, and the buffer is flushed before any other row
is reconciled, so the order of effects stays the sequential one. The
contract and every per-entry status are unchanged; a golden test runs
each batch through both paths against the same table.
/api/health took seconds after a start because register() awaited the
whole pg-boss worker start (queues, schedules, boot discovery) before
the server took a request. The worker, or a web process's producer, now
starts in the background after the blocking boot checks; the probe
answers at once with reason "starting" until it is up, and gives the
database two seconds before calling it not answering.
Room climate from a home automation system gets three types of its own:
INDOOR_CO2 (ppm), INDOOR_TEMPERATURE (celsius, shown in Fahrenheit for
imperial readers) and INDOOR_HUMIDITY (percent). They arrive through the
existing ingest token (measurements:write, source EXTERNAL), sit in the
environment cluster beside daylight, and the daily rollup carries their
mean, low and high like any spot reading. They never mix with the body
temperatures or with the outdoor environment channels.
The Home Assistant page documents the three types, a rest_command that
sends a room as one request, a single-room automation, and an
importable blueprint (docs/integrations/blueprints) that skips sensors
reporting unknown or non-numeric states, converts Fahrenheit, and
stamps readings to the minute so a retried push is refused rather than
written twice.
Six Apple Health signals leave the deferred list and ride
POST /api/measurements/batch and the export.zip import end to end:
EXERCISE_MINUTES, STAND_MINUTES and STAND_HOURS (the exercise and stand
rings), MINDFUL_MINUTES, STATE_OF_MIND_VALENCE (iOS 18 State of Mind,
-1..1) and AFIB_BURDEN (AFib History, percent of the week). The ring
types and mindful minutes are cumulative, so the daily rollup is their
sum; AFib burden is scaled strictly from Apple's fraction and a value
above 1 is refused rather than guessed at. The archive import takes
mindful minutes from the session's dates and skips idle stand hours.
Every registry carries them: units, ranges, labels in all locales,
categories, chart tokens, report catalogue (mindful minutes and the
valence fenced as sensitive), signal registry with FHIR codings, and
MCP exposure for the three ring types only.
Workout GPS tracks are now an opt-in, off by default
(PATCH /api/auth/me/workout-routes, a switch on Data & Privacy). While
it is off, the workout batch route and the export import keep every
workout and drop its track with the same per-entry status. With it on,
the import reads the referenced GPX files from the archive one bounded
member at a time, thins each to the point cap and stores it sealed.
The climb a track carries is written to Workout.elevationM at write
time, so the workout insight no longer opens the track, and the AI
egress guard now freezes the set of files that read route geometry.
A guardian acting on a managed profile can now connect and disconnect its
Withings, WHOOP, Oura, Polar, Strava and Google Health from the profile's
integration cards, and each card says whose record it connects for. The
connect starts with a fenced fetch that mints a state naming the profile;
the provider's callback lands tokens and data there only after checking
that the browser belongs to a guardian with a live MANAGE grant on it.
Without a provider app of its own, the profile borrows the guardian's,
with an audit row. Withings and Google Health callbacks read the session
directly, since a provider redirect cannot carry the record-session header.
The binding UI rulebook (card anatomy, spacing scale, text colour,
tokens, primitive mapping, enforcement) lived outside the repository,
so a contributor could follow a code comment to a file they could not
open. It now lives at docs/contributing/ui-standards.md, every comment
that cited it points there, and the docs index and ui-guidelines.md
say which document wins when they disagree.
The getting-started requirements now say a host needs 3 GB of RAM, 4 GB
recommended, and drop the advice to lower APP_MEMORY_LIMIT to 1g on a
2 GB host. The compose and .env example comments say the same.
At boot the app reads its cgroup memory limit (v2, then v1) and logs one
line when it is below MEMORY_FLOOR_MB. The support report adds the
container limit and a note below the floor, and Admin > System status
shows "below the supported floor" next to the existing low-heap flag.
The app keeps running either way.
Workouts from /v2/usercollection/workout land as Workout rows keyed on
the Oura id, with the activity mapped onto the canonical sport set and
the raw label, intensity and recording source kept in metadata. Heart
rate from /v2/usercollection/heartrate lands as PULSE, streamed page by
page and written in bounded batches, first write wins per instant. The
hourly window for heart rate is hours, not days: six by default,
reaching back to the newest reading after a gap, at most 72.
Both collections sit behind their own scopes (workout, heartrate). A
grant from before they were requested answers 401 naming the scope; that
is recorded as a scope note on the card, never as a failure or a
reconnect. Collection refusals carry Retry-After on a 429.
Add per-user columns for the labs, illness and checkups list presentation
and for preview-feature opt-ins, each with a GET plus a preserve-when-absent
write under /api/settings. The list hook now reads and writes the server
value and carries a browser-stored order up once when the server holds
none, so an existing arrangement follows the person to every device instead
of resetting.
Marc-André Bombeck