`medication_intake_events.idempotency_key` carried a GLOBAL unique from
0001_init. The value is client-supplied, so two people running the same
client can mint the same key — and on a global unique the second one is
simply refused by the constraint. Their dose is not written and nothing
tells them so. The bulk drain made it worse: its replay probe read the
key with no user scope, resolved the first user's row, and reported the
second user's entry as a duplicate carrying a stranger's row id.
Migration 0273 replaces the global unique with `(user_id,
idempotency_key)`, the grain every other client-supplied identity on the
table already uses. Both unscoped reads in the bulk drain now key on the
composite; the other four reads were already narrowed by user.
On existing data this is a relaxation, so nothing needs reconciling: the
old constraint is strictly stronger than the new one, and no pair of rows
can violate an index the previous one already forbade. The migration
still carries a reconciliation step for a database whose global index was
dropped by hand at some point. It never deletes or rewrites a dose — it
clears the dedup HINT on the later row of a colliding pair and keeps
every intake event, timestamp, attribution and inventory stamp intact.
The new index is created before the old one is dropped, so a failure
aborts with the old protection still in place.
A structural guard freezes the read side: no `medicationIntakeEvent`
query may filter on the key without narrowing by `userId`.
The disaster-recovery backup selected every Measurement scalar except the
aggregation fields, so a restore brought every `stats:` aggregate back with no
provenance and the Apple Health authority ladder came back flat: an export.xml
source-day estimate could then overwrite a native HealthKit statistic.
`aggregationProvenance` now rides the select, the disaster-recovery
serialization, the payload schema and the restore mapping, following
`externalSourceVersion` exactly — including staying out of the reduced
portable shape, which carries the human-readable subset only. The schema field
is optional, so every backup written before this parses unchanged.
Restoring one of those older backups no longer recreates the ambiguity
migration 0263 repaired: an Apple Health row under a `stats:` identity with no
provenance comes back as LEGACY_UNKNOWN, the sentinel the cumulative drain
already writes in that situation, so a native sync or archive reprocess can
repair it. The gate is scoped as 0263 scoped its backfill — other providers
mint `stats:` identities for their cumulative dailies and have never carried
provenance, and ordinary point rows keep NULL.
Migration 0263 added three columns beside the Apple Health `stats:*`
authority ladder. Only `aggregation_provenance` is ever read back: the
external-measurement reconciler selects it and refuses a lower-authority write
against a higher-authority canonical row, and the cumulative drain reads it
too. That column stays.
`aggregation_contributor_count` and `aggregation_selected_source_hash` are
written by three call sites and read by none. No select, no raw SQL, no export,
no backup, no API contract, no repair path. The import-time selection keeps its
deterministic tiebreak, which compares an in-memory map key computed per flush
rather than a value read back from the table.
Both are plain nullable scalars with no index, unique, check or foreign key, so
the drop is a catalog-only operation that touches no measured value.
Mood has been a first-class module here for a long time, with entries,
structured tags, rated factors and notes. The bridge to the external moodLog
service had nothing left to add, and its Settings card was withdrawn in
a56e6fe61 on the promise of a dedicated removal. This delivers it.
Removed: the hourly `moodlog-sync` queue, cron and handler binding; the
inbound webhook and its shared-secret handling; the reverse push and its four
call sites; the pull-sync module; the per-user credential routes and the
instance-wide toggle; the entry on the `/api/integrations/status` envelope,
the `IntegrationKey` union, the cadence map and the display-name table; the
boot-time secret rotation pass; 22 message keys per locale.
Migration 0271 drops the five `users` credential columns and
`app_settings.mood_log_global`, clears the ledger row for the provider, and
deletes the retired pg-boss schedule so no cron keeps enqueuing work no
handler drains.
Mood data is untouched. `mood_entries` is not named anywhere in the migration
and cannot be reached from it: the dropped columns carry no index, constraint
or foreign key, and `mood_entries.user_id` references `users(id)`, not any of
them. Rows the bridge imported keep `source = 'MOODLOG'`, and every read path
- list filter, source badge, exports, backups - still resolves that value.
`src/lib/validations/moodlog.ts` held the native mood-entry schemas alongside
the bridge's own; it is now `validations/mood.ts` and carries only the former.
The GET spread the optimistic-concurrency `updatedAt` into the response even
when the user had never saved preferences, which broke the strict-equality
contract that a never-saved read is exactly `DEFAULT_COACH_PREFS`. Mirror the
insights-layout never-saved GET: a null `coachPrefsJson` returns the pure
default with no token (the first save is the tokenless unconditional write), and
only a saved row and the PUT response carry `updatedAt`. Update the integration
round-trip to expect the token on the saved GET and PUT while keeping the
`coachPrefsJson` column assertion pure.
Merge the eight reviewed audit fixes that were cut a week ago and never
landed. Main has moved on since, so each fix was re-checked against the
current tree and each of the five conflicts resolved by understanding
both sides.
Kept, still open on main:
- Proxy early exits now carry the transport security header subset via a
shared helper. Reconciled with main's /api/health worker-only carve-out.
- Batch measurement ingest bounds the entry instant so a future-dated row
can no longer win "latest reading" or poison the rollup buckets.
- Web Push dials through the pinned safeFetch dispatcher with
requirePublicHost, closing the last user-controlled-host egress.
- Mood bulk upsert and its tag/factor links run in one transaction, so a
rejected rating rolls the mood row back instead of leaving a tagless row.
- Full backup carries NutrientIntakeDay with its unit and source. Folded
into main's portable-export / disaster-recovery split.
- Live daily-series fallback restores the min/max spread band the tiered
read already emitted.
- Coach stream aborts its in-flight reader on unmount and resets send state
on a conversation switch.
- Manual water writes share the day-bound helper, nutrient add clamps after
the increment, and the thiamin cap drops to a real supplement ceiling.
- Compliance point window widens symmetrically instead of wrapping overnight.
- Oura, Polar and Fitbit token helpers classify an empty or non-JSON 2xx
body as a handled transient error.
- WebAuthn relying-party fallback to localhost is gated to development.
- Chart-token strip list is derived from the allowlist and pinned by a test.
- Regression tests pin the snapshot-cache tenant key, the GlitchTip
path-secret redaction and the Withings credentials delete.
Superseded by main, tests re-pointed at the current shape:
- The intake CSV import was rewritten on main to run through a background
job with a single-active-job admission gate. The future-date bound is
re-applied at request validation and the per-user limiter is kept for
parity with the bulk twin; the old synchronous-write test is replaced.
Integration coverage for `validateSessionFromCookieValue` against real
Postgres: a live secret-backed session resolves to its user, an expired one
and an unknown/forged/empty value return null, and a legacy cuid cookie
resolves only while its row still carries no hashed secret. This is the
hashing + row lookup + expiry the callback's duplicate detection turns on,
which the unit suite stubs out.
Refs #600
The one-paragraph-per-workout claim is a unique index, the cascade is an FK,
and the daily-cap window is a SQL fact — a mocked upsert reports success for
all three regardless.
The contract now names which factors satisfy the fresh-factor routes and which
do not, carries `satisfiesFreshFactor` on the mint response so a client can pick
the ceremony before spending a proof, and describes the options endpoint's two
ceremony kinds. The second-factor status read is documented as plain
authentication — it carries no credential material, so it needs no elevation.
A step-up elevation minted from the account password satisfied the fresh-factor
routes. That is a second-factor bypass: a stolen token plus a known password
could rotate the recovery codes — which take no factor in the body — and then
spend one of them to disable the second factor outright. The web has never
allowed that chain, because a password login does not stamp mfaVerifiedAt.
The method is now an authorisation input. Only totp, webauthn, and passkey
satisfy a fresh-factor gate, which is exactly the set of ceremonies the web
stamps a session for; password reaches what a plain cookie session reaches and
stops. The mint gains TOTP and security-key arms so a TOTP-only account can
still manage its factor natively, and the response says which reach it bought.
The gate also returns a discriminated context. session.id meant a session row on
one arm and a token row on the other, and the disable route passed the token id
to a session-scoped query — excluding nothing, so it deleted every browser
session and revoked the caller's own device login. The transport is now in the
type and the compiler refuses the confusion.
Redemption splits into validate-then-commit so a 429, a 422, or a wrong code no
longer burns a proof against a five-per-fifteen-minute mint ceiling. The claim
is UPDATE ... RETURNING, so the statement that decides is the one that reports.
The mint is transactional behind a partial unique index. Sign-out-everywhere and
both password-reset paths clear elevations. The status read drops to plain
requireAuth — it carries no credential material — and every mutation stays gated.
The water quick-add validated calendar realism but never the upper day bound the
batch route applies, so a far-future day was accepted and — every read filters
`day >= since` with no ceiling — became the permanent latest day on the settings
card and the dashboard tile. Hoist the bound into one shared helper both routes
use.
`mode: "add"` bounded a single request to the catalog's plausible daily max but
nothing across requests, so repeated quick-adds drove the stored total
arbitrarily high while the batch route enforced the same cap strictly. Clamp
after the increment so the write stays atomic.
Thiamin's cap sat at exactly 1000 mg while its PRI is exactly 1.0 mg, so a
microgram-read-as-milligram sample landed on precisely the cap and the `>`
comparison let it through. 500 still clears any real supplement dose.
NutrientIntakeDay was in no export path, which contradicted the schema's own
reason for denormalising the unit column. Carry it in the full backup with its
unit and source — source is part of the composite key, so without it a restore
cannot tell a manual entry from a synced day total.
Backfill immunity is proved by replay, not by a hand-picked case: a ten-year
Apple Health export (12772 writes) and a 30-day provider re-sync (58 writes)
must both emit zero events, while a fresh sleep night, workout and first
weight of the day emit exactly one each.
The provider-isolation guard walks the worker's static import graph and fails
if any provider client or resolver becomes reachable. It carries two positive
controls, so a broken entry path cannot make it pass against an empty graph.
The integration test covers what unit mocks cannot: the unique index, the
ON CONFLICT DO NOTHING claim, the losing side of ten concurrent claims, the
forward-only marker, ciphertext at rest, the delete cascade, and the
retention window.
The three unit suites that stubbed `requireCookieAuth` / `requireFreshMfa`
directly now stub `requireMfaManagementAuth`, and the password-change suite
stubs the elevation revocation the route gained. Adds the unit checks that need
no database: the elevation TTL is pinned to the cookie path's step-up window so
the two cannot drift apart, the minted value is a prefixed CSPRNG secret whose
plaintext never reaches the row, and a value carrying another credential's
prefix is refused without costing a query.
Adds the structural guard that freezes the exact route set an elevation can
unlock, plus the behavioural suite that drives each threat case against a real
Postgres: a token alone mints nothing, an elevation minted for one token is
refused for another (same account included), a second redemption fails, a
concurrent redemption lets exactly one winner through, an expired one is
refused, an elevation never reaches requireAdmin, a wrong password is throttled
and audited without saying what was wrong, and the cookie flow is untouched.
The OpenAPI registry carries both new routes and names the accepting set in
prose, so the client implements against the contract rather than against a
description of it.
The assertion compared the cookie to the new session row's id, which only
held while the id WAS the cookie. Check the stored hash instead: it
proves the cookie authenticates the surviving row and that the row id
never reaches the client.
The cookie was the Session row's cuid. cuid is built for collision-
resistant identity across distributed writers, not for unguessability —
it encodes a millisecond timestamp and a process-local counter and never
touches a CSPRNG. A session cookie is a bearer credential and has to be
unguessable, so it now carries 32 random bytes and only the HMAC of that
value is stored, matching how api_tokens and refresh_tokens already hold
their secrets.
Compatibility is passive, and deliberately so. A backfill cannot help:
the cookie already sitting in a browser is the cuid, and nothing we
generate server-side can be pushed into it — backfilling would sign
everyone out at deploy. Upgrading a row in place on first use loses to
the parallel requests a single page load fires: each mints its own
secret, one write wins, and the losers cannot re-emit the winner's
secret because only a hash is stored, so the browser can end up holding
a value the row does not honour. Instead, rows minted before the column
keep a NULL token_hash and stay resolvable by id, and are denied the
sliding-expiry extension so they drain within the 30-day session
lifetime. Every login from this release forward is on a secret at once,
and an id stops authenticating the moment its row holds one.
Note for operators: API_TOKEN_HMAC_KEY now also keys session cookies, so
rotating it signs browser sessions out as well as invalidating tokens.
It is already Core-required, so no deployment gains a new prerequisite.
Two layers. The integration suite runs a real pg-boss against a real
Postgres and proves the semantics the policies rest on: that a bare
singletonKey under the default policy de-duplicates nothing, that
exclusive suppresses a duplicate while the first job is queued AND while
it is active, that short suppresses only while queued and admits a send
once the job is active, and that a distinct key always enqueues.
It also pins the two facts that make the fix reach a running instance:
createQueue does not change the policy of a queue that already exists,
and updateQueue refuses a policy change, so the reconcile is required.
The unit guard couples each enqueue site to its registrar entry. The key
and the policy live in different files with nothing in the type system
holding them together, so deleting an entry would leave the call site
looking correct while it silently stopped working.
Four structural guards freeze the invariants the change rests on: the set of
files that can resolve a Bearer token to a user, the single deliberate
any-valid-token posture, the mint sites with their scope sets, and the rule
that a declared scope is an exported constant rather than a literal.
Thirteen behavioural cases run against a real Postgres and the real resolver.
Four of them go red on the pre-fix tree — a narrow token reading the
full-backup export and the labs surface, and an MCP token on a REST read.
The rest pin the flows that must NOT change: the per-medication ingest pair
still works, the token without the per-medication grant is still refused, a
wildcard token still reaches the export and the batch ingest, an MCP token
still resolves on /mcp with its write scope intact, a declared fhir:read
still opens the FHIR face and nothing else, cookie sessions are unaffected,
and requireAdmin still refuses a wildcard Bearer.
The workout-batch case that asserted narrow tokens are admitted claimed the
iOS sync depended on it. It does not: workouts:ingest has no mint site
anywhere, and iOS holds a ["*"] token. Replaced with a case that asserts both
halves of the real contract.
Documentation updated for the enforcement rule and the retired mint.
The raw-SQL data backfills 0247 (WHOOP) and 0251 (Strava) rewrite
historical Workout.sport_type rows with no test — the thinnest-covered
class in the tree (audits L2). Add an integration test that seeds
pre-backfill rows, applies the EXACT migration SQL from disk against the
testcontainer Postgres, and asserts:
- canonical relabel across both value shapes (id placeholder + raw
label), case/normalisation, unknown→other fallback;
- the source guard (a WHOOP pass leaves Strava/MANUAL rows verbatim and
vice-versa);
- idempotency (a second apply reports zero affected rows).
Runs in CI's integration job; locally needs Docker/OrbStack via
pnpm test:integration.
Adds prev/next-day paging to the intraday pulse "shape of the day" chart
(TileHeader right slot, next disabled at today) so a user can step back
through prior days on the existing date-scoped route.
Raises DENSE_INTRADAY_RETENTION_DAYS from 14 to 90 — a self-hoster's own
storage is the trade-off for three months of full 10-minute-resolution
history instead of two weeks. Days outside the window keep folding to
hourly-mean stats: rows as before; loadIntradayPulse now falls back to
that hourly tier when a day's raw samples are gone, so the navigator
shows a coarser shape instead of an empty chart. The tension-window
read is suppressed on hourly days — it needs per-sample resolution to
stay honest. A new resolution field on the DTO ("tenMin" | "hourly")
drives a small note in the chart.
Replace the scalar document_id discriminator on a coach conversation with a
CoachConversationDocument join table and a sticky documentScoped flag, so a
conversation can be grounded on several stored documents at once while the
prompt-injection fence stays intact across the data-model change.
The fence is re-established at both enforcement points, fail-closed:
- Client: resolveCoachSendTarget routes any turn on a fenced conversation (or a
fresh chat with staged attachments) to the new /api/insights/chat/fenced
endpoint; the fenced body is strict and never carries scope, guided-question,
or prefill hints.
- Server: the tool route fetches with documentScoped:false AND 404s when the
fetched row still carries an attachment (flag/join drift), emitting a
fence_drift audit alarm. Document text now enters an LLM prompt in exactly one
code path — the fenced pipeline — with no tools and no health snapshot; a
module-graph test pins that the tool loop is unreachable from it.
The fenced turn fences each document separately with a server-built, scrubbed
header line (title and filename are untrusted text), runs a per-document egress
consent check on the single picked provider, and grounds numerically over the
live attachment union only. Attachments are capped at five documents and 192 KB
of combined text with honest truncation.
Attach and detach ride dedicated endpoints: attaching validates the document is
owned, live, indexed, and within cap, then flips the conversation fenced (the
one legal, privilege-reducing transition, audit-logged); detaching removes the
document but never clears the flag. The document-sheet chat keeps its public
wire contract and shares the same fenced pipeline over the join.
Migration 0245 adds the join table and the flag, backfills every legacy
single-document chat into a one-attachment fenced conversation, and drops the
scalar column in the same transaction — no dual-truth window.
The Apple Health ZIP import upserted each spot reading on the
(userId, type, source, externalId) index only. When two readings shared the
natural key (userId, type, measuredAt, source, sleepStage) with different
externalIds — a re-import of overlapping data, or two same-instant Apple
samples from different source apps — the externalId upsert missed and the
create collided on the second unique index (migration 0055), raising P2002
and aborting the entire import (issue #486, after the v1.28.36 dedup fix let
the upload run).
flushSpotBatch now wraps the per-row upsert: on P2002 it probes the occupying
row by the natural key (tombstone-inclusive) and adopts it in place — re-keys
to the incoming externalId, takes the new value/unit, resurrects if
soft-deleted — mirroring the v1.28.39 provider natural-key rescues (closest to
withings/sync-sleep). If no twin resolves it, the one reading is skipped and
the run continues; the catch sits in the per-row loop outside any transaction,
so a collision can never abort the batch or the import. Adopted rows count as
updated, skipped ones surface honestly. Workout + stats: paths verified unable
to hit this (no second unique index / deterministic externalId). Repro +
real-Postgres integration tests added.
Deferred perf fixes from the campaign's verified perf-correctness audit,
correctness-first (each with an output-equivalence test):
- comprehensive-generate fan-out: replaced per-type 400d findMany+JS-average
with DB prisma.aggregate(_avg) over the exact half-open windows the JS
reducer used (current 30d, lastMonth (60d,30d], lastYear (395d,365d]), same
rounding + null-on-empty, pLimit(4). SLEEP_DURATION stays on the raw path
(per-night reconstruction dedup). Byte-equality test vs the old algorithm.
- boot-discovery: replaced Prisma in-memory distinct with DB SELECT DISTINCT
in the 4 backfill discovery scans + migration 0243 (7 additive
partial indexes matching each predicate). Integration test asserts identical
discovered set against real Postgres.
- ChatBubble wrapped in React.memo with a comparator (onRegenerate compared by
presence — the thread rebuilds it each render), so only the streaming bubble
re-renders per token.
- doctor-report mood read narrowed to {score, tags} (the only fields the
collector uses); the 730d main measurement read left as-is — a blind bound
would corrupt clinical stats (documented deferred follow-up).
Migration 0243 additive-only.
The v1.28.33 rollup + staging fixes were correct but never reached the
reporter's case: the upload dedup keys on the content SHA-256 and the
lookup matched a previously FAILED job too, so re-uploading the same
export.zip returned the dead job's id and replayed its stale
failureReason without ever staging or sending again. The dedup now
excludes failed rows (short-circuits only on queued/unpacking/parsing/
upserting/done); the same bytes retry cleanly. A redundant staged upload
is unlinked on a dedup hit instead of leaking in the staging dir.
reconcileOrphanImportJobs no longer unconditionally fails every
non-terminal ImportJob at boot: it now checks pg-boss job state plus a
progress heartbeat (new import_jobs.updated_at, migration 0242), so a
split or multi-worker deployment cannot kill an import running in
another worker. The missing-staging message names the separate-container
cause explicitly, with a matching note in docs/self-hosting/scaling.md.
Client-rendered times fell back to the hardcoded display zone
(Europe/Berlin) because neither useFormatters() nor the legacy format.ts
helpers ever received users.timezone. A third localStorage mirror
(timezone-mirror.ts, following the time-format/date-format pattern)
carries the validated profile zone from /api/auth/me into both read
paths; the fallback chain is strictly profile → Europe/Berlin — never
the browser zone, so screen, PDF, and export cannot disagree.
makeFormatters validates its zone before Intl (a poison value falls back
instead of throwing inside every render).
Same pass, per the two-document audit: formatUpdatedLabel boundary and
clock now resolve one zone in every state (the undefined-tz browser hole
is closed); @db.Date calendar fields render UTC-pinned so profiles west
of UTC cannot shift a day; dose-history day headings and GLP-1 weekday
labels derive from the display zone; chart day/month labels are
encoding-derived and UTC-pinned (bucket math untouched, byte-identical
for Berlin profiles); the clinician share view renders in the patient's
timezone instead of the container's; Playwright pins Europe/Berlin.
Server day-key semantics, streak anchoring, insight cache keys, and
verbatim wall-clock strings are untouched by construction and pinned by
the new Auckland/New_York/Manila + DST test matrix.
Supplement-style intake written to Apple Health by nutrition apps —
vitamins, minerals, water, caffeine — can now sync as daily totals into a
dedicated NutrientIntakeDay store (migration 0241): composite (user, day,
nutrient) identity, re-post replaces, no soft delete, no rollup coupling.
A code-side catalog carries the 26 nutrient codes with canonical units,
plausibility caps, and EFSA dietary reference values (direction-aware;
chromium cites NIH ODS where EFSA declined a DRV).
Deliberately not a nutrition feature: no diary, no calories, no dashboard
surface. POST /api/nutrients/batch is gated behind the new opt-in
nutrients module (default off — with the module off the server refuses
ingest entirely), rate-limited, idempotency-wrapped, with per-entry
unit-mismatch and out-of-range skips. GET /api/nutrients serves a
catalog-ordered summary; the only visible surface is a read-only card
under Settings → Sources. Coach/correlation access is a later slice; the
iOS reading side is coordinated in the companion-app tracker.
Rollup writes were delete-then-createMany inside a transaction; under READ
COMMITTED a concurrent recompute racing the import's full fold on the same
(user, type, granularity, bucket, source) partition tripped the composite
PK — and the P2002 swallow dropped the loser's freshly computed aggregate,
leaving a stale bucket. persistRollupRows now writes chunked
INSERT … ON CONFLICT DO UPDATE with deterministic (type, source, bucket)
lock ordering inside one transaction: a race can neither fail the import
nor lose an aggregate, and previously-staled buckets self-heal on their
next recompute. The vanished-source delete and in-transaction guarantees
are preserved; the now-unreachable P2002 swallow is removed.
The masked ENOENT is gone with it: long imports outlived the queue's
15-minute expiry, pg-boss redelivered the still-running job, and the
redelivery re-opened the consumed staging file — overwriting the real
terminal state with a raw ENOENT. Import sends now carry retryLimit 0 +
6h expiry (a retry after the staging file is consumed can only mask),
deliveries for terminal ImportJob rows are no-ops, a genuinely missing
staging file reports an honest re-upload message, and the extracted XML
is cleaned up on failure too.
Long-term retention for the dense intra-day tier (PULSE, HRV, SpO2) folds
to per-local-hour means instead of a single daily mean: the last 14 days
keep every raw sample, older days keep up to 24 points per metric — the
day's shape survives permanently at ~9k rows/metric/year. Hourly rows key
as stats:<HK>:<YYYY-MM-DD>T<HH> anchored at local HH:30, adopt-in-place
against both unique indexes with the P2002 retry outside the transaction.
The pre-fold DAY-rollup recompute extends to HRV (mean fidelity, not only
min/max) and the post-fold recompute is gone for the whole tier. Derived
resting rows for proxy users still mint from the day's raw samples.
A one-shot rebuild reconstructs the hourly shape for every already-folded
day from the soft-deleted raw samples and retires the daily row in the
same transaction — at no instant are both grains live, so no
AVG-over-live-rows reader can double-count. Boot discovery pairs live
daily-grain rows with surviving tombstones (parameter-bound SQL, pinned
against real Postgres) and converges to zero once converted; days whose
tombstones were pruned keep their daily mean. PULSE/SpO2 DAY buckets keep
the true pre-fold stats; HRV DAY buckets recompute from the hourly rows
(strictly better than the collapsed fold-time bucket); WEEK/MONTH/YEAR
respans enqueue whole-bucket recomputes.
Cover the new DTO provenance end-to-end against a real Postgres: the index
source threads through the list and detail reads, and — the regression the
UI wave flagged — survives a metadata PATCH, which now reads the index row
rather than returning a stale null source.
Marc-André Bombeck