Merge pull request #919 from MBombeck/integration/post-1385

v1.38.6 — glucose in mmol/L, backups inside key rotation
This commit is contained in:
Marc-André Bombeck authored and GitHub committed 2026-09-03 08:17:59 +02:00
commit 3cf65c2c90
76 files changed
+4766 -458

No files matched your search

+182 -9
View File
@@ -1,6 +1,25 @@
# Changelog
## [Unreleased]
## [1.38.6] — 2026-09-03
Blood glucose can be shown in mmol/L, which it never could before, and the
weekly backup finally sits inside key rotation. Text the server writes now
arrives in the language you actually read the app in.
### Changed
- `greetingHour` is no longer part of the dashboard snapshot response. Any
client that decoded it as a required field needs the field made optional or
dropped; the `timezone` it sits beside is unchanged and is what the greeting
should be derived from.
- The rotation script says what it did NOT look at. A zero that means "nothing
left" and a zero that means "never looked" read the same on a summary, and an
operator acts on that zero by dropping a key. Each run now ends with the
count of registered columns it walked, names any it skipped, and exits
non-zero when the two disagree. The runbook's retire-the-key step was
rewritten around that signal instead of around "the script ran cleanly", and
it tells an operator who already rotated on an older release how to get their
backups back.
### Fixed
@@ -28,17 +47,160 @@
endpoints are required by their specifications to ignore parameters they do
not recognise, and an import file comes from another version or another tool
by definition.
- **The greeting could contradict the rest of the page.** Three separate
pieces of code answered the question "what hour is it" for the time-of-day
salutation. The dashboard snapshot computed one server-side and put it on
the payload so that clients would not have to; no client ever read it. The
insights hero worked it out from the profile timezone. The dashboard header
worked it out from a helper of its own that quietly fell back to the
device's clock whenever the runtime could not resolve the configured zone,
which is what happens to a zone the browser's timezone data does not yet
carry. At one instant, for one account, those three returned 20, 21 and 05,
so the header said good morning while the hero said good evening and every
date beneath them belonged to a third zone. The two greetings now read the
same helper, which stays on the configured zone and falls back to the same
default as the rest of the app rather than to the device. The server value
is gone: it is a clock reading, not a fact about the record, and the
snapshot body is served from cache for up to an hour, long enough for a
stored hour to name a salutation the clock had already left behind. What
the server resolves and every client reads is the timezone itself, which
travels on the same payload as before.
- **Some accounts had their generated text written in a language they do not
read.** Everything the app writes without a request in front of it — the
nightly briefing, the Coach nudge, the reminder messages — resolves its
language from one column on the user row, because a background pass has no
cookie and no browser to ask. Everything the user looks at resolves from the
request instead, and the request prefers the language cookie, then the
column, then what the browser asked for. So the two can disagree, and while
they do, nothing on screen looks wrong: the menus, the buttons and the
labels are all correct, and the disagreement shows up as a single generated
paragraph in the wrong language sitting inside an otherwise correct page.
That is how it was reported, on a page where the sentence above the health
score came back in English on an account read entirely in German.
Keeping the column in step was the browser's job, through a best-effort
request sent once when a page mounted and never checked. Anything that
interrupts it — the paint before sign-in, an offline moment, a tab closed
while it was still in flight — loses the write, and the next page load makes
the same unchecked attempt rather than noticing the column is still wrong,
so the wrong language can stand for the life of the account. The server now
keeps the two in step itself: when a request arrives carrying a language the
column does not agree with, the column is corrected, once, on the spot. It
needs nothing from the browser, it cannot be dropped in transit, and it
fixes accounts that have been diverged for months on their next page load.
Accounts that never opened the language picker at all are covered the same
way. Nobody's stored preference is guessed at: only a language the app was
actually being read in can reach the column.
- **The dashboard aggregate greeted some users in English on a German
instance.** The salutation and the streak label are translated on the
server for the native client, and that request carries no language cookie,
so the stored preference is the only signal. When an account had never
opened the language picker the column is empty, and the fallback stopped at
English instead of asking what language the instance is configured for.
It now follows the same order every background writer uses: the user's own
preference, then the operator's default, then English. This was the last
copy of the fallback that the nightly writers were moved off earlier.
- **A failing AI provider says what came back.** The provider health card in
the operator console could report a provider as failing without saying
whose problem it was. The ledger has recorded the HTTP status of every
failure since v1.11 and nothing read it. The card names it beside the
failure it belongs to now, which is the difference between a dead key the
operator has to replace and a provider having a bad afternoon. Blank where
there was no status to record, as with a connection that never completed.
- **Blood glucose can be shown in mmol/L. It never could before.** The
account has carried a glucose display unit since v1.2 and about thirty
places read it: the dashboard tiles, the glucose page, the targets panel,
the CSV and FHIR exports, the doctor report, the Coach's own notes, the
low-reading alert. Nothing could ever set it. There was no control, no
endpoint and no field on any form, so every account sat on the mg/dL
default and anyone who reads glucose in mmol/L got a number they had to
convert in their head on every screen. The unit is now a dropdown in
Settings under your profile, beside the metric/imperial one and
deliberately not folded into it, because metric countries are split on
which unit they read glucose in and one control would get half of them
wrong. Nothing already recorded changes: readings are stored in mg/dL and
stay stored in mg/dL, and the unit only picks how they are shown. The
entry form moved with it, which is the half that mattered. It now asks in
the unit you chose and converts on the way in, so a 5.3 typed by an mmol/L
reader is filed as 95 mg/dL instead of as 5.3, which is inside the
plausible range, passes without a word, and reads back on every surface as
a severe hypo. The measurements list and its edit sheet moved with it for
the same reason, so correcting a typo cannot rewrite a reading into the
other unit.
- **The same reading converted two different ways.** A glucose value
imported in mmol/L was multiplied by 18.016; one displayed in mmol/L was
divided by 18.0182. A value imported in mmol/L therefore did not come back
out as the number it went in as. Both ends take the same factor now, and
the test names the constant rather than repeating the digits, which is how
the two came apart to begin with.
- **An Apple Health import records which export it came from.** The import
status has always carried the instant the Health app stamped on the
archive, the API contract has always promised it, and it was always empty:
the element sat on the parser's list of things to skip. The parser reads
it now and writes it the moment it goes past, so a run that later fails
still says which export it was working from.
- **The weekly backup no longer takes the instance down with it.** The
compression that went out last release was real, and it was measured in the
wrong place. On a development machine with a multi-gigabyte heap the pass
finished; the container it actually runs in is capped at 1 GB, which gives
Node a 524 MB heap, and there the same pass died with
`Reached heap limit — JavaScript heap out of memory` about fifteen seconds
in. Because the job shares the application process, that was not one failed
backup: it was a restart, and every signed-in session on the host went with
it. The document is now written incrementally. The three tables that grow
without bound — readings, doses, mood entries — are read a page at a time and
serialised straight into the compressor and the cipher, every other section
is released as soon as its own JSON exists, and nothing between the database
rows and the stored value is ever held whole. On a seeded account of 445 000
readings the pass used to exhaust a 546 MB heap; the same account doubled to
890 000 now finishes inside 296 MB. The writer also watches its own memory
and stops at 80 percent of the limit, so an account too large for its host is
a backup that failed for that one account and is counted in the run's meta,
not a restart for everybody. Backups written under either older shape still
restore, which is checked both ways rather than asserted.
- **Backups were outside key rotation.** `DataBackup.data` holds every weekly
disaster-recovery snapshot and every uploaded pack, encrypted at rest like
everything else — but under a column called `data`, and the rotation registry
was keyed on the `*Encrypted` naming convention. So the rotation script never
read a backup, reported zero rows remaining, and the runbook told the
operator that zero meant the previous key was safe to retire. Following those
instructions to the letter made every stored backup permanently
undecryptable, which is the last thing that should break. The registry now
covers it, rotation walks it in bounded batches (one row is a whole
compressed account), and it re-seals the ciphertext without reading the
plaintext, so both stored envelopes — the plain JSON and the compressed one —
come back byte-identical, as will any later one. The same sweep found one
more: the idempotent-replay response cache, now rotated too.
- **Rotation crashed a third of the way through and said nothing.** Running the
script against a seeded database — rather than reading it — showed it dying
on the mood day-context note. That table is keyed on the entry it belongs to,
not on an `id`, and the walk asked for a column the database does not have.
Everything after it in the pass, the backups included, was never reached. The
walk now takes the model's actual key, and a check fails the build if a
registered column's key does not match the schema.
- The document index's verbatim text was rotating but never appeared in the
summary, so the new coverage check read it as skipped. It has its own line
now, with its own counts.
### Internal
- A column holding ciphertext is now recognised by what the code writes into
it, not by what it is called. A new guard traces the encryption helpers
through their wrappers, walks every Prisma write payload, and fails when a
column receives ciphertext without being registered for rotation — the
check that would have caught the backup blob nine releases ago. It asserts a
non-zero, anchored match count, so a matcher that quietly stops matching
fails instead of passing, and it was verified by breaking it three ways.
- The column-reader sweep was wrong in both directions and had been for as long as it existed. It answers "does this column have a consumer" for every column in the schema, and it is the tool this project leans on to catch a schema change that ships its writer and forgets its reader. Its idea of a write was any `field:` key anywhere, so `where: { ticketHash }` counted as writing `ticketHash`: all twelve columns it reported were a lookup key or a cron's discovery predicate being filtered on, every one a false alarm. And because it only ever reported a column that HAD a write, a column with no write at all was the one thing it could not see. It now understands which Prisma argument a key sits in, follows a payload assembled into a variable before the call, credits raw SQL against the mapped column name, and reports a column no code touches at all as its own category. Twelve false alarms became nineteen findings that each hold up, four of them worth acting on. The matcher underneath is pinned by a test that was checked by breaking it three ways.
- **Two columns have a reader and no writer.** The glucose unit preference is read across twenty-nine files, the dashboard and both exports and the doctor report and the FHIR resources among them, and no surface anywhere writes it, so an account cannot leave mg/dL. An import job's export date is described in the schema as parsed during unpack, is returned by the import status endpoint and is in the published contract, but the parser skips Apple's `ExportDate` element outright: the field has always been null.
- **Two columns have a writer and no reader.** The provider-health ledger records the HTTP status of a failed call and nothing reads it back, and the known-device ledger stamps a first-seen date that no query, page or export ever asks for.
## [1.38.5] — 2026-09-03
The weekly backup finishes again, the dashboard stops scanning the two
largest tables, and several settings now do what they say.
The weekly backup costs a fraction of the memory it did, the dashboard
stops scanning the two largest tables, and several settings now do what
they say.
### Security
@@ -62,12 +224,23 @@ largest tables, and several settings now do what they say.
which is four full copies of the record at the same moment. The stored copy
is now compressed before it is encrypted, which takes an order of magnitude
off everything after it, and the pass releases each stage before allocating
the next. A seeded 445 000-reading account went from dying of memory to
finishing in about twenty seconds, with a stored copy of 25 MB instead of
322 MB. The pass also has an explicit two-hour window now, so a genuinely
slow disk cannot be mistaken for a broken run. The nightly off-host copy is
built the same way and got the same treatment; files written before this
still restore, and so do backups already sitting in the database.
the next. A seeded 445 000-reading account, on a machine with
a generous heap, went from dying of memory to finishing in about twenty
seconds, with a stored copy of 25 MB instead of 322 MB. The pass also has an
explicit two-hour window now, so a genuinely slow disk cannot be mistaken for
a broken run. The nightly off-host copy is built the same way and got the
same treatment; files written before this still restore, and so do backups
already sitting in the database.
**That measurement does not carry to a small container, and this entry
originally over-claimed from it.** A container capped at 1 GB gives the
runtime a 524 MB heap, and against one of those the pass still runs out of
memory, roughly three times further along than before but short of the end.
Worse, it takes the whole process down with it, so a failed backup becomes an
outage for everyone else on the instance. Building the record incrementally
rather than whole is the actual fix and has not landed yet. Until it does, an
instance backing up a large record wants more than 1 GB.
- **A failed backup run is now visible on the backups page.** The weekly pass
can stop and leave no trace: the page listed whatever copies existed, and a
copy from six weeks ago carries a timestamp exactly like Sunday's. The page
+113 -6
View File
@@ -1,7 +1,7 @@
openapi: 3.1.0
info:
title: HealthLog API
version: 1.38.5
version: 1.38.6
description: >-
Self-hosted personal-health-tracking PWA — public API surface for the iOS native client and external ingest.
@@ -7759,6 +7759,62 @@ paths:
$ref: "#/components/schemas/ErrorEnvelope"
"422": *a4
"429": *a5
/api/auth/me/glucose-unit:
get:
tags:
- Auth
summary: Read the blood-glucose display unit
description: Which unit blood-glucose readings are rendered in — `mg/dL` or `mmol/L`. A separate choice from the
metric/imperial preference, because metric countries are split on which one they read glucose in. Canonical
storage is mg/dL on every reading and every ingest path; this changes presentation only. Any stored value other
than `mmol/L` resolves to `mg/dL`, including a null on an account that never set it.
responses:
"200":
description: The resolved unit.
content:
application/json:
schema:
$ref: "#/components/schemas/GetGlucoseUnitEnvelope"
"401": *a1
"422": *a4
"429": *a5
patch:
tags:
- Auth
summary: Set the blood-glucose display unit
description: >-
Hard-set, idempotent, audit-logged. Rate-limited 60 / min per user.
Nothing already stored is converted or reinterpreted: readings stay mg/dL, and a value typed into the entry form
in mmol/L is inverted to mg/dL before it is written.
The body is capped at 1 KB and must carry `Content-Type: application/json`. Anything larger is 413, a wrong
content type is 415, and a body that is not JSON at all is 400 — the same three the other scalars in this group
answer.
requestBody:
required: true
content:
application/json:
schema:
$ref: "#/components/schemas/GlucoseUnitPatchRequest"
responses:
"200":
description: The resolved next unit.
content:
application/json:
schema:
$ref: "#/components/schemas/PatchGlucoseUnitEnvelope"
"401": *a1
"413":
description: Request body exceeds 1024 bytes. Nothing was written.
content:
application/json:
schema:
$ref: "#/components/schemas/ErrorEnvelope"
"422": *a4
"429": *a5
/api/auth/me/documents-auto-ai-read:
get:
tags:
@@ -23141,6 +23197,17 @@ components:
required:
- unitPreference
description: Which display-unit branch to render. Presentation only — the stored measurement rows stay SI either way.
GlucoseUnitPatchRequest:
type: object
properties:
glucoseUnit:
type: string
enum:
- mg/dL
- mmol/L
required:
- glucoseUnit
description: Which unit blood-glucose readings are rendered in. Presentation only — every stored reading is mg/dL either way.
DocumentsAutoAiReadPatchRequest:
type: object
properties:
@@ -45799,6 +45866,51 @@ components:
- data
- error
additionalProperties: false
GetGlucoseUnitEnvelope:
type: object
properties:
data:
$ref: "#/components/schemas/GlucoseUnitResponse"
error:
type: "null"
meta:
type: object
properties:
requestId:
type: string
additionalProperties: false
required:
- data
- error
additionalProperties: false
GlucoseUnitResponse:
type: object
properties:
glucoseUnit:
type: string
enum:
- mg/dL
- mmol/L
required:
- glucoseUnit
additionalProperties: false
PatchGlucoseUnitEnvelope:
type: object
properties:
data:
$ref: "#/components/schemas/GlucoseUnitResponse"
error:
type: "null"
meta:
type: object
properties:
requestId:
type: string
additionalProperties: false
required:
- data
- error
additionalProperties: false
GetDocumentsAutoAiReadEnvelope:
type: object
properties:
@@ -48650,10 +48762,6 @@ components:
- type: "null"
onboardingTourCompleted:
type: boolean
greetingHour:
type: integer
minimum: -9007199254740991
maximum: 9007199254740991
required:
- username
- timezone
@@ -48662,7 +48770,6 @@ components:
- gender
- glucoseUnit
- onboardingTourCompleted
- greetingHour
additionalProperties: false
layout:
type: object
+53
View File
@@ -122,6 +122,59 @@ The full per-model reasoning lives in `src/lib/export/backup-plan.ts`, where
every excluded model carries a written verdict and a structural test refuses to
let a new model land without one.
## The weekly in-database backup (`data-backup`)
Separate from the off-host job above, and easy to confuse with it. A second
pg-boss job writes one `WEEKLY_AUTO` row per user into `data_backups.data` —
the same JSON document, gzipped and then encrypted under `ENCRYPTION_KEY` /
`ENCRYPTION_KEYS`, staying inside the instance. It is what
`/api/admin/backups/<id>/restore` reads.
### Container memory
This is the part that bites. The job runs inside the app process, so V8's heap
limit is the app's heap limit, and a container capped at 1 GB gives Node a
524 MB old-space limit by default. A long-lived record is bigger than it looks:
several hundred thousand measurements serialise to a JSON document of a few
hundred megabytes, and the writer used to need the object graph and that
document resident at the same time. On a seeded account of 445 000 measurements
under a 546 MB limit that is `FATAL ERROR: Reached heap limit` about thirty
seconds in — and since the job shares the process, the whole instance restarted
and every signed-in session on it went with it.
Two things changed, and both matter to an operator:
- **The writer streams.** The three tables that grow without bound —
measurements, intake events, mood entries — are read a page at a time and
serialised straight into gzip and the cipher, and every other section is
released as soon as its JSON exists. On the same fixture doubled to 890 000
measurements, the pass completes inside a 296 MB heap limit; before the
change, half that record did not fit in 546 MB.
- **It stops itself.** The writer watches its own live heap and aborts at 80 %
of V8's limit. That backup then fails for that one account, is counted in the
run's `users_failed` and `heap_budget_trips` meta, and the pass carries on
with everybody else. A memory failure is now a failed job rather than a
restart for every user on the host.
If `heap_budget_trips` is non-zero in `job.data_backup`, the answer is more
memory, not a retry: raise the container's limit, or set
`NODE_OPTIONS=--max-old-space-size=<MB>` to something under it.
### The stored column is the remaining ceiling
`data_backups.data` is a single `text` column, so the finished artifact has to
exist as one value before it can be written — that copy cannot be streamed
away. It is the only thing left in the job that grows with the record: at a
compressed blob of ~42 MB the pass peaks around 236 MB, and the growth from
there is roughly twice the blob's size (the base64 answer, plus the driver's
copy of it on the way to the wire). A blob past roughly 150 MB — an account
several times larger than any seen so far — will hit a 524 MB limit again, and
the fix at that point is to stop storing the artifact in one column: chunk it
across rows, or keep only the off-host copy. Reading it back has the same
shape, and worse: a restore parses the whole document, so the read path needs
several times the blob in heap. An operator restoring a very large account
should give the container more memory for the duration.
## Monthly restore drill (automatic)
Since v1.16.4 a pg-boss job (`data-restore-drill`, cron `11 4 1 * *` —
+55 -14
View File
@@ -41,35 +41,76 @@ under `v1` (the synthetic id assigned to the existing `ENCRYPTION_KEY`).
Restart the app. New writes are now encrypted under `v2`; existing
`v1`-keyed and legacy bare rows still decrypt because the `v1` entry is
retained.
3. **Run the rotation script** to re-encrypt every existing encrypted column
3. **Run the rotation script** to re-encrypt every registered encrypted column
under the new active key:
```
pnpm dlx tsx scripts/rotate-encryption-key.ts v2
```
The script is idempotent — running it again is a no-op for rows already
prefixed with `v2.`. It rotates every registered encrypted column across
all models, plus the two non-`*Encrypted` members:
`IntegrationStatus.lastError` and `CoachMessage.encryptedContent`
(Bytes). The canonical registry lives in
`src/lib/crypto/encrypted-columns.ts`; a guard test fails CI if any
`*Encrypted` column in the schema is missing from the registry or not
referenced by the script. The summary line at the
end shows `scanned`, `rotated`, and `errors` per table+field; treat
`errors > 0` as a hard failure and re-run after fixing the cause.
4. **Drop the old key (optional, recommended).** Once the script has run
cleanly:
prefixed with `v2.`. It rotates every column in the canonical registry
(`src/lib/crypto/encrypted-columns.ts`), which covers the `*Encrypted`
columns plus the ones whose names say nothing about their contents:
`IntegrationStatus.lastError`, `CoachMessage.encryptedContent`,
`NotificationChannel.config`, the OAuth `accessToken` / `refreshToken`
columns, the web-push `p256dh` / `auth` secrets, the idempotent-replay
`IdempotencyKey.responseBody`, and `DataBackup.data` — the whole-account
backup blob.
Read three things off the output before going further:
- the per-column line, `scanned` / `rotated` / `errors` / `dropped`. Treat
`errors > 0` as a hard failure and re-run after fixing the cause. A
`dropped` count is only ever a cache row the run could not read and
deleted rather than leave unreadable.
- `Columns walked: N/M registered`. The two numbers must match.
- a `NOT WALKED` block, if one appears. It lists registered columns this
run skipped, and the run exits non-zero. Rotation is incomplete; do not
go on to step 4.
Two guard tests keep the registry honest in CI: one scans the schema for
`*Encrypted` columns, the other derives the ciphertext-bearing columns from
the Prisma write payloads, which is what catches a column that holds
ciphertext under an ordinary name.
4. **Confirm nothing is left on the old key, THEN drop it.** This is the step
that destroys data if it is taken on a bad signal, so check the corpus
rather than the absence of complaints. Either re-run the script — a clean
second pass reports `rotated=0`, `errors=0` and a full `Columns walked`
line — or open **Admin → Encryption** and confirm the status view reports
zero rows outside the active key. Both read the same registry the rotation
walks.
A zero only means "nothing left" when the run also says it walked every
registered column. A zero from a run that skipped columns means "never
looked", and dropping the old key on it makes those rows permanently
undecryptable — `decrypt()` is fail-closed and there is no recovery path.
Only once the corpus reads clean:
```
ENCRYPTION_KEYS='{"v2":"<new key>"}'
ENCRYPTION_ACTIVE_KEY_ID="v2"
# ENCRYPTION_KEY can now be removed
```
Restart. The legacy single-key fallback is now disconnected; only `v2`
exists.
> **Backups are covered, and were not always.** `DataBackup.data` holds every
> weekly disaster-recovery snapshot and every uploaded pack, encrypted like
> everything else but under a column called `data`. It was outside the
> registry until v1.38.6, so a rotation before that release reported zero
> rows remaining without ever reading a backup. If you rotated on an older
> release and dropped the previous key, the stored backups are encrypted
> under the key you removed: put that key back into `ENCRYPTION_KEYS` and
> re-run the rotation on this release before removing it again.
## Adding a third key (v2 → v3)
Same procedure, just shift the labels — keep `v2` in the map until the
script reports zero `v2.`-prefixed rows remaining.
Same procedure, just shift the labels — keep `v2` in the map until a full
run (every registered column walked) reports zero `v2.`-prefixed rows
remaining.
```
ENCRYPTION_KEYS='{"v2":"<old>","v3":"<new>"}'
-1
View File
@@ -258,7 +258,6 @@ export function buildMockSnapshot(
gender: "MALE",
glucoseUnit: "mg/dL",
onboardingTourCompleted: true,
greetingHour: 9,
},
layout: DEFAULT_DASHBOARD_LAYOUT,
layoutCatalogue: buildLayoutCatalogue(),
+7
View File
@@ -5681,6 +5681,12 @@
"imperial": "Imperial",
"saved": "Einheiten-Einstellung gespeichert",
"saveError": "Einheiten-Einstellung konnte nicht gespeichert werden."
},
"glucoseUnit": {
"title": "Blutzucker",
"description": "In welcher Einheit Blutzuckerwerte angezeigt werden. Gespeichert werden sie so oder so gleich.",
"saved": "Blutzucker-Einheit gespeichert",
"saveError": "Die Blutzucker-Einheit konnte nicht gespeichert werden."
}
},
"withings": "Withings",
@@ -6501,6 +6507,7 @@
"trackedUsers": "{count} Nutzer erfasst",
"lastOk": "letzter Erfolg {at}",
"lastFailure": "letzter Fehlschlag {at}",
"lastFailureStatus": "HTTP {status}",
"statusHealthy": "In Ordnung",
"statusFailing": "{failing} von {tracked} scheitern, {run} in Folge",
"typeAdminOpenai": "Server-Schlüssel (Betreiber)",
+7
View File
@@ -5681,6 +5681,12 @@
"imperial": "Imperial",
"saved": "Unit preference saved",
"saveError": "Could not save your unit preference."
},
"glucoseUnit": {
"title": "Blood glucose",
"description": "Which unit blood-glucose readings are shown in. Readings are stored the same way either way.",
"saved": "Glucose unit saved",
"saveError": "Could not save your glucose unit."
}
},
"withings": "Withings",
@@ -6501,6 +6507,7 @@
"trackedUsers": "{count} users tracked",
"lastOk": "last success {at}",
"lastFailure": "last failure {at}",
"lastFailureStatus": "HTTP {status}",
"statusHealthy": "Healthy",
"statusFailing": "{failing} of {tracked} failing, {run} in a row",
"typeAdminOpenai": "Server key (operator)",
+7
View File
@@ -5681,6 +5681,12 @@
"imperial": "Imperial",
"saved": "Preferencia de unidades guardada",
"saveError": "No se pudo guardar tu preferencia de unidades."
},
"glucoseUnit": {
"title": "Glucosa en sangre",
"description": "La unidad en la que se muestran las lecturas de glucosa. Se guardan igual en ambos casos.",
"saved": "Unidad de glucosa guardada",
"saveError": "No se pudo guardar la unidad de glucosa."
}
},
"withings": "Withings",
@@ -6501,6 +6507,7 @@
"trackedUsers": "{count} usuarios registrados",
"lastOk": "último éxito {at}",
"lastFailure": "último fallo {at}",
"lastFailureStatus": "HTTP {status}",
"statusHealthy": "Correcto",
"statusFailing": "{failing} de {tracked} fallan, {run} seguidos",
"typeAdminOpenai": "Clave del servidor (operador)",
+7
View File
@@ -5681,6 +5681,12 @@
"imperial": "Impérial",
"saved": "Préférence d'unités enregistrée",
"saveError": "Impossible d'enregistrer votre préférence d'unités."
},
"glucoseUnit": {
"title": "Glycémie",
"description": "L'unité dans laquelle les mesures de glycémie sont affichées. L'enregistrement est le même dans les deux cas.",
"saved": "Unité de glycémie enregistrée",
"saveError": "Impossible d'enregistrer l'unité de glycémie."
}
},
"withings": "Withings",
@@ -6501,6 +6507,7 @@
"trackedUsers": "{count} utilisateurs suivis",
"lastOk": "dernier succès {at}",
"lastFailure": "dernier échec {at}",
"lastFailureStatus": "HTTP {status}",
"statusHealthy": "En bon état",
"statusFailing": "{failing} sur {tracked} en échec, {run} d'affilée",
"typeAdminOpenai": "Clé du serveur (opérateur)",
+7
View File
@@ -5681,6 +5681,12 @@
"imperial": "Imperiale",
"saved": "Preferenza unità salvata",
"saveError": "Impossibile salvare la preferenza delle unità."
},
"glucoseUnit": {
"title": "Glicemia",
"description": "L'unità in cui vengono mostrate le misurazioni della glicemia. Il salvataggio resta lo stesso in entrambi i casi.",
"saved": "Unità della glicemia salvata",
"saveError": "Impossibile salvare l'unità della glicemia."
}
},
"withings": "Withings",
@@ -6501,6 +6507,7 @@
"trackedUsers": "{count} utenti tracciati",
"lastOk": "ultimo successo {at}",
"lastFailure": "ultimo errore {at}",
"lastFailureStatus": "HTTP {status}",
"statusHealthy": "Regolare",
"statusFailing": "{failing} su {tracked} in errore, {run} di fila",
"typeAdminOpenai": "Chiave del server (operatore)",
+7
View File
@@ -5681,6 +5681,12 @@
"imperial": "야드파운드법",
"saved": "단위 설정을 저장했어요",
"saveError": "단위 설정을 저장하지 못했어요."
},
"glucoseUnit": {
"title": "혈당",
"description": "혈당 수치를 표시할 단위입니다. 어느 쪽을 골라도 저장 방식은 같습니다.",
"saved": "혈당 단위를 저장했습니다",
"saveError": "혈당 단위를 저장하지 못했습니다."
}
},
"withings": "Withings",
@@ -7083,6 +7089,7 @@
"trackedUsers": "{count}명 사용자 추적됨",
"lastOk": "마지막 성공 {at}",
"lastFailure": "마지막 실패 {at}",
"lastFailureStatus": "HTTP {status}",
"statusHealthy": "정상",
"statusFailing": "{tracked}개 중 {failing}개 실패, {run}회 연속",
"typeAdminOpenai": "서버 키 (운영자)",
+7
View File
@@ -5681,6 +5681,12 @@
"imperial": "Imperialne",
"saved": "Zapisano preferencję jednostek",
"saveError": "Nie udało się zapisać preferencji jednostek."
},
"glucoseUnit": {
"title": "Glukoza we krwi",
"description": "Jednostka, w której pokazywane są pomiary glukozy. Zapis pozostaje taki sam w obu przypadkach.",
"saved": "Zapisano jednostkę glukozy",
"saveError": "Nie udało się zapisać jednostki glukozy."
}
},
"withings": "Withings",
@@ -6501,6 +6507,7 @@
"trackedUsers": "Śledzeni użytkownicy: {count}",
"lastOk": "ostatni sukces {at}",
"lastFailure": "ostatnia awaria {at}",
"lastFailureStatus": "HTTP {status}",
"statusHealthy": "Sprawny",
"statusFailing": "{failing} z {tracked} zawodzi, {run} z rzędu",
"typeAdminOpenai": "Klucz serwera (operator)",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "healthlog",
"version": "1.38.5",
"version": "1.38.6",
"description": "Self-hosted personal-health-tracking PWA with Withings integration, AI insights, and doctor-report PDF export.",
"license": "PolyForm-Noncommercial-1.0.0",
"homepage": "https://healthlog.dev",
+31 -3
View File
@@ -1302,6 +1302,20 @@ model UserKnownDevice {
userId String @map("user_id")
deviceHash String @map("device_hash")
label String?
/// @internal: the row's own creation instant, supplied by this default —
/// `recordSignInDevice` (`src/lib/auth/login-alert.ts`) inserts the
/// fingerprint and the label and passes no timestamp. Nothing reads it. The
/// first sighting a USER sees is the `auth.login.new_device` audit row the
/// same function writes, rendered by the security-activity card, and that
/// row is removed on the audit-retention schedule. This column is the
/// ledger's own copy of the same instant, kept because the ledger outlives
/// the audit trail and an operator asking when a device first appeared on an
/// account has nowhere else to look. Deliberately reachable through the
/// database only: the ledger stores salted hashes and a coarse label
/// precisely so it has no user-facing surface. The backup does NOT select it
/// — the whole model is excluded (`src/lib/export/backup-plan.ts`), because
/// restoring a device-recognition ledger would pre-trust devices on a host
/// that has never seen them.
firstSeenAt DateTime @default(now()) @map("first_seen_at")
lastSeenAt DateTime @default(now()) @map("last_seen_at")
@@ -4968,8 +4982,16 @@ model ImportJob {
/// `APPLE_HEALTH_IMPORT_PARSER_REVISION`, so a job parsed under an older
/// revision never short-circuits a fresh upload of the same bytes.
parserRevision Int @default(1) @map("parser_revision")
/// Apple's `ExportDate` attribute string-formatted, parsed during
/// the unpack phase. NULL until the parser sees it.
/// The instant the Health app stamped on the archive itself — the
/// `value` attribute of the `<ExportDate>` element that opens
/// `export.xml`. Written mid-run by the parse phase the moment the
/// element goes past (`onExportDate` in
/// `src/lib/measurements/import-apple-health-export.ts`, persisted by
/// `src/lib/jobs/apple-health-import-worker.ts`), so a job that later
/// fails still records which export it was working from. Consumer: the
/// status endpoint (`src/app/api/import/apple-health-export/[jobId]/
/// status/route.ts`). NULL until the parser reaches the element, and
/// for an archive that omits it or writes an unreadable value.
exportedAt DateTime? @map("exported_at")
startedAt DateTime @default(now()) @map("started_at")
completedAt DateTime? @map("completed_at")
@@ -5062,7 +5084,13 @@ model ProviderHealth {
userId String @map("user_id")
providerType String @map("provider_type") // ProviderChainType tag
lastResult String @map("last_result") // "ok" | "hard_failed" | "auth_failed"
lastStatus Int? @map("last_status") // HTTP status of the last failure; null on ok/network
/// HTTP status the last failure came back with; null on a success and on
/// a network-class failure that never had a response to read one from.
/// Consumer: the operator readout (`src/app/api/admin/provider-health/
/// route.ts`), which carries it beside the failure instant it belongs to —
/// it is what separates a dead credential the operator has to replace from
/// a provider having a bad afternoon.
lastStatus Int? @map("last_status")
consecutiveFailures Int @default(0) @map("consecutive_failures")
lastOkAt DateTime? @map("last_ok_at")
lastFailureAt DateTime? @map("last_failure_at")
+1 -1
View File
@@ -36,7 +36,7 @@ try {
// v1.4.38.4 → v1.4.42. Do not hand-edit; bump `package.json` and rebuild.
const CACHE_VERSION =
(typeof self !== "undefined" && self.__APP_VERSION__) ||
/* @sw-version-fallback */ "v1.38.5";
/* @sw-version-fallback */ "v1.38.6";
const STATIC_CACHE = `healthlog-static-${CACHE_VERSION}`;
const PAGE_CACHE = `healthlog-pages-${CACHE_VERSION}`;
// v1.18.6 — read-only data cache for a curated allowlist of safe GET `/api/*`
+141 -23
View File
@@ -6,13 +6,16 @@
* already carries the active key id prefix are skipped.
*
* The set of columns this script rotates is the canonical registry in
* `src/lib/crypto/encrypted-columns.ts`. The guard test
* `src/lib/crypto/__tests__/encrypted-columns.test.ts` fails CI if any
* encrypted column in `prisma/schema.prisma` is missing from the registry,
* or if any registry column is not referenced here — so a new `*Encrypted`
* column can never silently skip rotation (which would make those rows
* permanently undecryptable once the legacy key is dropped, since decrypt is
* fail-closed).
* `src/lib/crypto/encrypted-columns.ts`. Two guard tests keep it honest:
* `encrypted-columns.test.ts` fails CI if a `*Encrypted`-named schema column
* is missing from the registry or is not referenced here, and
* `encrypted-column-writers.test.ts` derives the ciphertext-bearing columns
* from the Prisma write payloads instead of from names — the check that would
* have caught `DataBackup.data`, ciphertext under a column called `data`.
*
* The run ends by naming every registered column it did NOT walk and exiting
* non-zero, because an operator reads a zero here as permission to drop the
* previous key, and "nothing left" and "never looked" must not print alike.
*
* Usage:
* ENCRYPTION_KEYS='{"v1":"<old>","v2":"<new>"}' \
@@ -28,6 +31,11 @@ import {
rotateColumn,
type CorpusClient,
} from "@/lib/crypto/encryption-corpus";
import {
ENCRYPTED_COLUMNS,
encryptedColumnKey,
type EncryptedColumn,
} from "@/lib/crypto/encrypted-columns";
import { tokeniseAndHash } from "@/lib/documents/content-index";
const DATABASE_URL = process.env.DATABASE_URL;
@@ -55,6 +63,42 @@ interface RotationResult {
scanned: number;
rotated: number;
errors: number;
/** Unreadable rows deleted from a `disposable` column (cache entries only). */
dropped: number;
}
/**
* Look a column up in the canonical registry so the walk honours the flags
* recorded there (`batched`, `disposable`, `codecField`) rather than a second
* hand-written copy of them here.
*/
function registryColumn(model: string, field: string): EncryptedColumn {
const col = ENCRYPTED_COLUMNS.find(
(c) => c.model === model && c.field === field,
);
if (!col) {
throw new Error(
`${model}.${field} is not in the encrypted-column registry`,
);
}
return col;
}
/** Rotate one column through the shared registry-driven corpus walk. */
async function rotateRegistryColumn(
model: string,
field: string,
client: CorpusClient,
): Promise<RotationResult> {
const r = await rotateColumn(client, registryColumn(model, field));
return {
table: r.model,
field: r.field,
scanned: r.scanned,
rotated: r.rotated,
errors: r.errors,
dropped: r.dropped,
};
}
function shouldRotate(value: string | null): boolean {
@@ -89,6 +133,7 @@ async function rotateStringColumn(
scanned: rows.length,
rotated: 0,
errors: 0,
dropped: 0,
};
for (const row of rows) {
const v = row[field] as string | null;
@@ -131,6 +176,7 @@ async function rotateBytesColumn(
scanned: rows.length,
rotated: 0,
errors: 0,
dropped: 0,
};
for (const row of rows) {
const buf = row[field] as Uint8Array | null;
@@ -487,13 +533,14 @@ async function main() {
// v1.38 — the day-context note on a mood entry. Its own table, so its own
// walk; a context row exists only where somebody added one, and a row with
// no note carries NULL here and is skipped like every other nullable Bytes
// column.
// column. It goes through the registry-driven walk rather than the generic
// `id`-keyed one above: the table is keyed on `moodEntryId`, and selecting a
// column Prisma does not know threw mid-pass and abandoned every column
// after it — a rotation that reported nothing at all rather than a gap.
results.push(
await rotateBytesColumn(
"MoodContext",
"notesEncrypted",
prisma.moodContext,
),
await rotateRegistryColumn("MoodContext", "notesEncrypted", {
moodContext: prisma.moodContext,
} as unknown as CorpusClient),
);
// ───── v1.25 medication free-text notes (Bytes columns) ─────
@@ -601,6 +648,7 @@ async function main() {
scanned: docResult.scanned,
rotated: docResult.rotated,
errors: docResult.errors,
dropped: docResult.dropped,
});
}
// The staged extracted-fact payloads: the FHIR-staged clinical values and the
@@ -650,6 +698,19 @@ async function main() {
scanned: 0,
rotated: 0,
errors: 0,
dropped: 0,
};
// The verbatim sibling rides the same update, but it needs its own line in
// the summary: the coverage check reads the results to decide which
// registered columns this run actually looked at, and a column that
// rotates without ever being named there reads as skipped.
const verbatim: RotationResult = {
table: "DocumentContentIndex",
field: "verbatimTextEncrypted",
scanned: 0,
rotated: 0,
errors: 0,
dropped: 0,
};
let cursor: string | null = null;
// v1.27.33 (Document vault P4) — re-encrypt the string-shaped BYTEA into a
@@ -676,18 +737,19 @@ async function main() {
if (!buf || buf.byteLength === 0) continue;
const asString = Buffer.from(buf).toString("utf8");
if (!shouldRotate(asString)) continue;
// The "verbatimTextEncrypted" column (nullable; written together with
// "textEncrypted" so it shares the same key) rotates in the same
// update when present. Read outside the try so the catch can count it.
const verbatimBuf = row.verbatimTextEncrypted as Uint8Array | null;
const hasVerbatim = Boolean(verbatimBuf && verbatimBuf.byteLength > 0);
if (hasVerbatim) verbatim.scanned++;
try {
const plaintext = decrypt(asString);
const nextBytes = reEncryptBytes(asString);
const searchTokens = tokeniseAndHash(plaintext);
// The "verbatimTextEncrypted" column (nullable; written together with
// "textEncrypted" so it shares the same key) rotates in the same
// update when present.
const verbatimBuf = row.verbatimTextEncrypted as Uint8Array | null;
const verbatimNext =
verbatimBuf && verbatimBuf.byteLength > 0
? reEncryptBytes(Buffer.from(verbatimBuf).toString("utf8"))
: undefined;
const verbatimNext = hasVerbatim
? reEncryptBytes(Buffer.from(verbatimBuf!).toString("utf8"))
: undefined;
await prisma.documentContentIndex.update({
where: { id: row.id },
data: {
@@ -697,8 +759,10 @@ async function main() {
},
});
result.rotated++;
if (verbatimNext) verbatim.rotated++;
} catch (err) {
result.errors++;
if (hasVerbatim) verbatim.errors++;
console.error(
`[DocumentContentIndex.textEncrypted] row ${row.id}: ${(err as Error).message}`,
);
@@ -708,6 +772,7 @@ async function main() {
if (rows.length < 100) break;
}
results.push(result);
results.push(verbatim);
}
// ───── Document preview thumbnails (Bytes column) ─────
@@ -723,18 +788,71 @@ async function main() {
),
);
// ───── Whole-account backup blob (String, batched) ─────
// `DataBackup.data` holds `packBackupBlob()` output: AES-256-GCM ciphertext
// under a column that does NOT carry the `*Encrypted` suffix, which is why
// it sat outside rotation until v1.38.6. Missing it was the worst possible
// miss — the script reported zero rows remaining, the runbook told the
// operator that zero meant safe to drop the old key, and every stored backup
// became undecryptable. Walked in bounded id-cursor batches because one row
// is an entire compressed account, and re-sealed WITHOUT reading the
// plaintext, so both stored envelopes (the plain backup JSON and the
// `HLZ1:` gzip form) — and any later one — rotate unchanged.
results.push(
await rotateRegistryColumn("DataBackup", "data", {
dataBackup: prisma.dataBackup,
} as unknown as CorpusClient),
);
// ───── Idempotent-replay response cache (String, disposable) ─────
// `IdempotencyKey.responseBody` is the cached response body, encrypted
// because the PHI-returning creates echo their own decrypted DTO. Rotating
// it keeps a key drop from handing a replaying client a body it cannot
// parse; a row that cannot be read is deleted rather than counted as an
// error, because a cache miss only costs a re-run.
results.push(
await rotateRegistryColumn("IdempotencyKey", "responseBody", {
idempotencyKey: prisma.idempotencyKey,
} as unknown as CorpusClient),
);
console.log("\n=== Rotation summary ===");
let totalRotated = 0;
let totalErrors = 0;
let totalDropped = 0;
for (const r of results) {
console.log(
`${r.table}.${r.field}: scanned=${r.scanned} rotated=${r.rotated} errors=${r.errors}`,
`${r.table}.${r.field}: scanned=${r.scanned} rotated=${r.rotated} ` +
`errors=${r.errors} dropped=${r.dropped}`,
);
totalRotated += r.rotated;
totalErrors += r.errors;
totalDropped += r.dropped;
}
console.log(
`\nTOTAL rotated=${totalRotated} errors=${totalErrors} dropped=${totalDropped}`,
);
// ───── Coverage: what was NOT looked at ─────
// A zero that means "nothing left" and a zero that means "never looked" read
// the same on the summary above, and the operator acts on that zero by
// dropping the old key. So say which registered columns this run actually
// walked, and refuse to report success if any of them was skipped.
const walked = new Set(results.map((r) => `${r.table}.${r.field}`));
const notWalked = ENCRYPTED_COLUMNS.filter(
(c) => !walked.has(encryptedColumnKey(c)),
).map(encryptedColumnKey);
console.log(
`\nColumns walked: ${walked.size}/${ENCRYPTED_COLUMNS.length} registered`,
);
if (notWalked.length > 0) {
console.error(
`\nNOT WALKED (${notWalked.length}) — rotation is INCOMPLETE, do not ` +
`drop the previous key:\n ${notWalked.join("\n ")}`,
);
}
console.log(`\nTOTAL rotated=${totalRotated} errors=${totalErrors}`);
await prisma.$disconnect();
if (notWalked.length > 0) process.exit(4);
process.exit(totalErrors > 0 ? 3 : 0);
}
@@ -0,0 +1,131 @@
/**
* The time-of-day greeting has ONE evaluator.
*
* The dashboard header and the insights hero both open with a salutation
* bucketed by the wall-clock hour. For a long time three different pieces
* of code answered "what hour is it":
*
* - the dashboard snapshot builder, which computed a `greetingHour` in
* the user's zone and shipped it on the payload "so the client never
* has to run its own Intl" — and which no client ever read;
* - the insights hero, through the shared `hourInTz` (profile zone,
* Europe/Berlin when the zone will not resolve);
* - the dashboard header, through a local helper that fell back to the
* DEVICE clock whenever the zone would not resolve.
*
* At one instant, for one account, those three returned 20, 21 and 05.
*
* The server value lost. It is not a derived fact about the record, it is
* a clock read, and the snapshot body is served stale-while-revalidate for
* up to an hour — long enough for a baked-in hour to name a salutation the
* clock had left behind. What the server IS authoritative for is the zone,
* and that already reaches every client on the same payload.
*
* This guard fails if either end drifts back:
* T1 — the snapshot builder or its OpenAPI schema starts carrying a
* greeting hour again.
* T2 — a greeting surface reads a clock that is not `hourInTz` — the
* device zone via `Date#getHours`, or a hand-rolled `Intl` walk.
* T3 — behaviourally, the shared helper stays on the profile zone even
* when the device sits elsewhere and the zone will not resolve.
*
* T1/T2 are grep-shaped and assert a non-zero match count, so an empty
* sweep fails loudly instead of passing silently.
*/
import { describe, it, expect, afterAll, vi } from "vitest";
import { readFileSync } from "node:fs";
import { join } from "node:path";
import { hourInTz, DEFAULT_TIMEZONE } from "@/lib/tz/format";
const SRC = join(process.cwd(), "src");
const read = (rel: string) => readFileSync(join(SRC, rel), "utf8");
const SERVER_FILES = [
"lib/dashboard/snapshot.ts",
"lib/openapi/routes/insights/schemas.ts",
] as const;
/** Every surface that paints a time-of-day salutation. */
const GREETING_SURFACES = [
"components/dashboard/dashboard-header.tsx",
"components/insights/hero-strip.tsx",
] as const;
describe("T1 — the snapshot payload carries no wall-clock hour", () => {
it.each(SERVER_FILES)("%s declares no greeting hour", (rel) => {
const src = read(rel);
expect(src.length).toBeGreaterThan(0);
expect(src).not.toMatch(/greetingHour/);
expect(src).not.toMatch(/hourInTimezone/);
});
it("still resolves the zone the clients read", () => {
const src = read("lib/dashboard/snapshot.ts");
expect(src).toMatch(/timezone:\s*userTz/);
});
});
describe("T2 — every greeting surface reads the shared helper", () => {
it.each(GREETING_SURFACES)("%s derives its hour from hourInTz", (rel) => {
const src = read(rel);
expect(src.length).toBeGreaterThan(0);
// Imported from the shared module, not re-implemented next door.
expect(src).toMatch(
/import[\s\S]*?\bhourInTz\b[\s\S]*?from "@\/lib\/tz\/format"/,
);
const calls = src.match(/\bhourInTz\s*\(/g) ?? [];
expect(calls.length).toBeGreaterThan(0);
});
it.each(GREETING_SURFACES)("%s reads no device-local clock", (rel) => {
const src = read(rel);
// `getHours()` is the device zone by definition; a hand-rolled
// `Intl.DateTimeFormat` for the hour is the second evaluator this
// guard exists to prevent.
expect(src).not.toMatch(/\.getHours\s*\(/);
expect(src).not.toMatch(/Intl\.DateTimeFormat/);
expect(src).not.toMatch(/getHourForTimeZone/);
});
it("the retired header helper is gone from the tree", () => {
const src = read("components/dashboard/range-display.tsx");
expect(src.length).toBeGreaterThan(0);
expect(src).not.toMatch(/getHourForTimeZone/);
});
});
describe("T3 — the profile zone beats the device zone", () => {
const INSTANT = new Date("2026-03-01T20:30:00Z");
const originalTz = process.env.TZ;
afterAll(() => {
vi.useRealTimers();
process.env.TZ = originalTz;
});
it("follows the configured zone while the device sits ten hours away", () => {
process.env.TZ = "Asia/Tokyo";
vi.useFakeTimers();
vi.setSystemTime(INSTANT);
// The device says 05:30 (morning). The account is configured for
// Berlin, where it is 21:30 (evening).
expect(new Date().getHours()).toBe(5);
expect(hourInTz(new Date(), "Europe/Berlin")).toBe(21);
});
it("falls back to the shared default, never to the device clock", () => {
process.env.TZ = "Asia/Tokyo";
vi.useFakeTimers();
vi.setSystemTime(INSTANT);
// A zone this runtime cannot resolve — a recently added IANA entry on
// an older engine is the realistic case. The header helper used to
// answer with the device hour here, splitting the two greetings by
// sixteen hours for the same account at the same instant.
const unresolvable = hourInTz(new Date(), "Europe/Atlantis");
expect(unresolvable).toBe(hourInTz(new Date(), DEFAULT_TIMEZONE));
expect(unresolvable).not.toBe(new Date().getHours());
});
});
@@ -36,7 +36,12 @@ vi.mock("@/lib/db", () => ({
toJson: <T>(value: T) => value,
}));
vi.mock("@/lib/crypto", () => ({ decrypt: vi.fn(() => "{}") }));
vi.mock("@/lib/crypto", () => ({
decrypt: vi.fn(() => "{}"),
// The envelope reader asks which stored shape it was handed first.
isStreamCiphertext: () => false,
decryptStream: vi.fn(),
}));
vi.mock("@/lib/crypto/note-cipher", () => ({ encryptNote: vi.fn() }));
vi.mock("@/lib/validations/backup", () => ({
parseBackupPayload: vi.fn(),
@@ -30,8 +30,12 @@ vi.mock("@/lib/db", () => ({
}));
const decryptMock = vi.fn();
// The envelope reader asks which of the stored shapes it was handed before it
// decrypts anything, so a stub of this module has to answer that too.
vi.mock("@/lib/crypto", () => ({
decrypt: (...a: unknown[]) => decryptMock(...a),
isStreamCiphertext: () => false,
decryptStream: vi.fn(),
}));
const parseMock = vi.fn();
@@ -5,6 +5,11 @@
* provider type, failing counted from the LAST result only, the worst
* uninterrupted failure run taken across failing users, central types
* sorted first, and no per-user data in the response.
*
* It also pins the HTTP status the ledger has recorded on every failure
* since v1.11.0 and nothing read until now: it must describe the SAME
* failure the row's instant names, or the card tells the operator to go
* looking at the wrong thing.
*/
import { describe, it, expect, vi, beforeEach } from "vitest";
@@ -49,11 +54,13 @@ function group(
consecutiveFailures?: number;
lastFailureAt?: Date | null;
lastOkAt?: Date | null;
lastStatus?: number | null;
} = {},
) {
return {
providerType,
lastResult,
lastStatus: max.lastStatus ?? null,
_count: { _all: count },
_max: {
consecutiveFailures: max.consecutiveFailures ?? 0,
@@ -86,6 +93,7 @@ describe("GET /api/admin/provider-health", () => {
group("admin-openai", "hard_failed", 2, {
consecutiveFailures: 3334,
lastFailureAt: new Date("2026-08-27T06:00:00Z"),
lastStatus: 503,
}),
] as never);
@@ -99,10 +107,64 @@ describe("GET /api/admin/provider-health", () => {
maxConsecutiveFailures: 3334,
lastOkAt: "2026-08-27T07:00:00.000Z",
lastFailureAt: "2026-08-27T06:00:00.000Z",
lastFailureStatus: 503,
},
]);
});
it("reports the status of the newest failure, not the highest number", async () => {
// A 500 an hour ago and a 401 a minute ago: the operator needs the
// 401, because that is the one that will not clear on its own. Taking
// a MAX over the column would hand back the 500.
groupBy.mockResolvedValue([
group("openai", "hard_failed", 4, {
consecutiveFailures: 2,
lastFailureAt: new Date("2026-08-27T05:00:00Z"),
lastStatus: 500,
}),
group("openai", "auth_failed", 1, {
consecutiveFailures: 1,
lastFailureAt: new Date("2026-08-27T06:59:00Z"),
lastStatus: 401,
}),
] as never);
const res = await GET();
const body = await res.json();
expect(body.data.providers[0]).toMatchObject({
lastFailureAt: "2026-08-27T06:59:00.000Z",
lastFailureStatus: 401,
});
});
it("leaves the status out for a network-class failure that never had one", async () => {
groupBy.mockResolvedValue([
group("local", "hard_failed", 1, {
consecutiveFailures: 9,
lastFailureAt: new Date("2026-08-27T06:00:00Z"),
lastStatus: null,
}),
] as never);
const res = await GET();
const body = await res.json();
expect(body.data.providers[0]).toMatchObject({
lastFailureAt: "2026-08-27T06:00:00.000Z",
lastFailureStatus: null,
});
});
it("asks the ledger to split the fold by status rather than aggregate it", async () => {
groupBy.mockResolvedValue([] as never);
await GET();
expect(groupBy).toHaveBeenCalledWith(
expect.objectContaining({
by: ["providerType", "lastResult", "lastStatus"],
}),
);
});
it("counts auth_failed as failing and keeps an all-ok type at zero", async () => {
groupBy.mockResolvedValue([
group("codex", "auth_failed", 1, { consecutiveFailures: 7 }),
+23 -1
View File
@@ -15,6 +15,12 @@ export const dynamic = "force-dynamic";
* folds the ledger into one row per provider type so the console can show
* it. Read-only, admin-cookie-only, and it exposes counts and timestamps
* only — never which user a row belongs to.
*
* The ledger also records the HTTP status each failure came back with, and
* that is the difference between the two questions an operator actually has
* when a provider goes red: a 401 means the key is dead and only the operator
* can fix it, a 429 or a 5xx means the provider is having a day and waiting is
* the right move. The fold carries it beside the instant it belongs to.
*/
interface ProviderHealthSummary {
@@ -27,6 +33,13 @@ interface ProviderHealthSummary {
maxConsecutiveFailures: number;
lastOkAt: string | null;
lastFailureAt: string | null;
/**
* HTTP status the failure at `lastFailureAt` came back with. Null when
* that failure was a network-class one (no response to read a status
* from), and null when the user it belongs to has since recovered — a
* success clears the status along with the rest of the failure state.
*/
lastFailureStatus: number | null;
}
/** The two operator-managed tags sort first — they affect every user on the chain. */
@@ -37,7 +50,11 @@ export const GET = apiHandler(async () => {
annotate({ action: { name: "admin.provider-health.get" } });
const groups = await prisma.providerHealth.groupBy({
by: ["providerType", "lastResult"],
// `lastStatus` joins the grouping key rather than an aggregate: the
// MAX of two HTTP statuses is not a status, it is arithmetic on a
// label. Grouping by it splits each result bucket per status and the
// fold below picks the one belonging to the newest failure.
by: ["providerType", "lastResult", "lastStatus"],
_count: { _all: true },
_max: {
consecutiveFailures: true,
@@ -55,6 +72,7 @@ export const GET = apiHandler(async () => {
maxConsecutiveFailures: 0,
lastOkAt: null,
lastFailureAt: null,
lastFailureStatus: null,
};
row.tracked += g._count._all;
if (g.lastResult !== "ok") {
@@ -69,6 +87,10 @@ export const GET = apiHandler(async () => {
const fail = g._max.lastFailureAt?.toISOString() ?? null;
if (fail && (!row.lastFailureAt || fail > row.lastFailureAt)) {
row.lastFailureAt = fail;
// Tied to the instant, not taken from whichever group happens to
// have one: a status that describes a different failure than the
// one on screen is worse than no status at all.
row.lastFailureStatus = g.lastStatus;
}
byType.set(g.providerType, row);
}
@@ -0,0 +1,210 @@
import { beforeEach, describe, expect, it, vi } from "vitest";
/**
* The blood-glucose display unit — the writer the column never had.
*
* `User.glucoseUnit` shipped in v1.2 and about thirty surfaces read it, but
* no route, form or schema could ever set it, so every account stayed on the
* mg/dL default and the mmol/L half of the app was dead code. This pins the
* missing end: a PATCH writes the column and echoes the resolved state, a GET
* reads it back, and an unrecognised unit is refused rather than stored —
* a stray string in that column would resolve to mg/dL and quietly put every
* mmol/L reader back on the wrong number.
*/
vi.mock("@/lib/db", () => ({
prisma: {
user: {
findUnique: vi.fn(),
update: vi.fn(),
},
},
}));
vi.mock("@/lib/auth/session", () => ({ getSession: vi.fn() }));
vi.mock("@/lib/auth/audit", () => ({
auditLog: vi.fn().mockResolvedValue(undefined),
}));
vi.mock("@/lib/logging/transports", () => ({ emitIfSampled: vi.fn() }));
vi.mock("@/lib/db-compat", () => ({
ensureDbCompatibility: vi.fn().mockResolvedValue(undefined),
}));
vi.mock("@/lib/rate-limit", () => ({
checkRateLimit: vi.fn().mockResolvedValue({
allowed: true,
remaining: 59,
resetAt: Date.now() + 60_000,
}),
rateLimitHeaders: () => ({}),
}));
vi.mock("next/headers", () => ({
headers: vi.fn(async () => ({ get: () => null })),
cookies: vi.fn(async () => ({
get: () => undefined,
set: () => {},
delete: () => {},
})),
}));
import { GET, PATCH } from "../route";
import { prisma } from "@/lib/db";
import { getSession } from "@/lib/auth/session";
import { auditLog } from "@/lib/auth/audit";
import { checkRateLimit } from "@/lib/rate-limit";
const SESSION_OK = {
session: { id: "sess-1", expiresAt: new Date(Date.now() + 3_600_000) },
user: { id: "user-1", username: "testuser", role: "USER" as const },
};
function mkGet(): Request {
return new Request("http://localhost/api/auth/me/glucose-unit");
}
function mkPatch(body: unknown): Request {
return new Request("http://localhost/api/auth/me/glucose-unit", {
method: "PATCH",
body: JSON.stringify(body),
headers: { "Content-Type": "application/json" },
});
}
beforeEach(() => {
vi.clearAllMocks();
});
describe("GET /api/auth/me/glucose-unit", () => {
it("rejects an unauthenticated request with 401", async () => {
vi.mocked(getSession).mockResolvedValue(null);
const res = await (GET as (r: Request) => Promise<Response>)(mkGet());
expect(res.status).toBe(401);
expect(prisma.user.findUnique).not.toHaveBeenCalled();
});
it("resolves an account that never chose to mg/dL", async () => {
vi.mocked(getSession).mockResolvedValue(SESSION_OK as never);
vi.mocked(prisma.user.findUnique).mockResolvedValue({
glucoseUnit: null,
} as never);
const res = await (GET as (r: Request) => Promise<Response>)(mkGet());
expect(res.status).toBe(200);
const env = (await res.json()) as { data: { glucoseUnit: string } };
expect(env.data.glucoseUnit).toBe("mg/dL");
});
it("reads back the unit the account chose", async () => {
vi.mocked(getSession).mockResolvedValue(SESSION_OK as never);
vi.mocked(prisma.user.findUnique).mockResolvedValue({
glucoseUnit: "mmol/L",
} as never);
const res = await (GET as (r: Request) => Promise<Response>)(mkGet());
const env = (await res.json()) as { data: { glucoseUnit: string } };
expect(env.data.glucoseUnit).toBe("mmol/L");
});
});
describe("PATCH /api/auth/me/glucose-unit", () => {
it("rejects an unauthenticated request with 401", async () => {
vi.mocked(getSession).mockResolvedValue(null);
const res = await (PATCH as (r: Request) => Promise<Response>)(
mkPatch({ glucoseUnit: "mmol/L" }),
);
expect(res.status).toBe(401);
expect(prisma.user.update).not.toHaveBeenCalled();
});
it("moves an account to mmol/L, echoes it, and audits the transition", async () => {
vi.mocked(getSession).mockResolvedValue(SESSION_OK as never);
vi.mocked(prisma.user.findUnique).mockResolvedValue({
glucoseUnit: null,
} as never);
vi.mocked(prisma.user.update).mockResolvedValue({} as never);
const res = await (PATCH as (r: Request) => Promise<Response>)(
mkPatch({ glucoseUnit: "mmol/L" }),
);
expect(res.status).toBe(200);
const env = (await res.json()) as { data: { glucoseUnit: string } };
expect(env.data.glucoseUnit).toBe("mmol/L");
// The write must actually land, scoped to the caller, field-by-field.
expect(prisma.user.update).toHaveBeenCalledWith({
where: { id: "user-1" },
data: { glucoseUnit: "mmol/L" },
});
expect(auditLog).toHaveBeenCalledWith(
"user.glucose-unit.update",
expect.objectContaining({
userId: "user-1",
details: expect.objectContaining({
previous: "mg/dL",
next: "mmol/L",
}),
}),
);
});
it("moves an account back to mg/dL", async () => {
vi.mocked(getSession).mockResolvedValue(SESSION_OK as never);
vi.mocked(prisma.user.findUnique).mockResolvedValue({
glucoseUnit: "mmol/L",
} as never);
vi.mocked(prisma.user.update).mockResolvedValue({} as never);
const res = await (PATCH as (r: Request) => Promise<Response>)(
mkPatch({ glucoseUnit: "mg/dL" }),
);
expect(res.status).toBe(200);
expect(prisma.user.update).toHaveBeenCalledWith({
where: { id: "user-1" },
data: { glucoseUnit: "mg/dL" },
});
});
it("refuses a unit outside the two the app renders", async () => {
vi.mocked(getSession).mockResolvedValue(SESSION_OK as never);
for (const bad of ["mmol/l", "mgdl", "imperial", "", 1]) {
const res = await (PATCH as (r: Request) => Promise<Response>)(
mkPatch({ glucoseUnit: bad }),
);
expect(res.status).toBe(422);
}
expect(prisma.user.update).not.toHaveBeenCalled();
});
it("rejects a missing glucoseUnit field with 422", async () => {
vi.mocked(getSession).mockResolvedValue(SESSION_OK as never);
const res = await (PATCH as (r: Request) => Promise<Response>)(
mkPatch({ otherField: "mmol/L" }),
);
expect(res.status).toBe(422);
expect(prisma.user.update).not.toHaveBeenCalled();
});
it("returns 429 when the per-user rate-limit fires", async () => {
vi.mocked(getSession).mockResolvedValue(SESSION_OK as never);
vi.mocked(checkRateLimit).mockResolvedValueOnce({
allowed: false,
remaining: 0,
resetAt: Date.now() + 30_000,
});
const res = await (PATCH as (r: Request) => Promise<Response>)(
mkPatch({ glucoseUnit: "mmol/L" }),
);
expect(res.status).toBe(429);
expect(prisma.user.update).not.toHaveBeenCalled();
});
});
+122
View File
@@ -0,0 +1,122 @@
/**
* Per-user blood-glucose display unit.
*
* GET /api/auth/me/glucose-unit — current unit.
* PATCH /api/auth/me/glucose-unit — body
* `{ glucoseUnit: "mg/dL" | "mmol/L" }`.
*
* The column has existed since v1.2 and roughly thirty surfaces read it —
* the dashboard tiles, the glucose insight page, the targets panel, the CSV
* and FHIR exports, the doctor report, the Coach snapshot, the safety-floor
* push copy. None of it was reachable, because nothing could ever write the
* column: every account was NULL, and NULL resolves to mg/dL. For anyone in
* a country that reads glucose in mmol/L that was the wrong number on every
* screen. This is the missing end.
*
* Canonical storage stays mg/dL on every row and every ingest path. The unit
* selects a display branch, exactly like `unit-preference` next door; the
* entry form inverts through `toCanonicalMgdl` so a value TYPED in mmol/L
* still persists as mg/dL. Nothing already stored is reinterpreted.
*
* Mirrors the `unit-preference` per-user-scalar pattern: 60/min rate limit,
* `safeJson` with a 1 KB cap, Zod safeParse → 422 via `returnAllZodIssues`,
* audit-log row, field-by-field write. Idempotent — the endpoint always
* returns the resolved next state so the client can hard-set the optimistic
* update without an extra round-trip.
*/
import { apiHandler, requireAuth } from "@/lib/api-handler";
import {
apiError,
apiSuccess,
getClientIp,
returnAllZodIssues,
safeJson,
} from "@/lib/api-response";
import { annotate } from "@/lib/logging/context";
import { auditLog } from "@/lib/auth/audit";
import { prisma } from "@/lib/db";
import { checkRateLimit, rateLimitHeaders } from "@/lib/rate-limit";
import { resolveGlucoseUnit, type GlucoseUnit } from "@/lib/glucose";
import { glucoseUnitPatchSchema } from "@/lib/validations/user-prefs";
export const dynamic = "force-dynamic";
const PATCH_RATE_LIMIT = 60;
const PATCH_WINDOW_MS = 60_000;
type GlucoseUnitResponse = {
glucoseUnit: GlucoseUnit;
};
export const GET = apiHandler(async () => {
const { user } = await requireAuth();
annotate({ action: { name: "auth.me.glucose-unit.get" } });
const row = await prisma.user.findUnique({
where: { id: user.id },
select: { glucoseUnit: true },
});
const payload: GlucoseUnitResponse = {
glucoseUnit: resolveGlucoseUnit(row?.glucoseUnit),
};
return apiSuccess(payload);
});
export const PATCH = apiHandler(async (req: Request) => {
const { user } = await requireAuth();
const rl = await checkRateLimit(
`glucose-unit:patch:${user.id}`,
PATCH_RATE_LIMIT,
PATCH_WINDOW_MS,
);
if (!rl.allowed) {
const response = apiError("Too many requests", 429);
for (const [k, v] of Object.entries(rateLimitHeaders(rl))) {
response.headers.set(k, v);
}
return response;
}
// The body is a single enum value — bound the parse so a malformed or
// oversized payload is rejected before it is materialised.
const { data: body, error: jsonError } = await safeJson(req, {
maxBytes: 1024,
});
if (jsonError) return jsonError;
const parsed = glucoseUnitPatchSchema.safeParse(body);
if (!parsed.success) {
annotate({ action: { name: "auth.me.glucose-unit.patch.invalid_shape" } });
return returnAllZodIssues(parsed.error, 422);
}
const next = parsed.data.glucoseUnit;
const previous = await prisma.user.findUnique({
where: { id: user.id },
select: { glucoseUnit: true },
});
await prisma.user.update({
where: { id: user.id },
data: { glucoseUnit: next },
});
await auditLog("user.glucose-unit.update", {
userId: user.id,
ipAddress: getClientIp(req),
details: {
previous: resolveGlucoseUnit(previous?.glucoseUnit),
next,
},
});
annotate({
action: { name: "auth.me.glucose-unit.patch" },
meta: { glucoseUnit: next },
});
const payload: GlucoseUnitResponse = { glucoseUnit: next };
return apiSuccess(payload);
});
@@ -76,7 +76,6 @@ const SNAPSHOT_BODY = {
gender: "MALE",
glucoseUnit: "mg/dL",
onboardingTourCompleted: true,
greetingHour: 9,
},
layout: { version: 1, widgets: [] },
tiles: {
@@ -121,7 +120,12 @@ describe("GET /api/dashboard/snapshot", () => {
expect(json.data.briefingState).toBe("preparing");
expect(json.data.tiles).toBeDefined();
expect(json.data.extras).toBeNull();
expect(json.data.user.greetingHour).toBe(9);
// Inverted 2026-09: the payload no longer carries a wall-clock hour.
// The body is served stale for up to an hour, so a baked-in hour could
// name a salutation the clock had long left behind. Clients read the
// resolved `timezone` and their own live clock instead.
expect(json.data.user).not.toHaveProperty("greetingHour");
expect(json.data.user.timezone).toBe("Europe/Berlin");
// bfcache-friendly directive present.
expect(res.headers.get("Cache-Control")).toContain("private");
});
@@ -45,6 +45,12 @@ vi.mock("@/lib/db", () => ({
user: {
findUnique: vi.fn(),
},
// The greeting salutation is one of the two strings this route still
// translates server-side; `resolveJobLocale` reads the operator's
// configured default from here when the user's column is NULL.
appSettings: {
findUnique: vi.fn(async () => null),
},
// v1.4.38 W-F — the route now uses `$queryRaw` for the per-type
// latest reading, the 7-day rollup-bucket sparkline, and the
// 365-day distinct-day streak set. Three raw calls per request,
@@ -1089,6 +1095,46 @@ describe("GET /api/dashboard/summary", () => {
* source — `buildMoodDailySeries` for mood, `computeBmi` for BMI — so the
* client consumes a finished figure instead of re-deriving one.
*/
describe("GET /api/dashboard/summary — greeting locale", () => {
// The salutation and the streak label are translated on the server, and
// the native client is this route's only caller: a Bearer request carries
// no locale cookie, so `User.locale` is the sole per-user signal. When it
// is NULL the operator's configured default is the answer — the same
// fallback order every background writer uses. The coercion this replaces
// stopped at English, so a German instance greeted a user who never opened
// the language picker in English while the rest of their screen was German.
it("falls back to the operator default, not to English", async () => {
vi.mocked(getSession).mockResolvedValue(SESSION_OK as never);
vi.mocked(prisma.appSettings.findUnique).mockResolvedValue({
defaultLocale: "de",
} as never);
const res = await callGet(makeReq());
expect(res.status).toBe(200);
const body = (await res.json()) as {
data: { greeting: { salutation: string } };
};
expect(body.data.greeting.salutation).toBe("Hallo, testuser");
});
it("still prefers the user's own stored locale", async () => {
vi.mocked(getSession).mockResolvedValue({
...SESSION_OK,
user: { ...SESSION_OK.user, locale: "en" },
} as never);
vi.mocked(prisma.appSettings.findUnique).mockResolvedValue({
defaultLocale: "de",
} as never);
const res = await callGet(makeReq());
expect(res.status).toBe(200);
const body = (await res.json()) as {
data: { greeting: { salutation: string } };
};
expect(body.data.greeting.salutation).toBe("Hi, testuser");
});
});
describe("GET /api/dashboard/summary — mood + BMI cards", () => {
function cardsOf(body: unknown): Array<Record<string, unknown>> {
return (body as { data: { metrics: Array<Record<string, unknown>> } }).data
+16 -15
View File
@@ -61,7 +61,8 @@ import type { MeasurementType } from "@/generated/prisma/client";
import { measurementTypeEnum } from "@/lib/validations/measurement";
import { CUMULATIVE_HK_TYPES } from "@/lib/measurements/apple-health-mapping";
import { getServerTranslator } from "@/lib/i18n/server-translator";
import { defaultLocale, locales, type Locale } from "@/lib/i18n/config";
import { type Locale } from "@/lib/i18n/config";
import { resolveJobLocale } from "@/lib/i18n/job-locale";
import { userDayKey, DEFAULT_TIMEZONE } from "@/lib/tz/resolver";
import { cachedSwr, caches, type ServerCache } from "@/lib/cache/server-cache";
import { buildMedsTodayBlock } from "@/lib/dashboard/meds-today";
@@ -343,7 +344,8 @@ export const GET = apiHandler(async () => {
Awaited<ReturnType<typeof buildDashboardSummary>>
>,
`${user.id}|dashboard-summary`,
() => buildDashboardSummary(user.id, userTz, buildContext(user)),
async () =>
buildDashboardSummary(user.id, userTz, await buildContext(user)),
annotate,
);
@@ -383,23 +385,22 @@ interface SummaryBuilderContext {
locale: Locale;
}
function buildContext(user: {
async function buildContext(user: {
displayName: string | null;
username: string;
locale: string | null;
}): SummaryBuilderContext {
}): Promise<SummaryBuilderContext> {
const greetingName = user.displayName ?? user.username;
// v1.5.x — accept every shipped locale (de / en / es / fr / it / pl)
// so the greeting + streak label resolve against the user's
// configured language. The legacy narrowing dropped fr/es/it/pl back
// to the default; that bug is masked for the title/unit fields by
// the new key-based wire shape but still mattered for the strings
// that stay translated server-side.
const locale: Locale = (locales as readonly string[]).includes(
user.locale ?? "",
)
? (user.locale as Locale)
: defaultLocale;
// The greeting salutation and the streak label are the two strings this
// route still translates server-side, and the native client is its only
// caller — a Bearer request carries no locale cookie and no meaningful
// Accept-Language, so the stored column is the only per-user signal.
// That is exactly the shape `resolveJobLocale` exists for: stored user
// locale, then the operator's configured default, then English. The
// hand-rolled coercion this replaces stopped at English, so a German
// instance whose user never touched the language picker greeted them in
// English while every other string on the same screen was German.
const locale: Locale = await resolveJobLocale(user.locale);
return { greetingName, locale };
}
@@ -59,6 +59,7 @@ vi.mock("@/lib/rollups/measurement-rollups", async () => {
import { NextRequest } from "next/server";
import { POST } from "../route";
import { MGDL_PER_MMOL } from "@/lib/glucose";
import { checkRateLimit } from "@/lib/rate-limit";
import { prisma } from "@/lib/db";
import { recomputeBucketsForMeasurement } from "@/lib/rollups/measurement-rollups";
@@ -369,7 +370,7 @@ describe("POST /api/import/csv — contextless blood glucose", () => {
externalId: "sensor-1",
glucoseContext: null,
});
expect(row.value).toBeCloseTo(95.4848, 3);
expect(row.value).toBeCloseTo(5.3 * MGDL_PER_MMOL, 3);
expect((row.measuredAt as Date).toISOString()).toBe(
"2024-04-03T02:15:00.000Z",
);
+1 -3
View File
@@ -7,7 +7,7 @@ import { GlucoseClinicalPanel } from "@/components/insights/glucose/glucose-clin
import { useAuth } from "@/hooks/use-auth";
import { useModulePageGuard } from "@/hooks/use-module-page-guard";
import { useTranslations } from "@/lib/i18n/context";
import { resolveGlucoseUnit } from "@/lib/glucose";
import { MGDL_PER_MMOL, resolveGlucoseUnit } from "@/lib/glucose";
/**
* v1.7.0 — `/insights/blood-glucose`.
@@ -25,8 +25,6 @@ import { resolveGlucoseUnit } from "@/lib/glucose";
* exports. mg/dL-preference users are unaffected (scale = 1, integer
* precision).
*/
const MGDL_PER_MMOL = 18.0182;
export default function InsightsBlutzuckerPage() {
const { user } = useAuth();
const { t } = useTranslations();
@@ -0,0 +1,73 @@
import { describe, it, expect, vi } from "vitest";
import { renderToStaticMarkup } from "react-dom/server";
import { I18nProvider } from "@/lib/i18n/context";
/**
* The provider-health card and the HTTP status of the last failure.
*
* The ledger has recorded a status on every failure since v1.11.0 and no
* surface read it, so the card could say a provider was failing without
* saying whose problem it was. A 401 is a credential the operator has to
* replace; a 503 is the provider's afternoon. These pin that the status
* reaches the row, that it is left out when there was none to record,
* and that a healthy provider does not carry a stale one.
*/
const mockQueryState = vi.hoisted(() => ({
data: null as null | object,
isPending: false,
isError: false,
}));
vi.mock("@tanstack/react-query", () => ({
useQuery: () => ({
data: mockQueryState.data,
isPending: mockQueryState.isPending,
isError: mockQueryState.isError,
}),
}));
import { ProviderHealthSection } from "../provider-health-section";
function row(over: Record<string, unknown> = {}) {
return {
providerType: "admin-openai",
tracked: 5,
failing: 2,
maxConsecutiveFailures: 4,
lastOkAt: null,
lastFailureAt: "2026-08-27T06:00:00.000Z",
lastFailureStatus: 401,
...over,
};
}
function render(providers: object[]) {
mockQueryState.data = { providers };
mockQueryState.isPending = false;
mockQueryState.isError = false;
return renderToStaticMarkup(
<I18nProvider initialLocale="en">
<ProviderHealthSection />
</I18nProvider>,
);
}
describe("<ProviderHealthSection> — the last failure's HTTP status", () => {
it("names the status beside the failure it belongs to", () => {
expect(render([row()])).toContain("HTTP 401");
});
it("says nothing where the failure never had a status to record", () => {
expect(render([row({ lastFailureStatus: null })])).not.toContain("HTTP");
});
it("carries no status for a provider that is not failing", () => {
expect(
render([
row({ failing: 0, lastFailureAt: null, lastFailureStatus: 500 }),
]),
).not.toContain("HTTP");
});
});
@@ -18,6 +18,12 @@ import { formatDateTime } from "@/lib/format";
* read the database. The card shows one row per provider type: how many
* users' chains touch it, how many of them are currently failing, the worst
* uninterrupted failure run, and the last success / failure instants.
*
* The failure line also carries the HTTP status the ledger recorded for that
* failure, because "failing" on its own does not say whose problem it is. A
* 401 is a dead credential and waiting will not fix it; a 429 or a 503 is the
* provider's day and waiting is exactly right. Absent for a network-class
* failure, which never had a status to record.
*/
interface ProviderHealthRow {
@@ -27,6 +33,7 @@ interface ProviderHealthRow {
maxConsecutiveFailures: number;
lastOkAt: string | null;
lastFailureAt: string | null;
lastFailureStatus: number | null;
}
export function ProviderHealthSection() {
@@ -106,6 +113,13 @@ export function ProviderHealthSection() {
at: formatDateTime(p.lastFailureAt),
})}`
: ""}
{p.failing > 0 &&
p.lastFailureAt &&
p.lastFailureStatus !== null
? ` · ${t("admin.providerHealth.lastFailureStatus", {
status: p.lastFailureStatus,
})}`
: ""}
</span>
</div>
{p.failing === 0 ? (
@@ -25,7 +25,7 @@ import { useTranslations } from "@/lib/i18n/context";
import { useAuth } from "@/hooks/use-auth";
import { useMounted } from "@/hooks/use-mounted";
import { useRecordCapabilities } from "@/hooks/use-record-capabilities";
import { getHourForTimeZone } from "@/components/dashboard/range-display";
import { hourInTz } from "@/lib/tz/format";
import type { QuickEntryDialog } from "@/components/dashboard/quick-entry-sheets";
export function DashboardHeader({
@@ -47,9 +47,16 @@ export function DashboardHeader({
// hydrates — the text must match the name-less SSR output during
// hydration (React #418) and may only personalise from the first
// client re-render.
//
// The hour comes from the shared `hourInTz`, the same helper the
// insights hero greeting uses. Its predecessor fell back to the
// DEVICE clock whenever the runtime could not resolve the profile
// zone, so a zone this browser's ICU does not carry (a recently added
// one, an older engine) split the two greetings by hours while every
// other date on the page stayed on the profile zone.
const userTimezone = mounted ? user?.timezone : undefined;
const hour = useMemo(
() => (userTimezone ? getHourForTimeZone(userTimezone) : null),
() => (userTimezone ? hourInTz(new Date(), userTimezone) : null),
[userTimezone],
);
const timeGreeting =
@@ -46,24 +46,6 @@ export function toHoursSummary(s: DataSummary): DataSummary {
};
}
export function getHourForTimeZone(timeZone?: string): number {
const now = new Date();
if (!timeZone) return now.getHours();
try {
const parts = new Intl.DateTimeFormat("en-US", {
hour: "2-digit",
hour12: false,
timeZone,
}).formatToParts(now);
const hourPart = parts.find((part) => part.type === "hour")?.value;
const parsed = hourPart ? Number(hourPart) : Number.NaN;
return Number.isNaN(parsed) ? now.getHours() : parsed;
} catch {
return now.getHours();
}
}
export function getRangeColorClass(
value: number | null | undefined,
config: RangeDisplayConfig,
@@ -0,0 +1,98 @@
import { readFileSync } from "node:fs";
import { join } from "node:path";
import { describe, expect, it, vi } from "vitest";
import { renderToStaticMarkup } from "react-dom/server";
/**
* The manual entry form and the blood-glucose display unit.
*
* Glucose is stored in mg/dL and shown in whichever of mg/dL and mmol/L the
* account picked. Until the preference could be set at all, the form could
* hardcode "mg/dL" and be right by accident. It no longer can: a reader on
* mmol/L types 5.3, and 5.3 filed as mg/dL is a severe hypo that the
* plausibility band (20–800) accepts without a word and every surface then
* reads back as real. So the label and the stored number have to move
* together — the field asks in the account's unit and the submit path
* inverts to canonical mg/dL.
*
* The label half is rendered. The submit half is asserted against the source:
* this project runs SSR-only component tests (`@testing-library/react` is not
* a dependency), so nothing here can type into the field and press save. What
* the source assertion pins is the wiring — that the glucose branch inverts
* through `toCanonicalMgdl` rather than sending the typed number on — and
* `src/lib/__tests__/glucose.test.ts` proves the conversion itself.
*/
let glucoseUnit: string | null = null;
vi.mock("@/hooks/use-auth", () => ({
useAuth: () => ({
user: {
id: "u1",
username: "testuser",
role: "USER",
unitPreference: "metric",
glucoseUnit,
},
isAuthenticated: true,
isLoading: false,
}),
}));
vi.mock("@tanstack/react-query", () => ({
useQueryClient: () => ({
invalidateQueries: vi.fn(),
refetchQueries: vi.fn(),
}),
}));
vi.mock("@/lib/api/api-fetch", () => ({
apiPost: vi.fn().mockResolvedValue({}),
}));
import { I18nProvider } from "@/lib/i18n/context";
import { MeasurementForm } from "../measurement-form";
function render(): string {
return renderToStaticMarkup(
<I18nProvider initialLocale="en">
<MeasurementForm defaultType="BLOOD_GLUCOSE" />
</I18nProvider>,
);
}
const formSource = readFileSync(
join(process.cwd(), "src/components/measurements/measurement-form.tsx"),
"utf8",
);
describe("MeasurementForm — blood-glucose entry unit", () => {
it("asks in mg/dL for an account that never chose", () => {
glucoseUnit = null;
const html = render();
expect(html).toContain("mg/dL");
expect(html).not.toContain("mmol/L");
expect(html).toContain('placeholder="95"');
});
it("asks in mmol/L once the account chose it", () => {
glucoseUnit = "mmol/L";
const html = render();
expect(html).toContain("mmol/L");
// A mmol/L reader must not be shown a three-digit mg/dL example.
expect(html).toContain('placeholder="5.3"');
expect(html).not.toContain('placeholder="95"');
});
it("inverts a glucose entry to canonical mg/dL before it is sent", () => {
expect(formSource).toMatch(
/if \(isGlucoseMode\) \{\s*canonicalValue = toCanonicalMgdl\(typed, glucoseUnit\);/,
);
});
it("resolves the unit from the account rather than assuming one", () => {
expect(formSource).toMatch(
/const glucoseUnit = resolveGlucoseUnit\(user\?\.glucoseUnit\)/,
);
});
});
@@ -0,0 +1,117 @@
import { readFileSync } from "node:fs";
import { join } from "node:path";
import { describe, expect, it, vi } from "vitest";
import { renderToStaticMarkup } from "react-dom/server";
/**
* The measurements list and the blood-glucose display unit.
*
* Glucose is stored in mg/dL and rendered in the unit the account chose.
* The list is not part of the metric/imperial transform registry, so
* without a branch of its own a reader on mmol/L would see the dashboard
* tile say 5.3 and this list say 95 for the same reading.
*
* The edit sheet is the half that could do damage: its label follows the
* same resolution, so the number in the input is in the displayed unit and
* has to be inverted before the PATCH. Display and inversion are asserted
* together on purpose — converting one without the other rewrites a
* 95 mg/dL reading as 5.3 the first time somebody corrects a typo.
*/
const rows = [
{
id: "m1",
type: "BLOOD_GLUCOSE",
value: 95,
unit: "mg/dL",
source: "MANUAL",
measuredAt: "2026-05-15T10:00:00.000Z",
notes: null,
},
];
let glucoseUnit: string | null = null;
vi.mock("@tanstack/react-query", () => ({
useQuery: () => ({
data: { measurements: rows, meta: { total: 1 } },
isLoading: false,
}),
useQueryClient: () => ({ invalidateQueries: vi.fn() }),
useMutation: () => ({
mutate: vi.fn(),
mutateAsync: vi.fn(),
isPending: false,
isError: false,
error: null,
}),
}));
vi.mock("next/navigation", () => ({
useRouter: () => ({ push: vi.fn(), replace: vi.fn() }),
usePathname: () => "/measurements",
useSearchParams: () => new URLSearchParams(""),
}));
vi.mock("@/hooks/use-auth", () => ({
useAuth: () => ({
user: { id: "u1", username: "testuser", role: "USER", glucoseUnit },
isAuthenticated: true,
isLoading: false,
}),
}));
import { I18nProvider } from "@/lib/i18n/context";
import { MeasurementList } from "../measurement-list";
const listSource = readFileSync(
join(process.cwd(), "src/components/measurements/measurement-list.tsx"),
"utf8",
);
function render() {
return renderToStaticMarkup(
<I18nProvider initialLocale="en">
<MeasurementList />
</I18nProvider>,
);
}
describe("MeasurementList — blood-glucose display unit", () => {
it("shows the stored mg/dL reading as it stands for an mg/dL account", () => {
glucoseUnit = null;
const html = render();
expect(html).toContain("95");
expect(html).toContain("mg/dL");
expect(html).not.toContain("mmol/L");
});
it("converts the same reading for an mmol/L account", () => {
glucoseUnit = "mmol/L";
const html = render();
// 95 mg/dL ÷ 18.0182 = 5.3 mmol/L.
expect(html).toContain("5.3");
expect(html).toContain("mmol/L");
expect(html).not.toContain("95 mg/dL");
});
it("inverts an edited glucose value back to mg/dL before the save", () => {
expect(listSource).toMatch(
/editsConvertedGlucose\(editing\)[\s\S]{0,400}?canonicalValue = toCanonicalMgdl\(parsedValue, "mmol\/L"\)/,
);
});
it("seeds the edit input in the unit its own label names", () => {
expect(listSource).toMatch(
/const seedValue = editsConvertedGlucose\(measurement\)\s*\?\s*rd\.value/,
);
});
it("leaves an mg/dL edit on the identity path so a note fix cannot round the value", () => {
// `toCanonicalMgdl(v, "mg/dL")` rounds, so routing the mg/dL branch
// through it would quantise a stored 95.48 on an unrelated edit.
expect(listSource).toMatch(
/editsConvertedGlucose[\s\S]{0,300}?glucoseUnit === "mmol\/L"/,
);
});
});
@@ -26,6 +26,8 @@ import { Loader2, MoreHorizontal, Plus, RotateCcw } from "lucide-react";
import { toast } from "sonner";
import { useTranslations } from "@/lib/i18n/context";
import { useUnitDisplay } from "@/hooks/use-unit-display";
import { useAuth } from "@/hooks/use-auth";
import { resolveGlucoseUnit, toCanonicalMgdl } from "@/lib/glucose";
import {
entryValueToCanonical,
parseDecimalEntry,
@@ -246,6 +248,14 @@ export function MeasurementForm({
const recordName = useActiveRecordName();
const queryClient = useQueryClient();
const unitDisplay = useUnitDisplay();
const { user } = useAuth();
// Glucose is its own display axis, separate from metric/imperial: it is
// stored in mg/dL and rendered in whichever of mg/dL and mmol/L the
// account chose. The form has to ask in the same unit it labels, and
// invert on the way out, or a reader on mmol/L would file "5.5" as five
// and a half mg/dL — a number the plausibility band happily accepts and
// every surface then reads back as a severe hypo.
const glucoseUnit = resolveGlucoseUnit(user?.glucoseUnit);
// Normalize legacy BP types to combined mode
const normalizedDefault =
@@ -359,7 +369,9 @@ export function MeasurementForm({
return;
}
let canonicalValue = typed;
if (unitDisplay.isTransformed(type)) {
if (isGlucoseMode) {
canonicalValue = toCanonicalMgdl(typed, glucoseUnit);
} else if (unitDisplay.isTransformed(type)) {
const inverted = unitDisplay.fromDisplay(type, typed);
canonicalValue =
unitDisplay.preference === "imperial"
@@ -537,13 +549,15 @@ export function MeasurementForm({
// For a type with a metric/imperial transform the label follows the
// user's preference (kg↔lb, cm↔in, °C↔°F); everything else keeps
// its static catalogue unit.
unit: typeInfo
? unitDisplay.isTransformed(type)
? unitDisplay.unitFor(type)
: "unitKey" in typeInfo
? t(typeInfo.unitKey)
: typeInfo.unit
: "",
unit: isGlucoseMode
? glucoseUnit
: typeInfo
? unitDisplay.isTransformed(type)
? unitDisplay.unitFor(type)
: "unitKey" in typeInfo
? t(typeInfo.unitKey)
: typeInfo.unit
: "",
})}
>
<Input
@@ -558,14 +572,16 @@ export function MeasurementForm({
value={value}
onChange={(e) => setValue(e.target.value)}
placeholder={
typeInfo
? unitDisplay.preference === "imperial" &&
"placeholderImperial" in typeInfo
? typeInfo.placeholderImperial
: "placeholder" in typeInfo
? typeInfo.placeholder
: undefined
: undefined
isGlucoseMode && glucoseUnit === "mmol/L"
? "5.3"
: typeInfo
? unitDisplay.preference === "imperial" &&
"placeholderImperial" in typeInfo
? typeInfo.placeholderImperial
: "placeholder" in typeInfo
? typeInfo.placeholder
: undefined
: undefined
}
required
aria-required="true"
@@ -72,6 +72,11 @@ import { CUMULATIVE_DAY_SUM_TYPES } from "@/lib/measurements/cumulative-day-sum"
import { rawDisplayFractionDigits } from "@/lib/measurements/display-transform";
import { getUnitForType } from "@/lib/validations/measurement";
import { useUnitDisplay } from "@/hooks/use-unit-display";
import {
convertGlucose,
resolveGlucoseUnit,
toCanonicalMgdl,
} from "@/lib/glucose";
import {
invalidateKeys,
measurementDependentKeys,
@@ -303,17 +308,40 @@ export function MeasurementList({
// transform; a legacy non-canonical row (an old MCP write whose `unit` ≠ the
// type's canonical unit) is shown verbatim so its stored unit is never
// silently relabelled. Metric users take the identity path — unchanged.
// Glucose sits on its own display axis: stored mg/dL, rendered in whichever
// of mg/dL and mmol/L the account chose. It is not part of the
// metric/imperial registry, so it needs its own branch or a reader on
// mmol/L would see the dashboard tile and this list disagree about the
// same reading.
const glucoseUnit = resolveGlucoseUnit(user?.glucoseUnit);
// An edit whose input is NOT in the stored unit. Only the mmol/L branch
// qualifies: an mg/dL account edits in the unit the column holds, and
// routing it through the converter anyway would quantise a stored 95.48 to
// 95 the moment somebody opened the sheet to fix a note.
const editsConvertedGlucose = useCallback(
(m: { type: string; unit: string }) =>
m.type === "BLOOD_GLUCOSE" &&
glucoseUnit === "mmol/L" &&
m.unit === getUnitForType(m.type),
[glucoseUnit],
);
const rowDisplay = useCallback(
(m: { type: string; value: number; unit: string }) => {
if (m.unit !== getUnitForType(m.type)) {
return { value: m.value, unit: m.unit };
}
if (m.type === "BLOOD_GLUCOSE") {
return {
value: convertGlucose(m.value, glucoseUnit),
unit: glucoseUnit,
};
}
return {
value: unitDisplay.toDisplay(m.type, m.value),
unit: unitDisplay.unitFor(m.type),
};
},
[unitDisplay],
[unitDisplay, glucoseUnit],
);
// v1.28.42 (H3) — the desktop table and the mobile card list used to BOTH
// render (only CSS `display` hid one), so a dense cumulative (up to ~5000
@@ -791,10 +819,11 @@ export function MeasurementList({
// stored value, unchanged. Round the converted imperial seed so the input
// shows "210" rather than a long float.
const rd = rowDisplay(measurement);
const seedValue =
unitDisplay.preference === "imperial" &&
unitDisplay.isTransformed(measurement.type) &&
measurement.unit === getUnitForType(measurement.type)
const seedValue = editsConvertedGlucose(measurement)
? rd.value
: unitDisplay.preference === "imperial" &&
unitDisplay.isTransformed(measurement.type) &&
measurement.unit === getUnitForType(measurement.type)
? Math.round(rd.value * 100) / 100
: measurement.value;
const seed = {
@@ -852,7 +881,11 @@ export function MeasurementList({
// the PATCH so an imperial edit persists in the stored unit. Metric users
// and legacy non-canonical rows are the identity path — value unchanged.
let canonicalValue = parsedValue;
if (
if (editsConvertedGlucose(editing)) {
// The label above the input said mmol/L, so the number in it is
// mmol/L. Sending it on would rewrite a 95 mg/dL reading as 5.3.
canonicalValue = toCanonicalMgdl(parsedValue, "mmol/L");
} else if (
unitDisplay.isTransformed(editing.type) &&
editing.unit === getUnitForType(editing.type)
) {
@@ -0,0 +1,145 @@
import { readFileSync } from "node:fs";
import { join } from "node:path";
import { beforeEach, describe, expect, it, vi } from "vitest";
import { renderToStaticMarkup } from "react-dom/server";
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
import { I18nProvider } from "@/lib/i18n/context";
import type { AuthUser } from "@/hooks/use-auth";
/**
* Settings → Profile blood-glucose unit dropdown.
*
* The control the preference column never had. It sits beside the
* metric/imperial select and is deliberately not a branch of it: metric
* countries are split on which unit they read glucose in, so one dropdown
* would get one of them wrong.
*
* SSR-only, like the unit-system select next to it: the render pass pins the
* options and the selected value, and the mutation contract — endpoint,
* method, body — is asserted against the source, because nothing here can
* dispatch a change event.
*/
const authSpy = vi.fn<
() => { user: AuthUser | null; isAuthenticated: boolean }
>(() => ({ user: buildUser(null), isAuthenticated: true }));
vi.mock("@/hooks/use-auth", async () => {
const actual =
await vi.importActual<typeof import("@/hooks/use-auth")>(
"@/hooks/use-auth",
);
return { ...actual, useAuth: () => authSpy() };
});
function buildUser(glucoseUnit: string | null): AuthUser {
return {
id: "user-1",
username: "user",
email: null,
role: "USER",
heightCm: null,
dateOfBirth: null,
gender: null,
timezone: "Europe/Berlin",
onboardingCompletedAt: null,
onboardingTourCompleted: true,
onboardingTourProgress: null,
avatarUrl: null,
glucoseUnit,
unitPreference: "metric",
timeFormat: "AUTO",
dateFormat: "AUTO",
disableCoach: false,
fullName: null,
insurerName: null,
insurerIkNumber: null,
insuranceNumber: null,
lastReportPracticeName: null,
reportSelection: null,
cycleTrackingEnabled: false,
modules: {},
} as AuthUser;
}
import { GlucoseUnitSelect } from "../glucose-unit-select";
const componentSource = readFileSync(
join(process.cwd(), "src/components/settings/glucose-unit-select.tsx"),
"utf8",
);
beforeEach(() => {
authSpy.mockClear();
authSpy.mockImplementation(() => ({
user: buildUser(null),
isAuthenticated: true,
}));
});
function render(isAuthenticated = true): string {
const client = new QueryClient({
defaultOptions: { queries: { retry: 0 } },
});
return renderToStaticMarkup(
<QueryClientProvider client={client}>
<I18nProvider initialLocale="en">
<GlucoseUnitSelect isAuthenticated={isAuthenticated} />
</I18nProvider>
</QueryClientProvider>,
);
}
describe("Settings — GlucoseUnitSelect", () => {
it("offers both units the app renders", () => {
const html = render();
expect(html).toContain('data-testid="settings-glucose-unit-select"');
expect(html).toContain('value="mg/dL"');
expect(html).toContain('value="mmol/L"');
});
// React renders a controlled select's current value as `selected` on the
// option under SSR, not as an attribute on the `<select>`.
function selectedOption(html: string): string | null {
return /<option[^>]*selected[^>]*>/.exec(html)?.[0] ?? null;
}
it("shows mg/dL for an account that never chose", () => {
expect(selectedOption(render())).toContain('value="mg/dL"');
});
it("shows the unit an account already chose", () => {
authSpy.mockImplementation(() => ({
user: buildUser("mmol/L"),
isAuthenticated: true,
}));
expect(selectedOption(render())).toContain('value="mmol/L"');
});
it("disables the select when unauthenticated", () => {
const select = /<select[^>]*settings-glucose-unit-select[^>]*>/.exec(
render(false),
);
expect(select).not.toBeNull();
expect(select![0]).toContain("disabled");
});
it("PATCHes the glucose-unit endpoint field-by-field", () => {
expect(componentSource).toMatch(
/apiFetchRaw\(\s*"\/api\/auth\/me\/glucose-unit"[\s\S]{0,200}?method: "PATCH"[\s\S]{0,200}?JSON\.stringify\(\{ glucoseUnit: next \}\)/,
);
});
it("refreshes the account payload every glucose surface reads from", () => {
expect(componentSource).toMatch(/queryKey: queryKeys\.authMe\(\)/);
});
it("is actually mounted on the settings page", () => {
// A control nobody can reach is the same defect as no control.
const accountSection = readFileSync(
join(process.cwd(), "src/components/settings/account-section/index.tsx"),
"utf8",
);
expect(accountSection).toMatch(/<GlucoseUnitSelect\s/);
});
});
@@ -32,6 +32,7 @@ import { storeTimezone } from "@/lib/timezone-mirror";
import { TimeFormatSelect } from "@/components/settings/time-format-select";
import { DateFormatSelect } from "@/components/settings/date-format-select";
import { UnitPreferenceSelect } from "@/components/settings/unit-preference-select";
import { GlucoseUnitSelect } from "@/components/settings/glucose-unit-select";
import { detectBrowserTimezone } from "@/lib/tz/format";
import { apiFetchRaw } from "@/lib/api/api-fetch";
import {
@@ -421,14 +422,17 @@ export function AccountSection() {
</div>
</div>
{/* Timezone + unit system + hour format share one grid block —
all personal display preferences (like language above). The
unit and time-format dropdowns PATCH their own endpoints on
change; the timezone saves through the form's submit
handler. */}
{/* Timezone, unit system, glucose unit and the date/hour formats
share one grid block — all personal display preferences (like
language above). The glucose unit is its own dropdown rather
than a branch of the unit system because metric countries are
split on which unit they read glucose in. Every dropdown here
PATCHes its own endpoint on change; the timezone saves through
the form's submit handler. */}
<div className="grid gap-4 sm:grid-cols-2">
<TimezonePicker value={timezone} onChange={setTimezone} />
<UnitPreferenceSelect isAuthenticated={isAuthenticated} />
<GlucoseUnitSelect isAuthenticated={isAuthenticated} />
<TimeFormatSelect isAuthenticated={isAuthenticated} />
<DateFormatSelect isAuthenticated={isAuthenticated} />
</div>
@@ -0,0 +1,150 @@
"use client";
import { useEffect, useRef, useState } from "react";
import { useMutation, useQueryClient } from "@tanstack/react-query";
import { Label } from "@/components/ui/label";
import { NativeSelect } from "@/components/ui/native-select";
import { useAuth } from "@/hooks/use-auth";
import { useTranslations } from "@/lib/i18n/context";
import { queryKeys } from "@/lib/query-keys";
import { apiFetchRaw } from "@/lib/api/api-fetch";
import { resolveGlucoseUnit, type GlucoseUnit } from "@/lib/glucose";
/**
* Blood-glucose display unit as a Profile dropdown.
*
* The preference column and its thirty-odd readers shipped in v1.2 with no
* control to set them, so every account stayed on the mg/dL default and the
* mmol/L half of the app was unreachable. This is the control.
*
* It sits beside the metric/imperial dropdown because it is the same kind of
* setting — how a number is shown, not what is stored — but it is a separate
* choice: metric countries are split on which unit they read glucose in, so
* folding it into metric/imperial would get one of them wrong. The option
* labels are the unit symbols themselves, which read the same in every
* locale.
*
* Persistence mirrors the unit-system select: a change PATCHes
* `/api/auth/me/glucose-unit` immediately and invalidates `queryKeys.authMe()`
* plus `queryKeys.userGlucoseUnit()`, so the dashboard tiles, the glucose
* insight page and the targets panel re-render on the next /me refetch.
*/
export function GlucoseUnitSelect({
isAuthenticated,
id = "glucose-unit",
}: {
isAuthenticated: boolean;
id?: string;
}) {
const { t } = useTranslations();
const queryClient = useQueryClient();
const { user } = useAuth();
const [optimistic, setOptimistic] = useState<GlucoseUnit | null>(null);
const [msg, setMsg] = useState<string | null>(null);
const [msgType, setMsgType] = useState<"success" | "error" | null>(null);
const clearTimerRef = useRef<ReturnType<typeof setTimeout> | null>(null);
useEffect(() => {
return () => {
if (clearTimerRef.current !== null) {
clearTimeout(clearTimerRef.current);
clearTimerRef.current = null;
}
};
}, []);
function scheduleClear() {
if (clearTimerRef.current !== null) {
clearTimeout(clearTimerRef.current);
}
clearTimerRef.current = setTimeout(() => {
clearTimerRef.current = null;
setMsg(null);
setMsgType(null);
}, 3000);
}
const value: GlucoseUnit =
optimistic ?? resolveGlucoseUnit(user?.glucoseUnit);
const mutation = useMutation({
mutationFn: async (next: GlucoseUnit) => {
const res = await apiFetchRaw("/api/auth/me/glucose-unit", {
method: "PATCH",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ glucoseUnit: next }),
});
if (!res.ok) {
const json = (await res.json().catch(() => ({}))) as {
error?: string;
};
throw new Error(json.error ?? `HTTP ${res.status}`);
}
return next;
},
onSuccess: () => {
setMsg(t("settings.dashboard.glucoseUnit.saved"));
setMsgType("success");
queryClient.invalidateQueries({ queryKey: queryKeys.authMe() });
queryClient.invalidateQueries({
queryKey: queryKeys.userGlucoseUnit(),
});
setOptimistic(null);
scheduleClear();
},
onError: (err) => {
setOptimistic(null);
setMsg(
err instanceof Error
? err.message
: t("settings.dashboard.glucoseUnit.saveError"),
);
setMsgType("error");
scheduleClear();
},
});
function handleSelect(next: GlucoseUnit) {
if (next === value || mutation.isPending || !isAuthenticated) return;
setOptimistic(next);
setMsg(null);
setMsgType(null);
if (clearTimerRef.current !== null) {
clearTimeout(clearTimerRef.current);
clearTimerRef.current = null;
}
mutation.mutate(next);
}
return (
<div className="space-y-2">
<Label htmlFor={id}>{t("settings.dashboard.glucoseUnit.title")}</Label>
<NativeSelect
id={id}
data-testid="settings-glucose-unit-select"
value={value}
disabled={!isAuthenticated || mutation.isPending}
onChange={(e) => handleSelect(e.target.value as GlucoseUnit)}
>
{/* Unit symbols, not translated copy — they read the same
everywhere, and a locale that "translated" one would be
naming a different measurement. */}
<option value="mg/dL">mg/dL</option>
<option value="mmol/L">mmol/L</option>
</NativeSelect>
<p
role="status"
aria-live="polite"
className={
msgType === "error"
? "text-destructive text-xs"
: "text-muted-foreground text-xs"
}
>
{msg ?? t("settings.dashboard.glucoseUnit.description")}
</p>
</div>
);
}
@@ -0,0 +1,171 @@
/**
* `getSession` keeps `User.locale` in step with the language the browser is
* actually reading.
*
* The column is the only signal a cookie-blind path has — the nightly
* briefing warm, the Coach nudge, the reminder senders, the native dashboard
* aggregate. The screen in front of the user resolves from the
* `healthlog-locale` cookie first. When those two disagree nothing looks
* wrong until a job writes a sentence, and then one paragraph of an
* otherwise correct page comes back in the wrong language.
*
* Closing that used to be a fire-and-forget PUT from a mount effect, so a
* lost request left the divergence standing for the life of the account.
* These pin the server-side reconcile that replaced it as the mechanism.
*/
import { describe, it, expect, vi, beforeEach } from "vitest";
const cookieState = vi.hoisted(() => ({
sessionId: undefined as string | undefined,
locale: undefined as string | undefined,
set: vi.fn(),
delete: vi.fn(),
}));
const store = vi.hoisted(() => ({
row: null as Record<string, unknown> | null,
}));
vi.mock("@/lib/db", () => ({
prisma: {
session: {
findUnique: vi.fn(async () => store.row),
update: vi.fn(async () => ({})),
deleteMany: vi.fn(async () => ({})),
},
user: { update: vi.fn(async () => ({})) },
},
}));
vi.mock("@/lib/db-compat", () => ({
ensureDbCompatibility: vi.fn().mockResolvedValue(undefined),
}));
vi.mock("@/lib/logging/context", () => ({ getEvent: () => undefined }));
vi.mock("@/lib/auth/hmac", () => ({
hashToken: (raw: string) => `hash:${raw}`,
}));
vi.mock("next/headers", () => ({
cookies: vi.fn(async () => ({
get: (name: string) => {
if (name === "healthlog_session" && cookieState.sessionId) {
return { value: cookieState.sessionId };
}
if (name === "healthlog-locale" && cookieState.locale !== undefined) {
return { value: cookieState.locale };
}
return undefined;
},
set: cookieState.set,
delete: cookieState.delete,
})),
}));
import { prisma } from "@/lib/db";
import { getSession } from "../session";
const THIRTY_DAYS_MS = 30 * 24 * 60 * 60 * 1000;
function seedSession(userLocale: string | null) {
cookieState.sessionId = "hls_raw-secret";
store.row = {
id: "sess-1",
userId: "user-1",
tokenHash: "hash:hls_raw-secret",
// Far enough out that the sliding-expiry refresh stays quiet.
expiresAt: new Date(Date.now() + THIRTY_DAYS_MS),
lastActiveAt: new Date(),
actingAsUserId: null,
recordEpoch: 0,
user: { id: "user-1", locale: userLocale },
};
}
beforeEach(() => {
vi.clearAllMocks();
cookieState.locale = undefined;
store.row = null;
});
describe("getSession — locale column reconcile", () => {
it("writes the column when the browser reads a different language", async () => {
// The row the production instance actually held: a German reader whose
// column said English, so every generated sentence came back English.
seedSession("en");
cookieState.locale = "de";
const result = await getSession();
expect(prisma.user.update).toHaveBeenCalledWith({
where: { id: "user-1" },
data: { locale: "de" },
});
// The same request must not answer with the value just replaced.
expect(result?.user.locale).toBe("de");
});
it("backfills a column that was never written at all", async () => {
// The account that never opened the language picker: the cookie is the
// client's own mount-time write of the Accept-Language paint.
seedSession(null);
cookieState.locale = "de";
await getSession();
expect(prisma.user.update).toHaveBeenCalledWith({
where: { id: "user-1" },
data: { locale: "de" },
});
});
it("writes nothing once the two agree", async () => {
seedSession("de");
cookieState.locale = "de";
await getSession();
expect(prisma.user.update).not.toHaveBeenCalled();
});
it("ignores a cookie value that is not a shipped locale", async () => {
// The cookie is not HttpOnly, so anything can be in it. A value the
// resolver would later have to defend against never reaches the column.
seedSession("de");
cookieState.locale = "tlh";
const result = await getSession();
expect(prisma.user.update).not.toHaveBeenCalled();
expect(result?.user.locale).toBe("de");
});
it("re-emits the cookie instead of writing when the cookie is gone", async () => {
// Safari ITP expired the script-written copy; the column is the
// survivor, so it is the one that seeds the cookie, not the reverse.
seedSession("de");
cookieState.locale = undefined;
await getSession();
expect(prisma.user.update).not.toHaveBeenCalled();
expect(cookieState.set).toHaveBeenCalledWith(
"healthlog-locale",
"de",
expect.objectContaining({ httpOnly: false }),
);
});
it("does not block the request on the write", async () => {
// Fire-and-forget like the `lastActiveAt` stamp: a failing update must
// not turn locale drift into a failed session resolution.
seedSession("en");
cookieState.locale = "de";
vi.mocked(prisma.user.update).mockRejectedValueOnce(
new Error("db down") as never,
);
await expect(getSession()).resolves.not.toBeNull();
});
});
+55 -11
View File
@@ -325,26 +325,70 @@ export async function getSession(): Promise<{
.catch(() => {});
}
// Locale-cookie refresh: Safari's ITP expires the script-written
// `healthlog-locale` cookie after 7 days, so a returning user's first
// paint fell back to the browser language once a week. When the user
// has a persisted locale but the cookie is gone, re-emit it here —
// Set-Cookie is exempt from the ITP cap. Absent-only on purpose: a
// locale switch writes the cookie client-side before the PUT persists
// the column, and overwriting a *differing* cookie here would revert
// that fresh choice mid-flight. Fail-soft: in a server-component
// render the cookie store is read-only and `set` throws — the next
// route-handler request re-attempts.
const rawLocaleCookie = cookieStore.get(LOCALE_COOKIE)?.value;
const cookieLocale =
rawLocaleCookie && (locales as readonly string[]).includes(rawLocaleCookie)
? (rawLocaleCookie as Locale)
: null;
if (
session.user.locale &&
(locales as readonly string[]).includes(session.user.locale) &&
!cookieStore.get(LOCALE_COOKIE)
!cookieLocale
) {
// Locale-cookie refresh: Safari's ITP expires the script-written
// `healthlog-locale` cookie after 7 days, so a returning user's first
// paint fell back to the browser language once a week. When the user
// has a persisted locale but the cookie is gone, re-emit it here —
// Set-Cookie is exempt from the ITP cap. Fail-soft: in a
// server-component render the cookie store is read-only and `set`
// throws — the next route-handler request re-attempts.
try {
setLocaleCookie(cookieStore, session.user.locale as Locale);
} catch {
// Read-only cookie context — skip; nothing depends on this write.
}
} else if (cookieLocale && cookieLocale !== session.user.locale) {
// The column follows the language the browser is actually reading.
//
// `User.locale` is the ONLY signal every cookie-blind path has: the
// nightly briefing warm, the Coach nudge, the reminder senders and the
// native aggregate all resolve from the column alone, while the screen
// in front of the user resolves from this cookie first. The two can
// therefore disagree indefinitely, and the disagreement is invisible —
// it surfaces as one generated paragraph in the wrong language inside
// an otherwise correct page.
//
// Keeping them in step used to be the client's job: a fire-and-forget
// `PUT /api/auth/me/locale` from a mount effect, best-effort by
// construction. A 401 on the paint before sign-in, an offline blip, a
// tab closed mid-flight — every one of those loses the write silently,
// and the next mount repeats the same unreliable attempt rather than
// noticing the column is still wrong. So the column stayed on a
// language the user had left behind, for as long as the account existed.
//
// This arm closes it from the side that cannot miss. The cookie is only
// ever written alongside a committed UI locale (client `persistLocale`,
// or this function's own refresh above), so it is a statement about what
// the person is reading, not a guess: Accept-Language never reaches the
// column this way. Fire-and-forget like the `lastActiveAt` stamp, and
// self-limiting — once the two agree the condition is false and no
// further write is attempted.
//
// A second browser whose cookie still carries the older choice will move
// the column back. That is the same last-writer-wins the client PUT
// already had, and it is coherent: that browser is showing the older
// language too, because the cookie outranks the column on its first
// paint as well.
void prisma.user
.update({
where: { id: session.user.id },
data: { locale: cookieLocale },
})
.catch(() => {});
// Keep THIS request coherent — a handler resolving the locale from the
// returned row must not answer with the value we just replaced.
session.user.locale = cookieLocale;
}
return {
+136
View File
@@ -396,3 +396,139 @@ export function extractKeyId(encoded: string): string | null {
if (!/^[A-Za-z0-9_-]{1,32}$/.test(id)) return null;
return id;
}
// ─── Streaming codec ("stream1") ─────────────────────────────────────────────
//
// AES-256-GCM over a plaintext that is produced INCREMENTALLY and consumed as
// one base64 string. Written for the weekly backup blob, whose plaintext is
// hundreds of megabytes on a long-lived record: the string codec above has to
// hold the whole plaintext, the whole ciphertext AND the whole base64 at once,
// which is three full copies of a record that already does not fit.
//
// Layout: "~hlgcm1." + keyId + "." + base64(iv | ciphertext | authTag)
//
// Two deliberate differences from the string codec, and no others:
//
// - The tag rides at the END rather than in front of the ciphertext. GCM
// only produces it once the last block is in, so a leading tag is what
// makes the string codec un-streamable. Nothing about the authentication
// changes: the tag still covers every ciphertext byte and `decryptStream`
// still verifies it BEFORE returning a single plaintext byte.
// - The `~` prefix. Key ids match [A-Za-z0-9_-]{1,32} and a legacy payload is
// bare base64, so no value the string codec can ever write starts with a
// tilde. The two formats are disjoint by construction rather than by
// sniffing, which is why `decrypt()` below is left untouched.
//
// The write side never materialises the ciphertext: base64 is emitted in
// 3-byte-aligned pieces as the cipher produces them, with the sub-group
// remainder carried across chunks. The read side is deliberately NOT
// incremental — a caller that reads a backup parses it as one document
// anyway, and releasing unverified plaintext to save a copy would trade the
// authentication for memory.
/** Marker prefix of a streamed ciphertext. Disjoint from every other format. */
const STREAM_CODEC_PREFIX = "~hlgcm1.";
/** Incremental AES-256-GCM writer. Emits base64 pieces; concatenate in order. */
export interface StreamEncryptor {
/** The header, including base64(iv). Emit before any `update()` output. */
readonly header: string;
/** Encrypt one plaintext chunk into a base64 piece (may be empty). */
update(chunk: Buffer): string;
/** Flush the cipher and the auth tag. No further calls after this. */
final(): string;
}
/**
* Open a streaming encryptor under the ACTIVE key.
*
* The 12-byte IV is exactly four base64 groups, so it encodes standalone and
* the ciphertext continues on a group boundary — which is what lets the rest
* be emitted piecewise without ever holding the whole thing.
*/
export function createStreamEncryptor(): StreamEncryptor {
const { id, key } = getActiveKey();
const iv = randomBytes(IV_LENGTH);
const cipher = createCipheriv(ALGORITHM, key, iv);
// Bytes that did not fill a 3-byte base64 group in the previous chunk.
let carry = Buffer.alloc(0);
let done = false;
const emit = (bytes: Buffer): string => {
const buf = carry.byteLength > 0 ? Buffer.concat([carry, bytes]) : bytes;
const aligned = buf.byteLength - (buf.byteLength % 3);
carry = Buffer.from(buf.subarray(aligned));
return aligned === 0 ? "" : buf.subarray(0, aligned).toString("base64");
};
return {
header: `${STREAM_CODEC_PREFIX}${id}.${iv.toString("base64")}`,
update(chunk: Buffer): string {
if (done) throw new Error("Stream encryptor already finalised");
return emit(cipher.update(chunk));
},
final(): string {
if (done) throw new Error("Stream encryptor already finalised");
done = true;
// The tag is plaintext-independent trailing data, so it simply joins the
// byte stream; the last group is padded exactly once, here.
const tail = Buffer.concat([carry, cipher.final(), cipher.getAuthTag()]);
carry = Buffer.alloc(0);
return tail.toString("base64");
},
};
}
/** True when `stored` was written by `createStreamEncryptor`. */
export function isStreamCiphertext(stored: string): boolean {
return stored.startsWith(STREAM_CODEC_PREFIX);
}
/**
* Decrypt a streamed ciphertext back to its raw plaintext bytes.
*
* Fail-closed on every arm — a malformed header, an unconfigured key id, a
* truncated payload or a tag that does not verify all throw, and no plaintext
* is returned until `final()` has authenticated the whole ciphertext.
*/
export function decryptStream(stored: string): Buffer {
if (!isStreamCiphertext(stored)) {
throw new Error("Not a streamed ciphertext");
}
const rest = stored.slice(STREAM_CODEC_PREFIX.length);
const dot = rest.indexOf(".");
if (dot <= 0 || dot >= rest.length - 1) {
throw new Error("Streamed ciphertext has a malformed header");
}
const keyId = rest.slice(0, dot);
if (!/^[A-Za-z0-9_-]{1,32}$/.test(keyId)) {
throw new Error("Streamed ciphertext carries a malformed key id");
}
const key = getKeyById(keyId);
if (!key) {
throw new Error(
`Encryption key id '${keyId}' is not configured. Add it to ` +
`ENCRYPTION_KEYS before decrypting rows written under that key.`,
);
}
const packed = Buffer.from(rest.slice(dot + 1), "base64");
if (packed.byteLength < IV_LENGTH + AUTH_TAG_LENGTH) {
throw new Error("Streamed ciphertext is truncated");
}
const iv = packed.subarray(0, IV_LENGTH);
const tag = packed.subarray(packed.byteLength - AUTH_TAG_LENGTH);
const ct = packed.subarray(IV_LENGTH, packed.byteLength - AUTH_TAG_LENGTH);
const dec = createDecipheriv(ALGORITHM, key, iv);
dec.setAuthTag(tag);
return Buffer.concat([dec.update(ct), dec.final()]);
}
/** The key id a streamed ciphertext was written under, or null when unparsable. */
export function extractStreamKeyId(stored: string): string | null {
if (!isStreamCiphertext(stored)) return null;
const rest = stored.slice(STREAM_CODEC_PREFIX.length);
const dot = rest.indexOf(".");
if (dot <= 0) return null;
const keyId = rest.slice(0, dot);
return /^[A-Za-z0-9_-]{1,32}$/.test(keyId) ? keyId : null;
}
@@ -0,0 +1,461 @@
/**
* Derive the ciphertext-bearing columns FROM THE CODE, then assert the
* key-rotation registry covers every one of them.
*
* Why this exists next to `encrypted-columns.test.ts`. That guard scans
* `schema.prisma` for the `*Encrypted` suffix plus a hand-kept list of
* historically-named exceptions. The suffix is a naming convention, not a
* fact about the value, and the moment a column holding ciphertext is called
* something else the guard goes green while the column is unrotatable.
* `DataBackup.data` — the whole-account backup blob — sat outside rotation
* that way: encrypted on write, invisible to a name scan, and therefore
* reported as "zero rows remaining" by a run that never looked at it. The
* runbook then told the operator that zero meant the old key was safe to
* drop, which would have destroyed every backup in the deployment.
*
* How the derivation works, in three steps:
*
* 1. Start from the two primitives in `src/lib/crypto.ts` (`encrypt`,
* `encryptBytes`) and close over every function that RETURNS a value
* derived from one — `packBackupBlob`, `encryptNote`, the per-domain
* `encrypt*ToBytes` codecs, and so on. Taint propagates only through
* pure expressions; anything awaited or read back from Prisma is a
* round-trip, not a ciphertext this code produced.
* 2. Walk every `prisma|tx|db.<model>.<write>(...)` call, descend through
* the Prisma envelope keys (`data` / `create` / `update`, and relation
* writes resolved against the schema) into the payload objects, and
* record `<Model>.<field>` for each field whose value is a producer
* call or a variable derived from one.
* 3. Assert the result is a subset of the rotation registry.
*
* Known limit, stated so it is not mistaken for coverage: a payload assembled
* somewhere other than the call site (`data: updates` where `updates` is built
* field by field) resolves to nothing here. Those columns are covered by the
* name scan in the sibling guard; the two signals are deliberately
* independent, and a column has to escape BOTH to go unregistered. The scan
* asserts a non-zero, anchored match count, so a matcher that quietly stops
* matching fails instead of passing.
*/
import { describe, it, expect } from "vitest";
import { readFileSync, readdirSync, statSync } from "node:fs";
import { join } from "node:path";
import { ENCRYPTED_COLUMNS, encryptedColumnKey } from "../encrypted-columns";
const ROOT = join(__dirname, "../../../..");
const SRC = join(ROOT, "src");
const SCHEMA_PATH = join(ROOT, "prisma", "schema.prisma");
/** Prisma write methods whose argument carries a column payload. */
const WRITE_METHODS =
"create|createMany|createManyAndReturn|update|updateMany|upsert";
/** Prisma envelope keys whose value is a payload object, not a column. */
const ENVELOPE = new Set(["data", "create", "update"]);
/** Relation-write envelopes nested inside a payload value. */
const RELATION_ENVELOPE = new Set([
"create",
"update",
"upsert",
"createMany",
"connectOrCreate",
]);
/**
* Columns this scan derives that are deliberately NOT in the rotation
* registry. Empty today. An entry here is a written decision with a reason,
* never a way to quiet the guard.
*/
const NOT_ROTATED: ReadonlySet<string> = new Set<string>();
function sourceFiles(dir: string, out: string[] = []): string[] {
for (const entry of readdirSync(dir)) {
const p = join(dir, entry);
if (statSync(p).isDirectory()) {
if (entry === "generated" || entry === "__tests__") continue;
sourceFiles(p, out);
} else if (/\.tsx?$/.test(entry) && !/\.test\.tsx?$/.test(entry)) {
out.push(p);
}
}
return out;
}
/** Strip comments so prose about `encrypt()` never counts as a call. */
function stripComments(src: string): string {
return src
.replace(/\/\*[\s\S]*?\*\//g, "")
.replace(/(^|[^:"'`\\])\/\/.*$/gm, "$1");
}
const callRe = (names: string[]) =>
new RegExp(`(?<![.\\w])(?:${names.join("|")})\\s*\\(`);
/**
* Two rules for the same identifier, and the difference matters.
*
* Propagation is loose: a codec builds its result through `Buffer.from(ct)` and
* `new Uint8Array(encoded.byteLength)`, so a member read still carries the
* taint forward or the chain snaps one hop before the return.
*
* The final check at a payload field is strict — the identifier must be used
* WHOLE. `enc.expiresAt` off a tainted credentials object is a timestamp, not
* ciphertext, and counting it would file plaintext columns as encrypted.
*/
const tokenReLoose = (names: string[]) =>
new RegExp(`(?<![.\\w])(?:${names.join("|")})(?![\\w])`);
const tokenRe = (names: string[]) =>
new RegExp(`(?<![.\\w])(?:${names.join("|")})(?![\\w.])`);
/** Local identifiers whose value transitively derives from a producer call. */
function taintedIdentifiers(
body: string,
producers: string[],
loose: boolean,
): Set<string> {
const assigns = [
...body.matchAll(/(?:const|let|var)\s+(\w+)[^;=\n]*=\s*([^;]*)/g),
];
const tainted = new Set<string>();
for (let pass = 0; pass < 8; pass++) {
const before = tainted.size;
const producerCall = callRe(producers);
const strictToken = tainted.size ? tokenRe([...tainted]) : null;
const looseToken = tainted.size ? tokenReLoose([...tainted]) : null;
for (const [, name, init] of assigns) {
if (tainted.has(name)) continue;
if (producerCall.test(init)) {
tainted.add(name);
continue;
}
// Propagate through pure expressions only: a value awaited or read back
// from the database is a round-trip, not ciphertext this code made.
if (/\bawait\b|\b(?:prisma|tx|db)\./.test(init)) continue;
// A member read normally carries no taint (`creds.expiresAt` is a
// timestamp). The one exception is a buffer being CONSTRUCTED around a
// ciphertext — `new Uint8Array(encoded.byteLength)` then `.set(encoded)`
// is how every Bytes codec here builds its result, and reading the
// length off the ciphertext is the only link the initialiser shows.
const allowMemberRead = loose && /\bnew\s+[A-Z]|Buffer\./.test(init);
const token = allowMemberRead ? looseToken : strictToken;
if (token?.test(init)) tainted.add(name);
}
if (tainted.size === before) break;
}
return tainted;
}
interface Fn {
name: string;
body: string;
}
function declaredFunctions(src: string): Fn[] {
const out: Fn[] = [];
const heads = [
...src.matchAll(
/(?:^|\n)\s*(?:export\s+)?(?:async\s+)?function\s+(\w+)\s*[(<]/g,
),
];
for (let i = 0; i < heads.length; i++) {
const end = i + 1 < heads.length ? heads[i + 1].index! : src.length;
out.push({ name: heads[i][1], body: src.slice(heads[i].index!, end) });
}
for (const m of src.matchAll(
/(?:const|let)\s+(\w+)\s*(?::[^=\n]+)?=\s*(?:async\s*)?\([^)]*\)\s*(?::[^=\n]+)?=>/g,
)) {
// Bound the arrow at its own body. A fixed-size window instead would read
// the code AFTER the arrow as part of it, and a tiny id-resolving helper
// declared above a Prisma write would inherit that write's ciphertext.
const arrow = m.index! + m[0].length;
const brace = src.slice(arrow).search(/\S/) + arrow;
const end =
src[brace] === "{" ? balanced(src, brace) : endOfValue(src, brace);
out.push({ name: m[1], body: src.slice(m.index!, end + 1) });
}
return out;
}
/** Fixpoint: every function that returns something derived from `encrypt`. */
function ciphertextProducers(sources: string[]): Set<string> {
const funcs = sources.flatMap(declaredFunctions);
const producers = new Set(["encrypt", "encryptBytes"]);
for (let pass = 0; pass < 8; pass++) {
const before = producers.size;
for (const fn of funcs) {
if (producers.has(fn.name)) continue;
const names = [...producers];
const tainted = taintedIdentifiers(fn.body, names, true);
const producerCall = callRe(names);
const taintedToken = tainted.size ? tokenReLoose([...tainted]) : null;
const returnsCiphertext = [
...fn.body.matchAll(/\breturn\b([^;]*)/g),
].some((m) => producerCall.test(m[1]) || taintedToken?.test(m[1]));
if (returnsCiphertext) producers.add(fn.name);
}
if (producers.size === before) break;
}
return producers;
}
// ── brace / paren walking ────────────────────────────────────────────────
/** Index of the top-level comma (or closer) ending the value starting at `i`. */
function endOfValue(s: string, i: number): number {
let depth = 0;
while (i < s.length) {
const c = s[i];
if (c === '"' || c === "'" || c === "`") {
const quote = c;
i++;
while (i < s.length && s[i] !== quote) {
if (s[i] === "\\") i++;
i++;
}
i++;
continue;
}
if ("({[".includes(c)) depth++;
else if (")}]".includes(c)) {
if (depth === 0) return i;
depth--;
} else if (c === "," && depth === 0) return i;
i++;
}
return i;
}
/** Index of the closer matching the opener at `i`. */
function balanced(s: string, i: number): number {
const open = s[i];
const close = { "(": ")", "{": "}", "[": "]" }[open]!;
let depth = 0;
for (; i < s.length; i++) {
const c = s[i];
if (c === '"' || c === "'" || c === "`") {
const quote = c;
i++;
while (i < s.length && s[i] !== quote) {
if (s[i] === "\\") i++;
i++;
}
continue;
}
if (c === open) depth++;
else if (c === close && --depth === 0) return i;
}
return s.length - 1;
}
/** Top-level `key: value` pairs of an object literal (braces included). */
function objectPairs(obj: string): Array<[string, string, number]> {
const out: Array<[string, string, number]> = [];
let i = 1;
while (i < obj.length - 1) {
const m = /^[\s,]*(\w+|"[^"]*"|'[^']*')\s*:/.exec(obj.slice(i));
if (!m) {
const end = endOfValue(obj, i);
if (end <= i) break;
i = end + 1;
continue;
}
const start = i + m[0].length;
const end = endOfValue(obj, start);
out.push([m[1].replace(/^["']|["']$/g, ""), obj.slice(start, end), start]);
i = end + 1;
}
return out;
}
/** Object literals inside a payload value: `{…}`, `[{…}]`, or `map(() => ({…}))`. */
function payloadObjects(value: string): Array<[string, number]> {
const trimmed = value.trim();
if (trimmed.length === 0) return [];
const at = value.indexOf(trimmed[0]);
if (trimmed.startsWith("{")) {
return [[value.slice(at, balanced(value, at) + 1), at]];
}
if (trimmed.startsWith("[")) {
const arr = value.slice(at, balanced(value, at) + 1);
const out: Array<[string, number]> = [];
let i = 1;
while (i < arr.length - 1) {
while (i < arr.length && /[\s,]/.test(arr[i])) i++;
if (arr[i] === "{") {
const end = balanced(arr, i);
out.push([arr.slice(i, end + 1), at + i]);
i = end + 1;
} else {
const end = endOfValue(arr, i);
if (end <= i) break;
i = end + 1;
}
}
return out;
}
const out: Array<[string, number]> = [];
for (const m of value.matchAll(/=>\s*\(?\s*\{/g)) {
const brace = value.indexOf("{", m.index!);
out.push([value.slice(brace, balanced(value, brace) + 1), brace]);
}
return out;
}
// ── schema ───────────────────────────────────────────────────────────────
interface Schema {
models: Set<string>;
fieldType: Map<string, string>;
}
function parseSchema(): Schema {
const src = readFileSync(SCHEMA_PATH, "utf8");
const models = new Set<string>();
const fieldType = new Map<string, string>();
let model: string | null = null;
for (const raw of src.split("\n")) {
const line = raw.trim();
const head = /^model\s+(\w+)\s*\{/.exec(line);
if (head) {
model = head[1];
models.add(model);
continue;
}
if (line === "}") {
model = null;
continue;
}
if (!model || line === "" || line.startsWith("//")) continue;
const field = /^(\w+)\s+([A-Za-z]\w*)(\?|\[\])?/.exec(line);
if (field) fieldType.set(`${model}.${field[1]}`, field[2]);
}
return { models, fieldType };
}
// ── the scan ─────────────────────────────────────────────────────────────
function scanCiphertextWrites(): Map<string, Set<string>> {
const files = sourceFiles(SRC).map(
(f) => [f, stripComments(readFileSync(f, "utf8"))] as const,
);
const producers = [...ciphertextProducers(files.map(([, s]) => s))];
const schema = parseSchema();
const found = new Map<string, Set<string>>();
for (const [path, src] of files) {
// The rotation machinery re-encrypts what other code wrote; it is not a
// writer of new columns, and scanning it would fold its own generic
// `data: { [col.field]: … }` into the result.
if (path.includes("/lib/crypto/")) continue;
const tainted = taintedIdentifiers(src, producers, false);
const producerCall = callRe(producers);
const taintedToken = tainted.size ? tokenRe([...tainted]) : null;
const isCiphertext = (expr: string) =>
producerCall.test(expr) || Boolean(taintedToken?.test(expr));
const rel = path.slice(ROOT.length + 1);
const lineAt = (idx: number) => src.slice(0, idx).split("\n").length;
const record = (model: string, field: string, idx: number) => {
const key = `${model}.${field}`;
if (!found.has(key)) found.set(key, new Set());
found.get(key)!.add(`${rel}:${lineAt(idx)}`);
};
const descend = (
model: string,
value: string,
base: number,
depth = 0,
): void => {
if (depth > 4) return;
for (const [obj, objOffset] of payloadObjects(value)) {
for (const [field, expr, exprOffset] of objectPairs(obj)) {
const at = base + objOffset + exprOffset;
const type = schema.fieldType.get(`${model}.${field}`);
if (type && schema.models.has(type)) {
// Relation field: the nested payload belongs to the child model.
descend(type, expr, at, depth + 1);
continue;
}
if (!type) {
if (RELATION_ENVELOPE.has(field))
descend(model, expr, at, depth + 1);
continue;
}
if (isCiphertext(expr)) record(model, field, at);
}
}
};
const writes = new RegExp(
`\\b(?:prisma|tx|db)\\s*\\.\\s*(\\w+)\\s*\\.\\s*(?:${WRITE_METHODS})\\s*\\(`,
"g",
);
for (const m of src.matchAll(writes)) {
const model = m[1][0].toUpperCase() + m[1].slice(1);
if (!schema.models.has(model)) continue;
const paren = m.index! + m[0].length - 1;
const arg = src.slice(paren, balanced(src, paren) + 1);
const brace = arg.indexOf("{");
if (brace < 0) continue;
const obj = arg.slice(brace, balanced(arg, brace) + 1);
for (const [key, value, offset] of objectPairs(obj)) {
if (!ENVELOPE.has(key)) continue;
descend(model, value, paren + brace + offset);
}
}
}
return found;
}
describe("ciphertext columns derived from the code", () => {
const producers = ciphertextProducers(
sourceFiles(SRC).map((f) => stripComments(readFileSync(f, "utf8"))),
);
const written = scanCiphertextWrites();
it("closes over the encryption wrappers, not just the two primitives", () => {
// A closure that collapsed back to the seeds would make the write scan
// blind to every column written through a codec helper.
expect(producers.size).toBeGreaterThan(10);
for (const wrapper of [
"packBackupBlob",
"encryptNote",
"encryptToBytes",
"encryptCachedBody",
]) {
expect([...producers]).toContain(wrapper);
}
});
it("finds ciphertext writes, including the non-suffixed columns", () => {
// Anchored non-zero match count. An empty or collapsed result set is the
// failure mode this whole file exists to make impossible, so name the
// columns whose only signal is the write and not the column's name.
expect(written.size).toBeGreaterThan(30);
for (const anchor of [
"DataBackup.data",
"IdempotencyKey.responseBody",
"NotificationChannel.config",
"PushSubscription.p256dh",
"IntegrationStatus.lastError",
"WithingsConnection.accessToken",
]) {
expect(
[...written.keys()],
`${anchor} is written with ciphertext but the scan did not see it`,
).toContain(anchor);
}
});
it("registers every column the code writes ciphertext into", () => {
const registered = new Set(ENCRYPTED_COLUMNS.map(encryptedColumnKey));
const unregistered = [...written.keys()]
.filter((key) => !registered.has(key) && !NOT_ROTATED.has(key))
.sort()
.map((key) => `${key} (written at ${[...written.get(key)!][0]})`);
expect(
unregistered,
"columns written with AES-256-GCM ciphertext but absent from the " +
"key-rotation registry — dropping a retired key makes these rows " +
"permanently undecryptable",
).toEqual([]);
});
});
@@ -127,21 +127,29 @@ describe("encrypted-column registry", () => {
});
it("covers EVERY encrypted column in prisma/schema.prisma", () => {
const schemaEncrypted = parseSchemaColumns()
const schema = parseSchemaColumns();
const schemaEncrypted = schema
.filter(isEncryptedColumn)
.map((c) => `${c.model}.${c.field}`)
.sort();
const allSchemaColumns = new Set(
schema.map((c) => `${c.model}.${c.field}`),
);
const registered = ENCRYPTED_COLUMNS.map(encryptedColumnKey).sort();
// Any schema column the registry forgot is a rotation gap (data-loss
// risk on key drop). Any registry column missing from the schema is a
// stale entry. Both fail here.
//
// Staleness is checked against the WHOLE schema, not against the name
// scan: the registry legitimately carries columns this scan cannot see —
// `DataBackup.data` holds ciphertext under a name the convention does not
// reach — and the sibling guard `encrypted-column-writers.test.ts` is
// what finds those, by tracing the writers instead of the names.
const missingFromRegistry = schemaEncrypted.filter(
(k) => !registered.includes(k),
);
const staleInRegistry = registered.filter(
(k) => !schemaEncrypted.includes(k),
);
const staleInRegistry = registered.filter((k) => !allSchemaColumns.has(k));
expect(
missingFromRegistry,
"encrypted schema columns NOT wired into the rotation registry",
@@ -152,6 +160,42 @@ describe("encrypted-column registry", () => {
).toEqual([]);
});
it("declares the primary key for every model not keyed on `id`", () => {
// The rotation walk selects, orders by and addresses rows through this
// field. Get it wrong and Prisma rejects the select at runtime, which
// abandons the rest of the pass — a rotation that stops early looks the
// same from the outside as one that had nothing to do.
const src = readFileSync(SCHEMA_PATH, "utf8");
const primaryKey = new Map<string, string>();
let model: string | null = null;
for (const rawLine of src.split("\n")) {
const line = rawLine.trim();
const head = /^model\s+(\w+)\s*\{/.exec(line);
if (head) {
model = head[1];
continue;
}
if (line === "}") {
model = null;
continue;
}
if (!model) continue;
const field = /^(\w+)\s+[A-Za-z]\w*\??\s.*@id\b/.exec(line);
if (field) primaryKey.set(model, field[1]);
}
const wrong = (ENCRYPTED_COLUMNS as EncryptedColumn[])
.filter((c) => (c.pkField ?? "id") !== primaryKey.get(c.model))
.map(
(c) =>
`${encryptedColumnKey(c)} (schema key: ${primaryKey.get(c.model) ?? "none"})`,
);
expect(
wrong,
"registry entries whose pkField does not match the model's @id column",
).toEqual([]);
});
it("is fully referenced by the key-rotation script", () => {
const script = readFileSync(ROTATION_SCRIPT_PATH, "utf8");
// Field names are NOT unique across models (e.g. `noteEncrypted` lives on
@@ -45,9 +45,17 @@ function makeClient(seed: Record<string, Row[]>): {
if (client[key]) continue;
const model = col.model;
client[key] = {
findMany: async ({ select }) => {
// `take` / `cursor` are honoured so the batched-column walk terminates
// here the way it does against Postgres.
findMany: async ({ select, take, cursor, skip }) => {
const fields = Object.keys(select);
return store[model].map((row) => {
let rows = store[model];
if (cursor) {
const at = rows.findIndex((r) => r.id === cursor.id);
rows = at < 0 ? [] : rows.slice(at + (skip ?? 0));
}
if (take !== undefined) rows = rows.slice(0, take);
return rows.map((row) => {
const out: Record<string, unknown> = {};
for (const f of fields) out[f] = row[f];
return out;
@@ -58,6 +66,10 @@ function makeClient(seed: Record<string, Row[]>): {
if (row) Object.assign(row, data);
return row;
},
delete: async ({ where }) => {
const at = store[model].findIndex((r) => r.id === where.id);
return at < 0 ? null : store[model].splice(at, 1)[0];
},
};
}
return { client, store };
+69 -1
View File
@@ -24,6 +24,17 @@
* reads/writes the value directly.
* - "bytes" — the ciphertext is stored as a `Bytes` column; the script
* goes through a UTF-8 Buffer round-trip (matching the persistence layer).
*
* The `*Encrypted` suffix is NOT what makes a column belong here. It is a
* naming convention that happens to hold for most of them, and treating it as
* the definition is how `DataBackup.data` — the whole-account backup blob —
* went unregistered until v1.38.6: encrypted on write, invisible to
* a registry keyed on names, and therefore reported as "zero rows remaining"
* by a rotation that never looked at it. An operator following the runbook
* would then have dropped the old key and lost every backup. The guard test
* `encrypted-column-writers.test.ts` now derives the ciphertext-bearing set
* from the Prisma write payloads instead, so a column earns its place here by
* what the code stores in it, not by what it is called.
*/
export type EncryptedColumnKind = "string" | "bytes";
@@ -43,6 +54,30 @@ export interface EncryptedColumn {
* multi-megabyte blobs, so an unbounded `findMany` would balloon memory.
*/
readonly codecField?: string;
/**
* Walk this column in bounded id-cursor batches instead of one `findMany`.
* Set it on any column whose rows are blobs rather than short strings — a
* whole-account backup compresses to megabytes, and pulling every row of
* them into one array is how the weekly backup pass died before
* `packBackupBlob` existed. `codecField` implies batching.
*/
readonly batched?: boolean;
/**
* The column is a cache, not a record. Rotation still re-encrypts it, but a
* row it cannot read is DELETED rather than counted as an error: the value
* is reproducible, and leaving an undecryptable row behind would hand the
* next reader a body it cannot parse. Never set this on anything a user
* would notice the loss of.
*/
readonly disposable?: boolean;
/**
* The model's primary-key field, when it is not `id`. The rotation walk
* selects it, orders by it and addresses each row through it, so a model
* keyed on something else (`MoodContext.moodEntryId`) crashes the whole run
* without this — Prisma rejects the unknown `id` in the select, and every
* column after it in the pass is left un-rotated.
*/
readonly pkField?: string;
}
/**
@@ -226,7 +261,12 @@ export const ENCRYPTED_COLUMNS: readonly EncryptedColumn[] = [
// v1.38 — the one free-text note on a mood entry's day context. Same codec
// as the entry note beside it; one column for the whole context rather than
// one per section, so there is one rotation entry rather than four.
{ model: "MoodContext", field: "notesEncrypted", kind: "bytes" },
{
model: "MoodContext",
field: "notesEncrypted",
kind: "bytes",
pkField: "moodEntryId",
},
// ───── v1.25 medication free-text notes (Bytes columns) ─────
// Side-effect log note, dose-change titration note, and inventory-item
@@ -318,6 +358,34 @@ export const ENCRYPTED_COLUMNS: readonly EncryptedColumn[] = [
// codec `DocumentContentIndex.textEncrypted` uses. A scanned medical preview
// is PHI; rotation re-encrypts it on the standard Bytes walk.
{ model: "DocumentThumbnail", field: "thumbnailEncrypted", kind: "bytes" },
// ───── Whole-account backup blob (String, batched) ─────
// The weekly disaster-recovery snapshot and every manually uploaded pack.
// Named `data`, so the `*Encrypted` scan never saw it; the value is
// `packBackupBlob()` output, which is AES-256-GCM ciphertext like every
// other entry here. It is also the single worst column to miss: a backup
// that cannot be decrypted is the copy an operator reaches for precisely
// when nothing else works.
//
// Rotation re-encrypts the ciphertext WITHOUT touching the plaintext, so it
// is blind to the envelope inside: today a row is either the plain backup
// JSON or the `HLZ1:`-prefixed gzip form, and any further shape a future
// writer introduces rotates unchanged for the same reason. Batched, because
// a single row is megabytes.
{ model: "DataBackup", field: "data", kind: "string", batched: true },
// ───── Idempotent-replay response cache (String, disposable) ─────
// The cached response body, encrypted because the PHI-returning creates echo
// their own decrypted DTO. Rows live 24 hours. It is registered so a key
// drop cannot leave a reader holding an undecryptable body, and marked
// disposable so an unreadable row is dropped instead of failing the run —
// the cache is an accelerator, and a miss only costs a re-run.
{
model: "IdempotencyKey",
field: "responseBody",
kind: "string",
disposable: true,
},
] as const;
/** Stable `Model.field` key for a registry entry. */
+108 -40
View File
@@ -47,9 +47,10 @@ import {
export const LEGACY_BUCKET = "legacy";
/**
* Batch size for codec-dispatched blob columns (`codecField` present). The
* document vault's rows are up to cap-sized ciphertexts, so the walk is
* id-cursor paginated — at most this many blobs are in memory at once.
* Batch size for blob columns (`codecField` or `batched` set). The document
* vault's rows are up to cap-sized ciphertexts and a backup row is the whole
* account compressed, so the walk is id-cursor paginated — at most this many
* blobs are in memory at once.
*/
export const BLOB_ROTATION_BATCH_SIZE = 25;
@@ -63,9 +64,11 @@ interface ColumnDelegate {
skip?: number;
}) => Promise<Array<Record<string, unknown>>>;
update: (args: {
where: { id: string };
where: Record<string, string>;
data: Record<string, unknown>;
}) => Promise<unknown>;
/** Only ever called for a `disposable` column's unreadable rows. */
delete: (args: { where: Record<string, string> }) => Promise<unknown>;
}
/** The subset of the Prisma client we touch: one delegate per model. */
@@ -149,44 +152,75 @@ function reencryptBlob(value: Uint8Array, codec: string): Uint8Array {
}
/**
* Walk a codec-dispatched blob column in bounded id-cursor batches, invoking
* `onRow` per non-empty row. At most `BLOB_ROTATION_BATCH_SIZE` blobs are in
* memory per step; an interrupted run resumes safely on re-invocation because
* processing is idempotent (already-active rows are skipped by the callers).
* Walk a blob column in bounded id-cursor batches, invoking `onRow` per
* non-empty row. At most `BLOB_ROTATION_BATCH_SIZE` rows are in memory per
* step; an interrupted run resumes safely on re-invocation because processing
* is idempotent (already-active rows are skipped by the callers).
*
* `codec` is the row's own codec label for a codec-dispatched column and null
* for a plain `batched` one — a backup blob has one layout, just a large one.
*/
async function walkBlobColumn(
delegate: ColumnDelegate,
col: EncryptedColumn,
onRow: (row: {
id: string;
value: Uint8Array;
codec: string;
value: unknown;
codec: string | null;
}) => Promise<void> | void,
): Promise<void> {
const codecField = col.codecField!;
const codecField = col.codecField;
const pk = pkField(col);
let cursor: string | null = null;
for (;;) {
const rows = await delegate.findMany({
select: { id: true, [col.field]: true, [codecField]: true },
orderBy: { id: "asc" },
select: {
[pk]: true,
[col.field]: true,
...(codecField ? { [codecField]: true } : {}),
},
orderBy: { [pk]: "asc" },
take: BLOB_ROTATION_BATCH_SIZE,
...(cursor ? { cursor: { id: cursor }, skip: 1 } : {}),
...(cursor ? { cursor: { [pk]: cursor }, skip: 1 } : {}),
});
if (rows.length === 0) break;
for (const row of rows) {
const value = row[col.field] as Uint8Array | null;
if (!value || value.byteLength === 0) continue;
const value = row[col.field];
if (value == null) continue;
if (value instanceof Uint8Array && value.byteLength === 0) continue;
if (typeof value === "string" && value.length === 0) continue;
await onRow({
id: row.id as string,
id: row[pk] as string,
value,
codec: String(row[codecField] ?? ""),
codec: codecField ? String(row[codecField] ?? "") : null,
});
}
cursor = rows[rows.length - 1]!.id as string;
cursor = rows[rows.length - 1]![pk] as string;
if (rows.length < BLOB_ROTATION_BATCH_SIZE) break;
}
}
/** The model's primary-key field: `id` unless the registry says otherwise. */
function pkField(col: EncryptedColumn): string {
return col.pkField ?? "id";
}
/** True when the column's rows must be walked in bounded batches. */
function isBatched(col: EncryptedColumn): boolean {
return Boolean(col.codecField ?? col.batched);
}
/** The key id a walked row was written under, dispatching on its codec. */
function walkedKeyId(
value: unknown,
codec: string | null,
kind: EncryptedColumn["kind"],
): string | null {
if (codec !== null) return blobKeyId(value as Uint8Array, codec);
const ciphertext = toCiphertext(value, kind);
return ciphertext == null ? null : extractKeyId(ciphertext);
}
export interface ColumnScan {
model: string;
field: string;
@@ -208,16 +242,16 @@ export async function scanColumn(
const byKeyId: Record<string, number> = {};
let total = 0;
if (col.codecField) {
// Codec-dispatched blob column: bounded batches, per-row codec.
if (isBatched(col)) {
// Blob column: bounded batches, per-row codec where the column has one.
await walkBlobColumn(delegate, col, ({ value, codec }) => {
total += 1;
const id = blobKeyId(value, codec) ?? LEGACY_BUCKET;
const id = walkedKeyId(value, codec, col.kind) ?? LEGACY_BUCKET;
byKeyId[id] = (byKeyId[id] ?? 0) + 1;
});
} else {
const rows = await delegate.findMany({
select: { id: true, [col.field]: true },
select: { [pkField(col)]: true, [col.field]: true },
});
for (const row of rows) {
const ciphertext = toCiphertext(row[col.field], col.kind);
@@ -283,6 +317,8 @@ export interface RotationResult {
scanned: number;
rotated: number;
errors: number;
/** Unreadable rows deleted from a `disposable` column. */
dropped: number;
}
/** Re-encrypt one column's stale rows to the active key. */
@@ -297,51 +333,78 @@ export async function rotateColumn(
scanned: 0,
rotated: 0,
errors: 0,
dropped: 0,
};
if (col.codecField) {
// Codec-dispatched blob column: bounded id-cursor batches (never an
// unbounded blob findMany), re-encrypted under each row's OWN codec.
// Idempotent — rows already on the active key are skipped, so an
// interrupted run resumes cleanly on the next invocation.
/**
* A row that could not be re-encrypted. FAIL-CLOSED by default: count it and
* leave it untouched rather than dropping data. A `disposable` column takes
* the other branch — the value is a reproducible cache entry, and keeping an
* unreadable one only guarantees the next reader gets a body it cannot
* parse, so the row goes instead.
*/
const onUnreadable = async (id: string): Promise<void> => {
if (!col.disposable) {
result.errors += 1;
return;
}
try {
await delegate.delete({ where: { [pkField(col)]: id } });
result.dropped += 1;
} catch {
result.errors += 1;
}
};
if (isBatched(col)) {
// Blob column: bounded id-cursor batches (never an unbounded findMany),
// re-encrypted under each row's OWN codec where it has one. Idempotent —
// rows already on the active key are skipped, so an interrupted run
// resumes cleanly on the next invocation.
await walkBlobColumn(delegate, col, async ({ id, value, codec }) => {
result.scanned += 1;
if (blobKeyId(value, codec) === getActiveKeyId()) return;
if (walkedKeyId(value, codec, col.kind) === getActiveKeyId()) return;
try {
// The plaintext is never inspected, only re-sealed, so every envelope
// a stored value can legitimately carry survives rotation untouched.
const next =
codec !== null
? reencryptBlob(value as Uint8Array, codec)
: fromCiphertext(
encrypt(decrypt(toCiphertext(value, col.kind)!)),
col.kind,
);
await delegate.update({
where: { id },
data: { [col.field]: reencryptBlob(value, codec) },
where: { [pkField(col)]: id },
data: { [col.field]: next },
});
result.rotated += 1;
} catch {
// FAIL-CLOSED: unknown codec or a no-longer-configured key throws;
// count it and leave the row untouched rather than dropping data.
result.errors += 1;
await onUnreadable(id);
}
});
return result;
}
const pk = pkField(col);
const rows = await delegate.findMany({
select: { id: true, [col.field]: true },
select: { [pk]: true, [col.field]: true },
});
result.scanned = rows.length;
for (const row of rows) {
const ciphertext = toCiphertext(row[col.field], col.kind);
if (ciphertext == null || !shouldRotate(ciphertext)) continue;
const id = row.id as string;
const id = row[pk] as string;
try {
// ACTIVE-KEY-ONLY: encrypt() always writes the active key id.
const reencrypted = encrypt(decrypt(ciphertext));
await delegate.update({
where: { id },
where: { [pk]: id },
data: { [col.field]: fromCiphertext(reencrypted, col.kind) },
});
result.rotated += 1;
} catch {
// FAIL-CLOSED: a row under a no-longer-configured key throws on decrypt;
// count it and leave the row untouched rather than dropping data.
result.errors += 1;
await onUnreadable(id);
}
}
return result;
@@ -353,6 +416,8 @@ export interface CorpusRotation {
totalScanned: number;
totalRotated: number;
totalErrors: number;
/** Unreadable rows deleted from `disposable` columns. */
totalDropped: number;
}
/** Re-encrypt the whole corpus to the active key. Idempotent + active-key-only. */
@@ -367,10 +432,12 @@ export async function rotateCorpus(
let totalScanned = 0;
let totalRotated = 0;
let totalErrors = 0;
let totalDropped = 0;
for (const r of results) {
totalScanned += r.scanned;
totalRotated += r.rotated;
totalErrors += r.errors;
totalDropped += r.dropped;
}
return {
activeKeyId,
@@ -378,5 +445,6 @@ export async function rotateCorpus(
totalScanned,
totalRotated,
totalErrors,
totalDropped,
};
}
+5 -3
View File
@@ -258,9 +258,11 @@ describe("buildDashboardSnapshot — envelope shape", () => {
expect(snap.user.timezone).toBe("Europe/Berlin");
expect(snap.user.heightCm).toBe(180);
expect(snap.user.gender).toBe("MALE");
expect(typeof snap.user.greetingHour).toBe("number");
expect(snap.user.greetingHour).toBeGreaterThanOrEqual(0);
expect(snap.user.greetingHour).toBeLessThan(24);
// Inverted 2026-09: the builder used to compute a wall-clock hour for
// the greeting that no client ever read, and that a stale-while-
// revalidate serve could carry an hour past its bucket. The zone is
// the resolved value; the hour belongs to whoever holds a live clock.
expect(snap.user).not.toHaveProperty("greetingHour");
// tiles (fast phase) always present
expect(snap.tiles.summaries.WEIGHT.latest).toBe(80);
+8 -23
View File
@@ -212,6 +212,14 @@ export type BriefingState = "ready" | "preparing" | "disabled" | "no-provider";
export interface DashboardSnapshotUser {
username: string;
/**
* The user's configured zone — the ONE server-resolved input the
* greeting needs. The wall-clock hour itself is deliberately not on
* this payload: the body is served stale-while-revalidate for up to an
* hour, so a baked-in hour can name the wrong salutation long after
* the boundary passed, and every client already reads a live clock.
* Resolve the zone here, read the hour there.
*/
timezone: string;
heightCm: number | null;
dateOfBirth: string | null;
@@ -223,12 +231,6 @@ export interface DashboardSnapshotUser {
gender: ProfileSex;
glucoseUnit: string | null;
onboardingTourCompleted: boolean;
/**
* Server-computed wall-clock hour in the user's timezone so the
* client greeting never has to run its own `Intl.DateTimeFormat`
* before first paint.
*/
greetingHour: number;
}
/**
@@ -530,22 +532,6 @@ export interface SnapshotUserInput {
healthScoreConfigJson: unknown;
}
/** Wall-clock hour in `tz`, used for the server-side greeting. */
function hourInTimezone(now: Date, tz: string): number {
try {
const fmt = new Intl.DateTimeFormat("en-US", {
hour: "numeric",
hour12: false,
timeZone: tz,
});
const parsed = parseInt(fmt.format(now), 10);
// Intl emits "24" for midnight under hour12:false on some ICU builds.
return Number.isFinite(parsed) ? parsed % 24 : now.getUTCHours();
} catch {
return now.getUTCHours();
}
}
function enrichLastSeen(
raw: Record<string, { lastSeenAt: string } | null>,
nowMs: number,
@@ -1457,7 +1443,6 @@ export async function buildDashboardSnapshot(
gender,
glucoseUnit: user.glucoseUnit ?? null,
onboardingTourCompleted: user.onboardingTourCompleted,
greetingHour: hourInTimezone(now, userTz),
},
layout,
layoutCatalogue: buildLayoutCatalogue(layout),
+72 -1
View File
@@ -20,7 +20,20 @@ process.env.ENCRYPTION_KEY ??=
import { describe, expect, it } from "vitest";
import { encrypt } from "@/lib/crypto";
import { packBackupBlob, unpackBackupBlob } from "../backup-blob";
import {
packBackupBlob,
packBackupBlobStreaming,
unpackBackupBlob,
} from "../backup-blob";
/** Pack a whole string through the streaming writer, in small pieces. */
async function packStreamed(json: string, pieceSize = 4_096): Promise<string> {
return packBackupBlobStreaming(async (write) => {
for (let at = 0; at < json.length; at += pieceSize) {
await write(json.slice(at, at + pieceSize));
}
});
}
/** A record-shaped string: many rows, few distinct keys. */
function sampleJson(rows: number): string {
@@ -62,4 +75,62 @@ describe("backup blob envelope", () => {
const mangled = `${packed.slice(0, -8)}AAAAAAAA`;
expect(() => unpackBackupBlob(mangled)).toThrow();
});
it("reads back a streamed blob byte for byte", async () => {
const json = sampleJson(2_000);
expect(unpackBackupBlob(await packStreamed(json))).toBe(json);
});
it("is indifferent to where the producer's pieces fall", async () => {
// Base64 encodes three bytes at a time, so a writer that carries its
// sub-group remainder across chunks wrongly still round-trips at some
// piece sizes and corrupts at others. Cover every residue class.
// Small on purpose: a one-byte piece size means one write per character,
// and the point of the case is the residue class, not the volume.
const json = sampleJson(20);
for (const pieceSize of [1, 2, 3, 7, 64, 1_000, 1_001, json.length]) {
expect(unpackBackupBlob(await packStreamed(json, pieceSize))).toBe(json);
}
});
it("compresses the streamed form as hard as the buffered one", async () => {
const json = sampleJson(5_000);
expect((await packStreamed(json)).length).toBeLessThan(json.length / 4);
});
it("keeps the three stored shapes readable side by side", async () => {
// An operator's newest usable copy may predate either change, and the
// restore has to work for exactly that person. Plain, compressed and
// streamed all decode through the one entry point.
const json = sampleJson(50);
expect(unpackBackupBlob(encrypt(json))).toBe(json);
expect(unpackBackupBlob(packBackupBlob(json))).toBe(json);
expect(unpackBackupBlob(await packStreamed(json))).toBe(json);
});
it("cannot be confused for a legacy blob", async () => {
// The marker starts with a tilde, which is in neither the key-id charset
// nor the base64 alphabet, so no value the older writers can produce
// begins with it and the two families never have to be sniffed apart.
const streamed = await packStreamed(sampleJson(10));
expect(streamed.startsWith("~")).toBe(true);
expect(packBackupBlob(sampleJson(10)).startsWith("~")).toBe(false);
expect(encrypt("x").startsWith("~")).toBe(false);
});
it("refuses a streamed blob whose authentication tag was altered", async () => {
const streamed = await packStreamed(sampleJson(20));
// The tag is the last 16 bytes of the payload, so the tail of the base64.
const mangled = `${streamed.slice(0, -6)}${streamed.slice(-6) === "AAAAAA" ? "BBBBBB" : "AAAAAA"}`;
expect(() => unpackBackupBlob(mangled)).toThrow();
});
it("propagates a producer failure instead of storing a truncated blob", async () => {
await expect(
packBackupBlobStreaming(async (write) => {
await write('{"measurements":[');
throw new Error("row source failed");
}),
).rejects.toThrow("row source failed");
});
});
+115 -15
View File
@@ -13,22 +13,38 @@
*
* Compressing first is what makes the whole tail of that pipeline cheap. A
* health record's JSON is extremely repetitive — the same twenty keys per row,
* timestamps sharing a prefix — so gzip takes that 242 MB to 14 MB, and every
* copy after it shrinks with it. The same run then completes in about eight
* seconds with a peak heap of 434 MB. It is also the reason the stored row
* stops being a liability of its own: the backup lives INSIDE the database it
* would be needed to restore, so its size is not a cosmetic concern.
* timestamps sharing a prefix — so gzip takes that 242 MB to a few tens of
* megabytes, and every copy after it shrinks with it. It is also the reason
* the stored row stops being a liability of its own: the backup lives INSIDE
* the database it would be needed to restore, so its size is not a cosmetic
* concern.
*
* Both directions. Rows written before this envelope existed are plain
* `encrypt(json)` and have to keep restoring — an operator whose newest usable
* copy predates the fix is exactly the person who needs it to work. The marker
* sits inside the ciphertext rather than in front of it so the stored column
* keeps the one shape `decrypt()` already understands, key ids and all.
* Compressing first was not enough on its own. Even with a small stored blob,
* `packBackupBlob` still needs the whole JSON as one argument, and building
* that string is what exhausts the heap — measured under
* `--max-old-space-size=450` on the seeded account: `FATAL ERROR: Reached heap
* limit`. So the writer the weekly job actually uses is
* `packBackupBlobStreaming`, which never sees a complete copy of the JSON, the
* gzip output or the ciphertext: rows go in a page at a time, gzip and the
* cipher consume them as they arrive, and only the base64 answer accumulates,
* because the destination is a single `text` column and one value is what the
* column takes.
*
* Every direction reads. Three shapes exist in the wild and all three restore:
* the original `encrypt(json)`, the compressed `encrypt("HLZ1:" + gz)`, and
* the streamed `~hlgcm1.…` form written from now on. An operator whose newest
* usable copy predates any of this is exactly the person who needs it to work.
*/
import { Buffer } from "node:buffer";
import { gunzipSync, gzipSync } from "node:zlib";
import { createGzip, gunzipSync, gzipSync } from "node:zlib";
import { decrypt, encrypt } from "@/lib/crypto";
import {
createStreamEncryptor,
decrypt,
decryptStream,
encrypt,
isStreamCiphertext,
} from "@/lib/crypto";
/**
* Prefix of the DECRYPTED plaintext when the body is gzipped-then-base64'd.
@@ -43,14 +59,98 @@ export function packBackupBlob(json: string): string {
return encrypt(`${GZIP_MARKER}${compressed}`);
}
/**
* Produces the backup JSON in pieces. Every piece is written in order.
*
* Whatever it resolves to is ignored — the writer's own return value (the row
* counts) is the caller's business, not the envelope's.
*/
export type BackupJsonProducer = (
write: (chunk: string) => Promise<void>,
) => Promise<unknown>;
/**
* Serialised backup JSON, produced in pieces → the stored string.
*
* The pipeline is JSON piece → gzip → AES-256-GCM → base64, with nothing
* buffered end to end but the base64 answer. `producer` decides how big its
* pieces are; the gzip stream applies backpressure through the promise this
* hands back, so a fast producer cannot outrun the compressor and pile up
* chunks in the stream's internal queue.
*
* The one copy that cannot be avoided is the answer itself. `data_backups.data`
* is a single `text` column, so the row has to be one value, and one value has
* to exist as one string before the driver can bind it.
*/
export async function packBackupBlobStreaming(
producer: BackupJsonProducer,
): Promise<string> {
const encryptor = createStreamEncryptor();
const pieces: string[] = [encryptor.header];
const gzip = createGzip();
let failure: unknown = null;
gzip.on("data", (chunk: Buffer) => {
try {
const piece = encryptor.update(chunk);
if (piece !== "") pieces.push(piece);
} catch (err) {
failure ??= err;
gzip.destroy(err as Error);
}
});
const finished = new Promise<void>((resolve, reject) => {
gzip.on("end", resolve);
gzip.on("error", reject);
});
const write = async (chunk: string): Promise<void> => {
if (failure) throw failure;
if (gzip.write(chunk, "utf8")) return;
await new Promise<void>((resolve, reject) => {
const onDrain = () => {
gzip.off("error", onError);
resolve();
};
const onError = (err: Error) => {
gzip.off("drain", onDrain);
reject(err);
};
gzip.once("drain", onDrain);
gzip.once("error", onError);
});
};
try {
await producer(write);
} catch (err) {
gzip.destroy();
throw err;
}
gzip.end();
await finished;
if (failure) throw failure;
pieces.push(encryptor.final());
return pieces.join("");
}
/**
* A stored `DataBackup.data` string → the backup JSON.
*
* Fails closed on every arm: a bad key, a mangled ciphertext or a truncated
* gzip member throws rather than returning a partial document, because every
* caller goes on to parse the result as a whole account.
* Fails closed on every arm: a bad key, a mangled ciphertext, a tag that does
* not verify or a truncated gzip member throws rather than returning a partial
* document, because every caller goes on to parse the result as a whole
* account.
*/
export function unpackBackupBlob(stored: string): string {
// Streamed form. Always gzipped — the streaming writer has no other mode —
// and the tag is verified over the whole ciphertext before a byte of this
// is unpacked.
if (isStreamCiphertext(stored)) {
return gunzipSync(decryptStream(stored)).toString("utf8");
}
const plaintext = decrypt(stored);
if (!plaintext.startsWith(GZIP_MARKER)) return plaintext;
return gunzipSync(
+408 -186
View File
@@ -9,6 +9,16 @@
*
* The decrypted notes are surfaced here (the backup is the human-readable
* artefact); an admin restore re-encrypts them on re-insert.
*
* Two writers read this file, not one. `buildFullBackupPayload` materialises
* the whole payload, which is what the two export routes and every test want.
* `streamFullBackupJson` (`full-backup-stream.ts`) asks for the same payload
* with `deferBulk`, which leaves the three unbounded tables as markers and
* pulls their rows through itself — the weekly job takes that path because
* materialising a large record is what used to exhaust the container's heap.
* Both go through the same per-row serialisers below, so a column that reaches
* one reaches the other; the integration suite pins the two outputs byte for
* byte rather than trusting that sentence.
*/
import { Buffer } from "node:buffer";
import type { Prisma, PrismaClient } from "@/generated/prisma/client";
@@ -146,6 +156,14 @@ export interface FullBackupResult {
export interface FullBackupOptions {
purpose?: "portable-export" | "disaster-recovery";
exportedAt?: Date;
/**
* Declare the three unbounded tables instead of reading them.
*
* `measurements`, `intakeEvents` and `moodEntries` come back as
* `DeferredRows` markers and their counts as 0. Only `streamFullBackupJson`
* sets this; every other caller wants the arrays and gets them.
*/
deferBulk?: boolean;
}
/**
* Every restorable `Measurement` scalar. `userId` is deliberately absent — the
@@ -266,6 +284,35 @@ type BackupMoodContext = Prisma.MoodContextGetPayload<{
select: typeof MOOD_CONTEXT_BACKUP_SELECT;
}>;
type BackupMeasurement = Prisma.MeasurementGetPayload<{
select: typeof MEASUREMENT_BACKUP_SELECT;
}>;
type BackupIntakeEvent = Prisma.MedicationIntakeEventGetPayload<{
include: { medication: { select: { name: true } } };
}>;
/**
* The mood read, select and joins together. Named because two readers issue
* it — the materialising one and the paged one — and a select that lived in
* only one of them is exactly the drift the completeness guard cannot see.
*/
const MOOD_ENTRY_QUERY_SELECT = {
...MOOD_ENTRY_BACKUP_SELECT,
// Every link, not only the rated ones. This read used to filter
// `rating: { not: null }`, and a BINARY link carries a NULL rating by
// definition — so the present/absent tags, the kind most people pick,
// were absent from the file and could not come back from it.
tagLinks: { select: MOOD_ENTRY_TAG_LINK_SELECT },
// The day context, when the entry has one. Sparse by design — most
// entries carry none, and the join answers null on those.
context: { select: MOOD_CONTEXT_BACKUP_SELECT },
} as const satisfies Prisma.MoodEntrySelect;
type BackupMoodEntry = Prisma.MoodEntryGetPayload<{
select: typeof MOOD_ENTRY_QUERY_SELECT;
}>;
/**
* One context row, in whichever of the two wire shapes the caller asked for.
*
@@ -316,24 +363,339 @@ function serialiseMoodContext(
};
}
/**
* One measurement, in whichever of the two wire shapes the caller asked for.
*
* Named rather than inlined so the paged reader below and the materialising
* reader beside it cannot drift: both call this, so a column that reaches one
* shape reaches the other by construction. `measurement-backup-completeness.
* test.ts` proves a selected column reaches a payload by matching
* `<key>: measurement.`, which is why the parameter keeps that spelling.
*/
function serialiseMeasurement(
measurement: BackupMeasurement,
disasterRecovery: boolean,
) {
return disasterRecovery
? {
id: measurement.id,
type: measurement.type,
value: measurement.value,
valueMin: measurement.valueMin,
valueMax: measurement.valueMax,
unit: measurement.unit,
measuredAt: measurement.measuredAt.toISOString(),
source: measurement.source,
notes: measurement.notes,
notesEncrypted: measurement.notesEncrypted
? Buffer.from(measurement.notesEncrypted).toString("base64")
: null,
externalId: measurement.externalId,
externalSourceVersion: measurement.externalSourceVersion,
aggregationProvenance: measurement.aggregationProvenance,
glucoseContext: measurement.glucoseContext,
sleepStage: measurement.sleepStage,
rhythmClassification: measurement.rhythmClassification,
deviceType: measurement.deviceType,
syncVersion: measurement.syncVersion,
deletedAt: measurement.deletedAt?.toISOString() ?? null,
createdAt: measurement.createdAt.toISOString(),
updatedAt: measurement.updatedAt.toISOString(),
}
: {
id: measurement.id,
type: measurement.type,
value: measurement.value,
unit: measurement.unit,
measuredAt: measurement.measuredAt.toISOString(),
source: measurement.source,
notes: readNote(measurement.notesEncrypted, measurement.notes),
deletedAt: measurement.deletedAt?.toISOString() ?? null,
};
}
/** One intake event, as it rides the wire. */
function serialiseIntakeEvent(e: BackupIntakeEvent, disasterRecovery: boolean) {
return {
...(disasterRecovery
? {
id: e.id,
medicationId: e.medicationId,
autoMissed: e.autoMissed,
attributionSource: e.attributionSource,
idempotencyKey: e.idempotencyKey,
createdAt: e.createdAt.toISOString(),
injectionSite: e.injectionSite,
doseTaken: e.doseTaken,
inventoryConsumption: e.inventoryConsumption,
externalId: e.externalId,
updatedAt: e.updatedAt.toISOString(),
syncVersion: e.syncVersion,
deletedAt: e.deletedAt?.toISOString() ?? null,
}
: {}),
medication: e.medication.name,
scheduledFor: e.scheduledFor.toISOString(),
takenAt: e.takenAt?.toISOString() ?? null,
skipped: e.skipped,
source: e.source,
};
}
/**
* One mood entry, with its links partitioned and its day context folded in.
*
* The parameter is spelled out rather than the `e` every other mapper here
* uses, and deliberately: `mood-backup-completeness.test.ts` proves a
* selected column reaches a payload by matching `: moodEntry.`. A matcher
* on `: e.` would also match the intake-event mapper above it and could
* therefore never fail.
*/
function serialiseMoodEntry(
moodEntry: BackupMoodEntry,
disasterRecovery: boolean,
) {
// The two link arms partition the entry's links exhaustively: a link is
// a rated factor when its tag is RATED and it actually carries a score,
// and a structured tag otherwise. Total on purpose — a malformed link
// (a RATED tag whose rating went missing) rides as a tag key and comes
// back NAMED as unresolvable on restore, rather than falling between the
// two arms and disappearing the way every BINARY link used to.
const isRatedFactor = (link: (typeof moodEntry.tagLinks)[number]) =>
link.moodTag.kind === "RATED" && link.rating !== null;
const factors = moodEntry.tagLinks.filter(isRatedFactor).map((tagLink) => ({
key: tagLink.moodTag.key,
rating: tagLink.rating as number,
}));
// BINARY link keys. Their own array rather than a nullable rating inside
// `factors`: the restore resolves `factors` against `kind: "RATED"` and
// reports what it cannot resolve, so a rating-less entry in that array
// would collide with both. A separate key leaves every backup file
// written before this parsing and restoring exactly as it did.
const structuredTags = moodEntry.tagLinks
.filter((link) => !isRatedFactor(link))
.map((tagLink) => tagLink.moodTag.key);
return {
...(disasterRecovery
? {
note: moodEntry.note,
noteEncrypted: moodEntry.noteEncrypted
? Buffer.from(moodEntry.noteEncrypted).toString("base64")
: null,
syncedAt: moodEntry.syncedAt.toISOString(),
syncVersion: moodEntry.syncVersion,
createdAt: moodEntry.createdAt.toISOString(),
updatedAt: moodEntry.updatedAt.toISOString(),
}
: { note: readNote(moodEntry.noteEncrypted, moodEntry.note) }),
id: moodEntry.id,
date: moodEntry.date,
mood: moodEntry.mood,
score: moodEntry.score,
// The five level-A values, under the keys the write path takes. Both
// arms carry them: they are the entry's own answers, not a storage
// detail, and a portable export that dropped them would hand back a
// file describing the day less well than the row it came from.
a1: moodEntry.moodA1,
a2: moodEntry.stressA2,
a3: moodEntry.energyA3,
a4: moodEntry.connectionA4,
a5: moodEntry.stabilityA5,
tags: moodEntry.tags,
source: moodEntry.source,
loggedAt: moodEntry.moodLoggedAt.toISOString(),
externalId: moodEntry.externalId,
// The zone the `date` string is anchored to. Without it a restored row
// falls back to the legacy Europe/Berlin reading and an account that
// logged elsewhere gets its day boundaries quietly moved.
tz: moodEntry.tz,
deletedAt: moodEntry.deletedAt?.toISOString() ?? null,
factors,
structuredTags,
// The day context, or null. Null rather than an empty object, because
// "no context" and "a context whose every field is empty" are
// different facts and the restore has to keep them apart.
context: moodEntry.context
? serialiseMoodContext(moodEntry.context, disasterRecovery)
: null,
};
}
/**
* How many rows one page of a bulk table carries.
*
* Smaller than the measurement pager's 10 000 on purpose: an intake event
* drags its medication's name along and a mood entry drags its links and its
* day context, so a page of either costs several times what a page of
* measurements costs. The number is a memory budget, not a throughput knob.
*/
const BULK_PAGE_SIZE = 5_000;
/**
* Every backup measurement, one serialised row at a time.
*
* Keyset-paged through `iterateMeasurementPages`, so the caller decides
* whether to keep the rows (the materialising payload does) or hand each
* straight to a writer and forget it (the streaming writer does). Nothing here
* holds more than one page.
*/
export async function* iterateBackupMeasurements(
prisma: PrismaClient,
userId: string,
disasterRecovery: boolean,
): AsyncGenerator<Record<string, unknown>, void, void> {
const pages = iterateMeasurementPages(
prisma,
disasterRecovery ? { userId } : { userId, deletedAt: null },
MEASUREMENT_BACKUP_SELECT,
);
for await (const page of pages) {
for (const measurement of page) {
yield serialiseMeasurement(measurement, disasterRecovery);
}
}
}
/**
* Every backup intake event, one serialised row at a time.
*
* The sort is TOTAL — `scheduledFor desc` then `id desc` — where the
* materialising read used to order by the timestamp alone. A keyset cursor
* needs a total order to page correctly, and a partial one also let two
* exports of the same account emit two different files whenever two doses
* shared a scheduled minute, which every multi-dose day has.
*/
export async function* iterateBackupIntakeEvents(
prisma: PrismaClient,
userId: string,
disasterRecovery: boolean,
): AsyncGenerator<Record<string, unknown>, void, void> {
let cursorId: string | null = null;
for (;;) {
const page: BackupIntakeEvent[] =
await prisma.medicationIntakeEvent.findMany({
where: disasterRecovery ? { userId } : { userId, deletedAt: null },
include: { medication: { select: { name: true } } },
orderBy: [{ scheduledFor: "desc" }, { id: "desc" }],
take: BULK_PAGE_SIZE,
...(cursorId !== null ? { cursor: { id: cursorId }, skip: 1 } : {}),
});
if (page.length === 0) return;
for (const e of page) yield serialiseIntakeEvent(e, disasterRecovery);
if (page.length < BULK_PAGE_SIZE) return;
cursorId = page[page.length - 1]!.id;
}
}
/**
* Every backup mood entry, one serialised row at a time.
*
* Same total-order argument as the intake events above: `moodLoggedAt desc`
* alone is not a cursor, and two entries logged in the same millisecond are
* exactly what a bulk import produces.
*/
export async function* iterateBackupMoodEntries(
prisma: PrismaClient,
userId: string,
disasterRecovery: boolean,
): AsyncGenerator<Record<string, unknown>, void, void> {
let cursorId: string | null = null;
for (;;) {
const page: BackupMoodEntry[] = await prisma.moodEntry.findMany({
where: disasterRecovery ? { userId } : { userId, deletedAt: null },
orderBy: [{ moodLoggedAt: "desc" }, { id: "desc" }],
take: BULK_PAGE_SIZE,
...(cursorId !== null ? { cursor: { id: cursorId }, skip: 1 } : {}),
select: MOOD_ENTRY_QUERY_SELECT,
});
if (page.length === 0) return;
for (const moodEntry of page) {
yield serialiseMoodEntry(moodEntry, disasterRecovery);
}
if (page.length < BULK_PAGE_SIZE) return;
cursorId = page[page.length - 1]!.id;
}
}
/** Drain an async row source into an array. The materialising arm's only use. */
async function collect<T>(rows: AsyncIterable<T>): Promise<T[]> {
const out: T[] = [];
for await (const row of rows) out.push(row);
return out;
}
/**
* A bulk table the payload declares but has NOT read.
*
* `buildFullBackupPayload({ deferBulk: true })` puts one of these where the
* row array would sit. The streaming writer recognises it, opens the JSON
* array itself and pulls rows through page by page, so a table with half a
* million rows never exists as an array at all. Every other caller leaves
* `deferBulk` unset and gets the arrays, unchanged.
*
* The marker key is a symbol so it cannot collide with a payload field and so
* the deferred payload can never be handed to `JSON.stringify` by accident and
* come back looking merely empty — `streamFullBackupJson` is the only reader.
*/
const DEFERRED_ROWS = Symbol("healthlog.backup.deferredRows");
export interface DeferredRows {
readonly [DEFERRED_ROWS]: true;
/** Open the row source. Called exactly once, by the streaming writer. */
rows(): AsyncIterable<Record<string, unknown>>;
}
function deferredRows(
rows: () => AsyncIterable<Record<string, unknown>>,
): DeferredRows {
return { [DEFERRED_ROWS]: true, rows };
}
/** True when a payload value is a deferred bulk table rather than an array. */
export function isDeferredRows(value: unknown): value is DeferredRows {
return (
typeof value === "object" &&
value !== null &&
(value as Record<symbol, unknown>)[DEFERRED_ROWS] === true
);
}
/**
* The same payload, already serialised, with the object graph released.
*
* Its own frame on purpose, and both backup jobs go through it. The payload
* graph is the largest thing either job allocates — on an account with a few
* hundred thousand measurements it is several hundred megabytes of small
* objects — and neither job wants anything but the string. Returning from here
* makes the graph unreachable before the compress-and-encrypt step allocates
* anything of its own, which is part of why the weekly pass now fits in
* memory. Inlining these two statements back into a caller quietly undoes it.
* Built through the streaming writer rather than by stringifying a finished
* payload, so the object graph and the string never coexist: the three
* unbounded tables go past a page at a time and every other section is
* released as soon as its JSON exists. What is left is the answer itself,
* which a caller that wants a string necessarily holds.
*
* A caller that does NOT need the string — the weekly job, which only wants
* the compressed ciphertext — should go through `packBackupBlobStreaming` and
* `streamFullBackupJson` directly and never let the whole document exist at
* all. This function is for the callers that genuinely hand a JSON body on.
*/
export async function buildFullBackupJson(
prisma: PrismaClient,
userId: string,
options: FullBackupOptions = {},
): Promise<string> {
const { payload } = await buildFullBackupPayload(prisma, userId, options);
return JSON.stringify(payload);
// Imported here rather than at the top of the file: the writer imports this
// module for `buildFullBackupPayload`, and a static edge in both directions
// is a cycle. The call is once per backup, so the cost is nil.
const { streamFullBackupJson } =
await import("@/lib/export/full-backup-stream");
const pieces: string[] = [];
await streamFullBackupJson(
prisma,
userId,
(chunk) => {
pieces.push(chunk);
},
options,
);
return pieces.join("");
}
/**
@@ -347,6 +709,7 @@ export async function buildFullBackupPayload(
options: FullBackupOptions = {},
): Promise<FullBackupResult> {
const disasterRecovery = options.purpose === "disaster-recovery";
const deferBulk = options.deferBulk === true;
const [
appSettings,
measurements,
@@ -376,60 +739,9 @@ export async function buildFullBackupPayload(
disasterRecovery
? prisma.appSettings.findUnique({ where: { id: "singleton" } })
: Promise.resolve(null),
(async () => {
const rows: Array<Record<string, unknown>> = [];
const pages = iterateMeasurementPages(
prisma,
disasterRecovery ? { userId } : { userId, deletedAt: null },
MEASUREMENT_BACKUP_SELECT,
);
for await (const page of pages) {
for (const measurement of page) {
rows.push(
disasterRecovery
? {
id: measurement.id,
type: measurement.type,
value: measurement.value,
valueMin: measurement.valueMin,
valueMax: measurement.valueMax,
unit: measurement.unit,
measuredAt: measurement.measuredAt.toISOString(),
source: measurement.source,
notes: measurement.notes,
notesEncrypted: measurement.notesEncrypted
? Buffer.from(measurement.notesEncrypted).toString("base64")
: null,
externalId: measurement.externalId,
externalSourceVersion: measurement.externalSourceVersion,
aggregationProvenance: measurement.aggregationProvenance,
glucoseContext: measurement.glucoseContext,
sleepStage: measurement.sleepStage,
rhythmClassification: measurement.rhythmClassification,
deviceType: measurement.deviceType,
syncVersion: measurement.syncVersion,
deletedAt: measurement.deletedAt?.toISOString() ?? null,
createdAt: measurement.createdAt.toISOString(),
updatedAt: measurement.updatedAt.toISOString(),
}
: {
id: measurement.id,
type: measurement.type,
value: measurement.value,
unit: measurement.unit,
measuredAt: measurement.measuredAt.toISOString(),
source: measurement.source,
notes: readNote(
measurement.notesEncrypted,
measurement.notes,
),
deletedAt: measurement.deletedAt?.toISOString() ?? null,
},
);
}
}
return rows;
})(),
deferBulk
? null
: collect(iterateBackupMeasurements(prisma, userId, disasterRecovery)),
prisma.medication.findMany({
where: { userId },
// Side effects ride INSIDE their medication, exactly as the schedules
@@ -469,26 +781,12 @@ export async function buildFullBackupPayload(
},
},
}),
prisma.medicationIntakeEvent.findMany({
where: disasterRecovery ? { userId } : { userId, deletedAt: null },
include: { medication: { select: { name: true } } },
orderBy: { scheduledFor: "desc" },
}),
prisma.moodEntry.findMany({
where: disasterRecovery ? { userId } : { userId, deletedAt: null },
orderBy: { moodLoggedAt: "desc" },
select: {
...MOOD_ENTRY_BACKUP_SELECT,
// Every link, not only the rated ones. This read used to filter
// `rating: { not: null }`, and a BINARY link carries a NULL rating by
// definition — so the present/absent tags, the kind most people pick,
// were absent from the file and could not come back from it.
tagLinks: { select: MOOD_ENTRY_TAG_LINK_SELECT },
// The day context, when the entry has one. Sparse by design — most
// entries carry none, and the join answers null on those.
context: { select: MOOD_CONTEXT_BACKUP_SELECT },
},
}),
deferBulk
? null
: collect(iterateBackupIntakeEvents(prisma, userId, disasterRecovery)),
deferBulk
? null
: collect(iterateBackupMoodEntries(prisma, userId, disasterRecovery)),
// The account's OWN mood tags. The seeded catalogue is reference data every
// instance has; a tag the user made lives only here. Without it the restore
// cannot resolve the entry's factor keys — and unlike the cycle path, which
@@ -646,6 +944,27 @@ export async function buildFullBackupPayload(
const environmentSection: EnvironmentBackupSection = environment;
const ecgSection: EcgBackupSection = ecg;
// The three unbounded tables, either read into arrays or left as markers
// the streaming writer will pull through. One place decides, so the payload
// literal below reads the same in both modes.
const bulk = {
measurements: deferBulk
? deferredRows(() =>
iterateBackupMeasurements(prisma, userId, disasterRecovery),
)
: measurements!,
intakeEvents: deferBulk
? deferredRows(() =>
iterateBackupIntakeEvents(prisma, userId, disasterRecovery),
)
: intakeEvents!,
moodEntries: deferBulk
? deferredRows(() =>
iterateBackupMoodEntries(prisma, userId, disasterRecovery),
)
: moodEntries!,
};
// Where each archived era sits in its OWN drug's ordered list. Built once
// here rather than per row, and scoped per medication because the supersede
// pointer never crosses a drug — every route that writes it scopes the
@@ -674,7 +993,7 @@ export async function buildFullBackupPayload(
documentQuotaBytes: appSettings.documentQuotaBytes.toString(),
}
: null,
measurements,
measurements: bulk.measurements,
medications: medications.map((m) => ({
...(disasterRecovery
? {
@@ -903,105 +1222,8 @@ export async function buildFullBackupPayload(
primary: t.primary,
})),
})),
intakeEvents: intakeEvents.map((e) => ({
...(disasterRecovery
? {
id: e.id,
medicationId: e.medicationId,
autoMissed: e.autoMissed,
attributionSource: e.attributionSource,
idempotencyKey: e.idempotencyKey,
createdAt: e.createdAt.toISOString(),
injectionSite: e.injectionSite,
doseTaken: e.doseTaken,
inventoryConsumption: e.inventoryConsumption,
externalId: e.externalId,
updatedAt: e.updatedAt.toISOString(),
syncVersion: e.syncVersion,
deletedAt: e.deletedAt?.toISOString() ?? null,
}
: {}),
medication: e.medication.name,
scheduledFor: e.scheduledFor.toISOString(),
takenAt: e.takenAt?.toISOString() ?? null,
skipped: e.skipped,
source: e.source,
})),
// The parameter is spelled out rather than the `e` every other mapper here
// uses, and deliberately: `mood-backup-completeness.test.ts` proves a
// selected column reaches a payload by matching `: moodEntry.`. A matcher
// on `: e.` would also match the intake-event mapper above it and could
// therefore never fail.
moodEntries: moodEntries.map((moodEntry) => {
// The two link arms partition the entry's links exhaustively: a link is
// a rated factor when its tag is RATED and it actually carries a score,
// and a structured tag otherwise. Total on purpose — a malformed link
// (a RATED tag whose rating went missing) rides as a tag key and comes
// back NAMED as unresolvable on restore, rather than falling between the
// two arms and disappearing the way every BINARY link used to.
const isRatedFactor = (link: (typeof moodEntry.tagLinks)[number]) =>
link.moodTag.kind === "RATED" && link.rating !== null;
const factors = moodEntry.tagLinks
.filter(isRatedFactor)
.map((tagLink) => ({
key: tagLink.moodTag.key,
rating: tagLink.rating as number,
}));
// BINARY link keys. Their own array rather than a nullable rating inside
// `factors`: the restore resolves `factors` against `kind: "RATED"` and
// reports what it cannot resolve, so a rating-less entry in that array
// would collide with both. A separate key leaves every backup file
// written before this parsing and restoring exactly as it did.
const structuredTags = moodEntry.tagLinks
.filter((link) => !isRatedFactor(link))
.map((tagLink) => tagLink.moodTag.key);
return {
...(disasterRecovery
? {
note: moodEntry.note,
noteEncrypted: moodEntry.noteEncrypted
? Buffer.from(moodEntry.noteEncrypted).toString("base64")
: null,
syncedAt: moodEntry.syncedAt.toISOString(),
syncVersion: moodEntry.syncVersion,
createdAt: moodEntry.createdAt.toISOString(),
updatedAt: moodEntry.updatedAt.toISOString(),
}
: { note: readNote(moodEntry.noteEncrypted, moodEntry.note) }),
id: moodEntry.id,
date: moodEntry.date,
mood: moodEntry.mood,
score: moodEntry.score,
// The five level-A values, under the keys the write path takes. Both
// arms carry them: they are the entry's own answers, not a storage
// detail, and a portable export that dropped them would hand back a
// file describing the day less well than the row it came from.
a1: moodEntry.moodA1,
a2: moodEntry.stressA2,
a3: moodEntry.energyA3,
a4: moodEntry.connectionA4,
a5: moodEntry.stabilityA5,
tags: moodEntry.tags,
source: moodEntry.source,
loggedAt: moodEntry.moodLoggedAt.toISOString(),
externalId: moodEntry.externalId,
// The zone the `date` string is anchored to. Without it a restored row
// falls back to the legacy Europe/Berlin reading and an account that
// logged elsewhere gets its day boundaries quietly moved.
tz: moodEntry.tz,
deletedAt: moodEntry.deletedAt?.toISOString() ?? null,
factors,
structuredTags,
// The day context, or null. Null rather than an empty object, because
// "no context" and "a context whose every field is empty" are
// different facts and the restore has to keep them apart.
context: moodEntry.context
? serialiseMoodContext(moodEntry.context, disasterRecovery)
: null,
};
}),
intakeEvents: bulk.intakeEvents,
moodEntries: bulk.moodEntries,
customMoodTagCategories: customMoodTagCategories.map((category) => ({
id: category.id,
key: category.key,
@@ -1079,9 +1301,9 @@ export async function buildFullBackupPayload(
return {
payload,
counts: {
measurements: measurements.length,
measurements: measurements?.length ?? 0,
medications: medications.length,
intakeEvents: intakeEvents.length,
intakeEvents: intakeEvents?.length ?? 0,
medicationSideEffects: medications.reduce(
(total, m) => total + m.sideEffects.length,
0,
@@ -1111,7 +1333,7 @@ export async function buildFullBackupPayload(
(total, m) => total + m.scheduleRevisions.length,
0,
),
moodEntries: moodEntries.length,
moodEntries: moodEntries?.length ?? 0,
customMoodTagCategories: customMoodTagCategories.length,
hiddenMoodTags: hiddenMoodTags.length,
cycles: cycle.cycles.length,
+160
View File
@@ -0,0 +1,160 @@
/**
* The full-backup payload, written out incrementally instead of built.
*
* Why it exists. `buildFullBackupJson` has to hold two things at once that
* both scale with the record: the payload object graph, and the JSON string
* `JSON.stringify` makes of it. On a seeded account of 445 000 measurements,
* 30 000 mood entries and 60 000 intake events, that pair alone exhausts a
* 546 MB heap and the process dies with `Reached heap limit` — measured, not
* inferred, and reproducible with `--max-old-space-size=450`. The production
* container is capped at 1 GB, which puts V8's limit at 524 MB, so the weekly
* pass took the whole app down with it: every other user of that instance lost
* their session because one account's backup did not fit.
*
* What changed. The three tables that scale with a long-lived record —
* measurements, intake events, mood entries — are declared rather than read
* (`deferBulk`), and this writer pulls them through page by page, serialising
* one row at a time straight into the sink. Every other section is stringified
* and then RELEASED from the payload before the next one is touched, so the
* peak is the largest single section rather than the sum of all of them.
*
* The output is byte-identical to `JSON.stringify(payload)`. That is not a
* hope: `Object.entries` walks a plain object in insertion order, which is the
* order `JSON.stringify` uses, and `JSON.stringify([a, b])` is exactly
* `"[" + JSON.stringify(a) + "," + JSON.stringify(b) + "]"` for plain rows.
* The integration suite pins it against the materialising builder rather than
* trusting that paragraph.
*/
import v8 from "node:v8";
import type { PrismaClient } from "@/generated/prisma/client";
import {
buildFullBackupPayload,
isDeferredRows,
type FullBackupCounts,
type FullBackupOptions,
} from "@/lib/export/full-backup-payload";
/** Where the writer puts its pieces. Awaited, so a sink can apply backpressure. */
export type BackupJsonSink = (chunk: string) => void | Promise<void>;
/**
* How much serialised JSON to gather before handing it to the sink.
*
* Row-at-a-time writes would push half a million tiny strings through gzip and
* spend more time in framing than in compression; a batch this size keeps the
* resident buffer trivial and the write count in the low thousands.
*/
const FLUSH_BYTES = 256 * 1024;
/**
* Fraction of V8's heap limit the writer refuses to cross.
*
* The point is not to make an oversized account succeed — it is to make it
* FAIL AS A JOB. A backup that runs out of heap takes the process with it, and
* a pg-boss worker sharing the app process means one account's record can
* restart the instance for everybody. Because the work is now incremental
* there is a checkpoint every few hundred kilobytes, so the guard actually
* gets to run before V8 hits the wall; the old single `JSON.stringify` had no
* point at which anything could intervene.
*/
const HEAP_HIGH_WATER = 0.8;
/** Thrown when the writer stops itself short of the heap limit. */
export class BackupHeapBudgetExceededError extends Error {
constructor(usedBytes: number, ceilingBytes: number) {
super(
`Backup aborted at ${Math.round(usedBytes / 1024 / 1024)} MB of heap ` +
`(budget ${Math.round(ceilingBytes / 1024 / 1024)} MB). The record is ` +
`too large for this container's memory limit; raise the container's ` +
`memory or NODE_OPTIONS=--max-old-space-size.`,
);
this.name = "BackupHeapBudgetExceededError";
}
}
export interface StreamFullBackupOptions extends FullBackupOptions {
/**
* Heap ceiling in bytes. Defaults to 80 % of V8's limit for this process.
* Tests pass an explicit value; nothing else should need to.
*/
heapCeilingBytes?: number;
}
function defaultHeapCeiling(): number {
return Math.floor(v8.getHeapStatistics().heap_size_limit * HEAP_HIGH_WATER);
}
/**
* Write the full-backup JSON for `userId` into `sink`, and answer the counts.
*
* The counts for the three deferred tables are filled in as their rows go
* past, so a caller gets the same numbers the materialising builder reports.
*/
export async function streamFullBackupJson(
prisma: PrismaClient,
userId: string,
sink: BackupJsonSink,
options: StreamFullBackupOptions = {},
): Promise<FullBackupCounts> {
const ceiling = options.heapCeilingBytes ?? defaultHeapCeiling();
const { payload, counts } = await buildFullBackupPayload(prisma, userId, {
...options,
deferBulk: true,
});
let pending = "";
let pendingBytes = 0;
const flush = async (force: boolean): Promise<void> => {
if (pending === "" || (!force && pendingBytes < FLUSH_BYTES)) return;
const chunk = pending;
pending = "";
pendingBytes = 0;
await sink(chunk);
const used = process.memoryUsage().heapUsed;
if (used > ceiling) throw new BackupHeapBudgetExceededError(used, ceiling);
};
const write = async (piece: string): Promise<void> => {
pending += piece;
pendingBytes += piece.length;
await flush(false);
};
const bulkCounts: Record<string, number> = {};
await write("{");
let first = true;
// The walk is destructive: each section is released from the payload as soon
// as its JSON is in the sink, which is what keeps the peak at one section
// rather than at all of them. Nothing else reads this object.
for (const key of Object.keys(payload)) {
const value = payload[key];
// `JSON.stringify` omits an undefined-valued key; so does this.
if (value === undefined) continue;
if (!first) await write(",");
first = false;
await write(`${JSON.stringify(key)}:`);
if (isDeferredRows(value)) {
await write("[");
let rows = 0;
for await (const row of value.rows()) {
await write(
rows === 0 ? JSON.stringify(row) : `,${JSON.stringify(row)}`,
);
rows++;
}
await write("]");
bulkCounts[key] = rows;
} else {
await write(JSON.stringify(value));
}
delete payload[key];
}
await write("}");
await flush(true);
return { ...counts, ...bulkCounts } as FullBackupCounts;
}
+9 -1
View File
@@ -9,7 +9,15 @@
export type GlucoseUnit = "mg/dL" | "mmol/L";
const MGDL_PER_MMOL = 18.0182;
/**
* mg/dL per mmol/L, per the DGIM / DDG S3 guideline. Exported because it is
* the one factor for this conversion in the tree: the inbound alias resolver
* normalises a mmol/L cell with it, the glucose insight page scales its chart
* by its reciprocal, and the helpers below round with it. Two nearly-equal
* copies of a health conversion is how the same reading comes out as two
* different numbers on two screens.
*/
export const MGDL_PER_MMOL = 18.0182;
export function mgdlToMmol(mgdl: number): number {
return Math.round((mgdl / MGDL_PER_MMOL) * 10) / 10;
@@ -0,0 +1,100 @@
/**
* The UI language and the stored language are resolved from different
* evidence, and only one of them is ever shown to the person reading.
*
* `resolveServerLocale` / `resolveInitialLocale` answer a REQUEST: the
* `healthlog-locale` cookie first, then `User.locale`, then Accept-Language.
* Two of those three rungs are invisible to anything without a request, so
* a browser can render the whole app in German while the column says
* English.
*
* Everything written outside a request reads the column alone —
* `resolveJobLocale` for the nightly writers, `resolveLocaleForUser` for the
* Coach tool executor and the MCP reads. So a diverged column does not show
* up as a wrong menu or a wrong button. It shows up as one paragraph of
* generated prose in the wrong language, sitting in a page that is otherwise
* correct, which is exactly how it was reported.
*
* These are the two ways a browser gets a language the column never hears
* about. They are the reason `getSession` reconciles the column from the
* cookie: the client's fire-and-forget PUT is the only thing that ever
* closed this gap, and a request that never arrives leaves the divergence
* standing for as long as the account exists.
*/
import { describe, it, expect, vi, beforeEach } from "vitest";
const mockCookies = vi.fn();
const mockHeaders = vi.fn();
vi.mock("next/headers", () => ({
cookies: (...args: unknown[]) => mockCookies(...args),
headers: (...args: unknown[]) => mockHeaders(...args),
}));
const mockGetSessionUserLocale = vi.fn();
vi.mock("@/lib/auth/session", () => ({
getSessionUserLocale: (...args: unknown[]) =>
mockGetSessionUserLocale(...args),
}));
vi.mock("@/lib/app-settings", () => ({
getOperatorDefaultLocale: vi.fn(async () => null),
}));
import { resolveInitialLocale } from "@/lib/i18n/resolve-initial-locale";
import { resolveJobLocale } from "@/lib/i18n/job-locale";
function cookieStoreWith(value: string | undefined) {
return {
get: (name: string) =>
name === "healthlog-locale" && value !== undefined
? { name, value }
: undefined,
};
}
function headersWith(acceptLanguage: string | null) {
return {
get: (name: string) =>
name.toLowerCase() === "accept-language" ? acceptLanguage : null,
};
}
beforeEach(() => {
vi.clearAllMocks();
mockCookies.mockResolvedValue(cookieStoreWith(undefined));
mockHeaders.mockResolvedValue(headersWith(null));
mockGetSessionUserLocale.mockResolvedValue(null);
});
describe("a browser can read German while the column says English", () => {
it("Accept-Language alone: German UI, English background prose", async () => {
// The account has never opened the language picker, so the column is
// NULL, and the browser asks for German.
mockGetSessionUserLocale.mockResolvedValue(null);
mockHeaders.mockResolvedValue(headersWith("de-DE,de;q=0.9,en;q=0.5"));
await expect(resolveInitialLocale()).resolves.toBe("de");
await expect(resolveJobLocale(null)).resolves.toBe("en");
});
it("cookie alone: German UI, English background prose", async () => {
// The picker was used at some point, so the cookie carries German —
// but the column was left holding the language of the paint that
// preceded the choice.
mockCookies.mockResolvedValue(cookieStoreWith("de"));
mockGetSessionUserLocale.mockResolvedValue("en");
await expect(resolveInitialLocale()).resolves.toBe("de");
await expect(resolveJobLocale("en")).resolves.toBe("en");
});
it("the cookie outranks the column on every request that has one", async () => {
// Which is why the divergence is invisible: nothing the user looks at
// reads the column, so a wrong column produces no wrong screen until a
// job writes a sentence with it.
mockCookies.mockResolvedValue(cookieStoreWith("de"));
mockGetSessionUserLocale.mockResolvedValue("en");
await expect(resolveInitialLocale()).resolves.toBe("de");
expect(mockGetSessionUserLocale).not.toHaveBeenCalled();
});
});
@@ -0,0 +1,70 @@
/**
* Why the obvious suspect is not the cause.
*
* The tempting explanation for a column stuck on the wrong language is a
* race: first paint renders the default, the mount effect persists THAT,
* the client then flips to the real locale and persists again, and two
* unordered `keepalive` PUTs land in the wrong order. It would explain the
* symptom exactly, and it would mean the write needs ordering.
*
* It is not what happens. The provider takes the locale the SERVER already
* resolved — cookie, then column, then Accept-Language — and nothing on the
* mount path changes it afterwards, so the effect fires once, with the right
* value, and there is no second write to race with. The write is lost a
* simpler way: it is fire-and-forget, so a 401 on the paint before sign-in,
* an offline blip or a tab closed mid-flight drops it, and the next mount
* repeats the same unreliable attempt rather than noticing.
*
* These pin the two properties that rule the race out, so that a future
* change which does introduce a mount-time flip has to face this file.
*/
import { describe, it, expect } from "vitest";
import { readFileSync } from "node:fs";
import { join } from "node:path";
import { renderToStaticMarkup } from "react-dom/server";
import { I18nProvider, useTranslations } from "@/lib/i18n/context";
import deMessages from "../../../../messages/de.json";
function Read({ k }: { k: string }) {
const { t } = useTranslations();
return <span>{t(k)}</span>;
}
describe("first paint carries the resolved locale, not a default", () => {
it("renders the server-resolved locale on the very first pass", () => {
// If this ever rendered English first, the mount effect would persist
// English before the flip, and the race above would be real.
const html = renderToStaticMarkup(
<I18nProvider initialLocale="de" initialMessages={deMessages}>
<Read k="common.save" />
</I18nProvider>,
);
expect(html).toContain("Speichern");
expect(html).not.toContain(">Save<");
});
it("changes the active locale only through the switcher", () => {
const src = readFileSync(
join(process.cwd(), "src/lib/i18n/context.tsx"),
"utf8",
);
expect(src.length).toBeGreaterThan(0);
// Every `setActive` call in the file. One belongs to `setLocale` (the
// explicit switch); the others are the bundle backfill, which must
// carry the locale it was given and never a different one.
const setActiveCalls = src.match(/setActive\(/g) ?? [];
expect(setActiveCalls.length).toBeGreaterThan(0);
// The backfill keeps `prev` whenever the locale moved under it, so it
// can only ever swap MESSAGES, never the locale.
expect(src).toMatch(
/prev\.locale === locale \? \{ locale, messages: loaded \} : prev/,
);
// And the persistence effect keys on the locale alone, so one mount
// with one locale means one write.
expect(src).toMatch(/persistLocaleToServer\(locale\);\s*\}, \[locale\]\);/);
});
});
+13 -7
View File
@@ -109,13 +109,19 @@ function persistLocale(newLocale: Locale) {
document.documentElement.lang = newLocale;
}
// v1.25.0 — mirror the active locale onto `User.locale` so server-side
// background work (the proactive Coach nudge, the Telegram test message)
// renders in the user's language. The cookie/localStorage choice is
// invisible to a cron that has no request context; without this write the
// column stays null and those messages fall back to English. Fire-and-forget
// and idempotent on the server: a 401 on a public page, an offline blip or a
// no-op equal value all fail silently and never block the UI flip.
// Mirror the active locale onto `User.locale` so server-side background work
// (the proactive Coach nudge, the Telegram test message) renders in the
// user's language. The cookie/localStorage choice is invisible to a cron that
// has no request context.
//
// This is no longer what the column depends on. Being fire-and-forget, it
// loses the write on a 401, an offline blip or a tab closed mid-flight, and
// nothing retried — the column then held a language the user had left behind
// for good. `getSession` reconciles the column from the locale cookie on the
// next request instead, which is ordered by the request pipeline and needs no
// cooperation from the client. What this call still earns its place for is
// the server-set cookie the route emits in reply: Safari's ITP caps the
// script-written copy at 7 days, and a Set-Cookie is exempt.
function persistLocaleToServer(newLocale: Locale) {
void apiFetchRaw("/api/auth/me/locale", {
method: "PUT",
@@ -1,4 +1,5 @@
import { describe, it, expect } from "vitest";
import { MGDL_PER_MMOL, mgdlToMmol } from "@/lib/glucose";
import {
parseCsvMeasurements,
@@ -98,7 +99,12 @@ describe("parseCsvMeasurements — unit conversion", () => {
]);
expect(out.rows[0].status).toBe("ok");
expect(out.rows[0].row?.unit).toBe("mg/dL");
expect(out.rows[0].row?.value).toBeCloseTo(5.3 * 18.016, 3);
// The one factor the app converts glucose by, both directions. The
// literal that used to sit here was 18.016 while every display path
// divided by 18.0182, so a reading imported in mmol/L did not read back
// as the number it went in as.
expect(out.rows[0].row?.value).toBeCloseTo(5.3 * MGDL_PER_MMOL, 3);
expect(mgdlToMmol(out.rows[0].row?.value ?? 0)).toBe(5.3);
});
it("converts weight lb to canonical kg", () => {
@@ -176,7 +182,7 @@ describe("parseCsvMeasurements — glucose context", () => {
for (const result of out.rows) {
expect(result.row).not.toHaveProperty("glucoseContext");
expect(result.row?.unit).toBe("mg/dL");
expect(result.row?.value).toBeCloseTo(95.4848, 3);
expect(result.row?.value).toBeCloseTo(5.3 * MGDL_PER_MMOL, 3);
}
expect(out.rows[0].row?.measuredAt.toISOString()).toBe(
"2024-04-03T02:15:00.000Z",
@@ -0,0 +1,69 @@
/**
* `ImportJob.exportedAt` — the worker end of the wiring.
*
* The parse side is proved behaviourally in
* `src/lib/measurements/__tests__/import-apple-health-export.test.ts`:
* the parser hands the archive's own stamp to `onExportDate`. What that
* cannot show is whether anything persists it. The column, the status
* endpoint that returns it and the published contract that requires it
* all existed for a year while the one line that would produce the value
* was a `return`, so the half of the pipe worth pinning is this one.
*
* Source-shaped rather than run-shaped: driving the worker needs a real
* `export.zip`, a queue and a database, which the ECG contract test
* beside this one already declines to do for the same reason.
*/
import { readFileSync } from "node:fs";
import { join } from "node:path";
import { describe, expect, it } from "vitest";
const workerSource = readFileSync(
join(process.cwd(), "src/lib/jobs/apple-health-import-worker.ts"),
"utf8",
);
const parserSource = readFileSync(
join(process.cwd(), "src/lib/measurements/import-apple-health-export.ts"),
"utf8",
);
const statusRouteSource = readFileSync(
join(
process.cwd(),
"src/app/api/import/apple-health-export/[jobId]/status/route.ts",
),
"utf8",
);
describe("Apple Health import — the archive's export stamp reaches the column", () => {
it("subscribes to the parser's export-date hook", () => {
expect(workerSource).toMatch(/onExportDate:\s*async\s*\(exportedAt\)/);
});
it("writes the stamp onto the job row it is running", () => {
expect(workerSource).toMatch(
/onExportDate:[\s\S]{0,400}?prisma\s*\.?\s*importJob[\s\S]{0,200}?\.update\([\s\S]{0,200}?exportedAt/,
);
});
it("keeps the write best-effort so provenance cannot fail an import", () => {
expect(workerSource).toMatch(
/onExportDate:[\s\S]{0,400}?exportedAt[\s\S]{0,120}?\.catch\(/,
);
});
it("no longer discards the element that carries the stamp", () => {
// The ignore list is where `ExportDate` spent its first year.
const ignoreBlock = /Known elements we intentionally ignore/.exec(
parserSource,
);
expect(ignoreBlock).not.toBeNull();
const before = parserSource.slice(
Math.max(0, (ignoreBlock?.index ?? 0) - 400),
ignoreBlock?.index ?? 0,
);
expect(before).not.toMatch(/name === "ExportDate" \|\|/);
});
it("still surfaces the column the contract promises", () => {
expect(statusRouteSource).toMatch(/exportedAt:\s*row\.exportedAt/);
});
});
@@ -417,6 +417,16 @@ export async function handleAppleHealthImport(
},
});
},
// The instant the Health app stamped on the archive. Persisted as
// soon as the parser reads it, so the status endpoint answers with
// it for the rest of the run and a job that later fails still
// records which export it was working from. Best-effort: the
// stamp is provenance, and losing it must not fail an import.
onExportDate: async (exportedAt) => {
await prisma.importJob
.update({ where: { id: importJobId }, data: { exportedAt } })
.catch(() => {});
},
})),
ecg: {
discovered: 0,
+5
View File
@@ -48,6 +48,7 @@ export async function runEncryptionKeyRotation(): Promise<{
totalScanned: number;
totalRotated: number;
totalErrors: number;
totalDropped: number;
}> {
const prisma = getWorkerPrisma();
const out = await rotateCorpus(prisma as unknown as CorpusClient);
@@ -56,6 +57,7 @@ export async function runEncryptionKeyRotation(): Promise<{
totalScanned: out.totalScanned,
totalRotated: out.totalRotated,
totalErrors: out.totalErrors,
totalDropped: out.totalDropped,
};
}
@@ -70,6 +72,7 @@ export async function handleEncryptionKeyRotate(
evt.addMeta("rotate_scanned", result.totalScanned);
evt.addMeta("rotate_rotated", result.totalRotated);
evt.addMeta("rotate_errors", result.totalErrors);
evt.addMeta("rotate_dropped", result.totalDropped);
await auditLog("admin.encryption.rotate.completed", {
userId: requestedBy,
details: {
@@ -77,6 +80,7 @@ export async function handleEncryptionKeyRotate(
scanned: result.totalScanned,
rotated: result.totalRotated,
errors: result.totalErrors,
dropped: result.totalDropped,
},
});
// Per-row errors are the fail-closed skip of a row under a key the
@@ -86,6 +90,7 @@ export async function handleEncryptionKeyRotate(
rotate_scanned: result.totalScanned,
rotate_rotated: result.totalRotated,
rotate_errors: result.totalErrors,
rotate_dropped: result.totalDropped,
});
} catch (err) {
evt.addWarning(`encryption-key-rotate failed: ${err}`);
+2
View File
@@ -103,6 +103,7 @@ export const JOB_FACT_ALLOWLIST: ReadonlySet<string> = new Set([
"geo_backfill_skipped",
"geo_backfill_still_unresolved",
"geolite2_fetch_status",
"heap_budget_trips",
"host_metric_pruned",
"hourly_rows_upserted",
"imported",
@@ -172,6 +173,7 @@ export const JOB_FACT_ALLOWLIST: ReadonlySet<string> = new Set([
"retried",
"reviewed",
"rollup_failed",
"rotate_dropped",
"rotate_errors",
"rotate_rotated",
"rotate_scanned",
@@ -9,23 +9,29 @@ const mocks = vi.hoisted(() => ({
upsert: vi.fn(),
}));
// The handler takes the already-serialised form; the stand-in serialises
// whatever the payload mock was told to return, so this file keeps stubbing
// and asserting the PAYLOAD, which is what it is about.
// Only the payload builder is stubbed. The REAL streaming writer runs on top
// of it, so this file keeps stubbing and asserting the PAYLOAD — which is what
// it is about — while the framing that turns it into stored bytes is the
// framing the job actually uses. `isDeferredRows` rides along because the
// writer asks it about every section; nothing here defers.
vi.mock("@/lib/export/full-backup-payload", () => ({
buildFullBackupPayload: mocks.buildFullBackupPayload,
buildFullBackupJson: async (...args: unknown[]) =>
JSON.stringify(
((await mocks.buildFullBackupPayload(...args)) as { payload: unknown })
.payload,
),
isDeferredRows: () => false,
}));
// The envelope is exercised end-to-end in
// `src/lib/export/__tests__/backup-blob.test.ts`; here it stays transparent so
// the stored bytes can be read back as JSON.
vi.mock("@/lib/export/backup-blob", () => ({
packBackupBlob: mocks.packBlob,
packBackupBlobStreaming: async (
produce: (write: (chunk: string) => Promise<void>) => Promise<void>,
) => {
let out = "";
await produce(async (chunk) => {
out += chunk;
});
return mocks.packBlob(out);
},
}));
vi.mock("@/lib/logging/background", () => ({
@@ -95,7 +101,12 @@ describe("handleDataBackup canonical DR payload", () => {
expect(mocks.buildFullBackupPayload).toHaveBeenCalledWith(
prisma,
"user-dr",
{ purpose: "disaster-recovery" },
// `deferBulk` is the writer's own ask: it declares the unbounded tables
// rather than reading them, and pulls their rows through itself.
expect.objectContaining({
purpose: "disaster-recovery",
deferBulk: true,
}),
);
expect(mocks.upsert).toHaveBeenCalledOnce();
const encrypted = mocks.upsert.mock.calls[0]![0].create.data as string;
+26 -7
View File
@@ -6,8 +6,11 @@
*/
import { type Job } from "pg-boss";
import { recordError } from "@/lib/jobs/worker-status";
import { buildFullBackupJson } from "@/lib/export/full-backup-payload";
import { packBackupBlob } from "@/lib/export/backup-blob";
import { packBackupBlobStreaming } from "@/lib/export/backup-blob";
import {
BackupHeapBudgetExceededError,
streamFullBackupJson,
} from "@/lib/export/full-backup-stream";
import { jobDone, jobFailed, type JobOutcome } from "@/lib/jobs/job-outcome";
import { withBackgroundEvent } from "@/lib/logging/background";
import {
@@ -79,14 +82,19 @@ export async function handleDataBackup(
let backed = 0;
let usersFailed = 0;
let heapBudgetTrips = 0;
let largestBlobBytes = 0;
for (const user of users) {
try {
// Compressed, then encrypted (the record contains sensitive health
// information). Both legs live in `packBackupBlob`, which is also
// what the restore path reads back through.
const encryptedBackup = packBackupBlob(
await buildFullBackupJson(prisma, user.id, {
// Streamed, compressed, then encrypted (the record contains
// sensitive health information). Nothing between the database rows
// and this string exists as a whole: the payload goes into gzip a
// page at a time and the cipher consumes gzip's output as it comes,
// because a materialised copy of a large record is what used to take
// the process down. Both legs live in `packBackupBlobStreaming`,
// which is also what the restore path reads back through.
const encryptedBackup = await packBackupBlobStreaming((write) =>
streamFullBackupJson(prisma, user.id, write, {
purpose: "disaster-recovery",
}),
);
@@ -114,7 +122,14 @@ export async function handleDataBackup(
// One user's payload failing is that user's problem, not the pass's:
// it rides out as a count so the weekly run is not retried for the
// whole cohort.
//
// A record too large for this container's heap lands here too, and
// that is the whole point of the budget the writer enforces: before
// it existed the same condition was an uncatchable V8 abort that
// restarted the instance, so one account's size was a denial of
// service on every other account on the host.
usersFailed++;
if (err instanceof BackupHeapBudgetExceededError) heapBudgetTrips++;
evt.addWarning(`Failed for user ${user.id}: ${err}`);
}
}
@@ -122,6 +137,9 @@ export async function handleDataBackup(
// The stored size is the one number that says whether this pass is
// heading back towards the wall it hit before: it tracks the record.
evt.addMeta("data_backup_largest_blob_bytes", largestBlobBytes);
// How many accounts the writer stopped rather than let the process die.
// Non-zero means this host needs more memory, not a retry.
evt.addMeta("data_backup_heap_budget_trips", heapBudgetTrips);
evt.setBackground({
task_name: "job.data_backup",
result: { backed, total: users.length },
@@ -130,6 +148,7 @@ export async function handleDataBackup(
backed,
total: users.length,
users_failed: usersFailed,
heap_budget_trips: heapBudgetTrips,
});
} catch (err) {
evt.setError(err);
@@ -1226,3 +1226,55 @@ describe("streamParseExportXml — cycle module off", () => {
expect(result.totals.recordsRead).toBe(2);
});
});
/**
* `ImportJob.exportedAt` — the instant the Health app stamped on the
* archive. The status endpoint and the published contract have both
* promised it since v1.4.34, but `<ExportDate>` sat on the parser's
* ignore list, so the column could only ever answer null. These pin the
* capture: remove the `onExportDate` call from the parser and the first
* case goes red; drop the NaN guard and the third does.
*/
describe("streamParseExportXml — the archive's own export stamp", () => {
async function parseFor(xml: string) {
const tmp = mkdtempSync(join(tmpdir(), "healthlog-parser-export-date-"));
const xmlPath = join(tmp, "export.xml");
writeFileSync(xmlPath, xml);
const stamps: Date[] = [];
await streamParseExportXml({
xmlPath,
userId: "user-1",
userTimezone: "Europe/Berlin",
// eslint-disable-next-line @typescript-eslint/no-explicit-any
prisma: makeFakePrisma() as any,
onExportDate: (exportedAt) => {
stamps.push(exportedAt);
},
});
return stamps;
}
it("hands the caller the instant Apple wrote on the export", async () => {
const stamps = await parseFor(tinyExportXml());
expect(stamps).toHaveLength(1);
// "2026-05-15 14:32:01 +0200" — the fixture's own stamp.
expect(stamps[0]?.toISOString()).toBe("2026-05-15T12:32:01.000Z");
});
it("stays silent for an archive that carries no stamp", async () => {
const stamps = await parseFor(
`<?xml version="1.0" encoding="UTF-8"?>\n<HealthData locale="en_US"></HealthData>`,
);
expect(stamps).toEqual([]);
});
it("stays silent rather than guessing at an unreadable stamp", async () => {
const stamps = await parseFor(
`<?xml version="1.0" encoding="UTF-8"?>\n<HealthData locale="en_US">\n <ExportDate value="not a date"/>\n</HealthData>`,
);
expect(stamps).toEqual([]);
});
});
@@ -1,6 +1,7 @@
import { describe, it, expect } from "vitest";
import { resolveToCanonicalUnit } from "../unit-aliases";
import { MGDL_PER_MMOL, mgdlToMmol } from "@/lib/glucose";
describe("resolveToCanonicalUnit", () => {
it("passes the canonical unit through case-insensitively", () => {
@@ -53,10 +54,17 @@ describe("resolveToCanonicalUnit", () => {
});
});
it("converts glucose mmol/L to canonical mg/dL", () => {
it("converts glucose mmol/L to canonical mg/dL on the app's one factor", () => {
// This used to assert 18.016 while every display path divided by
// 18.0182, so a reading imported as mmol/L and read back as mmol/L did
// not return the number it went in as. Both ends take the same factor
// now, and the assertion names it rather than repeating a literal —
// pinning the constant twice is how the two drifted apart.
const out = resolveToCanonicalUnit("BLOOD_GLUCOSE", 5.3, "mmol/L");
expect(out?.unit).toBe("mg/dL");
expect(out?.value).toBeCloseTo(5.3 * 18.016, 3);
expect(out?.value).toBeCloseTo(5.3 * MGDL_PER_MMOL, 3);
// The round trip a mmol/L reader actually experiences.
expect(mgdlToMmol(out?.value ?? 0)).toBe(5.3);
});
it("refuses an unrecognised or type-mismatched unit (never mis-stores)", () => {
@@ -307,6 +307,13 @@ export interface StreamParseInput {
* continues even if the hook throws.
*/
onProgress?: (snapshot: ImportJobProgress) => Promise<void> | void;
/**
* Called once, mid-run, with the instant Apple stamped on the archive
* (`<ExportDate value="..."/>`, the first child of `<HealthData>`).
* Never called for an archive that omits the element or writes an
* unreadable value. Best-effort, like `onProgress`.
*/
onExportDate?: (exportedAt: Date) => Promise<void> | void;
/**
* Optional override of the spot-row flush batch size. Defaults to
* `SPOT_FLUSH_BATCH`. Lower values make the upsert path more
@@ -339,6 +346,7 @@ export async function streamParseExportXml(
userTimezone,
prisma,
onProgress,
onExportDate,
spotBatchSize = SPOT_FLUSH_BATCH,
workoutBatchSize = WORKOUT_FLUSH_BATCH,
} = input;
@@ -775,6 +783,21 @@ export async function streamParseExportXml(
let pendingError: Error | null = null;
let currentWorkout: PreparedWorkout | null = null;
// Apple stamps the archive with its own export instant as the first
// child of `<HealthData>`. The element used to be on the ignore list,
// so `ImportJob.exportedAt` — a column the status endpoint and the
// published contract both promise — could only ever answer null. The
// value is captured in the synchronous tag handler and handed to
// `onExportDate` from the async drain, so the worker can persist it
// while the run is still going rather than only on the terminal write.
let pendingExportDate: Date | null = null;
const flushExportDate = async (): Promise<void> => {
const stamped = pendingExportDate;
if (!stamped) return;
pendingExportDate = null;
if (onExportDate) await onExportDate(stamped);
};
parser.onerror = (err) => {
pendingError = err instanceof Error ? err : new Error(String(err));
};
@@ -983,8 +1006,17 @@ export async function streamParseExportXml(
return;
}
if (name === "ExportDate") {
// `value` carries Apple's own format ("2026-05-15 14:32:01 +0200"),
// the same shape every `Record` start/end date uses. An archive
// without the element, or with a value the runtime cannot read,
// leaves the column null rather than guessing at an instant.
const stamped = new Date(attrs.value ?? "");
if (!Number.isNaN(stamped.getTime())) pendingExportDate = stamped;
return;
}
if (
name === "ExportDate" ||
name === "Me" ||
name === "Correlation" ||
name === "ActivitySummary" ||
@@ -1049,9 +1081,11 @@ export async function streamParseExportXml(
}
// Drain any batches the parser filled to keep memory bounded.
const shouldFlush =
pendingExportDate !== null ||
spotBatch.length >= spotBatchSize ||
workoutBatch.length >= workoutBatchSize;
const drain = async (): Promise<void> => {
await flushExportDate();
if (spotBatch.length >= spotBatchSize) await flushSpotBatch();
if (workoutBatch.length >= workoutBatchSize)
await flushWorkoutBatch();
@@ -1090,6 +1124,10 @@ export async function streamParseExportXml(
await pipeline(readable, sink);
if (pendingError) throw pendingError;
// An archive small enough to arrive in a single chunk can finish
// before any drain ran; the stamp still has to reach the caller.
await flushExportDate();
await emitProgress("upserting");
// Drain any partial spot/workout batches that did not hit the flush
// threshold during the parse.
+2 -4
View File
@@ -17,9 +17,7 @@
* Pure + side-effect-free; safe to import from an API route or a script.
*/
import { getUnitForType } from "@/lib/validations/measurement";
/** Glucose mmol/L → mg/dL. */
const GLUCOSE_MMOL_TO_MGDL = 18.016;
import { MGDL_PER_MMOL } from "@/lib/glucose";
/** Exact pound → kilogram (1 lb = 0.45359237 kg). */
const LB_TO_KG = 0.45359237;
/** Exact inch → centimetre (1 in = 2.54 cm). */
@@ -87,7 +85,7 @@ export function resolveToCanonicalUnit(
// Glucose: mmol/L → mg/dL.
if (type === "BLOOD_GLUCOSE" && MMOL_ALIASES.has(lower)) {
return { value: value * GLUCOSE_MMOL_TO_MGDL, unit: canonical };
return { value: value * MGDL_PER_MMOL, unit: canonical };
}
// Body mass: lb → kg.
@@ -1030,7 +1030,6 @@ export const dashboardSnapshotResponse = z
gender: z.enum(["MALE", "FEMALE", "OTHER"]).nullable(),
glucoseUnit: z.string().nullable(),
onboardingTourCompleted: z.boolean(),
greetingHour: z.number().int(),
}),
layout: z.record(z.string(), z.unknown()),
// v1.7.0 — full 27-id widget catalogue (16 server-known + 11
+65
View File
@@ -26,6 +26,7 @@ import {
injectionSitePrefsPatchSchema,
labsLocalOcrPatchSchema,
unitPreferencePatchSchema,
glucoseUnitPatchSchema,
useCentralCodexPatchSchema,
} from "@/lib/validations/user-prefs";
import {
@@ -555,6 +556,16 @@ const unitPreferenceResponse = z
.object({ unitPreference: z.enum(["metric", "imperial"]) })
.meta({ id: "UnitPreferenceResponse" });
const glucoseUnitPatchRequest = glucoseUnitPatchSchema.meta({
id: "GlucoseUnitPatchRequest",
description:
"Which unit blood-glucose readings are rendered in. Presentation only — every stored reading is mg/dL either way.",
});
const glucoseUnitResponse = z
.object({ glucoseUnit: z.enum(["mg/dL", "mmol/L"]) })
.meta({ id: "GlucoseUnitResponse" });
const documentsAutoAiReadPatchRequest = documentsAutoAiReadPatchSchema.meta({
id: "DocumentsAutoAiReadPatchRequest",
description:
@@ -1577,6 +1588,60 @@ export const profilePaths: NonNullable<ZodOpenApiObject["paths"]> = {
},
},
},
"/api/auth/me/glucose-unit": {
get: {
tags: ["Auth"],
summary: "Read the blood-glucose display unit",
description:
"Which unit blood-glucose readings are rendered in — `mg/dL` or `mmol/L`. A separate choice from the metric/imperial preference, because metric countries are split on which one they read glucose in. Canonical storage is mg/dL on every reading and every ingest path; this changes presentation only. Any stored value other than `mmol/L` resolves to `mg/dL`, including a null on an account that never set it.",
responses: {
"200": {
description: "The resolved unit.",
content: {
"application/json": {
schema: dataEnvelope(
glucoseUnitResponse,
"GetGlucoseUnitEnvelope",
),
},
},
},
...stdResponses,
},
},
patch: {
tags: ["Auth"],
summary: "Set the blood-glucose display unit",
description:
"Hard-set, idempotent, audit-logged. Rate-limited 60 / min per user.\n\n" +
"Nothing already stored is converted or reinterpreted: readings stay mg/dL, and a value typed into the entry form in mmol/L is inverted to mg/dL before it is written.\n\n" +
"The body is capped at 1 KB and must carry `Content-Type: application/json`. Anything larger is 413, a wrong content type is 415, and a body that is not JSON at all is 400 — the same three the other scalars in this group answer.",
requestBody: {
required: true,
content: {
"application/json": { schema: glucoseUnitPatchRequest },
},
},
responses: {
"200": {
description: "The resolved next unit.",
content: {
"application/json": {
schema: dataEnvelope(
glucoseUnitResponse,
"PatchGlucoseUnitEnvelope",
),
},
},
},
"413": {
description: "Request body exceeds 1024 bytes. Nothing was written.",
content: { "application/json": { schema: errorEnvelope } },
},
...stdResponses,
},
},
},
"/api/auth/me/documents-auto-ai-read": {
get: {
tags: ["Auth"],
+9
View File
@@ -31,6 +31,15 @@ export const authKeys = {
* without a manual reload.
*/
userUnitPreference: () => ["user", "unit-preference"] as const,
/**
* The mg/dL vs mmol/L glucose display control behind
* `GET /api/auth/me/glucose-unit`. A separate axis from the
* metric/imperial preference above — metric countries are split on
* which one they read glucose in — so it carries its own key. The
* PATCH also invalidates `authMe()`, since every glucose surface
* resolves the unit from the account payload rather than from here.
*/
userGlucoseUnit: () => ["user", "glucose-unit"] as const,
passkeys: () => ["passkeys"] as const,
+11
View File
@@ -20,6 +20,17 @@ export const unitPreferencePatchSchema = z.object({
unitPreference: z.enum(["metric", "imperial"]),
});
/**
* `PATCH /api/auth/me/glucose-unit` — mg/dL vs mmol/L display branch.
*
* Not the metric/imperial preference: the two split along different lines.
* Germany is metric and reads glucose in mg/dL; the UK and the Nordics read
* it in mmol/L. Canonical storage stays mg/dL either way.
*/
export const glucoseUnitPatchSchema = z.object({
glucoseUnit: z.enum(["mg/dL", "mmol/L"]),
});
/** `PATCH /api/auth/me/documents-auto-ai-read` — the auto-read opt-in. */
export const documentsAutoAiReadPatchSchema = z.object({
documentsAutoAiRead: z.boolean(),
+37 -7
View File
@@ -66,7 +66,8 @@ import { buildFullBackupPayload } from "@/lib/export/full-backup-payload";
import { UNREADABLE_EXPORT_MARKER } from "@/lib/export/unreadable-marker";
import { decryptNoteFromBytes } from "@/lib/labs/store";
import { decryptContextFromBytes } from "@/lib/labs/biomarker-store";
import { packBackupBlob } from "@/lib/export/backup-blob";
import { packBackupBlobStreaming } from "@/lib/export/backup-blob";
import { streamFullBackupJson } from "@/lib/export/full-backup-stream";
import { TWO_ENDED_MODELS, type TwoEndedModel } from "@/lib/export/backup-plan";
import { POST } from "@/app/api/admin/backups/[id]/restore/route";
@@ -1252,10 +1253,36 @@ describe("every model the plan claims two-ended survives a real restore", () =>
"compare nothing against nothing and pass",
).toEqual([]);
// Pinned so the two writers below describe the same instant and can be
// compared byte for byte; `exportedAt` otherwise defaults to `new Date()`.
const exportedAt = new Date("2026-08-01T00:00:00.000Z");
const { payload } = await buildFullBackupPayload(prisma, OWNER_ID, {
purpose: "disaster-recovery",
exportedAt,
});
// The writer the weekly job actually uses never builds this payload: it
// pulls the unbounded tables through page by page and serialises each row
// straight into the cipher, because materialising them is what used to
// exhaust the heap. Two writers is two chances to drift, so the file it
// produces is pinned against the one above — every per-table assertion
// below then covers BOTH, since the row that gets restored comes from the
// streamed bytes.
let streamedJson = "";
await streamFullBackupJson(
prisma,
OWNER_ID,
(chunk) => {
streamedJson += chunk;
},
{ purpose: "disaster-recovery", exportedAt },
);
expect(
streamedJson,
"the streaming writer and the materialising builder disagree; the " +
"weekly backup no longer contains what this test proves restorable",
).toBe(JSON.stringify(payload));
// The tombstone has to be IN the file. A disaster-recovery payload that
// carries only live rows is smaller and cheaper and wrong: the restore
// wipes the account first, so a deletion that is not in the file is a
@@ -1292,12 +1319,15 @@ describe("every model the plan claims two-ended survives a real restore", () =>
data: {
userId: OWNER_ID,
type: "TWO_ENDED_ROUND_TRIP",
// The envelope the weekly worker actually writes: gzip, then
// encrypt. The three cases further down deliberately keep the plain
// `encrypt(json)` form, which is what every row written before this
// envelope existed looks like — an operator's newest usable copy may
// well be one of those, so both arms have to reach the restore route.
data: packBackupBlob(JSON.stringify(payload)),
// The envelope the weekly worker actually writes: stream the JSON
// through gzip and the cipher without ever holding a whole copy. The
// three cases further down deliberately keep the plain `encrypt(json)`
// form, which is what every row written before any of this existed
// looks like — an operator's newest usable copy may well be one of
// those, so every arm has to reach the restore route.
data: await packBackupBlobStreaming(async (write) => {
await write(streamedJson);
}),
},
});
const response = await POST(
@@ -0,0 +1,231 @@
/**
* The weekly backup, run against a record big enough to have killed the
* process, with its live memory measured rather than assumed.
*
* What this pins. The `data-backup` job used to build the whole payload as an
* object graph and then `JSON.stringify` it, so both the graph and the string
* were resident at once and both scaled with the record. On a seeded account
* of 445 000 measurements under `--max-old-space-size=450` that is
* `FATAL ERROR: Reached heap limit` about thirty seconds in — and because the
* job shares the app process, one account's size restarted the instance for
* everybody on it.
*
* Why it measures the way it does. `process.memoryUsage().heapUsed` on its own
* is not a measurement: V8 lets garbage float in proportion to the heap limit,
* and a test fork's limit is several times a container's, so the same code
* "peaks" at wildly different numbers depending on who is running it. Every
* reading below is taken after a forced collection, so what is compared is
* what each writer HOLDS.
*
* The two halves are the whole point. One says the streaming writer stays
* inside a budget; the other says the materialising builder does not fit that
* budget on the same fixture in the same process. Without the second, the
* budget could be any number at all and the first would still pass — green
* because nothing was measured rather than because something was proved.
*
* The fixture is seeded in one `INSERT … generate_series` rather than through
* Prisma. 120 000 rows through the client would dominate the runtime and prove
* nothing about the writer.
*/
import v8 from "node:v8";
import vm from "node:vm";
import { beforeAll, afterAll, describe, expect, it } from "vitest";
import {
packBackupBlobStreaming,
unpackBackupBlob,
} from "@/lib/export/backup-blob";
import {
BackupHeapBudgetExceededError,
streamFullBackupJson,
} from "@/lib/export/full-backup-stream";
import { buildFullBackupPayload } from "@/lib/export/full-backup-payload";
import { getPrismaClient, truncateAllTables } from "./setup";
const OWNER_ID = "backup-streaming-owner";
const MEASUREMENT_ROWS = 120_000;
const MOOD_ROWS = 8_000;
/**
* What the streaming writer may hold on top of the process's own baseline.
*
* Measured, not chosen: on this fixture the streaming writer holds 16 MB — a
* page of rows and the base64 answer so far — and materialising the same
* fixture holds 168 MB, the object graph and the document at once. The budget
* sits with a wide margin either side of it, which is what makes it a test
* rather than a coin toss.
*/
const STREAM_BUDGET_BYTES = 48 * 1024 * 1024;
const prisma = getPrismaClient();
/**
* A collection this process can ask for, without the runner having to be
* started with `--expose-gc`. The flag is turned on, the binding is pulled out
* of a throwaway context, and the flag is turned back off — so nothing about
* how the rest of the suite runs changes.
*/
const forceGc = ((): (() => void) => {
v8.setFlagsFromString("--expose-gc");
const gc = vm.runInNewContext("gc") as () => void;
v8.setFlagsFromString("--no-expose-gc");
return gc;
})();
/** Heap held after a forced collection. Garbage is not a measurement. */
function liveHeapBytes(): number {
// Twice: the first pass frees the objects, the second collects what the
// first pass's own bookkeeping released.
forceGc();
forceGc();
return process.memoryUsage().heapUsed;
}
async function seedLargeRecord(): Promise<void> {
await prisma.user.create({
data: { id: OWNER_ID, username: "backup-streaming-owner" },
});
// One statement, one round trip. Every row carries a note on one row in
// forty and ciphertext on one in twenty-five so the base64 arm of the
// serialiser is exercised at scale, and one in two hundred is a tombstone
// because the delta sync reads exactly those and a backup that drops them
// resurrects deleted readings on the next device sync.
await prisma.$executeRawUnsafe(
`INSERT INTO measurements (
id, user_id, type, value, unit, source, measured_at, notes,
notes_encrypted, external_id, created_at, updated_at, sync_version,
deleted_at)
SELECT
'sm' || lpad(g::text, 10, '0'),
$1,
'PULSE'::measurement_type,
60 + (g % 40),
'bpm',
'APPLE_HEALTH'::measurement_source,
timestamp '2019-01-01 00:00:00' + (g * interval '30 seconds'),
CASE WHEN g % 40 = 0 THEN 'a note recorded with reading ' || g END,
CASE WHEN g % 25 = 0
THEN decode(md5(g::text) || md5((g + 1)::text), 'hex') END,
'stream-' || g,
timestamp '2019-01-01 00:00:00' + (g * interval '30 seconds'),
timestamp '2019-01-01 00:00:00' + (g * interval '30 seconds'),
1,
CASE WHEN g % 200 = 0
THEN timestamp '2026-01-01 00:00:00' + (g * interval '1 second') END
FROM generate_series(1, ${MEASUREMENT_ROWS}) AS g`,
OWNER_ID,
);
await prisma.$executeRawUnsafe(
`INSERT INTO mood_entries (
id, user_id, date, mood, score, source, mood_logged_at, synced_at,
created_at, updated_at, tz, note, sync_version, deleted_at)
SELECT
'sd' || lpad(g::text, 10, '0'),
$1,
to_char(timestamp '2019-01-01' + (g * interval '1 hour'), 'YYYY-MM-DD'),
'okay', 3, 'MOODLOG',
timestamp '2019-01-01' + (g * interval '1 hour'),
now(), now(), now(), 'Europe/Berlin',
CASE WHEN g % 3 = 0 THEN 'a journal line about day ' || g END,
1,
CASE WHEN g % 150 = 0 THEN now() END
FROM generate_series(1, ${MOOD_ROWS}) AS g`,
OWNER_ID,
);
}
describe("weekly backup under a memory budget", () => {
beforeAll(async () => {
expect(
typeof forceGc,
"without a real collection every reading in this file measures " +
"uncollected garbage and nothing here can fail",
).toBe("function");
await truncateAllTables(prisma);
await seedLargeRecord();
}, 240_000);
afterAll(async () => {
await truncateAllTables(prisma);
});
it("writes a restorable blob while holding a bounded amount of the record", async () => {
const baseline = liveHeapBytes();
let peakHeld = 0;
let chunks = 0;
const blob = await packBackupBlobStreaming(async (write) => {
const counts = await streamFullBackupJson(
prisma,
OWNER_ID,
async (chunk) => {
await write(chunk);
// Sampled rather than continuous: a forced collection per chunk
// would dominate the runtime, and one in forty still lands inside
// every phase of the walk.
if (chunks++ % 40 === 0) {
peakHeld = Math.max(peakHeld, liveHeapBytes() - baseline);
}
},
// The purpose the weekly job asks for: tombstones and ciphertext ride
// verbatim, which is the arm that has to fit in memory.
{ purpose: "disaster-recovery" },
);
expect(counts.measurements).toBe(MEASUREMENT_ROWS);
expect(counts.moodEntries).toBe(MOOD_ROWS);
});
peakHeld = Math.max(peakHeld, liveHeapBytes() - baseline);
expect(
peakHeld,
`the streaming writer held ${Math.round(peakHeld / 1024 / 1024)} MB of ` +
"a record it is supposed to pass through a page at a time",
).toBeLessThan(STREAM_BUDGET_BYTES);
// A blob that writes but does not read is worse than none.
const restored = JSON.parse(unpackBackupBlob(blob)) as {
measurements: Array<{ deletedAt: string | null }>;
moodEntries: unknown[];
};
expect(restored.measurements).toHaveLength(MEASUREMENT_ROWS);
expect(restored.moodEntries).toHaveLength(MOOD_ROWS);
// Tombstones ride along, or the next device sync resurrects them.
expect(
restored.measurements.filter((row) => row.deletedAt !== null).length,
).toBe(Math.floor(MEASUREMENT_ROWS / 200));
}, 300_000);
it("would not fit that budget if the record were materialised", async () => {
const baseline = liveHeapBytes();
const { payload } = await buildFullBackupPayload(prisma, OWNER_ID, {
purpose: "disaster-recovery",
});
const json = JSON.stringify(payload);
// Both are deliberately still reachable at the reading: holding the
// graph and the document at the same time is exactly the shape that
// exhausted the container.
const held = liveHeapBytes() - baseline;
expect(json.length).toBeGreaterThan(0);
expect(payload.measurements).toHaveLength(MEASUREMENT_ROWS);
expect(
held,
"materialising this fixture no longer costs what the budget in this " +
"file assumes; re-measure the budget rather than widening it",
).toBeGreaterThan(STREAM_BUDGET_BYTES * 2);
}, 300_000);
it("fails as a job rather than as a process when the record does not fit", async () => {
// The budget the writer enforces on itself is the reason an oversized
// account is now a failed backup for that account instead of a restart
// for every account on the host.
await expect(
packBackupBlobStreaming((write) =>
streamFullBackupJson(prisma, OWNER_ID, write, {
purpose: "disaster-recovery",
heapCeilingBytes: 1,
}),
),
).rejects.toBeInstanceOf(BackupHeapBudgetExceededError);
}, 120_000);
});
@@ -18,6 +18,8 @@ import { beforeEach, describe, expect, it, vi } from "vitest";
import { cookieJar } from "./mock-next-headers";
import { getPrismaClient, truncateAllTables } from "./setup";
import { MGDL_PER_MMOL } from "@/lib/glucose";
const USER = "user-csv-glucose";
vi.mock("next/headers", async () => {
@@ -111,7 +113,10 @@ describe("POST /api/import/csv — contextless glucose (real Postgres)", () => {
expect(rows[0].glucoseContext).toBeNull();
expect(rows[0].source).toBe("IMPORT");
expect(rows[0].unit).toBe("mg/dL");
expect(rows[0].value).toBeCloseTo(95.4848, 3);
// Derived from the shared factor rather than pinned, for the same reason
// as below: a literal here re-pins whichever copy of the conversion the
// import path happened to use when it was written.
expect(rows[0].value).toBeCloseTo(5.3 * MGDL_PER_MMOL, 3);
expect(rows[0].externalId).toBe("sensor-1");
expect(rows[0].measuredAt.toISOString()).toBe("2024-04-03T02:15:00.000Z");
});
@@ -141,7 +146,10 @@ describe("POST /api/import/csv — contextless glucose (real Postgres)", () => {
});
expect(rows).toHaveLength(1);
expect(rows[0].glucoseContext).toBeNull();
expect(rows[0].value).toBeCloseTo(106.2944, 3);
// Derived, not pinned: this carried the import path's own copy of the
// conversion factor, so unifying the two conversions left it asserting a
// number the code no longer produces.
expect(rows[0].value).toBeCloseTo(5.9 * MGDL_PER_MMOL, 3);
});
it("keeps a named context, and still refuses one that is not in the enum", async () => {
@@ -0,0 +1,196 @@
/**
* Key rotation over the two ciphertext columns that carry no `*Encrypted` in
* their name, against real Postgres.
*
* `DataBackup.data` is the one that matters. It holds the whole account,
* compressed then encrypted, and it sat outside the rotation registry until
* v1.38.6 — the script reported zero rows remaining because it never looked,
* and the runbook reads a zero as permission to retire the previous key. This
* pins the fix at the level the bug lived at: seed real rows under an old key,
* rotate, and read the backup back.
*
* Both stored envelopes are seeded on purpose. A row written before
* `packBackupBlob` existed is `encrypt(json)`; one written after is the
* `HLZ1:`-prefixed gzip form. Rotation re-seals the ciphertext without ever
* looking at the plaintext, so both come back byte-identical — and so will
* whatever envelope a later writer introduces.
*
* `IdempotencyKey.responseBody` rides along as the disposable case: a row
* under a key the deployment no longer configures is DELETED rather than
* counted as an error, because the value is a 24h cache entry and leaving it
* behind only guarantees the next replay gets a body it cannot parse.
*/
import { beforeEach, describe, expect, it } from "vitest";
process.env.ENCRYPTION_KEYS = JSON.stringify({
v1: "1".repeat(64),
v2: "2".repeat(64),
});
process.env.ENCRYPTION_ACTIVE_KEY_ID = "v1";
delete process.env.ENCRYPTION_KEY;
import { _resetCryptoCacheForTests, encrypt, extractKeyId } from "@/lib/crypto";
import {
ENCRYPTED_COLUMNS,
encryptedColumnKey,
type EncryptedColumn,
} from "@/lib/crypto/encrypted-columns";
import {
rotateColumn,
type CorpusClient,
} from "@/lib/crypto/encryption-corpus";
import { packBackupBlob, unpackBackupBlob } from "@/lib/export/backup-blob";
import { getPrismaClient, truncateAllTables } from "./setup";
const TEST_USER_ID = "user-rotation-blobs";
const BACKUP_JSON = JSON.stringify({
schemaVersion: 7,
measurements: [{ type: "WEIGHT", value: 80.4, unit: "kg" }],
note: "üäö — a non-ASCII payload, so a mangled round-trip shows up",
});
function column(model: string, field: string): EncryptedColumn {
const col = ENCRYPTED_COLUMNS.find(
(c) => c.model === model && c.field === field,
);
if (!col) throw new Error(`${model}.${field} is not registered`);
return col;
}
/** Encrypt while `keyId` is active, then restore the caller's active id. */
function underKey(keyId: string, plaintext: string, pack = false): string {
const previous = process.env.ENCRYPTION_ACTIVE_KEY_ID;
process.env.ENCRYPTION_ACTIVE_KEY_ID = keyId;
_resetCryptoCacheForTests();
const out = pack ? packBackupBlob(plaintext) : encrypt(plaintext);
process.env.ENCRYPTION_ACTIVE_KEY_ID = previous;
_resetCryptoCacheForTests();
return out;
}
beforeEach(async () => {
process.env.ENCRYPTION_KEYS = JSON.stringify({
v1: "1".repeat(64),
v2: "2".repeat(64),
});
process.env.ENCRYPTION_ACTIVE_KEY_ID = "v2";
_resetCryptoCacheForTests();
await truncateAllTables(getPrismaClient());
await getPrismaClient().user.create({
data: {
id: TEST_USER_ID,
username: "rotation-blobs",
email: "rotation-blobs@example.test",
timezone: "Europe/Berlin",
},
});
});
describe("key rotation over the non-suffixed ciphertext columns", () => {
it("rotates DataBackup.data in both stored envelopes and keeps it readable", async () => {
const prisma = getPrismaClient();
const gzipped = await prisma.dataBackup.create({
data: {
userId: TEST_USER_ID,
type: "WEEKLY_AUTO",
data: underKey("v1", BACKUP_JSON, true),
},
});
// The pre-envelope shape: `encrypt(json)` with no `HLZ1:` marker.
const plainEnvelope = await prisma.dataBackup.create({
data: {
userId: TEST_USER_ID,
type: "MANUAL_UPLOAD_1",
data: underKey("v1", BACKUP_JSON),
},
});
const result = await rotateColumn(
{ dataBackup: prisma.dataBackup } as unknown as CorpusClient,
column("DataBackup", "data"),
);
expect(result.scanned).toBe(2);
expect(result.rotated).toBe(2);
expect(result.errors).toBe(0);
expect(result.dropped).toBe(0);
for (const id of [gzipped.id, plainEnvelope.id]) {
const row = await prisma.dataBackup.findUniqueOrThrow({ where: { id } });
expect(extractKeyId(row.data)).toBe("v2");
expect(unpackBackupBlob(row.data)).toBe(BACKUP_JSON);
}
// Idempotent: a second pass finds nothing left to do.
const again = await rotateColumn(
{ dataBackup: prisma.dataBackup } as unknown as CorpusClient,
column("DataBackup", "data"),
);
expect(again.scanned).toBe(2);
expect(again.rotated).toBe(0);
});
it("survives dropping the retired key once rotation has run", async () => {
const prisma = getPrismaClient();
const backup = await prisma.dataBackup.create({
data: {
userId: TEST_USER_ID,
type: "WEEKLY_AUTO",
data: underKey("v1", BACKUP_JSON, true),
},
});
await rotateColumn(
{ dataBackup: prisma.dataBackup } as unknown as CorpusClient,
column("DataBackup", "data"),
);
// What the runbook tells the operator to do next: retire v1 entirely.
process.env.ENCRYPTION_KEYS = JSON.stringify({ v2: "2".repeat(64) });
_resetCryptoCacheForTests();
const row = await prisma.dataBackup.findUniqueOrThrow({
where: { id: backup.id },
});
expect(unpackBackupBlob(row.data)).toBe(BACKUP_JSON);
});
it("drops an unreadable idempotency row instead of failing the run", async () => {
const prisma = getPrismaClient();
const seed = async (key: string, body: string) =>
prisma.idempotencyKey.create({
data: {
userId: TEST_USER_ID,
key,
method: "POST",
path: "/api/measurements",
responseStatus: 201,
responseBody: body,
expiresAt: new Date(Date.now() + 86_400_000),
},
});
const readable = await seed("k-readable", underKey("v1", '{"data":1}'));
// A key id the deployment no longer configures: fail-closed on decrypt.
const unreadable = await seed("k-unreadable", "v9.QUJDREVGR0hJSktM");
const result = await rotateColumn(
{ idempotencyKey: prisma.idempotencyKey } as unknown as CorpusClient,
column("IdempotencyKey", "responseBody"),
);
expect(result.rotated).toBe(1);
expect(result.dropped).toBe(1);
expect(result.errors).toBe(0);
const kept = await prisma.idempotencyKey.findUnique({
where: { id: readable.id },
});
expect(extractKeyId(kept!.responseBody)).toBe("v2");
expect(
await prisma.idempotencyKey.findUnique({ where: { id: unreadable.id } }),
).toBeNull();
});
it("keeps both columns in the registry the script and the job read", () => {
const keys = ENCRYPTED_COLUMNS.map(encryptedColumnKey);
expect(keys).toContain("DataBackup.data");
expect(keys).toContain("IdempotencyKey.responseBody");
});
});