tom/HealthLog
1
0 Fork 0
mirror of https://github.com/MBombeck/HealthLog.git synced 2026-11-08 12:32:43 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'main' into fix/ocr-review-date-validation

Browse Source
This commit is contained in:
Marc-André Bombeck authored and GitHub committed 2026-10-01 00:20:48 +02:00
2 parents eb7c3fafe4 c777e9e9e4
commit 02be4d3dff
61 files changed
+4387 -366
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/api/README.md
+7 -1
View File
@@ -66,7 +66,13 @@ before treating a warning as a regression.
`health:write`), audience-bound to `/mcp` and refused on REST writes;
- `POST /api/tokens/measurements` mints `measurements:write`, accepted by
`POST /api/measurements` and `POST /api/measurements/batch` only, on the
holder's own record only.
holder's own record only;
- `POST /api/tokens/workouts` mints `workouts:write`, accepted by
`POST /api/workouts/batch` only, on the holder's own record only; the
workouts it writes carry `source: EXTERNAL`;
- `POST /api/tokens/documents` mints `documents:write`, accepted by
`POST /api/documents/inbound` and `GET /api/documents/inbound/source`
only.
`POST /api/tokens` is **not** among them — that generic mint was removed in
v1.30.17. The path still lists (`GET`) and revokes (`DELETE /api/tokens/{id}`)
docs/api/openapi.yaml
+182 -23
View File
@@ -2134,10 +2134,10 @@ paths:
description: >-
Stamps the calling browser session as recently re-proved, so for the next five minutes it may run the actions
that ask for a fresh proof: GET /api/export/full-backup, POST /api/export/encrypted, POST /api/share-links, POST
/api/mcp/tokens, POST /api/tokens/measurements, POST /api/tokens/documents, and the admin backup download,
upload and restore, the admin data wipe and the admin password reset. Those answer 401 `auth.reproof.required`
with `meta.methods` (the proofs this account can give) when the session signed in or re-proved more than five
minutes ago.
/api/mcp/tokens, POST /api/tokens/measurements, POST /api/tokens/workouts, POST /api/tokens/documents, and the
admin backup download, upload and restore, the admin data wipe and the admin password reset. Those answer 401
`auth.reproof.required` with `meta.methods` (the proofs this account can give) when the session signed in or
re-proved more than five minutes ago.
The body takes the same shapes as POST /api/auth/step-up. On an account with a second factor only `totp`,
@@ -2977,10 +2977,10 @@ paths:
because that scope reached no ingest route while the pre-fail-closed default let it reach everything else. The
credentials that work are minted where they are scoped: the per-medication API-endpoint toggle, `POST
/api/mcp/tokens` for a connector, the sibling `POST /api/tokens/measurements` for third-party measurement
ingest, and `POST /api/tokens/documents` for document upload from another system. And unlike the revoke, this
read is NOT gated on the operator's instance-wide API switch: that switch governs the surfaces a token is for,
not a token's ability to authenticate, so tokens stay live while it is off and their owner has to be able to see
them.
ingest, `POST /api/tokens/workouts` for workout ingest from a bridge, and `POST /api/tokens/documents` for
document upload from another system. And unlike the revoke, this read is NOT gated on the operator's
instance-wide API switch: that switch governs the surfaces a token is for, not a token's ability to
authenticate, so tokens stay live while it is off and their owner has to be able to see them.
responses:
"200":
description: The caller's tokens, newest first.
@@ -3088,6 +3088,93 @@ paths:
application/json:
schema:
$ref: "#/components/schemas/ErrorEnvelope"
/api/tokens/workouts:
post:
tags:
- Auth
summary: Mint a workout-ingest token
description: >-
Mints a Bearer scoped to exactly `workouts:write` and audits the mint. THE RESPONSE CARRIES THE RAW TOKEN, once;
it is stored as an HMAC and no path re-reveals it.
**What it can do.** `POST /api/workouts/batch`, on its owner's own record, and nothing else. It is the door a
workout bridge pushes through: a watch vendor's sync relayed by a small service, or a script replaying an
exported history. Workouts it writes carry `source: EXTERNAL`, resolved from the credential rather than taken
from the body, and a batch in which any entry names a source is refused 422
(`workout.batch.source_not_permitted`) rather than relabelled. `EXTERNAL` is on no default source ladder, so
when the phone also recorded the same session the phone's copy is the one the workout list shows.
**What it cannot do.** List or read a workout, reach the measurement or document ingest routes, act on a shared
record, or mint another token. It is a separate scope from `measurements:write` on purpose: a token already
handed to a scale does not start accepting workouts, and a `measurements:write` token is refused 403 on the
workout batch. Every other route names no scope and refuses it 403.
Minting requires a COOKIE SESSION with a fresh proof, as for the measurement mint, for the reason it gives: a
day-lived native access token must not be able to leave behind a year-lived credential. Gated by the operator's
instance-wide API switch. Body capped at 16 KiB; 10 mints per user per minute, and at most 10 live workout
tokens held at once. Tokens appear in `GET /api/tokens` and are revoked at `DELETE /api/tokens/{id}`.
requestBody:
required: true
content:
application/json:
schema:
$ref: "#/components/schemas/CreateWorkoutTokenRequest"
responses:
"201":
description: Token minted. The `token` field is the only copy of the secret.
content:
application/json:
schema:
$ref: "#/components/schemas/CreateWorkoutTokenEnvelope"
"400":
description: Body is not parseable JSON.
content:
application/json:
schema:
$ref: "#/components/schemas/ErrorEnvelope"
"401": *a1
"403":
description: The operator has switched the API off instance-wide. Nothing was minted. A Bearer caller is refused 401
before the switch is read.
content:
application/json:
schema:
$ref: "#/components/schemas/ErrorEnvelope"
"409":
description: The account already holds 10 live workout tokens (`meta.errorCode` = `tokens.workouts.ceiling_reached`).
Nothing was minted. Revoke one at `DELETE /api/tokens/{id}`; only live tokens carrying `workouts:write`
count.
content:
application/json:
schema:
$ref: "#/components/schemas/ErrorEnvelope"
"413":
description: Body exceeds 16 KiB.
content:
application/json:
schema:
$ref: "#/components/schemas/ErrorEnvelope"
"415":
description: Content-Type is not `application/json`.
content:
application/json:
schema:
$ref: "#/components/schemas/ErrorEnvelope"
"422":
description: Validation failed. The envelope carries every offending issue.
content:
application/json:
schema:
$ref: "#/components/schemas/ErrorEnvelope"
"429":
description: More than 10 mints from this account in a minute. Nothing was minted.
content:
application/json:
schema:
$ref: "#/components/schemas/ErrorEnvelope"
/api/tokens/documents:
post:
tags:
@@ -4579,16 +4666,26 @@ paths:
tags:
- Measurements
summary: Typed workout batch ingest (v1.4.25 W16b)
description: 'Server-side ingest endpoint for HKWorkout records (iOS) and Withings activity rows (server-to-server). Up
to 100 workouts per call; nested route geometry (GeoJSON LineString) is capped at 20 000 points and stored in a
1:1 `WorkoutRoute` row keyed by `workoutId`. Idempotent via the `Idempotency-Key` header (replay window 24h).
description: >-
Server-side ingest endpoint for HKWorkout records (iOS) and Withings activity rows (server-to-server). Up to 100
workouts per call; nested route geometry (GeoJSON LineString) is capped at 20 000 points and stored in a 1:1
`WorkoutRoute` row keyed by `workoutId`. Idempotent via the `Idempotency-Key` header (replay window 24h).
Per-entry status (`inserted | duplicate | enriched | skipped`) lets the iOS sync cursor checkpoint accurately.
An entry matching a stored workout is first-write-wins for the workout row; when it carries a `samples` array
and the stored workout has no heart-rate series, the series is attached and the entry reports `enriched`
(repeatable: a workout that already has a series is a no-op). An entry whose `samples` array fails validation is
refused on its own with `reason: "invalid_samples"` — the rest of the batch still lands. The request body
ceiling is 5 MB enforced at the HTTP layer — clients above the ceiling receive a 413 with
`workout.batch.payload_too_large`.'
`workout.batch.payload_too_large`.
**Narrow token.** Also reachable with a `workouts:write` Bearer (minted at `POST /api/tokens/workouts`), the
supported way to push workouts from a bridge that is not the phone. This route is the scope's only one. Such a
credential writes only on its owner's own record; its workouts are attributed `EXTERNAL`, resolved from the
credential, and it may not name a source itself, so a batch in which any entry carries one, `MANUAL` and
`APPLE_HEALTH` alike, is refused 422 (`workout.batch.source_not_permitted`) and nothing is written. `EXTERNAL`
gives those writes their own `(userId, source, externalId)` namespace, so a bridge's ids can never collide with
the phone's. Every other narrow token, `measurements:write` included, is refused 403.
parameters:
- *a6
requestBody:
@@ -4618,7 +4715,14 @@ paths:
application/json:
schema:
$ref: "#/components/schemas/ErrorEnvelope"
"422": *a4
"422":
description: A narrow `workouts:write` credential sent a batch in which an entry names a source, any source
(`workout.batch.source_not_permitted`). Nothing was written; resending the batch unchanged is refused the
same way. Omit `source` and the server attributes the rows `EXTERNAL`.
content:
application/json:
schema:
$ref: "#/components/schemas/ErrorEnvelope"
"429": *a5
/api/devices:
post:
@@ -22919,6 +23023,21 @@ components:
permission array is a literal, so no request can widen it. `name` is what the token is listed under — name it
after the thing you are pasting it into, because that is what you will be reading when you decide whether to
revoke it. `expiresInDays` defaults to 365."
CreateWorkoutTokenRequest:
type: object
properties:
name:
type: string
minLength: 1
maxLength: 100
expiresInDays:
type: integer
minimum: 1
maximum: 365
required:
- name
description: "Mint a workout-ingest Bearer. No `scope` field: the endpoint mints one shape. `name` is what the token is
listed under; name it after the bridge it is pasted into. `expiresInDays` defaults to 365."
CreateDocumentTokenRequest:
type: object
properties:
@@ -31636,10 +31755,11 @@ components:
sharing.revoke.not_found, sharing.session.changed, sharing.switch.invalid,
sharing.switch.wrong_transport; source-priority — source-priority.body.invalid_json,
source-priority.body.invalid_shape; tokens — tokens.documents.ceiling_reached,
tokens.measurements.ceiling_reached; vaccination — vaccination.booster-invalid,
vaccination.booster-no-antigen, vaccination.booster-out-of-scope, vaccination.encounter-not-found,
vaccination.identity-required, vaccination.invalid, vaccination.link.invalid, vaccination.not-found,
vaccination.practitioner-not-found; workout — workout.batch.invalid, workout.batch.payload_too_large,
tokens.measurements.ceiling_reached, tokens.workouts.ceiling_reached; vaccination —
vaccination.booster-invalid, vaccination.booster-no-antigen, vaccination.booster-out-of-scope,
vaccination.encounter-not-found, vaccination.identity-required, vaccination.invalid,
vaccination.link.invalid, vaccination.not-found, vaccination.practitioner-not-found; workout —
workout.batch.invalid, workout.batch.payload_too_large, workout.batch.source_not_permitted,
workout.batch.too_large; (unprefixed) — about_me_conflict, ai_response_truncated, coach_prefs_conflict,
dashboard_layout_conflict, empty_file, health_score_config_conflict, insights_layout_conflict,
invalid_base_updated_at, json_carries_no_intake_time, json_not_an_array, medication_layout_conflict,
@@ -39650,6 +39770,42 @@ components:
- expiresAt
additionalProperties: false
description: The one response that carries a usable measurement-ingest Bearer value.
CreateWorkoutTokenEnvelope:
type: object
properties:
data:
$ref: "#/components/schemas/CreateWorkoutTokenResponse"
error:
type: "null"
meta:
type: object
properties:
requestId:
type: string
additionalProperties: false
required:
- data
- error
additionalProperties: false
CreateWorkoutTokenResponse:
type: object
properties:
token:
type: string
description: The raw `hlk_` Bearer, RETURNED EXACTLY ONCE. It is stored only as an HMAC-SHA256 hash and cannot be
retrieved later. Treat it as a secret.
name:
type: string
expiresAt:
type: string
format: date-time
pattern: ^(?:(?:\d\d[2468][048]|\d\d[13579][26]|\d\d0[48]|[02468][048]00|[13579][26]00)-02-29|\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\d|30)|(?:02)-(?:0[1-9]|1\d|2[0-8])))T(?:(?:[01]\d|2[0-3]):[0-5]\d(?::[0-5]\d(?:\.\d+)?)?(?:Z|([+-](?:[01]\d|2[0-3]):[0-5]\d)))$
required:
- token
- name
- expiresAt
additionalProperties: false
description: The one response that carries a usable workout-ingest Bearer value.
CreateDocumentTokenEnvelope:
type: object
properties:
@@ -40443,7 +40599,8 @@ components:
- EXTERNAL
description: Origin of the measurement. v1.4.23 added APPLE_HEALTH for the iOS HealthKit batch ingest path. v1.38.x
added EXTERNAL for rows pushed in under a narrow `measurements:write` Bearer — a scale, a watch bridge, a
home-automation rule. No client may name that value on a write; the server resolves it from the credential.
home-automation rule — and workouts pushed in under a narrow `workouts:write` Bearer carry it too. No client may
name that value on a write; the server resolves it from the credential.
SyncMeasurementTombstone:
type: object
properties:
@@ -40910,8 +41067,8 @@ components:
id:
type: string
description: Measurement row id; `sleep:<wake-day>` for a sleep night; `day:<local-day>` for a day bucket (`pulse` /
`glucose` beyond 90 days); `hour:<ISO start>` for an hour bucket (`pulse` within 90 days when the
window holds more than 10 000 readings).
`glucose` beyond 90 days); `hour:<ISO start>` for an hour bucket (`pulse` / `glucose` within 90 days
when the window holds more than 10 000 readings).
at:
type: string
format: date-time
@@ -40985,9 +41142,11 @@ components:
additionalProperties: false
description: "iOS-friendly per-kind time series: one point per reading (or per reconstructed night for `sleep`), an
explicit `unit` token, and a summary `stats` block. Sample-dense kinds are bucketed server-side: `pulse` and
`glucose` beyond 90 days per local day, and `pulse` within 90 days per local hour when the window holds more
than 10 000 readings (a bucket carries the mean as `value` and its range as `valueMin` / `valueMax`). `stats` is
always computed over the underlying readings, never over the buckets."
`glucose` beyond 90 days per local day, and `pulse` / `glucose` within 90 days per local hour when the window
holds more than 10 000 readings (a continuous glucose sensor, a watch recording heart rate every minute; a cuff
or fingerstick meter stays raw). A bucket carries the mean as `value`; a `pulse` bucket also carries its range
as `valueMin` / `valueMax`, and a `glucose` bucket is converted to the user's unit like a raw reading. `stats`
is always computed over the underlying readings, never over the buckets."
ListMeasurementsResponse:
type: object
properties:
messages/de.json
+12 -1
View File
@@ -533,7 +533,10 @@
"saveError": "Einnahme konnte nicht gespeichert werden.",
"emptyTitle": "Noch keine Medikamente angelegt",
"emptyDescription": "Lege zuerst ein Medikament mit Zeitfenster an, damit du die Einnahme protokollieren kannst.",
"emptyCta": "Medikament hinzufügen"
"emptyCta": "Medikament hinzufügen",
"noneActionableTitle": "Heute ist nichts einzunehmen",
"noneActionableDescription": "Deine Medikamente sind pausiert, sind nur dokumentiert, oder ihr Einnahmezeitraum hat noch nicht begonnen oder ist schon vorbei. Eine vergangene Einnahme kannst du weiterhin im Verlauf des Medikaments nachtragen.",
"noneActionableCta": "Zu den Medikamenten"
},
"quickEntryDiscard": {
"title": "Eintrag verwerfen?",
@@ -6172,6 +6175,13 @@
"scopeNote": "Es kann ausschließlich neue Werte in deine eigene Akte eintragen — deine Daten nicht lesen, nichts ändern oder löschen und kein weiteres Token erstellen. Werte werden als von einem externen Gerät gekennzeichnet, damit du sie später erkennst, und lassen sich jederzeit korrigieren.",
"createFailed": "Das Token konnte nicht erstellt werden. Bitte versuche es erneut."
},
"workoutsToken": {
"title": "Training-Tokens",
"description": "Erzeuge ein Token, damit eine Uhren-Brücke oder ein Sync-Skript deine Trainings eintragen kann.",
"detail": "Das Token wird nur einmal angezeigt.",
"scopeNote": "Es kann ausschließlich neue Trainings in deine eigene Akte eintragen. Es kann deine Daten nicht lesen, nichts ändern oder löschen und kein weiteres Token erstellen. Trainings, die darüber kommen, werden als von einem externen Gerät gekennzeichnet. Hat dein Telefon dieselbe Einheit aufgezeichnet, siehst du die Aufzeichnung des Telefons.",
"createFailed": "Das Token konnte nicht erstellt werden. Bitte versuche es erneut."
},
"documentsToken": {
"title": "Dokument-Tokens",
"description": "Erzeuge ein Token, damit Paperless-ngx, Papra oder das Import-Skript Dokumente in deine Ablage legen können.",
@@ -7291,6 +7301,7 @@
"previewTitle": "Inhalt der Sicherung",
"previewLoading": "Inhalt wird gelesen…",
"previewUnavailable": "Inhalt konnte nicht gelesen werden — die Wiederherstellung bleibt möglich.",
"previewStillReading": "Diese Sicherung stammt aus einer früheren Version, deshalb wird ihr Inhalt vollständig gelesen. Bei einem großen Datenbestand dauert das eine Weile. Öffne diesen Dialog in ein paar Minuten noch einmal, dann steht er hier. Die Wiederherstellung hängt nicht davon ab.",
"previewEmpty": "Die Datei enthält keine Datensätze.",
"previewOther": "Weitere Datensätze",
"previewMeasurements": "Messwerte",
messages/en.json
+12 -1
View File
@@ -533,7 +533,10 @@
"saveError": "Could not save medication intake.",
"emptyTitle": "No medications yet",
"emptyDescription": "Create a medication with a schedule first, then come back here to log the intake.",
"emptyCta": "Add medication"
"emptyCta": "Add medication",
"noneActionableTitle": "Nothing to take today",
"noneActionableDescription": "Your medications are paused, kept as a record, or their course has not started or has already ended. A past dose can still be added from a medication's history.",
"noneActionableCta": "Open medications"
},
"quickEntryDiscard": {
"title": "Discard this entry?",
@@ -6172,6 +6175,13 @@
"scopeNote": "It can only add new readings to your own record — it cannot read your data, change or delete anything, or create another token. Readings are marked as coming from an external device, so you can tell them apart later, and you can still correct them yourself.",
"createFailed": "The token could not be created. Please try again."
},
"workoutsToken": {
"title": "Workout tokens",
"description": "Mint a token so a watch bridge or a sync script can add your workouts.",
"detail": "The token is shown once.",
"scopeNote": "It can only add new workouts to your own record. It cannot read your data, change or delete anything, or create another token. Workouts it adds are marked as coming from an external device, and when your phone recorded the same session, you see the phone's copy.",
"createFailed": "The token could not be created. Please try again."
},
"documentsToken": {
"title": "Document tokens",
"description": "Mint a token so Paperless-ngx, Papra or the import script can add documents to your vault.",
@@ -7291,6 +7301,7 @@
"previewTitle": "What this backup contains",
"previewLoading": "Reading contents…",
"previewUnavailable": "Contents could not be read — restoring is still possible.",
"previewStillReading": "This backup was made by an earlier version, so its contents are being read in full, which takes a while for a large record. Open this dialog again in a few minutes to see them. Restoring does not depend on it.",
"previewEmpty": "The file contains no records.",
"previewOther": "Other records",
"previewMeasurements": "Measurements",
messages/es.json
+12 -1
View File
@@ -533,7 +533,10 @@
"saveError": "No se ha podido guardar la toma.",
"emptyTitle": "Aún no hay medicamentos",
"emptyDescription": "Crea primero un medicamento con horario, después vuelve aquí para registrar la toma.",
"emptyCta": "Añadir medicamento"
"emptyCta": "Añadir medicamento",
"noneActionableTitle": "Nada que tomar hoy",
"noneActionableDescription": "Tus medicamentos están en pausa, se guardan solo como registro, o su tratamiento aún no ha empezado o ya ha terminado. Puedes seguir añadiendo una toma pasada desde el historial del medicamento.",
"noneActionableCta": "Abrir medicamentos"
},
"quickEntryDiscard": {
"title": "¿Descartar esta entrada?",
@@ -6172,6 +6175,13 @@
"scopeNote": "Solo puede añadir mediciones nuevas a tu propio historial: no puede leer tus datos, ni modificar o borrar nada, ni crear otro token. Las mediciones se marcan como procedentes de un dispositivo externo, así las distingues más tarde, y puedes corregirlas igualmente.",
"createFailed": "No se pudo crear el token. Inténtalo de nuevo."
},
"workoutsToken": {
"title": "Tokens de entrenamientos",
"description": "Crea un token para que un puente de tu reloj o un script de sincronización añadan tus entrenamientos.",
"detail": "El token se muestra una sola vez.",
"scopeNote": "Solo puede añadir entrenamientos nuevos a tu propio historial. No puede leer tus datos, ni modificar o borrar nada, ni crear otro token. Los entrenamientos que añade se marcan como procedentes de un dispositivo externo, y si tu teléfono registró la misma sesión, ves la copia del teléfono.",
"createFailed": "No se pudo crear el token. Inténtalo de nuevo."
},
"documentsToken": {
"title": "Tokens de documentos",
"description": "Crea un token para que Paperless-ngx, Papra o el script de importación añadan documentos a tu archivo.",
@@ -7291,6 +7301,7 @@
"previewTitle": "Contenido de la copia",
"previewLoading": "Leyendo el contenido…",
"previewUnavailable": "No se pudo leer el contenido — restaurar sigue siendo posible.",
"previewStillReading": "Esta copia se hizo con una versión anterior, así que su contenido se está leyendo completo, y con un historial grande eso lleva un rato. Vuelve a abrir este diálogo en unos minutos para verlo. Restaurar no depende de ello.",
"previewEmpty": "El archivo no contiene registros.",
"previewOther": "Otros registros",
"previewMeasurements": "Mediciones",
messages/fr.json
+12 -1
View File
@@ -533,7 +533,10 @@
"saveError": "Impossible d’enregistrer la prise.",
"emptyTitle": "Aucun médicament pour l’instant",
"emptyDescription": "Crée d’abord un médicament avec un horaire, puis reviens ici pour saisir la prise.",
"emptyCta": "Ajouter un médicament"
"emptyCta": "Ajouter un médicament",
"noneActionableTitle": "Rien à prendre aujourd’hui",
"noneActionableDescription": "Tes médicaments sont en pause, conservés comme simple fiche, ou leur traitement n’a pas encore commencé ou est déjà terminé. Tu peux toujours ajouter une prise passée depuis l’historique du médicament.",
"noneActionableCta": "Ouvrir les médicaments"
},
"quickEntryDiscard": {
"title": "Abandonner cette saisie ?",
@@ -6172,6 +6175,13 @@
"scopeNote": "Il peut uniquement ajouter de nouvelles mesures à votre propre dossier : il ne peut pas lire vos données, ni modifier ou supprimer quoi que ce soit, ni créer un autre jeton. Les mesures sont signalées comme provenant d'un appareil externe, ce qui vous permet de les reconnaître plus tard, et vous pouvez toujours les corriger.",
"createFailed": "Le jeton n'a pas pu être créé. Veuillez réessayer."
},
"workoutsToken": {
"title": "Jetons d'entraînements",
"description": "Créez un jeton pour qu'une passerelle de montre ou un script de synchronisation ajoutent vos entraînements.",
"detail": "Le jeton n'est affiché qu'une fois.",
"scopeNote": "Il peut uniquement ajouter de nouveaux entraînements à votre propre dossier. Il ne peut pas lire vos données, ni modifier ou supprimer quoi que ce soit, ni créer un autre jeton. Les entraînements qu'il ajoute sont signalés comme provenant d'un appareil externe, et si votre téléphone a enregistré la même séance, c'est la copie du téléphone que vous voyez.",
"createFailed": "Le jeton n'a pas pu être créé. Veuillez réessayer."
},
"documentsToken": {
"title": "Jetons de documents",
"description": "Créez un jeton pour que Paperless-ngx, Papra ou le script d'import ajoutent des documents à votre coffre.",
@@ -7291,6 +7301,7 @@
"previewTitle": "Contenu de la sauvegarde",
"previewLoading": "Lecture du contenu…",
"previewUnavailable": "Le contenu n'a pas pu être lu — la restauration reste possible.",
"previewStillReading": "Cette sauvegarde a été créée par une version antérieure, son contenu est donc lu en entier, ce qui prend un moment pour un gros dossier. Rouvrez cette fenêtre dans quelques minutes pour le voir. La restauration n'en dépend pas.",
"previewEmpty": "Le fichier ne contient aucun enregistrement.",
"previewOther": "Autres enregistrements",
"previewMeasurements": "Mesures",
messages/it.json
+12 -1
View File
@@ -533,7 +533,10 @@
"saveError": "Impossibile salvare l’assunzione.",
"emptyTitle": "Ancora nessun farmaco",
"emptyDescription": "Crea prima un farmaco con un orario, poi torna qui per registrarne l’assunzione.",
"emptyCta": "Aggiungi farmaco"
"emptyCta": "Aggiungi farmaco",
"noneActionableTitle": "Niente da prendere oggi",
"noneActionableDescription": "I tuoi farmaci sono in pausa, conservati solo come scheda, oppure la terapia non è ancora iniziata o è già terminata. Puoi comunque aggiungere un’assunzione passata dalla cronologia del farmaco.",
"noneActionableCta": "Apri i farmaci"
},
"quickEntryDiscard": {
"title": "Scartare questa voce?",
@@ -6172,6 +6175,13 @@
"scopeNote": "Può soltanto aggiungere nuove misurazioni alla tua cartella: non può leggere i tuoi dati, modificare o eliminare nulla, né creare un altro token. Le misurazioni vengono contrassegnate come provenienti da un dispositivo esterno, così le riconosci in seguito, e puoi comunque correggerle.",
"createFailed": "Non è stato possibile creare il token. Riprova."
},
"workoutsToken": {
"title": "Token per allenamenti",
"description": "Crea un token perché un ponte dell'orologio o uno script di sincronizzazione aggiungano i tuoi allenamenti.",
"detail": "Il token viene mostrato una sola volta.",
"scopeNote": "Può soltanto aggiungere nuovi allenamenti alla tua cartella. Non può leggere i tuoi dati, modificare o eliminare nulla, né creare un altro token. Gli allenamenti che aggiunge vengono contrassegnati come provenienti da un dispositivo esterno e, se il telefono ha registrato la stessa sessione, vedi la copia del telefono.",
"createFailed": "Non è stato possibile creare il token. Riprova."
},
"documentsToken": {
"title": "Token per documenti",
"description": "Crea un token perché Paperless-ngx, Papra o lo script di importazione aggiungano documenti al tuo archivio.",
@@ -7291,6 +7301,7 @@
"previewTitle": "Contenuto del backup",
"previewLoading": "Lettura del contenuto…",
"previewUnavailable": "Impossibile leggere il contenuto — il ripristino resta possibile.",
"previewStillReading": "Questo backup è stato creato da una versione precedente, quindi il suo contenuto viene letto per intero, e con un archivio grande ci vuole un po'. Riapri questa finestra tra qualche minuto per vederlo. Il ripristino non dipende da questo.",
"previewEmpty": "Il file non contiene record.",
"previewOther": "Altri record",
"previewMeasurements": "Misurazioni",
messages/ko.json
+12 -1
View File
@@ -533,7 +533,10 @@
"saveError": "복용 기록을 저장하지 못했어요.",
"emptyTitle": "등록된 약이 없어요",
"emptyDescription": "먼저 일정이 있는 약을 만든 뒤, 여기로 돌아와 복용을 기록하세요.",
"emptyCta": "약 추가"
"emptyCta": "약 추가",
"noneActionableTitle": "오늘 복용할 약이 없어요",
"noneActionableDescription": "약이 일시 중지되었거나 기록용으로만 보관 중이거나, 복용 기간이 아직 시작되지 않았거나 이미 끝났어요. 지난 복용은 약의 기록에서 계속 추가할 수 있어요.",
"noneActionableCta": "약 목록 열기"
},
"quickEntryDiscard": {
"title": "이 기록을 버릴까요?",
@@ -6172,6 +6175,13 @@
"scopeNote": "본인 기록에 새 측정값을 추가하는 것만 할 수 있어요. 데이터를 읽거나, 무언가를 고치거나 지우거나, 다른 토큰을 만들 수는 없어요. 측정값은 외부 기기에서 온 것으로 표시되니 나중에 구분할 수 있고, 언제든 고칠 수 있어요.",
"createFailed": "토큰을 만들지 못했어요. 다시 시도해 주세요."
},
"workoutsToken": {
"title": "운동 토큰",
"description": "시계 연동 브리지나 동기화 스크립트가 운동 기록을 추가할 수 있도록 토큰을 만들어요.",
"detail": "토큰은 한 번만 표시돼요.",
"scopeNote": "본인 기록에 새 운동을 추가하는 것만 할 수 있어요. 데이터를 읽거나, 무언가를 고치거나 지우거나, 다른 토큰을 만들 수는 없어요. 이 토큰으로 추가된 운동은 외부 기기에서 온 것으로 표시되고, 휴대폰이 같은 운동을 기록했다면 휴대폰의 기록이 보여요.",
"createFailed": "토큰을 만들지 못했어요. 다시 시도해 주세요."
},
"documentsToken": {
"title": "문서 토큰",
"description": "Paperless-ngx, Papra 또는 가져오기 스크립트가 문서함에 문서를 추가할 수 있도록 토큰을 만들어요.",
@@ -7270,6 +7280,7 @@
"previewTitle": "이 백업에 담긴 내용",
"previewLoading": "내용을 읽는 중…",
"previewUnavailable": "내용을 읽지 못했어요 — 복원은 그래도 할 수 있어요.",
"previewStillReading": "이 백업은 이전 버전에서 만들어져서 내용을 처음부터 끝까지 읽고 있어요. 기록이 많으면 시간이 좀 걸려요. 몇 분 뒤에 이 창을 다시 열면 내용이 보여요. 복원은 이것과 상관없이 할 수 있어요.",
"previewEmpty": "파일에 기록이 없어요.",
"previewOther": "기타 기록",
"previewMeasurements": "측정",
messages/pl.json
+12 -1
View File
@@ -533,7 +533,10 @@
"saveError": "Nie udało się zapisać przyjęcia.",
"emptyTitle": "Brak leków",
"emptyDescription": "Najpierw dodaj lek z harmonogramem, a potem wróć tutaj, aby zarejestrować jego przyjęcie.",
"emptyCta": "Dodaj lek"
"emptyCta": "Dodaj lek",
"noneActionableTitle": "Dziś nic do przyjęcia",
"noneActionableDescription": "Twoje leki są wstrzymane, zapisane tylko jako wpis albo ich kuracja jeszcze się nie zaczęła lub już się skończyła. Przeszłą dawkę nadal możesz dodać w historii leku.",
"noneActionableCta": "Otwórz leki"
},
"quickEntryDiscard": {
"title": "Odrzucić ten wpis?",
@@ -6172,6 +6175,13 @@
"scopeNote": "Może wyłącznie dodawać nowe pomiary do Twojej dokumentacji — nie odczyta danych, niczego nie zmieni ani nie usunie i nie utworzy kolejnego tokenu. Pomiary są oznaczane jako pochodzące z urządzenia zewnętrznego, dzięki czemu później je rozpoznasz, i nadal możesz je poprawić.",
"createFailed": "Nie udało się utworzyć tokenu. Spróbuj ponownie."
},
"workoutsToken": {
"title": "Tokeny treningów",
"description": "Utwórz token, aby mostek zegarka lub skrypt synchronizacji mogły dodawać Twoje treningi.",
"detail": "Token pokazujemy tylko raz.",
"scopeNote": "Może wyłącznie dodawać nowe treningi do Twojej dokumentacji. Nie odczyta danych, niczego nie zmieni ani nie usunie i nie utworzy kolejnego tokenu. Dodane przez niego treningi są oznaczane jako pochodzące z urządzenia zewnętrznego, a jeśli telefon zapisał tę samą sesję, widzisz zapis z telefonu.",
"createFailed": "Nie udało się utworzyć tokenu. Spróbuj ponownie."
},
"documentsToken": {
"title": "Tokeny dokumentów",
"description": "Utwórz token, aby Paperless-ngx, Papra lub skrypt importu mogły dodawać dokumenty do Twojego archiwum.",
@@ -7291,6 +7301,7 @@
"previewTitle": "Zawartość kopii",
"previewLoading": "Odczytywanie zawartości…",
"previewUnavailable": "Nie udało się odczytać zawartości — przywracanie jest nadal możliwe.",
"previewStillReading": "Ta kopia powstała we wcześniejszej wersji, więc jej zawartość jest odczytywana w całości, co przy dużych danych chwilę trwa. Otwórz to okno ponownie za kilka minut, aby ją zobaczyć. Przywracanie od tego nie zależy.",
"previewEmpty": "Plik nie zawiera rekordów.",
"previewOther": "Inne rekordy",
"previewMeasurements": "Pomiary",
pnpm-lock.yaml
Generated
+27 -22
View File
@@ -27,6 +27,11 @@ overrides:
valibot@<1.4.2: ^1.4.2
'@humanfs/node@<0.16.8': ^0.16.8
esbuild@>=0.27.3 <0.28.1: ^0.28.1
'@grpc/grpc-js@>=1.14.0 <1.14.5': ^1.14.5
brace-expansion@<1.1.21: ^1.1.21
brace-expansion@>=2.0.0 <2.1.7: ^2.1.7
brace-expansion@>=4.0.0 <5.0.12: ^5.0.12
undici@>=7.0.0 <7.29.1: ^7.29.1
importers:
@@ -679,8 +684,8 @@ packages:
'@floating-ui/utils@0.2.12':
resolution: {integrity: sha512-HpCo8tmWzLVad5s2d19EhAz5zqrrQ6s69qd6moPMQvkOuSwDT1YgRfWSVuc4ennqrgv3OHppiOGMQ7oC13yIww==}
'@grpc/grpc-js@1.14.4':
resolution: {integrity: sha512-k9Dj3DV/itK9D06Y8f190Qgop7/Ui+D0njFV3LHMPwPT75DpXLQohE9Wmz0QElrJnzsjB7KPWiKJbOl7IPDArQ==}
'@grpc/grpc-js@1.14.5':
resolution: {integrity: sha512-7VZM+SVdEcUUqSQeNI3zM8Qs/BhQKZndPo2h5VkYkAM8Iz0wJIa8mKV5ekQGqG8UUsnkQ0NMxIxwkIHYvj0qOw==}
engines: {node: '>=12.10.0'}
'@grpc/proto-loader@0.7.15':
@@ -3525,14 +3530,14 @@ packages:
bowser@2.14.1:
resolution: {integrity: sha512-tzPjzCxygAKWFOJP011oxFHs57HzIhOEracIgAePE4pqB3LikALKnSzUyU4MGs9/iCEUuHlAJTjTc5M+u7YEGg==}
brace-expansion@1.1.18:
resolution: {integrity: sha512-Edep/X9fGqVNmzKBVsDYIOtD+z1tuezV70LBjdCst9Tqu76lsnvRiZ6oTic1n+/BIwX6QDGAO94PN4N2SADvtw==}
brace-expansion@1.1.21:
resolution: {integrity: sha512-9zeA+KLZNNzglF2TPKRQEDyx6Yby7daAkuy8MiPzpXPsYDWi/DRM8jmwUDxokQjYqBpv5DgPiwD4h4ZZSy1Ujw==}
brace-expansion@2.1.4:
resolution: {integrity: sha512-hGfVzPxthbf3+2yjg/RBs60cB0FhqBS/zvdV/4wn4/BmN0bNMMHPc4V/BbFieqf1TKAGGAHnY4eSjajCl0f2Xg==}
brace-expansion@2.1.7:
resolution: {integrity: sha512-uZbew1NqdmPDTMJ8ah1y+b+9QEJrfkXFk3RcTQw3X0jW/xRUvFKsg1CfQdSYGdTbXZWExtU3J3ccxtnfw1Fi0g==}
brace-expansion@5.0.9:
resolution: {integrity: sha512-ScQ4IuvIEF1TMlP7Zt+vjJ//9zlPb2SDcxWxM3bk8s6t6GGdJ7KO1dCcTidOPJKePW30LE/2cT7wCyPho9/Wxg==}
brace-expansion@5.0.12:
resolution: {integrity: sha512-YovQ3rzhaLMIrDjNDMkNS01tea93qhEhG5xy8f6+R0l+dw3Ki+5sCoIoI942iuLZTHWogWktgwVDhU09iNEimQ==}
engines: {node: 20 || >=22}
braces@3.0.3:
@@ -6604,8 +6609,8 @@ packages:
undici-types@8.9.0:
resolution: {integrity: sha512-KTDyRTYX8sWmKXAikPHHSyc63CRPETMctyjKFupcC6OBLXT3xsN0e9aF7m+mIXutFWpUXuedtowG7iLOzp0kQg==}
undici@7.29.0:
resolution: {integrity: sha512-IDxfleLmmbSskfWSUATiN1nfn2rDuvnMOqb5CWR92iIfojA0Ud+ulOAAEQ57LPr9rWmsreUyf5lwyao+7GNNVw==}
undici@7.30.0:
resolution: {integrity: sha512-dkrQXeHSaoamnItlYbmzG0wFYrM0ZwDxCIg0A7aKjTyyhh9svRzCNFEzV+Vm05/yehjCzjDZ31KXfGEjYSztDQ==}
engines: {node: '>=20.18.1'}
undici@8.11.2:
@@ -7451,7 +7456,7 @@ snapshots:
open: 8.4.2
picomatch: 4.0.5
systeminformation: 5.33.1
undici: 7.29.0
undici: 7.30.0
which: 4.0.0
yocto-spinner: 1.2.2
@@ -7583,7 +7588,7 @@ snapshots:
'@floating-ui/utils@0.2.12': {}
'@grpc/grpc-js@1.14.4':
'@grpc/grpc-js@1.14.5':
dependencies:
'@grpc/proto-loader': 0.8.1
'@js-sdsl/ordered-map': 4.4.2
@@ -10292,16 +10297,16 @@ snapshots:
bowser@2.14.1: {}
brace-expansion@1.1.18:
brace-expansion@1.1.21:
dependencies:
balanced-match: 1.0.2
concat-map: 0.0.1
brace-expansion@2.1.4:
brace-expansion@2.1.7:
dependencies:
balanced-match: 1.0.2
brace-expansion@5.0.9:
brace-expansion@5.0.12:
dependencies:
balanced-match: 4.0.4
@@ -10715,7 +10720,7 @@ snapshots:
dockerode@5.0.1(supports-color@7.2.0):
dependencies:
'@balena/dockerignore': 1.0.2
'@grpc/grpc-js': 1.14.4
'@grpc/grpc-js': 1.14.5
'@grpc/proto-loader': 0.7.15
docker-modem: 5.0.7(supports-color@7.2.0)
protobufjs: 7.6.5
@@ -12121,19 +12126,19 @@ snapshots:
minimatch@10.2.6:
dependencies:
brace-expansion: 5.0.9
brace-expansion: 5.0.12
minimatch@3.1.5:
dependencies:
brace-expansion: 1.1.18
brace-expansion: 1.1.21
minimatch@5.1.9:
dependencies:
brace-expansion: 2.1.4
brace-expansion: 2.1.7
minimatch@9.0.9:
dependencies:
brace-expansion: 2.1.4
brace-expansion: 2.1.7
minimist@1.2.8: {}
@@ -13176,7 +13181,7 @@ snapshots:
tailwind-merge: 3.6.0
ts-morph: 26.0.0
tsconfig-paths: 4.2.0
undici: 7.29.0
undici: 7.30.0
validate-npm-package-name: 7.0.2
zod: 3.25.76
zod-to-json-schema: 3.25.2(zod@3.25.76)
@@ -13705,7 +13710,7 @@ snapshots:
undici-types@8.9.0: {}
undici@7.29.0: {}
undici@7.30.0: {}
undici@8.11.2: {}
pnpm-workspace.yaml
+9
View File
@@ -83,6 +83,15 @@ overrides:
# patched line is the same minor, so nothing downstream moves.
"esbuild@>=0.27.3 <0.28.1": "^0.28.1"
# Development-only copies (test containers, build tooling) that the
# dependency alerts flag: gRPC under the test containers, brace-expansion on
# three major lines, and undici 7 next to the app's own undici 8.
"@grpc/grpc-js@>=1.14.0 <1.14.5": "^1.14.5"
"brace-expansion@<1.1.21": "^1.1.21"
"brace-expansion@>=2.0.0 <2.1.7": "^2.1.7"
"brace-expansion@>=4.0.0 <5.0.12": "^5.0.12"
"undici@>=7.0.0 <7.29.1": "^7.29.1"
allowBuilds:
"@prisma/engines": true
prisma: true
prisma/migrations/0364_data_backup_preview/migration.sql
+5
View File
@@ -0,0 +1,5 @@
-- What the restore preview shows for a stored backup, worked out while the
-- copy is written, so the preview no longer reads the whole copy inside a web
-- request. Nullable: a copy written before this release has none until its
-- first preview has read it once.
ALTER TABLE "data_backups" ADD COLUMN IF NOT EXISTS "preview" JSONB;
prisma/schema.prisma
+12
View File
@@ -5032,6 +5032,18 @@ model DataBackup {
/// Whether `innerKeyIds` was recorded; false on every copy written before
/// v1.39.3. Consumer: the same status read (`backups.unrecorded`).
innerKeyIdsRecorded Boolean @default(false) @map("inner_key_ids_recorded")
/// What the restore preview shows for this copy, worked out while the copy
/// was written (`src/lib/export/backup-preview.ts`): the section counts, the
/// schema version, and the inner key ids with the sections they sit in and
/// one sample value each, so the key verdict is taken again against this
/// server's keys every time it is read. It names the copy it describes and
/// is ignored once the copy changes. Plain JSON on purpose: counts and dates
/// only, and the samples are ciphertext already, which a key rotation must
/// not re-seal because they stand for what is inside the copy. NULL for a
/// copy written before this column existed, until its first preview has
/// read it whole.
/// Consumer: `GET /api/admin/backups/{id}/summary`.
preview Json?
createdAt DateTime @default(now()) @map("created_at")
user User @relation(fields: [userId], references: [id], onDelete: Cascade)
src/__tests__/bearer-scope-enforcement-guard.test.ts
+26
View File
@@ -152,6 +152,9 @@ describe("T3 — the mint sites are frozen", () => {
// Document upload from another system (#1038). Same shape as the
// measurement mint: a literal, one scope, one route that accepts it.
"app/api/tokens/documents/route.ts": "[DOCUMENTS_WRITE_SCOPE]",
// Workout ingest from a bridge (#1054). Same shape again: a literal, one
// scope, one route that accepts it.
"app/api/tokens/workouts/route.ts": "[WORKOUTS_WRITE_SCOPE]",
// MCP, audience-bound to `/mcp`. `health:write` requires explicit consent.
"app/api/mcp/tokens/route.ts": "SCOPE_HEALTH_READ / SCOPE_HEALTH_WRITE",
"app/api/mcp/oauth/token/route.ts":
@@ -179,6 +182,7 @@ describe("T3 — the mint sites are frozen", () => {
"app/api/medications/[id]/api-endpoint/route.ts",
"app/api/tokens/measurements/route.ts",
"app/api/tokens/documents/route.ts",
"app/api/tokens/workouts/route.ts",
"app/api/mcp/tokens/route.ts",
"app/api/mcp/oauth/token/route.ts",
];
@@ -234,6 +238,7 @@ describe("T4 — declared scopes are exported constants, never string literals",
"DOCUMENTS_WRITE_SCOPE",
"FHIR_READ_SCOPE",
"MEASUREMENTS_WRITE_SCOPE",
"WORKOUTS_WRITE_SCOPE",
];
const used = new Set<string>();
for (const rel of sourceFiles()) {
@@ -284,6 +289,27 @@ describe("T4b — `documents:write` opens exactly two doors", () => {
});
});
describe("T4c — `workouts:write` opens exactly one door", () => {
it("only the workout batch declares the scope, and only on its POST", () => {
// The set of files naming the scope IS its reach. The batch ingest and
// nothing else: not the workout list or detail, which would hand a
// bridge the history it has no business reading back.
const declaring = filesMatching(
/requireAuth\(\s*WORKOUTS_WRITE_SCOPE\s*\)|requireRecordAuth\([^)]*\bscope:\s*WORKOUTS_WRITE_SCOPE\b/,
);
expect(declaring).toEqual(["app/api/workouts/batch/route.ts"]);
const batch = read("app/api/workouts/batch/route.ts");
expect(
batch.match(/requireAuth\(\s*WORKOUTS_WRITE_SCOPE\s*\)/g),
).toHaveLength(1);
expect(
[...batch.matchAll(/export const (GET|POST|PUT|PATCH|DELETE)\b/g)].map(
(m) => m[1],
),
).toEqual(["POST"]);
});
});
describe("T5 — a delegable route naming a Bearer scope is frozen", () => {
/**
* Routes that admit a narrow token on a DELEGABLE surface, and the scope each
src/__tests__/recent-proof-surface-guard.test.ts
+1
View File
@@ -114,6 +114,7 @@ const COOKIE_PROOF_ROUTES = [
"app/api/auth/reproof/route.ts",
"app/api/tokens/documents/route.ts",
"app/api/tokens/measurements/route.ts",
"app/api/tokens/workouts/route.ts",
].sort();
describe("the recent-proof gate", () => {
src/app/api/admin/backups/[id]/summary/__tests__/route.test.ts
+229 -26
View File
@@ -1,11 +1,14 @@
/**
* v1.37.20 — restore preview: the summary read decrypts and validates the
* stored file through the SAME helpers the restore uses, and only ever
* reads. Watched red: written against the route before its annotate landed;
* the not-found and decrypt-failure arms were asserted first.
* Restore preview. A copy written with a preview is answered from it without
* opening the copy (#1031); a copy from before previews is read whole once,
* through the same helpers the restore uses, and what that read found is kept
* on the row. The verdicts (schema version, keys) are taken again on every
* read. Only a file that fails the schema is a 422 with that message; any
* other failure is the server's, not the file's.
*/
import { describe, it, expect, vi, beforeEach } from "vitest";
import { NextRequest } from "next/server";
import { ZodError } from "zod/v4";
vi.mock("@/lib/api-handler", () => ({
apiHandler: <T extends (...args: unknown[]) => unknown>(fn: T) => fn,
@@ -23,9 +26,15 @@ vi.mock("@/lib/api-handler", () => ({
vi.mock("@/lib/logging/context", () => ({ annotate: vi.fn() }));
const findUniqueMock = vi.fn();
const updateManyMock = vi.fn();
const chunkCountMock = vi.fn();
vi.mock("@/lib/db", () => ({
prisma: {
dataBackup: { findUnique: (...a: unknown[]) => findUniqueMock(...a) },
dataBackup: {
findUnique: (...a: unknown[]) => findUniqueMock(...a),
updateMany: (...a: unknown[]) => updateManyMock(...a),
},
dataBackupChunk: { count: (...a: unknown[]) => chunkCountMock(...a) },
},
}));
@@ -34,16 +43,32 @@ const decryptMock = vi.fn();
// decrypts anything, so a stub of this module has to answer that too.
vi.mock("@/lib/crypto", () => ({
decrypt: (...a: unknown[]) => decryptMock(...a),
decryptBytes: vi.fn(),
getConfiguredKeyIds: () => ["v1"],
isStreamCiphertext: () => false,
decryptStream: vi.fn(),
}));
const openSpy = vi.fn();
vi.mock("@/lib/export/stored-backup", async (importOriginal) => {
const actual =
await importOriginal<typeof import("@/lib/export/stored-backup")>();
return {
...actual,
openStoredBackup: (...a: Parameters<typeof actual.openStoredBackup>) => {
openSpy(...a);
return actual.openStoredBackup(...a);
},
};
});
const parseMock = vi.fn();
const summarizeMock = vi.fn();
const compatibleMock = vi.fn((version: string) => version !== "99");
vi.mock("@/lib/validations/backup", () => ({
parseBackupPayload: (...a: unknown[]) => parseMock(...a),
summarizeBackup: (...a: unknown[]) => summarizeMock(...a),
isCompatibleSchemaVersion: vi.fn(() => true),
isCompatibleSchemaVersion: (version: string) => compatibleMock(version),
// The measurements are read as a stream and counted one by one.
BACKUP_SCHEMA_VERSION: "2",
backupMeasurementSchema: {
@@ -52,52 +77,201 @@ vi.mock("@/lib/validations/backup", () => ({
}));
import { GET } from "../route";
import { storedCopyIdentity } from "@/lib/export/backup-preview";
const request = new NextRequest(
"http://localhost/api/admin/backups/b1/summary",
);
const params = { params: Promise.resolve({ id: "b1" }) };
const params = () => ({ params: Promise.resolve({ id: "b1" }) });
const ROW = {
id: "b1",
userId: "u1",
data: "ciphertext",
chunkCount: null,
chunkStreamId: null,
createdAt: new Date("2026-08-01T00:00:00Z"),
user: { id: "u1", username: "self-hoster" },
};
function storedPreview(overrides: Record<string, unknown> = {}) {
return {
version: 1,
copy: storedCopyIdentity(ROW),
summary: {
schemaVersion: "2",
userId: "u1",
exportedAt: "2026-08-01T00:00:00.000Z",
measurements: 1_800_000,
moodEntries: 3,
},
keys: [],
...overrides,
};
}
beforeEach(() => {
vi.clearAllMocks();
findUniqueMock.mockResolvedValue({
id: "b1",
userId: "u1",
data: "ciphertext",
chunkCount: null,
chunkStreamId: null,
createdAt: new Date("2026-08-01T00:00:00Z"),
user: { id: "u1", username: "self-hoster" },
});
findUniqueMock.mockResolvedValue({ ...ROW, preview: null });
updateManyMock.mockResolvedValue({ count: 1 });
// Twelve measurements in the file: the preview counts them from the stream.
decryptMock.mockReturnValue(
JSON.stringify({
measurements: Array.from({ length: 12 }, (_, i) => ({ id: `m${i}` })),
}),
);
parseMock.mockReturnValue({ schemaVersion: 3 });
summarizeMock.mockReturnValue({ measurements: 0, moodEntries: 3 });
parseMock.mockReturnValue({ schemaVersion: "2" });
summarizeMock.mockReturnValue({
schemaVersion: "2",
userId: "u1",
exportedAt: "2026-08-01T00:00:00.000Z",
measurements: 0,
moodEntries: 3,
});
});
describe("GET /api/admin/backups/[id]/summary", () => {
it("answers the file's counts without writing anything", async () => {
const res = await GET(request, params);
describe("GET /api/admin/backups/[id]/summary, from the stored preview", () => {
it("answers from the preview without opening the copy", async () => {
findUniqueMock.mockResolvedValue({ ...ROW, preview: storedPreview() });
const res = await GET(request, params());
expect(res.status).toBe(200);
const body = await res.json();
expect(body.data.summary).toEqual({ measurements: 12, moodEntries: 3 });
expect(body.data.summary).toMatchObject({
measurements: 1_800_000,
moodEntries: 3,
});
expect(body.data.owner).toBe("self-hoster");
expect(openSpy).not.toHaveBeenCalled();
expect(decryptMock).not.toHaveBeenCalled();
expect(updateManyMock).not.toHaveBeenCalled();
});
it("takes the schema verdict again against this release", async () => {
findUniqueMock.mockResolvedValue({
...ROW,
preview: storedPreview({
summary: {
schemaVersion: "99",
userId: "u1",
exportedAt: "2026-08-01T00:00:00.000Z",
measurements: 1,
},
}),
});
const res = await GET(request, params());
expect(res.status).toBe(422);
expect(openSpy).not.toHaveBeenCalled();
});
it("takes the key verdict again against this server's keys", async () => {
findUniqueMock.mockResolvedValue({
...ROW,
preview: storedPreview({
keys: [
{
keyId: "v7",
count: 4,
sections: ["moodEntries"],
sample: { value: "v7.AAAA", form: "string" },
},
// Only in the instance settings, which the restore leaves out.
{
keyId: "v8",
count: 1,
sections: ["appSettings"],
sample: null,
},
],
}),
});
const res = await GET(request, params());
expect(res.status).toBe(422);
expect(await res.json()).toMatchObject({
meta: { errorCode: "backup.key.missing", keyIds: ["v7"] },
});
expect(openSpy).not.toHaveBeenCalled();
});
it("still counts the pieces, and refuses a copy that lost one", async () => {
const chunked = {
...ROW,
data: null,
chunkCount: 3,
chunkStreamId: "s1",
};
findUniqueMock.mockResolvedValue({
...chunked,
preview: storedPreview({ copy: storedCopyIdentity(chunked) }),
});
chunkCountMock.mockResolvedValue(3);
expect((await GET(request, params())).status).toBe(200);
chunkCountMock.mockResolvedValue(2);
const res = await GET(request, params());
expect(res.status).toBe(422);
expect(await res.json()).toMatchObject({
meta: { errorCode: "backup.payload.undecryptable" },
});
expect(openSpy).not.toHaveBeenCalled();
});
it("ignores a preview that describes another copy", async () => {
findUniqueMock.mockResolvedValue({
...ROW,
preview: storedPreview({ copy: "chunks:other:3" }),
});
const res = await GET(request, params());
expect(res.status).toBe(200);
expect((await res.json()).data.summary.measurements).toBe(12);
expect(openSpy).toHaveBeenCalledTimes(1);
});
});
describe("GET /api/admin/backups/[id]/summary, a copy without a preview", () => {
it("reads the copy once and keeps what it found on the row", async () => {
const res = await GET(request, params());
expect(res.status).toBe(200);
const body = await res.json();
expect(body.data.summary).toMatchObject({
measurements: 12,
moodEntries: 3,
});
expect(openSpy).toHaveBeenCalledTimes(1);
await vi.waitFor(() => expect(updateManyMock).toHaveBeenCalledTimes(1));
const [call] = updateManyMock.mock.calls[0] as [
{ where: Record<string, unknown>; data: { preview: { copy: string } } },
];
// Only onto the copy it was read from.
expect(call.where).toEqual({
id: "b1",
chunkStreamId: null,
chunkCount: null,
});
expect(call.data.preview.copy).toBe(storedCopyIdentity(ROW));
});
it("shares one read between requests for the same copy", async () => {
const [a, b] = await Promise.all([
GET(request, params()),
GET(request, params()),
]);
expect(a.status).toBe(200);
expect(b.status).toBe(200);
expect(openSpy).toHaveBeenCalledTimes(1);
});
it("404s an unknown backup id", async () => {
findUniqueMock.mockResolvedValue(null);
await expect(GET(request, params)).rejects.toMatchObject({ status: 404 });
await expect(GET(request, params())).rejects.toMatchObject({
status: 404,
});
});
it("refuses a file it cannot decrypt with the documented 422", async () => {
decryptMock.mockImplementation(() => {
throw new Error("bad key");
});
const res = await GET(request, params);
const res = await GET(request, params());
// Bad stored input — a key dropped from `ENCRYPTION_KEYS`, or bytes that
// are not the ones written — and not a fault in this process, so it does
// not answer 500 and does not reach the error reporter as one.
@@ -105,13 +279,42 @@ describe("GET /api/admin/backups/[id]/summary", () => {
expect(await res.json()).toMatchObject({
meta: { errorCode: "backup.payload.undecryptable" },
});
expect(updateManyMock).not.toHaveBeenCalled();
});
it("422s a file that fails schema validation", async () => {
parseMock.mockImplementation(() => {
throw new Error("nope");
throw new ZodError([]);
});
const res = await GET(request, params);
const res = await GET(request, params());
expect(res.status).toBe(422);
expect((await res.json()).error).toBe(
"Backup payload failed schema validation",
);
expect(updateManyMock).not.toHaveBeenCalled();
});
it("422s a file that is not JSON", async () => {
decryptMock.mockReturnValue("{ not json");
const res = await GET(request, params());
expect(res.status).toBe(422);
expect((await res.json()).error).toBe(
"Backup payload failed schema validation",
);
});
it("does not call a server failure a schema failure", async () => {
parseMock.mockImplementation(() => {
throw new Error("out of memory");
});
// Thrown to the handler, which answers 500 and reports it.
await expect(GET(request, params())).rejects.toThrow("out of memory");
expect(updateManyMock).not.toHaveBeenCalled();
});
it("still answers when keeping the preview fails", async () => {
updateManyMock.mockRejectedValue(new Error("db gone"));
const res = await GET(request, params());
expect(res.status).toBe(200);
});
});
src/app/api/admin/backups/[id]/summary/route.ts
+200 -46
View File
@@ -4,33 +4,57 @@
*
* The restore dialog used to ask for the typed confirmation on nothing but
* a filename and a date; the counts only appeared in the audit row after
* the rows were already replaced. This read decrypts and schema-validates
* the stored file exactly the way the restore itself does (same decrypt,
* same `parseBackupPayload`, same compatibility gate) and answers with the
* `summarizeBackup` counts — so what the operator confirms is what the
* restore will write, derived from the same code path. Read-only; nothing
* is touched.
* the rows were already replaced. This answers with the `summarizeBackup`
* counts of the stored copy, and refuses the way the restore would refuse:
* a schema version this release cannot restore, or a key the copy's inner
* ciphertext needs and this server does not have.
*
* The counts are worked out while the copy is written and kept on its row
* (`DataBackup.preview`, `src/lib/export/backup-preview.ts`), so this answers
* without opening the copy. Reading the copy here took two minutes for an
* account of 1.8 million readings on slow hardware, long after the dialog had
* given up (#1031). Both verdicts are taken again on every read, from what
* the preview keeps, because the release and the server's keys can both
* change after the copy was written. The copy's pieces are still counted
* against the row (no piece is read), so a copy that lost one since is
* refused here, as the restore and the download refuse it.
*
* A copy written before previews existed has none. That one is read whole,
* once: the preview worked out from it is stored on the row, so the next
* request answers at once, and concurrent requests for the same copy share
* the one read.
*/
import { NextRequest } from "next/server";
import { ZodError } from "zod/v4";
import { Prisma } from "@/generated/prisma/client";
import { prisma } from "@/lib/db";
import { apiHandler, HttpError, requireAdmin } from "@/lib/api-handler";
import { apiError, apiSuccess } from "@/lib/api-response";
import {
checkStoredBackupShape,
isStoredBackupReadError,
openStoredBackup,
storedBackupRefusal,
STORED_BACKUP_SELECT,
type StoredBackupRef,
} from "@/lib/export/stored-backup";
import {
assessBackupKeys,
BACKUP_KEY_MISSING_CODE,
BackupKeyIdCollector,
describeBackupKeyProblem,
} from "@/lib/export/backup-key-ids";
import { BackupJsonError } from "@/lib/export/backup-json-scan";
import {
buildBackupPreview,
storedCopyIdentity,
storedPreviewFor,
type BackupPreview,
} from "@/lib/export/backup-preview";
import {
readStreamedBackup,
type BackupSource,
type StreamedBackup,
StreamedBackupInvalidError,
} from "@/lib/export/streamed-backup";
import { annotate } from "@/lib/logging/context";
import {
@@ -43,6 +67,114 @@ export const dynamic = "force-dynamic";
type RouteParams = { params: Promise<{ id: string }> };
/** Why a copy could not be summarised, as the route answers it. */
type Refusal = { status: number; message: string; code?: string };
type ReadOutcome = { preview: BackupPreview } | { refusal: Refusal };
/**
* Reads of a copy without a preview that are running now, by copy. A second
* request for the same copy (the dialog opened again while the first read is
* still going) waits for the first instead of reading it all again.
*/
const inFlight = new Map<string, Promise<ReadOutcome>>();
/** Whether a failure is the file itself not being a backup this reads. */
function isPayloadInvalid(err: unknown): boolean {
return (
err instanceof BackupJsonError ||
err instanceof StreamedBackupInvalidError ||
err instanceof ZodError
);
}
/**
* Read a copy whole and work out its preview: the way every preview was
* taken before previews were stored. Throws for anything that is neither a
* read refusal nor a file that fails the schema, which the handler answers
* as a server fault.
*/
async function readPreview(
backup: StoredBackupRef,
copy: string,
): Promise<ReadOutcome> {
const refused = (err: unknown): ReadOutcome => {
const refusal = storedBackupRefusal(err);
return {
refusal: {
status: refusal.status,
message: refusal.message,
code: refusal.code,
},
};
};
// Opened and read as a stream: a large record's JSON is longer than any
// string V8 can hold (#1031).
let source;
try {
source = await openStoredBackup(prisma, backup);
} catch (err) {
// Same refusal the restore and the download give, for the same reason:
// a copy this instance cannot open is bad stored input, and the preview
// is the first place an operator meets it.
return refused(err);
}
try {
const streamed = await readStreamedBackup(source);
const payload = parseBackupPayload(streamed.raw);
const summary = {
...summarizeBackup(payload),
measurements: streamed.measurementCount,
};
return { preview: buildBackupPreview(copy, summary, streamed.keys) };
} catch (err) {
if (isStoredBackupReadError(err)) return refused(err);
// Only a file that is not a backup this release reads is a 422. Anything
// else is this server failing, not the file, and is answered as such.
if (isPayloadInvalid(err)) {
return {
refusal: {
status: 422,
message: "Backup payload failed schema validation",
},
};
}
throw err;
}
}
/**
* Keep a preview worked out by reading the copy, so the next request answers
* without reading it. Written only while the row still holds the copy it was
* read from; a failure here costs the next request the same read, nothing
* more, so it is noted and not raised.
*/
async function keepPreview(
backup: StoredBackupRef,
preview: BackupPreview,
): Promise<void> {
try {
await prisma.dataBackup.updateMany({
where: {
id: backup.id,
chunkStreamId: backup.chunkStreamId,
chunkCount: backup.chunkCount,
},
data: { preview: preview as unknown as Prisma.InputJsonValue },
});
} catch (err) {
annotate({
action: { name: "admin.backups.summary.keep_failed" },
meta: {
backupId: backup.id,
error: err instanceof Error ? err.name : "unknown",
},
});
}
}
export const GET = apiHandler(
async (_request: NextRequest, { params }: RouteParams) => {
const { user: admin } = await requireAdmin();
@@ -53,6 +185,7 @@ export const GET = apiHandler(
where: { id },
select: {
...STORED_BACKUP_SELECT,
preview: true,
createdAt: true,
user: { select: { id: true, username: true } },
},
@@ -61,41 +194,58 @@ export const GET = apiHandler(
throw new HttpError(404, "Backup not found");
}
// Opened and read as a stream: a large record's JSON is longer than any
// string V8 can hold (#1031).
let source: BackupSource;
try {
source = await openStoredBackup(prisma, backup);
} catch (err) {
// Same refusal the restore and the download give, for the same reason:
// a copy this instance cannot open is bad stored input, and the preview
// is the first place an operator meets it.
const refusal = storedBackupRefusal(err);
return apiError(refusal.message, refusal.status, {
errorCode: refusal.code,
});
}
let payload;
let measurementCount: number;
let streamed: StreamedBackup;
try {
streamed = await readStreamedBackup(source);
measurementCount = streamed.measurementCount;
payload = parseBackupPayload(streamed.raw);
} catch (err) {
if (isStoredBackupReadError(err)) {
let preview = storedPreviewFor(backup.preview, backup);
const fromStore = preview !== null;
if (preview) {
// The counts were kept; the copy's pieces are still counted, so a copy
// that lost one since is refused here as the restore would refuse it.
// Tampering inside a piece is found by the restore's own open, before
// it deletes anything.
try {
await checkStoredBackupShape(prisma, backup);
} catch (err) {
if (!isStoredBackupReadError(err)) throw err;
const refusal = storedBackupRefusal(err);
return apiError(refusal.message, refusal.status, {
errorCode: refusal.code,
return refusalResponse({
status: refusal.status,
message: refusal.message,
code: refusal.code,
});
}
return apiError("Backup payload failed schema validation", 422);
}
if (!preview) {
const copy = storedCopyIdentity(backup);
if (copy === null) {
// A row holding both forms, or neither: the open refuses it, with
// the reason.
const outcome = await readPreview(backup, "");
if ("refusal" in outcome) return refusalResponse(outcome.refusal);
throw new Error("A stored backup with no nameable copy was read");
}
const flight = `${backup.id}:${copy}`;
let pending = inFlight.get(flight);
if (!pending) {
pending = readPreview(backup, copy).finally(() => {
inFlight.delete(flight);
});
inFlight.set(flight, pending);
// Kept whatever the verdicts below say: they are taken again on
// every read, and the counts do not change.
void pending.then(
(outcome) =>
"preview" in outcome ? keepPreview(backup, outcome.preview) : null,
() => null,
);
}
const outcome = await pending;
if ("refusal" in outcome) return refusalResponse(outcome.refusal);
preview = outcome.preview;
}
if (!isCompatibleSchemaVersion(payload.schemaVersion)) {
const { summary } = preview;
if (!isCompatibleSchemaVersion(summary.schemaVersion)) {
return apiError(
`Backup schema version ${payload.schemaVersion} is not restorable by this release`,
`Backup schema version ${summary.schemaVersion} is not restorable by this release`,
422,
);
}
@@ -105,9 +255,10 @@ export const GET = apiHandler(
// does not have. The instance settings are left out here, as the restore
// leaves them out unless asked; the restore itself checks them when they
// are asked for.
const keyVerdict = assessBackupKeys(streamed.keys, {
ignoreSections: new Set(["appSettings"]),
});
const keyVerdict = assessBackupKeys(
BackupKeyIdCollector.fromStored(preview.keys),
{ ignoreSections: new Set(["appSettings"]) },
);
const keyProblem = describeBackupKeyProblem(keyVerdict);
if (keyProblem) {
return apiError(keyProblem, 422, {
@@ -116,14 +267,9 @@ export const GET = apiHandler(
});
}
const summary = {
...summarizeBackup(payload),
measurements: measurementCount,
};
annotate({
action: { name: "admin.backups.summary" },
meta: { backupId: id, ownerId: backup.userId },
meta: { backupId: id, ownerId: backup.userId, fromStore },
});
return apiSuccess({
@@ -133,3 +279,11 @@ export const GET = apiHandler(
});
},
);
function refusalResponse(refusal: Refusal) {
return apiError(
refusal.message,
refusal.status,
refusal.code ? { errorCode: refusal.code } : undefined,
);
}
src/app/api/measurements/series/__tests__/route.test.ts
+79
View File
@@ -631,6 +631,85 @@ describe("GET /api/measurements/series — dense-kind day-bucketing (v1.28.25)",
expect(bucketSql.slice(1)).toContain("hour");
});
it("hour-buckets a CGM glucose window too dense to send raw, in the user's unit", async () => {
vi.mocked(getSession).mockResolvedValue(SESSION_OK as never);
vi.mocked(prisma.user.findUnique).mockResolvedValue({
glucoseUnit: "mmol/L",
} as never);
// A sensor reading every five minutes for 90 days.
vi.mocked(prisma.measurement.count).mockResolvedValue(25_920 as never);
vi.mocked(prisma.$queryRaw)
.mockResolvedValueOnce([
{
bucket_start: new Date("2026-06-01T08:00:00Z"),
mean: 100,
min_value: 90,
max_value: 112,
},
] as never)
.mockResolvedValueOnce([
{ n: 25_920, mean: 118.004, min: 48, max: 260, sd: 36.004 },
] as never);
const res = await GET(req("kind=glucose&days=90"));
expect(res.status).toBe(200);
const body = (await res.json()) as {
data: {
unit: string;
points: Array<Record<string, unknown>>;
stats: {
count: number;
mean: number;
min: number;
max: number;
stdDev: number;
};
};
};
expect(prisma.measurement.findMany).not.toHaveBeenCalled();
expect(vi.mocked(prisma.measurement.count).mock.calls[0]?.[0]).toEqual(
expect.objectContaining({
where: expect.objectContaining({ type: "BLOOD_GLUCOSE" }),
}),
);
expect(body.data.unit).toBe("mmol/L");
// Converted once at serialization; no band on a glucose point.
expect(body.data.points).toEqual([
{
id: "hour:2026-06-01T08:00:00.000Z",
at: "2026-06-01T08:00:00.000Z",
value: 5.5,
secondary: null,
},
]);
// Stats describe every reading (mg/dL aggregate, converted once).
expect(body.data.stats).toEqual({
count: 25_920,
mean: 6.5,
min: 2.7,
max: 14.4,
stdDev: 2,
});
const bucketSql = vi.mocked(prisma.$queryRaw).mock.calls[0] as unknown[];
expect(bucketSql.slice(1)).toContain("hour");
});
it("keeps fingerstick glucose at or under the row cap raw", async () => {
vi.mocked(getSession).mockResolvedValue(SESSION_OK as never);
vi.mocked(prisma.measurement.count).mockResolvedValue(360 as never);
vi.mocked(prisma.measurement.findMany).mockResolvedValue([
{ id: "g1", value: 104, measuredAt: new Date("2026-06-01T07:12:00Z") },
] as never);
const res = await GET(req("kind=glucose&days=90"));
const body = (await res.json()) as {
data: { points: Array<Record<string, unknown>> };
};
expect(prisma.$queryRaw).not.toHaveBeenCalled();
expect(body.data.points).toEqual([
{ id: "g1", at: "2026-06-01T07:12:00.000Z", value: 104, secondary: null },
]);
});
it("keeps a pulse window at or under the row cap raw", async () => {
vi.mocked(getSession).mockResolvedValue(SESSION_OK as never);
vi.mocked(prisma.measurement.count).mockResolvedValue(10_000 as never);
src/app/api/measurements/series/route.ts
+14 -11
View File
@@ -52,17 +52,20 @@ const DENSE_SERIES_RAW_WINDOW_DAYS = 90;
const DENSE_SERIES_KINDS: ReadonlySet<string> = new Set(["glucose", "pulse"]);
/**
* #1023 — inside the raw window, a pulse series with more rows than this is
* bucketed per local hour in SQL instead of sent raw.
* #1023 — inside the raw window, a pulse or glucose series with more rows
* than this is bucketed per local hour in SQL instead of sent raw.
*
* A watch that records heart rate once a minute puts 43 000 rows in the
* default 30-day window and 130 000 in 90 days: a 15 MB response the client
* could not draw at that resolution anyway, built from as many objects on the
* server. Hourly buckets carry the hour's mean with its low/high band, the
* shape an hourly-bucket import already has, and cap the answer at 24 points
* a day. The cap sits far above what sparse sources produce (a cuff, hourly
* buckets over 90 days, a watch sampling every five minutes over 30 days), so
* those series stay raw and unchanged.
* server. A CGM reading every five minutes puts 26 000 glucose rows in 90
* days, the same problem at a smaller scale. Hourly buckets carry the hour's
* mean (pulse also its low/high band, the shape an hourly-bucket import
* already has) and cap the answer at 24 points a day; the statistics stay
* computed over every raw reading. The cap sits far above what sparse sources
* produce (a cuff, fingerstick glucose, hourly buckets over 90 days, a watch
* sampling every five minutes over 30 days), so those series stay raw and
* unchanged.
*/
const DENSE_SERIES_RAW_ROW_CAP = 10_000;
@@ -368,18 +371,18 @@ export const GET = apiHandler(async (request: NextRequest) => {
});
} else if (
(DENSE_SERIES_KINDS.has(kind) && days > DENSE_SERIES_RAW_WINDOW_DAYS) ||
(kind === "pulse" &&
(DENSE_SERIES_KINDS.has(kind) &&
(await prisma.measurement.count({
where: {
userId: user.id,
type: "PULSE",
type: KIND_TO_TYPE[kind],
measuredAt: { gte: since },
deletedAt: null,
},
})) > DENSE_SERIES_RAW_ROW_CAP)
) {
// Day buckets past the raw window; hour buckets for a pulse stream too
// dense to send raw inside it.
// Day buckets past the raw window; hour buckets for a pulse or CGM
// stream too dense to send raw inside it.
const grain = days > DENSE_SERIES_RAW_WINDOW_DAYS ? "day" : "hour";
// v1.28.25 — long-window read of a sample-dense kind (CGM glucose,
// per-sample / hourly pulse). Day-bucket in SQL instead of walking
src/app/api/tokens/workouts/__tests__/route.test.ts
+245
View File
@@ -0,0 +1,245 @@
/**
* The workout-ingest mint (#1054).
*
* The assertion that matters most is not the 201 — it is the exact
* `permissions` array handed to `issueApiToken`. That helper defaults to
* `["*"]` when the property is absent, so an edit that drops one line turns a
* user-facing endpoint into a cookie-equivalent credential factory, and a test
* that only checked the status code would go on passing. The structural guard
* catches a literal `permissions: ["*"` and would not catch the omission, so
* this is where that hole is closed.
*/
import { describe, it, expect, vi, beforeEach } from "vitest";
vi.mock("@/lib/api-handler", () => ({
apiHandler: (fn: unknown) => fn,
requireCookieAuth: vi.fn(),
assertRecentCookieProof: vi.fn(),
}));
vi.mock("@/lib/auth/issue-token", () => ({
issueApiToken: vi.fn(),
}));
vi.mock("@/lib/app-settings", () => ({
isApiGloballyEnabled: vi.fn(),
}));
vi.mock("@/lib/rate-limit", () => ({
checkRateLimit: vi.fn(),
}));
vi.mock("@/lib/auth/audit", () => ({
auditLog: vi.fn().mockResolvedValue(undefined),
}));
vi.mock("@/lib/logging/context", () => ({
annotate: vi.fn(),
getEvent: vi.fn(() => null),
}));
vi.mock("@/lib/db", () => ({
prisma: { apiToken: { count: vi.fn() } },
}));
import { POST } from "../route";
import { assertRecentCookieProof, requireCookieAuth } from "@/lib/api-handler";
import { issueApiToken } from "@/lib/auth/issue-token";
import { isApiGloballyEnabled } from "@/lib/app-settings";
import { checkRateLimit } from "@/lib/rate-limit";
import { auditLog } from "@/lib/auth/audit";
import { prisma } from "@/lib/db";
import { WORKOUTS_WRITE_SCOPE } from "@/lib/workouts/scopes";
const USER = { id: "user-1", role: "USER" as const };
const EXPIRES = new Date("2027-08-31T00:00:00.000Z");
function req(body: unknown): Request {
return new Request("https://health.example/api/tokens/workouts", {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify(body),
});
}
beforeEach(() => {
vi.resetAllMocks();
vi.mocked(requireCookieAuth).mockResolvedValue({
user: USER,
session: { id: "sess-1" },
} as never);
vi.mocked(assertRecentCookieProof).mockResolvedValue(undefined);
vi.mocked(isApiGloballyEnabled).mockResolvedValue(true);
vi.mocked(checkRateLimit).mockResolvedValue({ allowed: true } as never);
vi.mocked(auditLog).mockResolvedValue(undefined as never);
// Below the ceiling by default; the cases that care set their own count.
vi.mocked(prisma.apiToken.count).mockResolvedValue(0 as never);
vi.mocked(issueApiToken).mockResolvedValue({
token: "hlk_" + "a".repeat(64),
expiresAt: EXPIRES,
tokenId: "token-1",
name: "Watch bridge",
} as never);
});
describe("the fresh proof in front of the mint", () => {
it("mints nothing when the session carries no recent proof", async () => {
vi.mocked(assertRecentCookieProof).mockRejectedValue(
new Error("auth.reproof.required"),
);
await expect(POST(req({ name: "Scale" }) as never)).rejects.toThrow(
"auth.reproof.required",
);
expect(issueApiToken).not.toHaveBeenCalled();
});
it("asks about the caller's own session", async () => {
await POST(req({ name: "Scale" }) as never);
expect(assertRecentCookieProof).toHaveBeenCalledWith(
expect.objectContaining({ id: expect.any(String) }),
"sess-1",
);
});
});
describe("what it mints", () => {
it("names exactly the workout scope, and no wildcard", async () => {
const res = await POST(req({ name: "Watch bridge" }) as never);
expect(res.status).toBe(201);
const opts = vi.mocked(issueApiToken).mock.calls[0][0];
// Exact equality, not `toContain`: an extra scope is a widening, and this
// is the assertion standing between the endpoint and one.
expect(opts.permissions).toEqual([WORKOUTS_WRITE_SCOPE]);
expect(opts.userId).toBe(USER.id);
});
it("returns the raw token once, with its name and expiry", async () => {
const res = await POST(req({ name: "Watch bridge" }) as never);
const json = await res.json();
expect(json.data.token).toMatch(/^hlk_[0-9a-f]{64}$/);
expect(json.data.name).toBe("Watch bridge");
expect(json.data.expiresAt).toBeDefined();
});
it("expires by default rather than living forever", async () => {
await POST(req({ name: "Export replay" }) as never);
const opts = vi.mocked(issueApiToken).mock.calls[0][0];
expect(opts.expiresInDays).toBe(365);
});
it("honours an explicit lifetime", async () => {
await POST(req({ name: "Export replay", expiresInDays: 30 }) as never);
expect(vi.mocked(issueApiToken).mock.calls[0][0].expiresInDays).toBe(30);
});
it("audits the mint with the token id and the scope", async () => {
await POST(req({ name: "Export replay" }) as never);
expect(auditLog).toHaveBeenCalledWith(
"tokens.workouts.create",
expect.objectContaining({
userId: USER.id,
details: expect.objectContaining({
tokenId: "token-1",
scope: WORKOUTS_WRITE_SCOPE,
}),
}),
);
});
it("counts only live tokens carrying this scope", async () => {
// The whole shape of the query, asserted rather than assumed. Counting
// every row would let the `["*"]` tokens a login mints consume the budget
// and refuse a mint for a reason the person cannot see; counting revoked
// or expired rows would mean revoking never frees a slot.
await POST(req({ name: "Export replay" }) as never);
const where = vi.mocked(prisma.apiToken.count).mock.calls[0][0]?.where;
expect(where).toMatchObject({
userId: USER.id,
revoked: false,
permissions: { has: WORKOUTS_WRITE_SCOPE },
});
expect(where?.OR).toEqual([
{ expiresAt: null },
{ expiresAt: { gt: expect.any(Date) } },
]);
});
});
describe("when it refuses", () => {
it("403s while the operator has the API switched off", async () => {
vi.mocked(isApiGloballyEnabled).mockResolvedValue(false);
const res = await POST(req({ name: "Export replay" }) as never);
expect(res.status).toBe(403);
expect(issueApiToken).not.toHaveBeenCalled();
});
it("429s past the mint rate limit, without minting", async () => {
vi.mocked(checkRateLimit).mockResolvedValue({ allowed: false } as never);
const res = await POST(req({ name: "Export replay" }) as never);
expect(res.status).toBe(429);
expect(issueApiToken).not.toHaveBeenCalled();
});
it("422s on a missing name", async () => {
const res = await POST(req({}) as never);
expect(res.status).toBe(422);
expect(issueApiToken).not.toHaveBeenCalled();
});
it("422s on a lifetime outside the permitted band", async () => {
const res = await POST(
req({ name: "Export replay", expiresInDays: 4000 }) as never,
);
expect(res.status).toBe(422);
expect(issueApiToken).not.toHaveBeenCalled();
});
it("409s at the live-token ceiling, without minting", async () => {
vi.mocked(prisma.apiToken.count).mockResolvedValue(10 as never);
const res = await POST(req({ name: "Export replay" }) as never);
// 409, not 429: a conflict with the account's current state, whose remedy
// is revoking rather than waiting. A 429 would say the opposite.
expect(res.status).toBe(409);
expect((await res.json()).meta?.errorCode).toBe(
"tokens.workouts.ceiling_reached",
);
expect(issueApiToken).not.toHaveBeenCalled();
});
it("still mints on the last free slot", async () => {
// The off-by-one that would make the ceiling nine. Worth pinning, because
// `>=` versus `>` is invisible in review and only one of them is right.
vi.mocked(prisma.apiToken.count).mockResolvedValue(9 as never);
const res = await POST(req({ name: "Export replay" }) as never);
expect(res.status).toBe(201);
});
it("checks the ceiling only after the body validates", async () => {
// A malformed request should hear about the malformation, not the
// ceiling — and should not spend a query establishing it.
vi.mocked(prisma.apiToken.count).mockResolvedValue(10 as never);
const res = await POST(req({}) as never);
expect(res.status).toBe(422);
expect(prisma.apiToken.count).not.toHaveBeenCalled();
});
it("ignores a scope the body tries to smuggle in", async () => {
// There is no `scope` field to validate, so an unknown key is simply not
// read — the permission array is a literal either way. Asserted because
// "the field does not exist" is the security property, and a schema change
// that added one would silently make this body meaningful.
await POST(
req({
name: "Export replay",
scope: "read_write",
permissions: ["*"],
}) as never,
);
expect(vi.mocked(issueApiToken).mock.calls[0][0].permissions).toEqual([
WORKOUTS_WRITE_SCOPE,
]);
});
});
src/app/api/tokens/workouts/route.ts
+135
View File
@@ -0,0 +1,135 @@
/**
* `POST /api/tokens/workouts` — mint a Bearer that can push workouts into the
* holder's own record and do nothing else (#1054).
*
* The door a workout bridge pushes through: a watch vendor's sync relayed by a
* small service, or a script replaying an exported history. It is the
* measurement-ingest mint (`../measurements/route.ts`) copied for a third
* scope, and every argument made there holds here — read that file for the
* reasoning behind each guard. In short:
*
* - `permissions` is a literal, so no request shape reaches it;
* - `workouts:write` is accepted by `POST /api/workouts/batch` alone, on the
* holder's own record; the rows it writes carry `EXTERNAL` provenance, and
* it cannot list or read a workout back, reach any other route, or mint
* another token;
* - minting takes a cookie session, because a short-lived native access token
* must not be able to leave behind a credential that lives a year;
* - ten mints a minute and ten live tokens bound a runaway loop.
*
* No listing and no revoke of its own: `GET /api/tokens` lists every token
* with its permissions and `DELETE /api/tokens/[id]` revokes one.
*/
import { NextRequest } from "next/server";
import {
apiHandler,
assertRecentCookieProof,
requireCookieAuth,
} from "@/lib/api-handler";
import {
apiError,
apiSuccess,
getClientIp,
returnAllZodIssues,
safeJson,
} from "@/lib/api-response";
import { annotate } from "@/lib/logging/context";
import { auditLog } from "@/lib/auth/audit";
import { prisma } from "@/lib/db";
import { issueApiToken } from "@/lib/auth/issue-token";
import { isApiGloballyEnabled } from "@/lib/app-settings";
import { checkRateLimit } from "@/lib/rate-limit";
import { WORKOUTS_WRITE_SCOPE } from "@/lib/workouts/scopes";
import { createWorkoutTokenSchema } from "@/lib/validations/tokens";
/**
* Days a token lives when the caller names no lifetime. A year, like the
* measurement token: it is pasted into a bridge that runs unattended, where a
* quiet expiry surfaces as "my runs stopped showing up sometime in the spring".
*/
const DEFAULT_EXPIRY_DAYS = 365;
/** Mints per user per minute. */
const MINT_RATE_LIMIT_MAX = 10;
const MINT_RATE_LIMIT_WINDOW_MS = 60 * 1000;
/** Live workout tokens an account may hold at once; catches a loop. */
const MAX_LIVE_TOKENS = 10;
export const POST = apiHandler(async (request: NextRequest) => {
const { user, session } = await requireCookieAuth();
// And a fresh proof on top of the session: the token lives far longer than
// a stolen session would, so minting one needs the person, not just the
// browser.
await assertRecentCookieProof(user, session.id);
annotate({ action: { name: "tokens.workouts.create" } });
if (!(await isApiGloballyEnabled())) {
return apiError("API is globally disabled", 403);
}
const rl = await checkRateLimit(
`tokens:workouts:mint:${user.id}`,
MINT_RATE_LIMIT_MAX,
MINT_RATE_LIMIT_WINDOW_MS,
);
if (!rl.allowed) {
return apiError("Too many token mints, try again later", 429);
}
const { data: body, error: jsonError } = await safeJson(request, {
maxBytes: 16 * 1024,
});
if (jsonError) return jsonError;
const parsed = createWorkoutTokenSchema.safeParse(body);
if (!parsed.success) {
return returnAllZodIssues(parsed.error, 422);
}
// Counted live and per scope, after validation and before the create; the
// count-then-create race is bounded by the mint bucket above. See the
// measurement mint for the full argument.
const live = await prisma.apiToken.count({
where: {
userId: user.id,
revoked: false,
permissions: { has: WORKOUTS_WRITE_SCOPE },
OR: [{ expiresAt: null }, { expiresAt: { gt: new Date() } }],
},
});
if (live >= MAX_LIVE_TOKENS) {
annotate({
action: { name: "tokens.workouts.create" },
meta: { outcome: "ceiling_reached", live_token_count: live },
});
return apiError(
`You already have ${MAX_LIVE_TOKENS} workout tokens. Revoke one before creating another.`,
409,
{ errorCode: "tokens.workouts.ceiling_reached" },
);
}
const issued = await issueApiToken({
userId: user.id,
name: parsed.data.name,
// A literal, never spread and never derived from the body. `issueApiToken`
// defaults to `["*"]` when this property is absent, which is why the unit
// suite asserts this array and not merely the 201.
permissions: [WORKOUTS_WRITE_SCOPE],
expiresInDays: parsed.data.expiresInDays ?? DEFAULT_EXPIRY_DAYS,
});
await auditLog("tokens.workouts.create", {
userId: user.id,
ipAddress: getClientIp(request),
details: { tokenId: issued.tokenId, scope: WORKOUTS_WRITE_SCOPE },
});
// The raw token, once. It is stored as an HMAC and no path re-reveals it.
return apiSuccess(
{ token: issued.token, name: issued.name, expiresAt: issued.expiresAt },
201,
);
});
src/app/api/workouts/__tests__/batch-scope.test.ts
+265
View File
@@ -0,0 +1,265 @@
/**
* Who may call `POST /api/workouts/batch` (#1054).
*
* The route names `workouts:write`, so the set of credentials it admits is:
* a cookie session, a wildcard (`["*"]`) token, and a narrow token carrying
* exactly that scope. Every other narrow token — `measurements:write`
* included, however adjacent it looks — is refused 403 by the fail-closed
* default.
*
* The real `requireAuth` and the real `resolveBearerToken` run here; only the
* token row lookup is stubbed, so the admission decision under test is the
* production one rather than a re-implementation of it.
*
* The second half pins what a scoped caller writes: its rows are `EXTERNAL`,
* resolved from the credential, and a body naming any source is refused 422
* rather than relabelled. A session and a wildcard token keep the source they
* send.
*/
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { NextRequest } from "next/server";
vi.mock("@/lib/db", () => ({
prisma: {
apiToken: { findUnique: vi.fn(), update: vi.fn() },
user: { findUnique: vi.fn() },
workout: {
findMany: vi.fn(),
createManyAndReturn: vi.fn(),
},
workoutRoute: { createMany: vi.fn() },
workoutSamples: { createManyAndReturn: vi.fn() },
$transaction: vi.fn(async (fn: unknown) => {
if (typeof fn === "function") {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
return (fn as any)(prisma as unknown as { workout: unknown });
}
}),
},
}));
vi.mock("@/lib/auth/session", () => ({ getSession: vi.fn() }));
vi.mock("@/lib/auth/audit", () => ({
auditLog: vi.fn().mockResolvedValue(undefined),
}));
vi.mock("@/lib/logging/transports", () => ({ emitIfSampled: vi.fn() }));
vi.mock("@/lib/db-compat", () => ({
ensureDbCompatibility: vi.fn().mockResolvedValue(undefined),
}));
vi.mock("@/lib/rate-limit", () => ({ checkRateLimit: vi.fn() }));
vi.mock("@/lib/jobs/pr-detection", () => ({
enqueuePrDetection: vi.fn().mockResolvedValue(undefined),
}));
vi.mock("@/lib/arrivals/emit-shared", () => ({
emitDataArrival: vi.fn().mockResolvedValue(undefined),
}));
vi.mock("next/headers", () => ({
headers: vi.fn(async () => ({ get: () => null })),
cookies: vi.fn(async () => ({
get: () => undefined,
set: () => {},
delete: () => {},
})),
}));
import { POST } from "../batch/route";
import { prisma } from "@/lib/db";
import { getSession } from "@/lib/auth/session";
import { headers } from "next/headers";
import { checkRateLimit } from "@/lib/rate-limit";
import { auditLog } from "@/lib/auth/audit";
import { MEASUREMENTS_WRITE_SCOPE } from "@/lib/measurements/scopes";
import { DOCUMENTS_WRITE_SCOPE } from "@/lib/documents/scopes";
import { WORKOUTS_WRITE_SCOPE } from "@/lib/workouts/scopes";
const USER = {
id: "user-1",
username: "tester",
role: "USER" as const,
sourcePriorityJson: null,
};
const TOKEN = "hlk_" + "c".repeat(64);
function makeRequest(body: unknown): NextRequest {
return new NextRequest("http://localhost/api/workouts/batch", {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify(body),
});
}
function workout(
externalId: string,
overrides: Record<string, unknown> = {},
): Record<string, unknown> {
return {
sportType: "running",
startedAt: "2026-09-20T06:30:00.000Z",
endedAt: "2026-09-20T07:15:00.000Z",
externalId,
...overrides,
};
}
/** Present `TOKEN` as a Bearer whose row carries `permissions`. */
function armToken(permissions: string[]) {
vi.mocked(getSession).mockResolvedValue(null as never);
vi.mocked(headers).mockResolvedValue({
get: (name: string) =>
name.toLowerCase() === "authorization" ? `Bearer ${TOKEN}` : null,
} as never);
vi.mocked(prisma.apiToken.findUnique).mockResolvedValue({
id: "tok-1",
userId: USER.id,
permissions,
revoked: false,
expiresAt: new Date(Date.now() + 86_400_000),
} as never);
}
function armSession() {
vi.mocked(getSession).mockResolvedValue({
session: { id: "sess-1", expiresAt: new Date(Date.now() + 3_600_000) },
user: USER,
} as never);
}
/** The rows the route handed to `INSERT ... RETURNING`. */
function insertedRows(): Array<Record<string, unknown>> {
return vi
.mocked(prisma.workout.createManyAndReturn)
.mock.calls.flatMap(([args]) => {
const data = (args as { data: unknown }).data;
return (Array.isArray(data) ? data : [data]) as Array<
Record<string, unknown>
>;
});
}
beforeEach(() => {
vi.resetAllMocks();
vi.stubEnv("API_TOKEN_HMAC_KEY", "k".repeat(64));
vi.mocked(headers).mockResolvedValue({ get: () => null } as never);
vi.mocked(getSession).mockResolvedValue(null as never);
vi.mocked(checkRateLimit).mockResolvedValue({
allowed: true,
limit: 60,
remaining: 60,
resetAt: Date.now() + 60_000,
});
vi.mocked(prisma.apiToken.update).mockResolvedValue({} as never);
// The refusal path chains `.catch` on the audit write, so the reset mock
// has to hand back a promise again.
vi.mocked(auditLog).mockResolvedValue(undefined as never);
// One row serves both lookups: the resolver's user fetch and the route's
// source-priority read.
vi.mocked(prisma.user.findUnique).mockResolvedValue(USER as never);
vi.mocked(prisma.workout.findMany).mockResolvedValue([]);
vi.mocked(prisma.workout.createManyAndReturn).mockImplementation(
(async (args: { data: Array<Record<string, unknown>> }) =>
args.data.map((row, index) => ({
id: `w-${index}`,
source: row.source,
externalId: row.externalId,
}))) as never,
);
});
afterEach(() => {
vi.unstubAllEnvs();
});
describe("POST /api/workouts/batch — which credentials it admits", () => {
it("admits a narrow workouts:write token", async () => {
armToken([WORKOUTS_WRITE_SCOPE]);
const res = await POST(makeRequest({ workouts: [workout("s-1")] }));
expect(res.status).toBe(200);
const body = await res.json();
expect(body.data.inserted).toBe(1);
});
it("still admits a cookie session", async () => {
armSession();
const res = await POST(
makeRequest({ workouts: [workout("s-1", { source: "APPLE_HEALTH" })] }),
);
expect(res.status).toBe(200);
});
it("still admits a wildcard token", async () => {
armToken(["*"]);
const res = await POST(
makeRequest({ workouts: [workout("s-1", { source: "APPLE_HEALTH" })] }),
);
expect(res.status).toBe(200);
});
it.each([
["measurements:write", [MEASUREMENTS_WRITE_SCOPE]],
["documents:write", [DOCUMENTS_WRITE_SCOPE]],
["medication:ingest", ["medication:ingest", "medication:m-1:ingest"]],
["health:read", ["health:read"]],
])("refuses a %s token with 403 and writes nothing", async (_, perms) => {
armToken(perms);
const res = await POST(makeRequest({ workouts: [workout("s-1")] }));
expect(res.status).toBe(403);
expect(prisma.workout.createManyAndReturn).not.toHaveBeenCalled();
});
});
describe("POST /api/workouts/batch — what a scoped caller writes", () => {
it("attributes its rows to EXTERNAL", async () => {
armToken([WORKOUTS_WRITE_SCOPE]);
const res = await POST(
makeRequest({ workouts: [workout("s-1"), workout("s-2")] }),
);
expect(res.status).toBe(200);
const rows = insertedRows();
expect(rows).toHaveLength(2);
expect(rows.map((r) => r.source)).toEqual(["EXTERNAL", "EXTERNAL"]);
});
it("probes for duplicates in the EXTERNAL namespace", async () => {
armToken([WORKOUTS_WRITE_SCOPE]);
await POST(makeRequest({ workouts: [workout("s-1")] }));
const probe = vi
.mocked(prisma.workout.findMany)
.mock.calls.map(([args]) => args as { where?: { OR?: unknown } })
.find((args) => args.where?.OR !== undefined);
expect(probe?.where?.OR).toEqual([
{ source: "EXTERNAL", externalId: "s-1" },
]);
});
it.each(["APPLE_HEALTH", "MANUAL"])(
"refuses a body naming %s with 422 and writes nothing",
async (source) => {
armToken([WORKOUTS_WRITE_SCOPE]);
const res = await POST(
makeRequest({
workouts: [workout("s-1"), workout("s-2", { source })],
}),
);
expect(res.status).toBe(422);
const body = await res.json();
expect(body.meta?.errorCode).toBe("workout.batch.source_not_permitted");
expect(prisma.workout.createManyAndReturn).not.toHaveBeenCalled();
},
);
it("leaves the source a wildcard caller sends alone", async () => {
armToken(["*"]);
await POST(
makeRequest({ workouts: [workout("s-1", { source: "APPLE_HEALTH" })] }),
);
expect(insertedRows().map((r) => r.source)).toEqual(["APPLE_HEALTH"]);
});
it("leaves the source a session sends alone", async () => {
armSession();
await POST(
makeRequest({ workouts: [workout("s-1", { source: "MANUAL" })] }),
);
expect(insertedRows().map((r) => r.source)).toEqual(["MANUAL"]);
});
});
src/app/api/workouts/batch/route.ts
+67 -8
View File
@@ -1,7 +1,7 @@
/**
* `POST /api/workouts/batch` — typed workout ingest with nested route.
*
* Consumers (v1.5):
* Consumers:
* 1. The native iOS app draining its HealthKit observer queue —
* maps `HKWorkout` + `HKWorkoutRoute` samples into the request
* shape locked by `createBatchWorkoutSchema` in
@@ -15,12 +15,18 @@
* (MANUAL / APPLE_HEALTH), so this server path must write its
* `WITHINGS`-sourced rows directly via `prisma.workout.upsert`
* (as the WHOOP / Fitbit syncs already do), not through this route.
* 3. A third-party bridge holding a narrow `workouts:write` token
* (#1054) — a watch vendor's sync relayed by a small service. Its
* rows are attributed `EXTERNAL`, resolved from the credential; see
* the source guard in the handler.
*
* Cross-cutting concerns mirror `/api/measurements/batch` so the iOS
* sync engine can re-use the same retry / cursor plumbing:
* - `requireAuth()` — cookie + Bearer (narrow-scope-token safe per
* v1.4.25 W10 fix-C: when the route declares no scope, any
* authenticated token passes).
* - `requireAuth` naming `WORKOUTS_WRITE_SCOPE` — a cookie session, a
* wildcard (`["*"]`) token, or a narrow `workouts:write` token. Every
* other narrow token is refused 403: under the fail-closed default a
* route admits exactly the scope it names, so a `measurements:write`
* credential does not reach this route.
* - `withIdempotency` — `Idempotency-Key` header replays the cached
* response on retry (24h window).
* - `checkRateLimit("workouts:batch:${user.id}", 60, 60s)` — same
@@ -59,7 +65,7 @@
import { NextRequest } from "next/server";
import { prisma } from "@/lib/db";
import { apiHandler, requireAuth } from "@/lib/api-handler";
import { apiHandler, isScopedCredential, requireAuth } from "@/lib/api-handler";
import { annotate } from "@/lib/logging/context";
import { auditLog } from "@/lib/auth/audit";
import {
@@ -85,6 +91,8 @@ import {
type UnstableExternalIdShape,
} from "@/lib/validations/external-id";
import { dedupeWorkoutBatch } from "@/lib/workouts/canonical-rows";
import { WORKOUTS_WRITE_SCOPE } from "@/lib/workouts/scopes";
import { EXTERNAL_SOURCE } from "@/lib/measurements/external-source";
import { encryptRouteGeometry } from "@/lib/workouts/route-geometry-cipher";
import { Prisma, type MeasurementSource } from "@/generated/prisma/client";
@@ -169,7 +177,17 @@ async function emitWorkoutArrivals(
export const POST = apiHandler(withIdempotency<[NextRequest]>(postBatch));
async function postBatch(request: NextRequest): Promise<Response> {
const { user } = await requireAuth();
// Declares the workout-ingest scope, so a narrow `workouts:write` token
// reaches this route alongside cookie sessions and the wildcard tokens a
// native login mints. The only route that names it. `requireAuth` refuses
// any acting-account carrier for every credential shape, so the scoped token
// is confined to its owner's own record without an arm of its own here.
const auth = await requireAuth(WORKOUTS_WRITE_SCOPE);
const { user } = auth;
// A scoped credential is a bridge, not the native client: the source of
// its rows is resolved from the credential below rather than from the body.
const scoped = isScopedCredential(auth);
// v1.4.43 W9 — resolve the per-user source-priority blob once at
// the start of the handler so the write-time canonical-row picker
@@ -268,6 +286,27 @@ async function postBatch(request: NextRequest): Promise<Response> {
});
}
// A scoped credential may attribute nothing: its rows are `EXTERNAL`,
// resolved from the credential. Refused loudly rather than relabelled, the
// measurement batch's rule and for its reasons. `APPLE_HEALTH` is half the
// `(userId, source, externalId)` dedup key the phone writes into and ranks
// first in the cross-source picker, so a bridge naming it would land in the
// phone's namespace and could shadow the phone's own copy of a session;
// `MANUAL` would make a bridged workout indistinguishable from one entered
// by hand. Checked on the raw body because the schema defaults an absent
// `source` to `MANUAL`, which would make "named" and "omitted" look alike.
if (scoped && namesAnySource(rawBody)) {
annotate({
action: { name: "workout.batch.ingest" },
meta: { outcome: "source_not_permitted" },
});
return apiError(
"This credential resolves the source itself; omit the field",
422,
{ errorCode: "workout.batch.source_not_permitted" },
);
}
const { workouts } = parsed.data;
const refusedSampleIndices = new Set(parsed.refusedSampleIndices);
@@ -359,7 +398,11 @@ async function postBatch(request: NextRequest): Promise<Response> {
stepCount: w.stepCount ?? null,
elevationM: w.elevationM ?? null,
pauseDurationSec: w.pauseDurationSec ?? null,
source: w.source,
// `EXTERNAL` under a scoped credential, and nothing else: the guard
// above refused any body that named a source, so nothing asserted is
// discarded here. It also moves the dedup key into its own namespace,
// so a bridge's ids can never collide with the phone's.
source: scoped ? EXTERNAL_SOURCE : w.source,
externalId: w.externalId ?? null,
externalSourceVersion: w.externalSourceVersion ?? null,
metadata: w.metadata
@@ -373,7 +416,10 @@ async function postBatch(request: NextRequest): Promise<Response> {
route,
samples,
dedupKey: w.externalId
? { source: w.source, externalId: w.externalId }
? {
source: scoped ? EXTERNAL_SOURCE : w.source,
externalId: w.externalId,
}
: null,
},
];
@@ -719,3 +765,16 @@ async function postBatch(request: NextRequest): Promise<Response> {
entries: results,
});
}
/** Whether any entry of a raw batch body names a `source` at all. */
function namesAnySource(rawBody: unknown): boolean {
if (typeof rawBody !== "object" || rawBody === null) return false;
const workouts = (rawBody as { workouts?: unknown }).workouts;
if (!Array.isArray(workouts)) return false;
return workouts.some(
(w) =>
typeof w === "object" &&
w !== null &&
(w as { source?: unknown }).source !== undefined,
);
}
src/components/admin/__tests__/backups-restore-preview.test.tsx
+136
View File
@@ -0,0 +1,136 @@
/**
* The restore dialog's contents box asks once and says honestly why it has
* nothing to show.
*
* A copy from an earlier version is read whole on the server, which took two
* minutes for a large record; the dialog stopped waiting, retried, and the
* server read the whole copy a second time (#1031). Pinned here: the preview
* query is never retried and carries its own timeout, and a timeout reads as
* "still being read", not as "could not be read".
*
* Mutation check: drop `retry: false` and the first case goes red; classify a
* timeout as unavailable and the second goes red.
*/
import { describe, expect, it, vi } from "vitest";
import { renderToStaticMarkup } from "react-dom/server";
import { ApiError } from "@/lib/api/api-fetch";
import { queryKeys } from "@/lib/query-keys";
const mocks = vi.hoisted(() => ({
queries: [] as Array<Record<string, unknown>>,
apiGet: vi.fn(async () => ({ summary: {} })),
}));
vi.mock("next/navigation", () => ({
useRouter: () => ({ push: vi.fn(), replace: vi.fn() }),
usePathname: () => "/admin/backups",
}));
vi.mock("sonner", () => ({ toast: { success: vi.fn(), error: vi.fn() } }));
vi.mock("@/lib/api/api-fetch", async (importOriginal) => {
const actual = await importOriginal<typeof import("@/lib/api/api-fetch")>();
return { ...actual, apiGet: mocks.apiGet };
});
vi.mock("@tanstack/react-query", () => ({
useQuery: (options: Record<string, unknown>) => {
mocks.queries.push(options);
return {
data: {
rows: [
{
id: "b1",
userId: "u1",
username: "self-hoster",
type: "WEEKLY_AUTO",
sizeBytes: 1024,
createdAt: "2026-09-01T03:00:00.000Z",
},
],
retentionDays: 30,
schedule: {
lastSuccessAt: "2026-09-01T03:00:00.000Z",
lastSuccessAgeDays: 2,
staleAfterDays: 10,
stale: false,
lastRun: null,
lastRunFailed: false,
},
},
isLoading: false,
isError: false,
};
},
useQueryClient: () => ({ invalidateQueries: vi.fn() }),
useMutation: () => ({
mutate: vi.fn(),
mutateAsync: vi.fn(),
isPending: false,
isError: false,
error: null,
variables: undefined,
}),
}));
vi.mock("@/hooks/use-auth", () => ({
useAuth: () => ({
user: { id: "u1", username: "testuser", role: "ADMIN" },
isAuthenticated: true,
isLoading: false,
refetch: vi.fn(),
}),
}));
import { I18nProvider } from "@/lib/i18n/context";
import { BackupsSection, restorePreviewFailure } from "../backups-section";
describe("restore preview query", () => {
it("is never retried and waits a bounded time", async () => {
renderToStaticMarkup(
<I18nProvider initialLocale="en">
<BackupsSection />
</I18nProvider>,
);
const key = JSON.stringify(queryKeys.adminBackupSummary("b1"));
const preview = mocks.queries.find(
(options) => JSON.stringify(options.queryKey) === key,
);
expect(preview).toBeDefined();
expect(preview!.retry).toBe(false);
await (preview!.queryFn as () => Promise<unknown>)();
const [path, init] = mocks.apiGet.mock.calls[0] as unknown as [
string,
RequestInit,
];
expect(path).toBe("/api/admin/backups/b1/summary");
expect(init.signal).toBeInstanceOf(AbortSignal);
});
});
describe("restorePreviewFailure", () => {
it("reads a timeout as a copy still being read", () => {
const timeout = new DOMException("signal timed out", "TimeoutError");
expect(restorePreviewFailure(timeout)).toBe("stillReading");
});
it("names a missing key", () => {
expect(
restorePreviewFailure(
new ApiError("key", 422, {
errorCode: "backup.key.missing",
keyIds: ["v1"],
}),
),
).toBe("keyMissing");
});
it("calls everything else unavailable", () => {
expect(restorePreviewFailure(new ApiError("boom", 500))).toBe(
"unavailable",
);
expect(restorePreviewFailure(new TypeError("offline"))).toBe("unavailable");
});
});
src/components/admin/backups-section.tsx
+50 -6
View File
@@ -90,6 +90,39 @@ import {
* Mirrors the wipe dialog's pattern but adds the typed gate because the
* blast radius is bigger (re-creates rows, not just deletes).
*/
/**
* How long the restore dialog waits for the contents of a copy. A copy's
* counts are kept with it and come back at once; this only runs out on a copy
* from an earlier version that the server is still reading whole, and stays
* under the minute a reverse proxy commonly allows a request.
*/
const PREVIEW_TIMEOUT_MS = 55_000;
/**
* What the restore dialog says when the contents did not come back: the key
* the copy needs and this server lacks, a copy from an earlier version still
* being read when the dialog stopped waiting, or anything else.
*/
export function restorePreviewFailure(
error: unknown,
): "keyMissing" | "stillReading" | "unavailable" {
if (
error instanceof ApiError &&
error.meta?.errorCode === "backup.key.missing"
) {
return "keyMissing";
}
// `AbortSignal.timeout` rejects the fetch with a DOMException of this name.
if (
typeof error === "object" &&
error !== null &&
(error as { name?: unknown }).name === "TimeoutError"
) {
return "stillReading";
}
return "unavailable";
}
/**
* v1.37.20 — the sections a restore preview names, in render order. Each
* entry sums one or more `BackupSummary` counters under one human label;
@@ -151,18 +184,25 @@ function RestoreRowDialog({
const [withInstanceSettings, setWithInstanceSettings] = useState(false);
// v1.37.20 — restore preview: fetch what the file contains the moment the
// dialog opens, so the typed confirmation is an informed one. Derived from
// the same decrypt + schema path the restore itself runs. A failed preview
// never blocks the restore — it states its own absence instead.
// dialog opens, so the typed confirmation is an informed one. A failed
// preview never blocks the restore — it states its own absence instead.
//
// Never retried. The server answers from counts kept with the copy, at
// once; only a copy from an earlier version is read whole, and that read
// goes on (and is kept) after this request gives up. A retry would only
// start the same read a second time (#1031).
const preview = useQuery({
queryKey: queryKeys.adminBackupSummary(row.id),
queryFn: () =>
apiGet<{ summary: Record<string, unknown> }>(
`/api/admin/backups/${row.id}/summary`,
{ signal: AbortSignal.timeout(PREVIEW_TIMEOUT_MS) },
),
enabled: open,
staleTime: 60_000,
retry: false,
});
const previewFailure = restorePreviewFailure(preview.error);
const numericSummary: Record<string, number> = {};
if (preview.data?.summary) {
for (const [key, value] of Object.entries(preview.data.summary)) {
@@ -239,17 +279,21 @@ function RestoreRowDialog({
</p>
)}
{preview.isError &&
(preview.error instanceof ApiError &&
preview.error.meta?.errorCode === "backup.key.missing" ? (
(previewFailure === "keyMissing" &&
preview.error instanceof ApiError ? (
// The one preview failure the restore would repeat: say which
// key, so the operator can put it back before trying.
<p role="alert" className="text-destructive">
{t("admin.section.backups.previewKeyMissing", {
keys: Array.isArray(preview.error.meta.keyIds)
keys: Array.isArray(preview.error.meta?.keyIds)
? preview.error.meta.keyIds.map(String).join(", ")
: "",
})}
</p>
) : previewFailure === "stillReading" ? (
<p className="text-muted-foreground">
{t("admin.section.backups.previewStillReading")}
</p>
) : (
<p className="text-muted-foreground">
{t("admin.section.backups.previewUnavailable")}
src/components/dashboard/__tests__/medication-intake-quick-add.test.tsx
+43
View File
@@ -71,6 +71,7 @@ function makeMed(
name: `Med ${id}`,
dose: "5 mg",
active: true,
intakeActionable: true,
schedules: [],
lastTakenAt: null,
todayEventCount: 0,
@@ -316,6 +317,48 @@ describe("<MedicationIntakeQuickAdd> — SSR contract", () => {
}
});
// #1040 — the picker offers what the server calls actionable, nothing
// it re-derives. An ended course is still `active` (nobody paused it),
// which is exactly the case the old `active && trackIntake` filter let
// through.
it("does not offer an ended course, even though it is still active (#1040)", () => {
mockUseQuery.mockReturnValue({
data: [
makeMed("ended", {
name: "Alpha",
active: true,
intakeActionable: false,
}),
makeMed("current", { name: "Zulu" }),
],
isLoading: false,
});
const html = renderSSR(<MedicationIntakeQuickAdd />);
// Alphabetical fallback would land on "Alpha" if it were offered.
expect(html).toMatch(
/data-slot="select-value"[^>]*>[\s\S]*?Zulu[\s\S]*?<\/span>/,
);
expect(html).not.toContain("Alpha");
});
it("shows the empty state when no medication is actionable today (#1040)", () => {
mockUseQuery.mockReturnValue({
data: [
makeMed("ended", { name: "Alpha", intakeActionable: false }),
makeMed("upcoming", { name: "Beta", intakeActionable: false }),
],
isLoading: false,
});
const html = renderSSR(<MedicationIntakeQuickAdd />);
expect(html).toContain('data-testid="medication-intake-quick-add-empty"');
expect(html).not.toContain(
'data-testid="medication-intake-quick-add-form"',
);
// The user has medications, so "no medications yet" would be untrue.
expect(html).toContain("Nothing to take today");
expect(html).not.toContain("No medications yet");
});
it("does not leak raw i18n keys in either locale", () => {
mockUseQuery.mockReturnValue({ data: [], isLoading: false });
const en = renderSSR(<MedicationIntakeQuickAdd />, "en");
src/components/dashboard/medication-intake-quick-add.tsx
+26 -8
View File
@@ -81,8 +81,13 @@ interface Schedule extends ScheduleWindowInput {
export interface MedicationOption extends DefaultMedicationOption {
dose: string;
schedules: Schedule[];
/** v1.39.1 (#1033) — false: kept as a record, not offered for a dose. */
trackIntake?: boolean;
/**
* The server's answer to "does this medication offer a dose today"
* (active, intake tracked, course includes today; see
* `src/lib/medications/intake-actionable.ts`). The list read always
* carries it, so the picker renders from it and re-derives nothing.
*/
intakeActionable: boolean;
}
interface MedicationIntakeQuickAddProps {
@@ -141,9 +146,10 @@ export function MedicationIntakeQuickAdd({
const medications = useMemo(
() =>
Array.isArray(medicationsRaw)
? // v1.39.1 (#1033) — a medication with intake tracking off is
// kept as a record; the dose capture does not offer it.
medicationsRaw.filter((m) => m.active && m.trackIntake !== false)
? // #1040 — offer exactly what the cards, the table and "take
// all" offer: a paused, record-only, not-yet-started or ended
// course is not a dose to capture now.
medicationsRaw.filter((m) => m.intakeActionable === true)
: [],
[medicationsRaw],
);
@@ -314,6 +320,10 @@ export function MedicationIntakeQuickAdd({
// anlegen</Link>` so the footer-slot promise holds across both
// branches.
if (!medicationsLoading && medications.length === 0) {
// Medications exist, none offers a dose today: say that, rather than
// "no medications yet", which would be untrue.
const noneActionable =
Array.isArray(medicationsRaw) && medicationsRaw.length > 0;
const emptyFooter = (
<div className="flex w-full items-center justify-end gap-2">
{onCancel && (
@@ -328,7 +338,9 @@ export function MedicationIntakeQuickAdd({
)}
<Button asChild className="min-h-11 sm:min-h-9">
<Link href="/medications">
{t("dashboard.medicationIntakeQuickAdd.emptyCta")}
{noneActionable
? t("dashboard.medicationIntakeQuickAdd.noneActionableCta")
: t("dashboard.medicationIntakeQuickAdd.emptyCta")}
</Link>
</Button>
</div>
@@ -344,10 +356,16 @@ export function MedicationIntakeQuickAdd({
<Pill className="text-muted-foreground h-6 w-6" aria-hidden="true" />
<div className="space-y-1">
<p className="text-sm font-medium">
{t("dashboard.medicationIntakeQuickAdd.emptyTitle")}
{noneActionable
? t("dashboard.medicationIntakeQuickAdd.noneActionableTitle")
: t("dashboard.medicationIntakeQuickAdd.emptyTitle")}
</p>
<p className="text-muted-foreground text-xs">
{t("dashboard.medicationIntakeQuickAdd.emptyDescription")}
{noneActionable
? t(
"dashboard.medicationIntakeQuickAdd.noneActionableDescription",
)
: t("dashboard.medicationIntakeQuickAdd.emptyDescription")}
</p>
</div>
</div>
src/components/settings/api-section.tsx
+35 -1
View File
@@ -3,6 +3,7 @@
import { useState } from "react";
import { useQuery, useQueryClient } from "@tanstack/react-query";
import {
Activity,
Check,
ChevronDown,
Copy,
@@ -62,6 +63,7 @@ export function ApiSection() {
<div className="space-y-6">
<ApiEndpointsCard />
<MeasurementTokensCard />
<WorkoutTokensCard />
<DocumentTokensCard />
<ApiTokensCard />
</div>
@@ -95,6 +97,29 @@ function MeasurementTokensCard() {
);
}
/**
* Mint a token a workout bridge pushes through (#1054): a watch vendor's sync
* relayed by a small service, or a script replaying an export. Same card,
* same no-list reasoning as the measurement one above.
*/
function WorkoutTokensCard() {
const { t } = useTranslations();
return (
<IngestTokenCard
endpoint="/api/tokens/workouts"
icon={Activity}
slot="settings-workouts-token-created"
copy={{
title: t("settings.workoutsToken.title"),
description: t("settings.workoutsToken.description"),
detail: t("settings.workoutsToken.detail"),
scopeNote: t("settings.workoutsToken.scopeNote"),
createFailed: t("settings.workoutsToken.createFailed"),
}}
/>
);
}
/**
* Mint a token another document system uploads through (#1038): a Paperless
* workflow or the import script. Same card, same no-list reasoning as the
@@ -125,7 +150,10 @@ function IngestTokenCard({
slot,
copy,
}: {
endpoint: "/api/tokens/measurements" | "/api/tokens/documents";
endpoint:
| "/api/tokens/measurements"
| "/api/tokens/workouts"
| "/api/tokens/documents";
icon: LucideIcon;
slot: string;
copy: {
@@ -296,6 +324,12 @@ function ApiEndpointsCard() {
auth: "Authorization: Bearer hlk_...",
example: `{ "entries": [ { "hkIdentifier": "...", "value": 1, ... } ] }`,
},
{
method: "POST",
path: "/api/workouts/batch",
auth: "Authorization: Bearer hlk_...",
example: `{ "workouts": [ { "sportType": "running", "startedAt": "...", "endedAt": "...", "externalId": "..." } ] }`,
},
{
method: "POST",
path: "/api/documents/inbound",
src/components/settings/security-section/__tests__/recent-proof-callers.test.tsx
+2
View File
@@ -71,6 +71,8 @@ describe.each([
["components/settings/share-link-create-form.tsx", "/api/share-links"],
["components/settings/mcp-section.tsx", "/api/mcp/tokens"],
["components/settings/api-section.tsx", "/api/tokens/measurements"],
["components/settings/api-section.tsx", "/api/tokens/documents"],
["components/settings/api-section.tsx", "/api/tokens/workouts"],
["components/admin/backups-section.tsx", "/download"],
["components/admin/backups-section.tsx", "/restore"],
["components/admin/backups-section.tsx", "/api/admin/backups/upload"],
src/lib/export/__tests__/backup-preview.test.ts
+222
View File
@@ -0,0 +1,222 @@
/**
* The preview worked out while a copy is written has to say what the preview
* route used to derive by reading the stored copy: the same counts, the same
* key uses. Held here against that derivation over the same file, fed in
* pieces that split values, strings and multi-byte characters.
*/
import { afterEach, describe, expect, it } from "vitest";
import { _resetCryptoCacheForTests, encrypt } from "@/lib/crypto";
import {
BackupKeyIdCollector,
assessBackupKeys,
} from "@/lib/export/backup-key-ids";
import {
buildBackupPreview,
createBackupPreviewScanner,
storedCopyIdentity,
storedPreviewFor,
} from "@/lib/export/backup-preview";
import { backupPayloadSchema, summarizeBackup } from "@/lib/validations/backup";
const ts = "2026-07-19T07:00:00.000Z";
/** A bytes-column value holding the string codec under key id `v3`. */
const sealedV3 = Buffer.from(`v3.${"QUJD".repeat(20)}`, "utf8").toString(
"base64",
);
const FILE = {
schemaVersion: "2",
exportedAt: ts,
userId: "u1",
measurements: [1, 2, 3].map((i) => ({
id: `m${i}`,
type: "WEIGHT",
value: 70 + i,
unit: "kg",
measuredAt: ts,
source: "MANUAL",
notes: "Grüße, ☕",
...(i === 2 ? { notesEncrypted: sealedV3 } : {}),
})),
intakeEvents: [
{ medication: "Example", scheduledFor: ts, source: "WEB" },
{ medication: "Example", scheduledFor: ts, source: "WEB" },
],
moodEntries: [
{
date: "2026-05-08",
mood: "GUT",
score: 4,
source: "MOODLOG",
loggedAt: ts,
},
],
labResults: [{ analyte: "HbA1c", unit: "%", takenAt: ts, source: "MANUAL" }],
illnessEpisodes: [
{
id: "ep-1",
label: "Cold",
type: "INFECTION",
lifecycle: "ACUTE",
onsetAt: ts,
dayLogs: [
{ date: "2026-04-01", symptoms: [] },
{ date: "2026-04-02", symptoms: [] },
],
},
],
};
async function scan(pieces: Array<string | Uint8Array>) {
const scanner = createBackupPreviewScanner();
for (const piece of pieces) await scanner.feed(piece);
return scanner.finish();
}
function split(text: string, size: number): Uint8Array[] {
const bytes = Buffer.from(text, "utf8");
const out: Uint8Array[] = [];
for (let at = 0; at < bytes.length; at += size) {
out.push(bytes.subarray(at, at + size));
}
return out;
}
describe("createBackupPreviewScanner", () => {
const json = JSON.stringify(FILE);
// What the preview route derived from a stored copy.
const expected = summarizeBackup(backupPayloadSchema.parse(FILE));
const expectedKeys = new BackupKeyIdCollector();
expectedKeys.visit(FILE);
it.each([1, 7, 64, 100_000])(
"counts what the full read counts, in pieces of %i bytes",
async (size) => {
const result = await scan(split(json, size));
expect(result).not.toBeNull();
expect(result!.summary).toEqual(expected);
expect(result!.summary.measurements).toBe(3);
expect(result!.summary.intakeEvents).toBe(2);
expect(result!.summary.moodEntries).toBe(1);
expect(result!.summary.illnessDayLogs).toBe(2);
expect(result!.keys.toStored()).toEqual(expectedKeys.toStored());
expect(result!.keys.keyIds()).toEqual(["v3"]);
},
);
it("takes text pieces the way the weekly writer hands them over", async () => {
const result = await scan([json.slice(0, 50), json.slice(50)]);
expect(result!.summary).toEqual(expected);
});
it("answers null for a file that is not a backup, and never holds the writer", async () => {
expect(await scan(["{ not json", "more", "and more"])).toBeNull();
expect(await scan([JSON.stringify({ schemaVersion: 2 })])).toBeNull();
expect(await scan([json.slice(0, 40)])).toBeNull();
});
});
describe("storedPreviewFor", () => {
const chunked = { data: null, chunkCount: 4, chunkStreamId: "s1" };
const preview = () =>
buildBackupPreview(
storedCopyIdentity(chunked)!,
summarizeBackup(backupPayloadSchema.parse(FILE)),
new BackupKeyIdCollector(),
);
it("reads a preview of the copy the row holds", () => {
const stored = JSON.parse(JSON.stringify(preview()));
expect(storedPreviewFor(stored, chunked)?.summary.measurements).toBe(3);
});
it("ignores a preview of another copy", () => {
expect(
storedPreviewFor(preview(), { ...chunked, chunkStreamId: "s2" }),
).toBeNull();
expect(
storedPreviewFor(preview(), { ...chunked, chunkCount: 5 }),
).toBeNull();
});
it("describes nothing on a row holding both forms, or none", () => {
expect(storedPreviewFor(preview(), { ...chunked, data: "x" })).toBeNull();
expect(
storedPreviewFor(preview(), {
data: null,
chunkCount: null,
chunkStreamId: null,
}),
).toBeNull();
});
it("ignores a malformed preview", () => {
expect(storedPreviewFor({ version: 1 }, chunked)).toBeNull();
expect(storedPreviewFor(null, chunked)).toBeNull();
});
it("names a single stored value by its length and head", () => {
const a = storedCopyIdentity({
data: "v1.aaaa",
chunkCount: null,
chunkStreamId: null,
});
const b = storedCopyIdentity({
data: "v1.aaab",
chunkCount: null,
chunkStreamId: null,
});
expect(a).toMatch(/^value:[0-9a-f]{64}$/);
expect(a).not.toBe(b);
});
});
describe("BackupKeyIdCollector.toStored / fromStored", () => {
const saved = {
key: process.env.ENCRYPTION_KEY,
keys: process.env.ENCRYPTION_KEYS,
active: process.env.ENCRYPTION_ACTIVE_KEY_ID,
};
function useKeys(keys: Record<string, string>, active: string) {
delete process.env.ENCRYPTION_KEY;
process.env.ENCRYPTION_KEYS = JSON.stringify(keys);
process.env.ENCRYPTION_ACTIVE_KEY_ID = active;
_resetCryptoCacheForTests();
}
afterEach(() => {
for (const [name, value] of [
["ENCRYPTION_KEY", saved.key],
["ENCRYPTION_KEYS", saved.keys],
["ENCRYPTION_ACTIVE_KEY_ID", saved.active],
] as const) {
if (value === undefined) delete process.env[name];
else process.env[name] = value;
}
_resetCryptoCacheForTests();
});
it("keeps what the verdict needs, so it can be taken again later", () => {
// Written under `old` and `other` on the host that made the copy.
useKeys({ old: "11".repeat(32), other: "22".repeat(32) }, "old");
const collector = new BackupKeyIdCollector();
collector.add("moodEntries", encrypt("a note"));
process.env.ENCRYPTION_ACTIVE_KEY_ID = "other";
_resetCryptoCacheForTests();
collector.add("labResults", encrypt("another note"));
collector.add("appSettings", encrypt("a setting"));
const stored = JSON.parse(JSON.stringify(collector.toStored()));
const options = { ignoreSections: new Set(["appSettings"]) };
// Later, on a host that dropped `old` and holds different material
// under `other`.
useKeys({ other: "33".repeat(32) }, "other");
const back = BackupKeyIdCollector.fromStored(stored);
const verdict = assessBackupKeys(back, options);
expect(verdict.missing).toEqual(["old"]);
expect(verdict.unreadable).toEqual(["other"]);
expect(verdict).toEqual(assessBackupKeys(collector, options));
expect(back.toStored()).toEqual(collector.toStored());
});
});
src/lib/export/backup-key-ids.ts
+37
View File
@@ -185,6 +185,43 @@ export class BackupKeyIdCollector {
entries(): ReadonlyMap<string, KeyUse> {
return this.uses;
}
/**
* What was collected, as plain data, for a copy's stored preview
* (`backup-preview.ts`): enough to hold it against this host's keys again
* later without reading the copy.
*/
toStored(): StoredKeyUse[] {
return [...this.uses.entries()]
.sort(([a], [b]) => a.localeCompare(b))
.map(([keyId, use]) => ({
keyId,
count: use.count,
sections: [...use.sections].sort(),
sample: use.sample ? { ...use.sample } : null,
}));
}
/** The collector {@link toStored} was taken from. */
static fromStored(stored: readonly StoredKeyUse[]): BackupKeyIdCollector {
const collector = new BackupKeyIdCollector();
for (const entry of stored) {
collector.uses.set(entry.keyId, {
count: entry.count,
sections: new Set(entry.sections),
sample: entry.sample ? { ...entry.sample } : null,
});
}
return collector;
}
}
/** One key id a copy needs, as {@link BackupKeyIdCollector.toStored} keeps it. */
export interface StoredKeyUse {
keyId: string;
count: number;
sections: string[];
sample: { value: string; form: InnerCiphertextForm } | null;
}
export interface BackupKeyAssessment {
src/lib/export/backup-preview.ts
+223
View File
@@ -0,0 +1,223 @@
/**
* What the restore preview shows for a stored backup, worked out while the
* copy is written and kept on its row (`DataBackup.preview`).
*
* Why. The preview used to read the whole stored copy inside the request that
* asked for it: open every piece, parse every measurement, validate the rest.
* For an account of 1.8 million readings on slow hardware that took two
* minutes, the dialog gave up long before, and the operator was told the
* contents could not be read (#1031). The copy passes through this process
* once anyway, on its way into the pieces, so the counts are taken there.
*
* What is kept. The `summarizeBackup` counts of the copy, its schema version,
* and the inner key ids with the sections they sit in and one sample value
* each. The verdicts are not kept: whether this release can restore the
* schema version, and whether this server holds the keys the copy needs, can
* both change after the copy was written, so the route takes them again from
* these inputs on every read.
*
* Which copy. A preview names the copy it describes (`copy`), and a reader
* ignores one that names another: a copy replaced by a writer that does not
* know about previews (an older release, after a downgrade) must not be
* described by the numbers of the one it replaced.
*/
import { createHash } from "node:crypto";
import { z } from "zod/v4";
import {
BackupKeyIdCollector,
type StoredKeyUse,
} from "@/lib/export/backup-key-ids";
import { scanBackupJson } from "@/lib/export/backup-json-scan";
import type { StoredBackupRef } from "@/lib/export/stored-backup";
import {
backupPayloadSchema,
summarizeBackup,
type BackupSummary,
} from "@/lib/validations/backup";
export interface BackupPreview {
version: 1;
/** The copy this describes (`storedCopyIdentity`). */
copy: string;
summary: BackupSummary;
keys: StoredKeyUse[];
}
const storedPreviewSchema = z.object({
version: z.literal(1),
copy: z.string().min(1),
summary: z
.object({
schemaVersion: z.string(),
userId: z.string(),
exportedAt: z.string(),
measurements: z.number(),
})
.catchall(z.union([z.string(), z.number()])),
keys: z.array(
z.object({
keyId: z.string(),
count: z.number().int().nonnegative(),
sections: z.array(z.string()),
sample: z
.object({
value: z.string(),
form: z.enum(["string", "bytes-string", "binary"]),
})
.nullable(),
}),
),
});
/**
* How a single stored value is told apart from another: its length and its
* first characters, which carry the key id and the random nonce of the
* envelope. Hashed, so the preview does not repeat the envelope's head.
*/
export function singleValueIdentity(length: number, head: string): string {
return `value:${createHash("sha256").update(`${length}:${head}`).digest("hex")}`;
}
/** How many leading characters of a single stored value the identity reads. */
export const SINGLE_VALUE_IDENTITY_HEAD = 128;
/**
* The name of the copy a row holds now, or null for a row nothing may
* describe: one holding both forms, or neither.
*/
export function storedCopyIdentity(
backup: Pick<StoredBackupRef, "data" | "chunkCount" | "chunkStreamId">,
): string | null {
if (backup.data != null && backup.chunkStreamId != null) return null;
if (backup.chunkStreamId != null) {
return `chunks:${backup.chunkStreamId}:${backup.chunkCount ?? ""}`;
}
if (backup.data != null) {
return singleValueIdentity(
backup.data.length,
backup.data.slice(0, SINGLE_VALUE_IDENTITY_HEAD),
);
}
return null;
}
export function buildBackupPreview(
copy: string,
summary: BackupSummary,
keys: BackupKeyIdCollector,
): BackupPreview {
return { version: 1, copy, summary, keys: keys.toStored() };
}
/**
* The stored preview of a row, when it describes the copy the row holds now;
* null when there is none, it is malformed, or it describes another copy.
*/
export function storedPreviewFor(
stored: unknown,
backup: Pick<StoredBackupRef, "data" | "chunkCount" | "chunkStreamId">,
): BackupPreview | null {
const copy = storedCopyIdentity(backup);
if (copy === null || stored == null) return null;
const parsed = storedPreviewSchema.safeParse(stored);
if (!parsed.success || parsed.data.copy !== copy) return null;
return parsed.data as unknown as BackupPreview;
}
/**
* The sections that grow with a record. Their elements are counted and their
* key ids read as they pass, and never kept, so the scan holds no more of the
* copy than the writer itself does (`full-backup-stream.ts`).
*/
const STREAMED = new Set(["measurements", "intakeEvents", "moodEntries"]);
export interface BackupPreviewScanner {
/**
* Hand over the next piece of the JSON. Resolves once the scan has taken
* it, so a writer never runs more than a piece ahead of the scan.
*/
feed(piece: string | Uint8Array): Promise<void>;
/**
* The counts and key uses, once every piece has been fed; null when the
* JSON was not a backup the schema accepts. Never rejects: a copy is
* stored whether or not its preview could be worked out.
*/
finish(): Promise<{
summary: BackupSummary;
keys: BackupKeyIdCollector;
} | null>;
}
/**
* Work out a preview from a copy's JSON as it is written. The counts are the
* ones the preview route derives from the stored copy (`summarizeBackup` over
* the parsed file), the three bulk sections counted rather than parsed.
*/
export function createBackupPreviewScanner(): BackupPreviewScanner {
const queue: Array<{ chunk: Uint8Array; taken: () => void }> = [];
let ended = false;
let stopped = false;
let wake: (() => void) | null = null;
const signal = () => {
const w = wake;
wake = null;
w?.();
};
async function* pieces(): AsyncGenerator<Uint8Array> {
for (;;) {
const next = queue.shift();
if (next) {
next.taken();
yield next.chunk;
continue;
}
if (ended) return;
await new Promise<void>((resolve) => {
wake = resolve;
});
}
}
const keys = new BackupKeyIdCollector();
const result = scanBackupJson(pieces(), {
streamKeys: STREAMED,
onElement: (key, element) => keys.visit(element, key),
})
.then(({ document, streamedCounts }) => {
keys.visit(document);
const parsed = backupPayloadSchema.safeParse(document);
if (!parsed.success) return null;
const summary: BackupSummary = {
...summarizeBackup(parsed.data),
measurements: streamedCounts.measurements ?? 0,
intakeEvents: streamedCounts.intakeEvents ?? 0,
moodEntries: streamedCounts.moodEntries ?? 0,
};
return { summary, keys };
})
.catch(() => null)
.finally(() => {
// A scan that stopped early takes nothing more: release the writer.
stopped = true;
for (const waiting of queue.splice(0)) waiting.taken();
});
return {
feed(piece) {
if (stopped) return Promise.resolve();
const chunk =
typeof piece === "string" ? Buffer.from(piece, "utf8") : piece;
return new Promise<void>((taken) => {
queue.push({ chunk, taken });
signal();
});
},
finish() {
ended = true;
signal();
return result;
},
};
}
src/lib/export/store-backup-blob.ts
+66 -5
View File
@@ -23,7 +23,7 @@
* this module exists to remove. A model create is planned once for its shape
* and keeps nothing.
*/
import type { PrismaClient } from "@/generated/prisma/client";
import { Prisma, type PrismaClient } from "@/generated/prisma/client";
import { isStreamCiphertext, openStreamDecryptor } from "@/lib/crypto";
import {
@@ -35,6 +35,14 @@ import {
type PackBackupOptions,
} from "@/lib/export/backup-blob";
import { createBackupKeyIdTextScanner } from "@/lib/export/backup-key-ids";
import {
buildBackupPreview,
createBackupPreviewScanner,
SINGLE_VALUE_IDENTITY_HEAD,
singleValueIdentity,
storedCopyIdentity,
type BackupPreview,
} from "@/lib/export/backup-preview";
import {
BackupIntegrityError,
newChunkStreamId,
@@ -118,6 +126,9 @@ export async function storeBackupBlob(
// what is inside them, so this is what tells an operator how long a
// retired key is still needed (`GET /api/admin/encryption/status`).
const keyScanner = createBackupKeyIdTextScanner();
// And what the restore preview shows, from the same pass, so the
// preview never has to read the copy back (`backup-preview.ts`).
const previewScanner = createBackupPreviewScanner();
const { chunks, bytes } = await packBackupChunks(
async (sealed, seq) => {
await tx.dataBackupChunk.create({
@@ -129,10 +140,12 @@ export async function storeBackupBlob(
(write) =>
producer(async (chunk) => {
keyScanner.feed(chunk);
await previewScanner.feed(chunk);
await write(chunk);
}),
options,
);
const scanned = await previewScanner.finish();
// Again at the end: a restore may have been queued while this ran. The
// window left is the few milliseconds to the commit, and a restore
@@ -147,6 +160,18 @@ export async function storeBackupBlob(
chunkStreamId: streamId,
innerKeyIds: keyScanner.keyIds(),
innerKeyIdsRecorded: true,
// Prisma's JSON null clears a preview left by the previous copy.
preview: scanned
? (buildBackupPreview(
storedCopyIdentity({
data: null,
chunkCount: chunks,
chunkStreamId: streamId,
})!,
scanned.summary,
scanned.keys,
) as unknown as Prisma.InputJsonValue)
: Prisma.DbNull,
createdAt: new Date(),
},
});
@@ -156,6 +181,17 @@ export async function storeBackupBlob(
);
}
/** A single value's stored preview, when it describes that value. */
function previewOfSingleValue(
stored: unknown,
len: number,
head: string,
): BackupPreview | null {
if (!stored || typeof stored !== "object") return null;
const preview = stored as BackupPreview;
return preview.copy === singleValueIdentity(len, head) ? preview : null;
}
/**
* The first characters of a single stored value, which carry its key id, or
* null when the row no longer holds one. For the key-rotation scan, which must
@@ -208,10 +244,16 @@ export async function convertSingleValueBackup(
`SET LOCAL idle_in_transaction_session_timeout = '${STORE_IDLE_TIMEOUT}'`,
);
const [row] = await tx.$queryRaw<
Array<{ head: string; len: number; stream_id: string | null }>
Array<{
head: string;
len: number;
stream_id: string | null;
preview: unknown;
}>
>`
SELECT left(data, 128) AS head, length(data)::int AS len,
chunk_stream_id AS stream_id
SELECT left(data, ${SINGLE_VALUE_IDENTITY_HEAD}::int) AS head,
length(data)::int AS len,
chunk_stream_id AS stream_id, preview
FROM data_backups
WHERE id = ${id} AND data IS NOT NULL
FOR UPDATE
@@ -272,9 +314,28 @@ export async function convertSingleValueBackup(
// limit on writing a new one.
{ maxBytes: Number.MAX_SAFE_INTEGER },
);
// The content is the same copy in another form, so a preview of it
// still holds; it is moved over to the new form's name.
const preview = previewOfSingleValue(row.preview, row.len, row.head);
await tx.dataBackup.update({
where: { id },
data: { data: null, chunkCount: chunks, chunkStreamId: streamId },
data: {
data: null,
chunkCount: chunks,
chunkStreamId: streamId,
...(preview
? {
preview: {
...preview,
copy: storedCopyIdentity({
data: null,
chunkCount: chunks,
chunkStreamId: streamId,
})!,
} as unknown as Prisma.InputJsonValue,
}
: {}),
},
});
return "converted";
},
src/lib/export/stored-backup.ts
+66 -18
View File
@@ -141,6 +141,71 @@ async function* openedChunks(
}
}
/**
* Refuse a chunked copy whose row does not list as many pieces as are stored:
* one missing, or one this code never wrote. A count, without reading a
* piece; `openStoredBackup` goes on to authenticate every piece.
*/
async function checkStoredBackupPieceCount(
prisma: ChunkReader,
id: string,
streamId: string,
count: number,
): Promise<void> {
if (count < 1) {
throw new BackupIntegrityError("The stored copy lists no pieces.");
}
// Pieces beyond the count would be ignored by the reads, but they are not
// something this code ever writes, so their presence is refused too.
const stored = await prisma.dataBackupChunk.count({
where: { backupId: id },
});
if (stored !== count) {
const now = await prisma.dataBackup.findUnique({
where: { id },
select: { chunkStreamId: true },
});
if (now?.chunkStreamId !== streamId) throw new BackupReplacedError();
throw new BackupIntegrityError(
`The stored copy lists ${count} pieces but ${stored} are stored.`,
);
}
}
/**
* The checks on a stored copy that need no piece read: both forms, a count
* without a stream id or the reverse, no content, and the pieces a chunked
* copy lists against those stored. Throws what `openStoredBackup` would
* throw for the same row. For a reader that answers from what was kept about
* the copy (the restore preview) and must still not describe a copy the
* restore would refuse on sight.
*/
export async function checkStoredBackupShape(
prisma: ChunkReader,
backup: StoredBackupRef,
): Promise<void> {
if (backup.data != null && backup.chunkStreamId != null) {
throw new BackupFormsConflictError();
}
if ((backup.chunkStreamId == null) !== (backup.chunkCount == null)) {
throw new BackupIntegrityError(
"The stored copy lists pieces without saying how many, or the reverse.",
);
}
if (backup.chunkStreamId == null || backup.chunkCount == null) {
if (backup.data == null) {
throw new BackupIntegrityError("The stored copy has no content.");
}
return;
}
await checkStoredBackupPieceCount(
prisma,
backup.id,
backup.chunkStreamId,
backup.chunkCount,
);
}
/**
* Open a stored copy as a source of its JSON bytes, openable as many times as
* the caller needs. Rejects when the copy cannot be read whole: a key that is
@@ -166,24 +231,7 @@ export async function openStoredBackup(
return openBackupBlob(backup.data);
}
const { id, chunkStreamId: streamId, chunkCount: count } = backup;
if (count < 1) {
throw new BackupIntegrityError("The stored copy lists no pieces.");
}
// Pieces beyond the count would be ignored by the reads below, but they are
// not something this code ever writes, so their presence is refused too.
const stored = await prisma.dataBackupChunk.count({
where: { backupId: id },
});
if (stored !== count) {
const now = await prisma.dataBackup.findUnique({
where: { id },
select: { chunkStreamId: true },
});
if (now?.chunkStreamId !== streamId) throw new BackupReplacedError();
throw new BackupIntegrityError(
`The stored copy lists ${count} pieces but ${stored} are stored.`,
);
}
await checkStoredBackupPieceCount(prisma, id, streamId, count);
// The whole copy, checked once before anyone reads a byte of it.
for await (const piece of openedChunks(prisma, id, streamId, count)) {
void piece;
src/lib/google-health/client.ts
+32 -1
View File
@@ -121,6 +121,28 @@ export async function runWithGoogleHealthClientOutcome<T>(
return { result, outcome };
}
/**
* A stop condition for the walks below, set by a caller that runs under a
* time limit (the full-history backfill, which runs under its pg-boss job's
* budget and abort signal). A walk asks it before each request and, when it
* says stop, ends there and reports itself truncated, the same outcome as
* reaching the page ceiling: the resource is incomplete, the cycle is not
* stamped, and the reason shown is that only part of the history was
* processed. Outside such a scope the walks never stop early.
*/
const stopStorage = new AsyncLocalStorage<() => boolean>();
function stopRequested(): boolean {
return stopStorage.getStore()?.() === true;
}
export async function runWithGoogleHealthStop<T>(
shouldStop: () => boolean,
fn: () => Promise<T>,
): Promise<T> {
return stopStorage.run(shouldStop, fn);
}
export const GOOGLE_HEALTH_API_BASE = "https://health.googleapis.com/v4";
const GOOGLE_OAUTH_AUTH_URL = "https://accounts.google.com/o/oauth2/v2/auth";
const GOOGLE_OAUTH_TOKEN_URL = "https://oauth2.googleapis.com/token";
@@ -769,8 +791,13 @@ export async function forEachDataPointPage(
let fetched = 0;
let pageToken: string | null | undefined;
let pageCount = 0;
let stopped = false;
do {
if (stopRequested()) {
stopped = true;
break;
}
const params = new URLSearchParams({ pageSize: String(pageSize) });
if (query.start) {
const { field, bound } = incrementalFilter(
@@ -830,7 +857,7 @@ export async function forEachDataPointPage(
await onPage(page);
} while (pageToken && pageCount < maxPages);
notePagination(pageCount, fetched, Boolean(pageToken));
notePagination(pageCount, fetched, stopped || Boolean(pageToken));
};
// The fallback only exists for filtered daily-summary reads — every other
@@ -1055,6 +1082,10 @@ export async function fetchDailyRollUp(
let totalPages = 0;
let truncated = false;
for (const chunk of chunkCivilRange(startCivil, endCivil, maxDays)) {
if (stopRequested()) {
truncated = true;
break;
}
let pageToken: string | null | undefined;
let pageCount = 0;
do {
src/lib/jobs/__tests__/google-health-backfill.test.ts
+42
View File
@@ -27,6 +27,12 @@ vi.mock("@/lib/logging/context", () => ({
getEvent: () => null,
}));
const { safeFetchMock } = vi.hoisted(() => ({ safeFetchMock: vi.fn() }));
vi.mock("@/lib/safe-fetch", async (importOriginal) => {
const actual = await importOriginal<typeof import("@/lib/safe-fetch")>();
return { ...actual, safeFetch: safeFetchMock };
});
vi.mock("@/lib/integrations/status", () => ({
isReauthRequired: isReauthRequiredMock,
}));
@@ -39,6 +45,11 @@ vi.mock("@/lib/google-health/sync", () => ({
}));
import { runGoogleHealthBackfillForUser } from "../google-health-backfill";
import {
GOOGLE_HEALTH_DATA_TYPES,
forEachDataPointPage,
runWithGoogleHealthClientOutcome,
} from "@/lib/google-health/client";
beforeEach(() => {
vi.clearAllMocks();
@@ -83,4 +94,35 @@ describe("runGoogleHealthBackfillForUser — verdict-gated marker", () => {
expect(syncUserGoogleHealthMock).not.toHaveBeenCalled();
expect(prismaMock.googleHealthConnection.update).not.toHaveBeenCalled();
});
it("stops the walk when its budget runs out and throws without stamping", async () => {
// An endless heart-rate stream: every page names a next one.
safeFetchMock.mockImplementation(async () => ({
status: 200,
ok: true,
json: async () => ({ dataPoints: [{}], nextPageToken: "more" }),
}));
// The real walk under the sync mock, reporting what the real sync reports
// for a truncated resource.
syncUserGoogleHealthMock.mockImplementation(async () => {
const { outcome } = await runWithGoogleHealthClientOutcome(() =>
forEachDataPointPage(
GOOGLE_HEALTH_DATA_TYPES.heartRate,
"token",
"fetchHeartRate",
{ maxPages: 10_000 },
() => {},
),
);
return { imported: outcome.fetched, failed: outcome.truncated };
});
let asked = 0;
const shouldStop = () => ++asked > 3;
await expect(
runGoogleHealthBackfillForUser("u1", shouldStop),
).rejects.toThrow(/incomplete/);
expect(safeFetchMock).toHaveBeenCalledTimes(3);
expect(prismaMock.googleHealthConnection.update).not.toHaveBeenCalled();
});
});
src/lib/jobs/__tests__/integration-backfill-admission.test.ts
+9
View File
@@ -6,6 +6,7 @@ import {
INTEGRATION_BACKFILL_ADMISSION_GROUP,
INTEGRATION_BACKFILL_ADMISSION_QUEUE,
INTEGRATION_BACKFILL_BOOT_ORDER,
INTEGRATION_BACKFILL_EXPIRE_SECONDS,
INTEGRATION_BACKFILL_GLOBAL_CONCURRENCY,
INTEGRATION_BACKFILL_STAGGER_SECONDS,
bootStaggerSecondsFor,
@@ -109,12 +110,20 @@ describe("shared full-history backfill admission", () => {
expect.objectContaining({
singletonKey: "sleep-timeline-backfill|WHOOP|u1",
group: INTEGRATION_BACKFILL_ADMISSION_GROUP,
expireInSeconds: INTEGRATION_BACKFILL_EXPIRE_SECONDS,
}),
);
expect(first).toBe("job-1");
expect(duplicate).toBeNull();
});
it("gives the admission job an expiry sized for a long import, under pg-boss's 24-hour ceiling", () => {
// pg-boss's default is 900 s; a full-history heart-rate import runs for
// hours, and past the expiry pg-boss retries the job beside the run.
expect(INTEGRATION_BACKFILL_EXPIRE_SECONDS).toBe(16 * 60 * 60);
expect(INTEGRATION_BACKFILL_EXPIRE_SECONDS).toBeLessThan(24 * 60 * 60);
});
it("keeps source enqueue singleton keys while applying stagger only to boot discovery", () => {
expect(integrationBackfillSourceOptions("whoop-backfill|u1", 60)).toEqual(
expect.objectContaining({
src/lib/jobs/__tests__/measurement-tombstone-cleanup.test.ts
+84 -10
View File
@@ -21,19 +21,43 @@ import { TOMBSTONE_RETENTION_DAYS } from "@/lib/auth/native-client";
const DAY_MS = 86_400_000;
/**
* A client whose interactive transaction hands the callback a client with the
* same delegates, and whose restore-lock probe answers `held`.
*/
function mockClient(
measurement: { findMany: unknown; deleteMany: unknown },
held: (userId: string) => boolean = () => true,
) {
const lockCalls: string[] = [];
const tx = {
measurement,
$queryRaw: vi.fn(async (_strings: TemplateStringsArray, key: string) => {
const userId = key.replace(/^backup-restore:/, "");
lockCalls.push(userId);
return [{ held: held(userId) }];
}),
};
const prisma = {
measurement,
$transaction: vi.fn(async (fn: (client: typeof tx) => unknown) => fn(tx)),
} as unknown as Parameters<typeof cleanupExpiredMeasurementTombstones>[0];
return { prisma, lockCalls };
}
describe("cleanupExpiredMeasurementTombstones", () => {
it("prunes only tombstones older than the retention horizon", async () => {
const now = new Date("2026-06-01T00:00:00.000Z");
const findMany = vi
.fn()
.mockResolvedValueOnce([{ id: "a" }, { id: "b" }, { id: "c" }]);
const findMany = vi.fn().mockResolvedValueOnce([
{ id: "a", userId: "u1" },
{ id: "b", userId: "u1" },
{ id: "c", userId: "u1" },
]);
const deleteMany = vi.fn().mockResolvedValue({ count: 3 });
const prisma = {
measurement: { findMany, deleteMany },
} as unknown as Parameters<typeof cleanupExpiredMeasurementTombstones>[0];
const { prisma } = mockClient({ findMany, deleteMany });
const outcome = await cleanupExpiredMeasurementTombstones(prisma, now);
expect(outcome).toEqual({ deleted: 3, drained: true });
expect(outcome).toEqual({ deleted: 3, drained: true, deferredAccounts: 0 });
expect(findMany).toHaveBeenCalledTimes(1);
const where = findMany.mock.calls[0][0].where as {
@@ -61,12 +85,11 @@ describe("cleanupExpiredMeasurementTombstones", () => {
// to stop at the cap rather than loop forever.
const fullBatch = Array.from({ length: PURGE_BATCH_SIZE }, (_, i) => ({
id: `row-${i}`,
userId: "u1",
}));
const findMany = vi.fn().mockResolvedValue(fullBatch);
const deleteMany = vi.fn().mockResolvedValue({ count: PURGE_BATCH_SIZE });
const prisma = {
measurement: { findMany, deleteMany },
} as unknown as Parameters<typeof cleanupExpiredMeasurementTombstones>[0];
const { prisma } = mockClient({ findMany, deleteMany });
const outcome = await cleanupExpiredMeasurementTombstones(prisma, now);
@@ -81,6 +104,57 @@ describe("cleanupExpiredMeasurementTombstones", () => {
});
});
describe("cleanupExpiredMeasurementTombstones — accounts under restore", () => {
it("deletes one statement per account per batch, each under the account's restore lock", async () => {
const findMany = vi.fn().mockResolvedValueOnce([
{ id: "a1", userId: "a" },
{ id: "b1", userId: "b" },
{ id: "a2", userId: "a" },
]);
const deleteMany = vi.fn(async ({ where }) => ({
count: (where.id.in as string[]).length,
}));
const { prisma, lockCalls } = mockClient({ findMany, deleteMany });
const outcome = await cleanupExpiredMeasurementTombstones(prisma);
expect(outcome).toEqual({ deleted: 3, drained: true, deferredAccounts: 0 });
expect(lockCalls).toEqual(["a", "b"]);
expect(deleteMany.mock.calls.map((call) => call[0].where)).toEqual([
{ id: { in: ["a1", "a2"] } },
{ id: { in: ["b1"] } },
]);
});
it("leaves an account under restore and keeps it out of the rest of the run", async () => {
const full = (userId: string) =>
Array.from({ length: PURGE_BATCH_SIZE }, (_, i) => ({
id: `${userId}-${i}`,
userId,
}));
const findMany = vi
.fn()
.mockResolvedValueOnce(full("restoring"))
.mockResolvedValueOnce([{ id: "other-1", userId: "other" }]);
const deleteMany = vi.fn(async ({ where }) => ({
count: (where.id.in as string[]).length,
}));
const { prisma } = mockClient(
{ findMany, deleteMany },
(userId) => userId !== "restoring",
);
const outcome = await cleanupExpiredMeasurementTombstones(prisma);
expect(outcome).toEqual({ deleted: 1, drained: true, deferredAccounts: 1 });
expect(findMany.mock.calls[0][0].where).not.toHaveProperty("userId");
expect(findMany.mock.calls[1][0].where.userId).toEqual({
notIn: ["restoring"],
});
expect(deleteMany).toHaveBeenCalledTimes(1);
});
});
describe("reminder-worker — measurement-tombstone-cleanup schedule", () => {
// v1.18.1 — the cleanup wiring moved out of the 2143-LOC reminder-worker
// boot file into the maintenance registrar. The dead-queue guard follows it.
src/lib/jobs/google-health-backfill.ts
+12 -3
View File
@@ -18,6 +18,7 @@ import { integrationBackfillSourceOptions } from "@/lib/jobs/integration-backfil
import { isReauthRequired } from "@/lib/integrations/status";
import { GOOGLE_HEALTH_INTEGRATION_KEY } from "@/lib/google-health/sync-core";
import { syncUserGoogleHealth } from "@/lib/google-health/sync";
import { runWithGoogleHealthStop } from "@/lib/google-health/client";
export const GOOGLE_HEALTH_BACKFILL_QUEUE = "google-health-backfill";
@@ -47,9 +48,17 @@ export interface GoogleHealthBackfillPayload {
* just burn the retry budget on the same no-op: return WITHOUT stamping — the
* boot discovery re-enqueues the account on the next boot, and the stamp lands
* once the user reconnects and a clean walk completes.
*
* `shouldStop` is the admission job's budget (`jobBudget`). A per-minute
* heart-rate history runs to thousands of pages, and the walk asks it before
* every page: once it says stop, the walk ends where it is, the cycle reports
* itself truncated, and this throws like any other incomplete run, without
* the stamp. The rows already written stay; the retry or the next boot walks
* again and the key-stable upserts overwrite them.
*/
export async function runGoogleHealthBackfillForUser(
userId: string,
shouldStop: () => boolean = () => false,
): Promise<{ imported: number }> {
if (await isReauthRequired(userId, GOOGLE_HEALTH_INTEGRATION_KEY)) {
annotate({
@@ -61,9 +70,9 @@ export async function runGoogleHealthBackfillForUser(
return { imported: 0 };
}
const { imported, failed } = await syncUserGoogleHealth(userId, {
fullSync: true,
});
const { imported, failed } = await runWithGoogleHealthStop(shouldStop, () =>
syncUserGoogleHealth(userId, { fullSync: true }),
);
if (failed) {
// Surface through the pg-boss retry path (retryLimit 3, backoff). If the
src/lib/jobs/integration-backfill-admission.ts
+39 -2
View File
@@ -49,6 +49,27 @@ const RETRY_OPTIONS = {
retryBackoff: true,
} as const;
/**
* The admission job's expiry: sixteen hours, so a legitimately long import
* is never declared dead underneath itself.
*
* Without it every admission job took pg-boss's default of fifteen minutes.
* A full-history import of a per-minute heart-rate stream is thousands of
* pages (a page is 1 000 readings, about seventeen hours of a watch's
* history, so five years is some 2 600 pages), and at a few seconds a page on
* a slow link that is hours. Past the expiry pg-boss fails the job and retries
* it while the handler keeps running (`integration-backfill-lock.ts`), and the
* import that could not finish in fifteen minutes started over beside itself.
*
* The runs that honour the job's budget (`jobBudget`, three quarters of this)
* stop on their own at twelve hours, which is still some 8 000 pages at five
* seconds each, and are retried or re-offered at the next boot. The expiry is
* also what ends a run that hangs: the lane is freed for the next import, and
* the import lock keeps a retry from starting beside the stuck run. pg-boss
* refuses an expiry of 24 hours or more.
*/
export const INTEGRATION_BACKFILL_EXPIRE_SECONDS = 16 * 60 * 60;
export function bootStaggerSecondsFor(kind: IntegrationBackfillKind): number {
return (
(INTEGRATION_BACKFILL_BOOT_ORDER.indexOf(kind) + 1) *
@@ -83,9 +104,25 @@ export async function enqueueIntegrationBackfillAdmission(
boss: BossSender,
payload: IntegrationBackfillAdmissionPayload,
): Promise<string | null> {
return boss.send(INTEGRATION_BACKFILL_ADMISSION_QUEUE, payload, {
return boss.send(
INTEGRATION_BACKFILL_ADMISSION_QUEUE,
payload,
integrationBackfillAdmissionSendOptions(payload),
);
}
/**
* The send options of one admission job. The options ride on the send, not on
* `createQueue`, whose `ON CONFLICT DO NOTHING` leaves an existing queue's
* defaults as they were.
*/
export function integrationBackfillAdmissionSendOptions(
payload: IntegrationBackfillAdmissionPayload,
): SendOptions {
return {
...RETRY_OPTIONS,
expireInSeconds: INTEGRATION_BACKFILL_EXPIRE_SECONDS,
singletonKey: integrationBackfillAdmissionSingletonKey(payload),
group: INTEGRATION_BACKFILL_ADMISSION_GROUP,
});
};
}
src/lib/jobs/integration-backfill-drain.ts
+188
View File
@@ -0,0 +1,188 @@
/**
* The handler of the shared full-history admission queue.
*
* Every provider's backfill and repair reaches this one queue, and the worker
* runs it one job at a time across the whole instance
* (`register-integration-sync.ts`). Each job runs under two guards:
*
* - The import lock (`withIntegrationBackfillLock`), keyed on the job's
* identity. A retry that pg-boss starts after the job's expiry while the
* first run is still going finds the lock taken and does nothing, so one
* account's import of one kind never runs twice at once.
* - The job's budget (`jobBudget`), handed to the runners that can stop
* part-way. A run that stops on its budget throws without stamping its
* completion marker, so it is retried, and after that the next boot's
* discovery offers it again.
*/
import type { Job } from "pg-boss";
import { runWhoopBackfillForUser } from "@/lib/jobs/whoop-backfill";
import { runFitbitBackfillForUser } from "@/lib/jobs/fitbit-backfill";
import { runGoogleHealthBackfillForUser } from "@/lib/jobs/google-health-backfill";
import { runGoogleHealthSleepRepairForUser } from "@/lib/jobs/google-health-sleep-repair";
import { runFitbitSleepRepairForUser } from "@/lib/jobs/fitbit-sleep-repair";
import { runStravaBackfillForUser } from "@/lib/jobs/strava-backfill";
import { runSleepTimelineBackfillForUser } from "@/lib/jobs/sleep-timeline-backfill";
import { runLabBiomarkerBackfillForUser } from "@/lib/jobs/lab-biomarker-backfill";
import {
integrationBackfillAdmissionSingletonKey,
type IntegrationBackfillAdmissionPayload,
} from "@/lib/jobs/integration-backfill-admission";
import { withIntegrationBackfillLock } from "@/lib/jobs/integration-backfill-lock";
import { jobBudget } from "@/lib/jobs/job-budget";
import { jobDone, type JobOutcome } from "@/lib/jobs/job-outcome";
import { workerLog } from "@/lib/jobs/reminder/shared";
import { withBackgroundEvent } from "@/lib/logging/background";
interface RunCounts {
imported: number;
removed: number;
deleted: number;
markers: number;
linked: number;
}
async function runAdmitted(
payload: IntegrationBackfillAdmissionPayload,
shouldStop: () => boolean,
totals: RunCounts,
): Promise<void> {
const { kind, data } = payload;
const { userId } = data;
switch (kind) {
case "whoop-backfill": {
const { imported } = await runWhoopBackfillForUser(userId);
totals.imported += imported;
workerLog("info", `[whoop-backfill] user=${userId} imported=${imported}`);
return;
}
case "fitbit-backfill": {
const { imported } = await runFitbitBackfillForUser(userId);
totals.imported += imported;
workerLog(
"info",
`[fitbit-backfill] user=${userId} imported=${imported}`,
);
return;
}
case "google-health-backfill": {
const { imported } = await runGoogleHealthBackfillForUser(
userId,
shouldStop,
);
totals.imported += imported;
workerLog(
"info",
`[google-health-backfill] user=${userId} imported=${imported}`,
);
return;
}
case "google-health-sleep-repair": {
const { imported } = await runGoogleHealthSleepRepairForUser(userId);
totals.imported += imported;
workerLog(
"info",
`[google-health-sleep-repair] user=${userId} imported=${imported}`,
);
return;
}
case "fitbit-sleep-repair": {
const { imported, removed } = await runFitbitSleepRepairForUser(userId);
totals.imported += imported;
totals.removed += removed;
workerLog(
"info",
`[fitbit-sleep-repair] user=${userId} imported=${imported} removed=${removed}`,
);
return;
}
case "sleep-timeline-backfill": {
if (!data.provider) {
throw new Error(
"sleep-timeline backfill admission requires a provider",
);
}
const { deleted, imported } = await runSleepTimelineBackfillForUser(
userId,
data.provider,
);
totals.deleted += deleted;
totals.imported += imported;
workerLog(
"info",
`[sleep-timeline-backfill] user=${userId} provider=${data.provider} deleted=${deleted} imported=${imported}`,
);
return;
}
case "lab-biomarker-backfill": {
const { markers, linked } = await runLabBiomarkerBackfillForUser(userId);
totals.markers += markers;
totals.linked += linked;
workerLog(
"info",
`[lab-biomarker-backfill] user=${userId} markers=${markers} linked=${linked}`,
);
return;
}
case "strava-backfill": {
const { imported } = await runStravaBackfillForUser(userId);
totals.imported += imported;
workerLog(
"info",
`[strava-backfill] user=${userId} imported=${imported}`,
);
return;
}
}
}
/**
* Drain a batch of admission jobs. There is no per-job error isolation: a
* runner that throws fails the job and the admission retry policy applies. So
* reaching the end means every admitted job either did its work or found its
* import already running, and the counts each runner returns are what the
* pass reports.
*/
export async function drainIntegrationBackfillAdmission(
jobs: Job<IntegrationBackfillAdmissionPayload>[],
): Promise<JobOutcome> {
const shouldStop = jobBudget(jobs);
const totals: RunCounts = {
imported: 0,
removed: 0,
deleted: 0,
markers: 0,
linked: 0,
};
let alreadyRunning = 0;
for (const job of jobs) {
const identity = integrationBackfillAdmissionSingletonKey(job.data);
const run = await withIntegrationBackfillLock(identity, () =>
runAdmitted(job.data, shouldStop, totals),
);
if (!run.ran) {
alreadyRunning += 1;
await withBackgroundEvent(
"integration_backfill.already_running",
async (event) => {
event.addMeta("identity", identity);
event.addMeta("job_id", job.id);
event.addMeta(
"reason",
"an earlier delivery of this import still holds its lock; this delivery does nothing",
);
event.elevateLevel("warn");
},
);
}
}
return jobDone({
jobs: jobs.length,
already_running: alreadyRunning,
imported: totals.imported,
removed: totals.removed,
deleted: totals.deleted,
markers: totals.markers,
linked: totals.linked,
});
}
src/lib/jobs/integration-backfill-lock.ts
+70
View File
@@ -0,0 +1,70 @@
/**
* One full-history import per account and kind at a time, across processes.
*
* The admission queue runs one import at a time (`groupConcurrency: 1`), but
* pg-boss only holds that line while the job is inside its expiry. Once the
* expiry passes, pg-boss fails the job, fires its abort signal, frees the
* worker slot and the group slot, and retries the job after its retry delay.
* It does not stop the handler. Measured against pg-boss 12.34
* (`tests/integration/integration-backfill-lane-expiry.test.ts`): a handler
* that ignores the signal was still running when the retry of the same job
* started beside it in the same worker, so the lane's "one at a time" did not
* hold for the one case it exists for, an import too long for its expiry.
*
* So each run holds a session-level advisory lock keyed on the job's identity
* (kind, provider where there is one, account) for as long as it runs. A
* delivery that finds the lock taken does no work: the run that holds it is
* still importing, and it either stamps its completion marker or leaves the
* account for the next boot's discovery. The lock lives on a connection of its
* own, outside the Prisma pool, because a session lock has to stay on one
* connection for the whole run and a pooled one is handed back between
* statements. Ending the connection releases the lock, and so does the
* process dying, so a crash never leaves an account locked.
*
* One extra connection per running import; the lane admits one import per
* process at a time, so that is one connection.
*/
import { Client } from "pg";
/** The lock key for one admission job's identity. */
function integrationBackfillLockKey(identity: string): string {
return `integration-backfill:${identity}`;
}
/** What a guarded run came to. */
type GuardedRun<T> = { ran: true; result: T } | { ran: false };
/**
* Run `run` while holding the import lock for `identity`. Resolves
* `{ ran: false }` at once, without calling `run`, when another run holds it.
* Rejects exactly as `run` does otherwise.
*/
export async function withIntegrationBackfillLock<T>(
identity: string,
run: () => Promise<T>,
connectionString: string | undefined = process.env.DATABASE_URL,
): Promise<GuardedRun<T>> {
const client = new Client({ connectionString, keepAlive: true });
// An idle client whose server goes away emits `error`; unhandled, that
// takes the whole worker down. The run itself notices the database is
// gone through its own queries.
client.on("error", () => {});
await client.connect();
try {
const key = integrationBackfillLockKey(identity);
const { rows } = await client.query<{ held: boolean }>(
"SELECT pg_try_advisory_lock(hashtextextended($1, 0)) AS held",
[key],
);
if (!rows[0]?.held) return { ran: false };
try {
return { ran: true, result: await run() };
} finally {
await client
.query("SELECT pg_advisory_unlock(hashtextextended($1, 0))", [key])
.catch(() => {});
}
} finally {
await client.end().catch(() => {});
}
}
src/lib/jobs/job-outcome.ts
+6
View File
@@ -43,6 +43,9 @@ export type JobFacts = Readonly<Record<string, JobFact>>;
*/
export const JOB_FACT_ALLOWLIST: ReadonlySet<string> = new Set([
"access_tokens_deleted",
// An admission delivery that found its import still running from an
// earlier delivery of the same job and did nothing.
"already_running",
"appointment_addresses_cleared",
"assessments_warmed",
"auto_resolved",
@@ -68,6 +71,9 @@ export const JOB_FACT_ALLOWLIST: ReadonlySet<string> = new Set([
"days_skipped",
"days_skipped_no_tombstones",
"days_stored",
// Tombstone purge: accounts left for the next run because a restore of
// the account was running.
"deferred_accounts",
"deleted",
"dense_days_consolidated",
"dense_rows_soft_deleted",
src/lib/jobs/measurement-tombstone-cleanup.ts
+140 -39
View File
@@ -18,9 +18,17 @@
* (`NATIVE_REFRESH_TOKEN_DAYS` + margin) so it moves automatically if the
* refresh-token lifetime changes — the two never drift.
*/
import type { PrismaClient } from "@/generated/prisma/client";
import type { Prisma, PrismaClient } from "@/generated/prisma/client";
import { TOMBSTONE_RETENTION_DAYS } from "@/lib/auth/native-client";
import { purgeInBatches, type PurgeOutcome } from "@/lib/jobs/purge-batch";
import {
AccountRestoreInProgressError,
holdAccountAgainstRestore,
} from "@/lib/export/restore-lock";
import {
PURGE_BATCH_SIZE,
PURGE_MAX_BATCHES,
type PurgeOutcome,
} from "@/lib/jobs/purge-batch";
const DAY_MS = 86_400_000;
@@ -28,6 +36,103 @@ function tombstoneCutoff(now: Date): Date {
return new Date(now.getTime() - TOMBSTONE_RETENTION_DAYS * DAY_MS);
}
/** A purge run, plus the accounts it left for the next run. */
export interface TombstonePurgeOutcome extends PurgeOutcome {
/**
* Accounts whose tombstones were left in place because a restore of the
* account was running. The next night purges them.
*/
deferredAccounts: number;
}
type TombstoneRow = { id: string; userId: string };
interface TombstonePurgeOptions {
prisma: PrismaClient;
/** Up to `take` expired tombstones, none of them owned by `skipUserIds`. */
findRows: (take: number, skipUserIds: string[]) => Promise<TombstoneRow[]>;
/** Remove exactly those ids inside `tx`; returns the rows removed. */
deleteIds: (tx: Prisma.TransactionClient, ids: string[]) => Promise<number>;
batchSize?: number;
maxBatches?: number;
}
/**
* Walk the expired tombstones in bounded batches, deleting each batch one
* account at a time under the account's restore lock.
*
* A restore clears every row of the account, tombstones included, in one
* long transaction. The purge used to delete a batch spanning accounts in one
* statement, so on an account under restore it waited on the restore's row
* locks, for as long as the restore ran and well past `statement_timeout`,
* or, holding some of the account's rows while the restore held others,
* deadlocked with it. So each account's share of a batch is deleted in a
* transaction that first takes the restore lock in shared mode, the way the
* consolidation passes do (`restore-lock.ts`). An account under restore is
* refused at once, dropped from the rest of the run, and purged the next
* night, when its tombstones are whatever the restore left.
*
* Still one delete statement per account per batch, never one per row.
*/
async function purgeTombstonesByAccount({
prisma,
findRows,
deleteIds,
batchSize = PURGE_BATCH_SIZE,
maxBatches = PURGE_MAX_BATCHES,
}: TombstonePurgeOptions): Promise<TombstonePurgeOutcome> {
let deleted = 0;
const deferred = new Set<string>();
const outcome = (drained: boolean): TombstonePurgeOutcome => ({
deleted,
drained,
deferredAccounts: deferred.size,
});
for (let batch = 0; batch < maxBatches; batch++) {
const rows = await findRows(batchSize, [...deferred]);
if (rows.length === 0) return outcome(true);
const byAccount = new Map<string, string[]>();
for (const row of rows) {
const ids = byAccount.get(row.userId);
if (ids) ids.push(row.id);
else byAccount.set(row.userId, [row.id]);
}
for (const [userId, ids] of byAccount) {
try {
deleted += await prisma.$transaction(
async (tx) => {
await holdAccountAgainstRestore(tx, userId);
return deleteIds(tx, ids);
},
// A full batch of one account is one statement over thousands of
// rows; the interactive default of five seconds is too tight for
// it on a slow disk. The connection's statement_timeout bounds it.
{ timeout: 60_000 },
);
} catch (err) {
if (!(err instanceof AccountRestoreInProgressError)) throw err;
deferred.add(userId);
}
}
// A short batch means the predicate is exhausted, the deferred accounts
// aside; asking again would cost a round-trip to learn nothing.
if (rows.length < batchSize) return outcome(true);
}
return outcome(false);
}
/** The retention predicate, minus the accounts a run has deferred. */
function expiredTombstones(cutoff: Date, skipUserIds: string[]) {
return {
deletedAt: { not: null, lt: cutoff },
...(skipUserIds.length > 0 ? { userId: { notIn: skipUserIds } } : {}),
};
}
/**
* Hard-delete soft-deleted measurement rows whose `deletedAt` is older than
* the retention horizon.
@@ -44,20 +149,18 @@ function tombstoneCutoff(now: Date): Date {
export async function cleanupExpiredMeasurementTombstones(
prisma: PrismaClient,
now: Date = new Date(),
): Promise<PurgeOutcome> {
): Promise<TombstonePurgeOutcome> {
const cutoff = tombstoneCutoff(now);
return purgeInBatches({
findIds: async (take) =>
(
await prisma.measurement.findMany({
where: { deletedAt: { not: null, lt: cutoff } },
select: { id: true },
take,
})
).map((row) => row.id),
deleteIds: async (ids) =>
(await prisma.measurement.deleteMany({ where: { id: { in: ids } } }))
.count,
return purgeTombstonesByAccount({
prisma,
findRows: (take, skipUserIds) =>
prisma.measurement.findMany({
where: expiredTombstones(cutoff, skipUserIds),
select: { id: true, userId: true },
take,
}),
deleteIds: async (tx, ids) =>
(await tx.measurement.deleteMany({ where: { id: { in: ids } } })).count,
});
}
@@ -65,19 +168,18 @@ export async function cleanupExpiredMeasurementTombstones(
export async function cleanupExpiredMoodTombstones(
prisma: PrismaClient,
now: Date = new Date(),
): Promise<PurgeOutcome> {
): Promise<TombstonePurgeOutcome> {
const cutoff = tombstoneCutoff(now);
return purgeInBatches({
findIds: async (take) =>
(
await prisma.moodEntry.findMany({
where: { deletedAt: { not: null, lt: cutoff } },
select: { id: true },
take,
})
).map((row) => row.id),
deleteIds: async (ids) =>
(await prisma.moodEntry.deleteMany({ where: { id: { in: ids } } })).count,
return purgeTombstonesByAccount({
prisma,
findRows: (take, skipUserIds) =>
prisma.moodEntry.findMany({
where: expiredTombstones(cutoff, skipUserIds),
select: { id: true, userId: true },
take,
}),
deleteIds: async (tx, ids) =>
(await tx.moodEntry.deleteMany({ where: { id: { in: ids } } })).count,
});
}
@@ -88,20 +190,19 @@ export async function cleanupExpiredMoodTombstones(
export async function cleanupExpiredIntakeTombstones(
prisma: PrismaClient,
now: Date = new Date(),
): Promise<PurgeOutcome> {
): Promise<TombstonePurgeOutcome> {
const cutoff = tombstoneCutoff(now);
return purgeInBatches({
findIds: async (take) =>
return purgeTombstonesByAccount({
prisma,
findRows: (take, skipUserIds) =>
prisma.medicationIntakeEvent.findMany({
where: expiredTombstones(cutoff, skipUserIds),
select: { id: true, userId: true },
take,
}),
deleteIds: async (tx, ids) =>
(
await prisma.medicationIntakeEvent.findMany({
where: { deletedAt: { not: null, lt: cutoff } },
select: { id: true },
take,
})
).map((row) => row.id),
deleteIds: async (ids) =>
(
await prisma.medicationIntakeEvent.deleteMany({
await tx.medicationIntakeEvent.deleteMany({
where: { id: { in: ids } },
})
).count,
src/lib/jobs/reminder/__tests__/cleanup-handlers-visibility.test.ts
+15 -3
View File
@@ -62,15 +62,16 @@ import {
} from "../cleanup-handlers";
const DRAINED = { deleted: 1, drained: true };
const TOMBSTONES_DRAINED = { ...DRAINED, deferredAccounts: 0 };
beforeEach(() => {
vi.clearAllMocks();
for (const key of Object.keys(h.meta)) delete h.meta[key];
h.warnings.length = 0;
h.cleanupOldAuditLogs.mockResolvedValue(DRAINED);
h.cleanupExpiredMeasurementTombstones.mockResolvedValue(DRAINED);
h.cleanupExpiredMoodTombstones.mockResolvedValue(DRAINED);
h.cleanupExpiredIntakeTombstones.mockResolvedValue(DRAINED);
h.cleanupExpiredMeasurementTombstones.mockResolvedValue(TOMBSTONES_DRAINED);
h.cleanupExpiredMoodTombstones.mockResolvedValue(TOMBSTONES_DRAINED);
h.cleanupExpiredIntakeTombstones.mockResolvedValue(TOMBSTONES_DRAINED);
});
describe("audit-log cleanup handler", () => {
@@ -115,6 +116,7 @@ describe("tombstone cleanup handler", () => {
h.cleanupExpiredMoodTombstones.mockResolvedValue({
deleted: 200_000,
drained: false,
deferredAccounts: 0,
});
await handleMeasurementTombstoneCleanup([]);
expect(h.meta.tombstone_cleanup_drained).toBe(false);
@@ -122,6 +124,16 @@ describe("tombstone cleanup handler", () => {
expect(h.warnings.join(" ")).toContain("backlog remains");
});
it("reports the accounts it left because a restore was running", async () => {
h.cleanupExpiredMeasurementTombstones.mockResolvedValue({
...TOMBSTONES_DRAINED,
deferredAccounts: 2,
});
const outcome = await handleMeasurementTombstoneCleanup([]);
expect(h.meta.tombstone_cleanup_deferred_accounts).toBe(2);
expect(outcome).toMatchObject({ did: { deferred_accounts: 2 } });
});
it("stays quiet when all three legs drained", async () => {
await handleMeasurementTombstoneCleanup([]);
expect(h.meta.tombstone_cleanup_drained).toBe(true);
src/lib/jobs/reminder/cleanup-handlers.ts
+8
View File
@@ -254,6 +254,13 @@ export async function handleMeasurementTombstoneCleanup(
// outside, which is how a purge that never finished stayed invisible.
const drained = measurements.drained && mood.drained && intakes.drained;
evt.addMeta("tombstone_cleanup_drained", drained);
// Accounts under restore are left alone and purged the next night.
const deferredAccounts = Math.max(
measurements.deferredAccounts,
mood.deferredAccounts,
intakes.deferredAccounts,
);
evt.addMeta("tombstone_cleanup_deferred_accounts", deferredAccounts);
if (!drained) {
evt.addWarning(
"tombstone-cleanup stopped at the batch cap; a backlog remains for the next run",
@@ -264,6 +271,7 @@ export async function handleMeasurementTombstoneCleanup(
mood_pruned: mood.deleted,
intakes_pruned: intakes.deleted,
drained,
deferred_accounts: deferredAccounts,
});
} catch (err) {
// Rethrow. A retention purge that fails silently every night looks
src/lib/jobs/reminder/register-integration-sync.ts
+2 -119
View File
@@ -17,56 +17,48 @@ import { PgBoss } from "pg-boss";
import {
WHOOP_BACKFILL_QUEUE,
WHOOP_BACKFILL_CONCURRENCY,
runWhoopBackfillForUser,
enqueueBootTimeWhoopBackfill,
type WhoopBackfillPayload,
} from "@/lib/jobs/whoop-backfill";
import {
FITBIT_BACKFILL_QUEUE,
FITBIT_BACKFILL_CONCURRENCY,
runFitbitBackfillForUser,
enqueueBootTimeFitbitBackfill,
type FitbitBackfillPayload,
} from "@/lib/jobs/fitbit-backfill";
import {
GOOGLE_HEALTH_BACKFILL_QUEUE,
GOOGLE_HEALTH_BACKFILL_CONCURRENCY,
runGoogleHealthBackfillForUser,
enqueueBootTimeGoogleHealthBackfill,
type GoogleHealthBackfillPayload,
} from "@/lib/jobs/google-health-backfill";
import {
GOOGLE_HEALTH_SLEEP_REPAIR_QUEUE,
GOOGLE_HEALTH_SLEEP_REPAIR_CONCURRENCY,
runGoogleHealthSleepRepairForUser,
enqueueBootTimeGoogleHealthSleepRepair,
type GoogleHealthSleepRepairPayload,
} from "@/lib/jobs/google-health-sleep-repair";
import {
FITBIT_SLEEP_REPAIR_QUEUE,
FITBIT_SLEEP_REPAIR_CONCURRENCY,
runFitbitSleepRepairForUser,
enqueueBootTimeFitbitSleepRepair,
type FitbitSleepRepairPayload,
} from "@/lib/jobs/fitbit-sleep-repair";
import {
STRAVA_BACKFILL_QUEUE,
STRAVA_BACKFILL_CONCURRENCY,
runStravaBackfillForUser,
enqueueBootTimeStravaBackfill,
type StravaBackfillPayload,
} from "@/lib/jobs/strava-backfill";
import {
SLEEP_TIMELINE_BACKFILL_QUEUE,
SLEEP_TIMELINE_BACKFILL_CONCURRENCY,
runSleepTimelineBackfillForUser,
enqueueBootTimeSleepTimelineBackfill,
type SleepTimelineBackfillPayload,
} from "@/lib/jobs/sleep-timeline-backfill";
import {
LAB_BIOMARKER_BACKFILL_QUEUE,
LAB_BIOMARKER_BACKFILL_CONCURRENCY,
runLabBiomarkerBackfillForUser,
enqueueBootTimeLabBiomarkerBackfill,
type LabBiomarkerBackfillPayload,
} from "@/lib/jobs/lab-biomarker-backfill";
@@ -77,6 +69,7 @@ import {
enqueueIntegrationBackfillAdmission,
type IntegrationBackfillAdmissionPayload,
} from "@/lib/jobs/integration-backfill-admission";
import { drainIntegrationBackfillAdmission } from "@/lib/jobs/integration-backfill-drain";
import { workerLog } from "./shared";
import { jobDone } from "@/lib/jobs/job-outcome";
import {
@@ -482,117 +475,7 @@ export async function registerIntegrationSyncQueues(
localConcurrency: INTEGRATION_BACKFILL_GLOBAL_CONCURRENCY,
groupConcurrency: INTEGRATION_BACKFILL_GLOBAL_CONCURRENCY,
},
async (jobs) => {
// The drain has no per-job error isolation: a runner that throws fails
// the job and the admission retry policy applies. So reaching the end
// means every admitted job did its work, and the counts each runner
// already returns are what the pass reports.
let importedTotal = 0;
let removedTotal = 0;
let deletedTotal = 0;
let markersTotal = 0;
let linkedTotal = 0;
for (const job of jobs) {
const { kind, data } = job.data;
const { userId } = data;
switch (kind) {
case "whoop-backfill": {
const { imported } = await runWhoopBackfillForUser(userId);
importedTotal += imported;
workerLog(
"info",
`[whoop-backfill] user=${userId} imported=${imported}`,
);
break;
}
case "fitbit-backfill": {
const { imported } = await runFitbitBackfillForUser(userId);
importedTotal += imported;
workerLog(
"info",
`[fitbit-backfill] user=${userId} imported=${imported}`,
);
break;
}
case "google-health-backfill": {
const { imported } = await runGoogleHealthBackfillForUser(userId);
importedTotal += imported;
workerLog(
"info",
`[google-health-backfill] user=${userId} imported=${imported}`,
);
break;
}
case "google-health-sleep-repair": {
const { imported } =
await runGoogleHealthSleepRepairForUser(userId);
importedTotal += imported;
workerLog(
"info",
`[google-health-sleep-repair] user=${userId} imported=${imported}`,
);
break;
}
case "fitbit-sleep-repair": {
const { imported, removed } =
await runFitbitSleepRepairForUser(userId);
importedTotal += imported;
removedTotal += removed;
workerLog(
"info",
`[fitbit-sleep-repair] user=${userId} imported=${imported} removed=${removed}`,
);
break;
}
case "sleep-timeline-backfill": {
if (!data.provider) {
throw new Error(
"sleep-timeline backfill admission requires a provider",
);
}
const { deleted, imported } = await runSleepTimelineBackfillForUser(
userId,
data.provider,
);
deletedTotal += deleted;
importedTotal += imported;
workerLog(
"info",
`[sleep-timeline-backfill] user=${userId} provider=${data.provider} deleted=${deleted} imported=${imported}`,
);
break;
}
case "lab-biomarker-backfill": {
const { markers, linked } =
await runLabBiomarkerBackfillForUser(userId);
markersTotal += markers;
linkedTotal += linked;
workerLog(
"info",
`[lab-biomarker-backfill] user=${userId} markers=${markers} linked=${linked}`,
);
break;
}
case "strava-backfill": {
const { imported } = await runStravaBackfillForUser(userId);
importedTotal += imported;
workerLog(
"info",
`[strava-backfill] user=${userId} imported=${imported}`,
);
break;
}
}
}
return jobDone({
jobs: jobs.length,
imported: importedTotal,
removed: removedTotal,
deleted: deletedTotal,
markers: markersTotal,
linked: linkedTotal,
});
},
drainIntegrationBackfillAdmission,
);
await createAndWork<WithingsSyncPayload>(
src/lib/openapi/error-codes.ts
+2
View File
@@ -339,6 +339,7 @@ export const ERROR_CODE_CATALOGUE: Readonly<Record<string, readonly string[]>> =
tokens: [
"tokens.documents.ceiling_reached",
"tokens.measurements.ceiling_reached",
"tokens.workouts.ceiling_reached",
],
vaccination: [
"vaccination.booster-invalid",
@@ -354,6 +355,7 @@ export const ERROR_CODE_CATALOGUE: Readonly<Record<string, readonly string[]>> =
workout: [
"workout.batch.invalid",
"workout.batch.payload_too_large",
"workout.batch.source_not_permitted",
"workout.batch.too_large",
],
"(unprefixed)": [
src/lib/openapi/routes/auth.ts
+89 -2
View File
@@ -33,6 +33,7 @@ import { oidcNativeTokenSchema } from "@/lib/validations/oidc-native";
import {
createDocumentTokenSchema,
createMeasurementTokenSchema,
createWorkoutTokenSchema,
} from "@/lib/validations/tokens";
import { nativeHandoffTokenSchema } from "@/lib/validations/native-handoff";
import {
@@ -489,6 +490,28 @@ const createMeasurementTokenResponse = z
"The one response that carries a usable measurement-ingest Bearer value.",
});
const createWorkoutTokenRequest = createWorkoutTokenSchema.meta({
id: "CreateWorkoutTokenRequest",
description:
"Mint a workout-ingest Bearer. No `scope` field: the endpoint mints one shape. `name` is what the token is listed under; name it after the bridge it is pasted into. `expiresInDays` defaults to 365.",
});
const createWorkoutTokenResponse = z
.object({
token: z
.string()
.describe(
"The raw `hlk_` Bearer, RETURNED EXACTLY ONCE. It is stored only as an HMAC-SHA256 hash and cannot be retrieved later. Treat it as a secret.",
),
name: z.string(),
expiresAt: z.iso.datetime({ offset: true }),
})
.meta({
id: "CreateWorkoutTokenResponse",
description:
"The one response that carries a usable workout-ingest Bearer value.",
});
const createDocumentTokenRequest = createDocumentTokenSchema.meta({
id: "CreateDocumentTokenRequest",
description:
@@ -881,7 +904,7 @@ export const authPaths: NonNullable<ZodOpenApiObject["paths"]> = {
tags: ["Auth"],
summary: "Re-prove a credential on a browser session (cookie only)",
description:
"Stamps the calling browser session as recently re-proved, so for the next five minutes it may run the actions that ask for a fresh proof: GET /api/export/full-backup, POST /api/export/encrypted, POST /api/share-links, POST /api/mcp/tokens, POST /api/tokens/measurements, POST /api/tokens/documents, and the admin backup download, upload and restore, the admin data wipe and the admin password reset. Those answer 401 `auth.reproof.required` with `meta.methods` (the proofs this account can give) when the session signed in or re-proved more than five minutes ago.\n\n" +
"Stamps the calling browser session as recently re-proved, so for the next five minutes it may run the actions that ask for a fresh proof: GET /api/export/full-backup, POST /api/export/encrypted, POST /api/share-links, POST /api/mcp/tokens, POST /api/tokens/measurements, POST /api/tokens/workouts, POST /api/tokens/documents, and the admin backup download, upload and restore, the admin data wipe and the admin password reset. Those answer 401 `auth.reproof.required` with `meta.methods` (the proofs this account can give) when the session signed in or re-proved more than five minutes ago.\n\n" +
'The body takes the same shapes as POST /api/auth/step-up. On an account with a second factor only `totp`, `webauthn` (a security key) or `passkey` are accepted, because a password has never stood in for the second factor; on one without, `password` or `passkey`. A method the account cannot use here is refused with 422 `auth.reproof.too_weak` and `meta.methods`, before any guess is spent. A second-factor or passkey proof also refreshes the session\'s second-factor stamp, exactly as signing in again would. Assertions begin at POST /api/auth/passkey/register-options with `{ method: "passkey" | "webauthn" }`.\n\n' +
"Proofs draw on the account's shared re-proof budget (five per fifteen minutes, 429 once spent). A proof that does not verify is 401 `auth.reproof.failed` and is audited. Bearer callers use POST /api/auth/step-up instead.",
requestBody: {
@@ -1687,7 +1710,7 @@ export const authPaths: NonNullable<ZodOpenApiObject["paths"]> = {
summary: "List the caller's API tokens",
description:
"Every `ApiToken` row belonging to the caller, newest first, revoked ones included — presence in this list is not validity, `revoked` is. The hash is never returned and there is no path that re-reveals a token's plaintext.\n\n" +
"Three things the name understates. The list is not limited to tokens a person minted from the settings surface: a native login mints a wildcard access token as an `ApiToken` row, so a signed-in phone shows up here too. There is no GENERIC mint any more — the POST at this exact path issued `[\"medication:ingest\"]` and was removed, because that scope reached no ingest route while the pre-fail-closed default let it reach everything else. The credentials that work are minted where they are scoped: the per-medication API-endpoint toggle, `POST /api/mcp/tokens` for a connector, the sibling `POST /api/tokens/measurements` for third-party measurement ingest, and `POST /api/tokens/documents` for document upload from another system. And unlike the revoke, this read is NOT gated on the operator's instance-wide API switch: that switch governs the surfaces a token is for, not a token's ability to authenticate, so tokens stay live while it is off and their owner has to be able to see them.",
"Three things the name understates. The list is not limited to tokens a person minted from the settings surface: a native login mints a wildcard access token as an `ApiToken` row, so a signed-in phone shows up here too. There is no GENERIC mint any more — the POST at this exact path issued `[\"medication:ingest\"]` and was removed, because that scope reached no ingest route while the pre-fail-closed default let it reach everything else. The credentials that work are minted where they are scoped: the per-medication API-endpoint toggle, `POST /api/mcp/tokens` for a connector, the sibling `POST /api/tokens/measurements` for third-party measurement ingest, `POST /api/tokens/workouts` for workout ingest from a bridge, and `POST /api/tokens/documents` for document upload from another system. And unlike the revoke, this read is NOT gated on the operator's instance-wide API switch: that switch governs the surfaces a token is for, not a token's ability to authenticate, so tokens stay live while it is off and their owner has to be able to see them.",
responses: {
"200": {
description: "The caller's tokens, newest first.",
@@ -1771,6 +1794,70 @@ export const authPaths: NonNullable<ZodOpenApiObject["paths"]> = {
},
},
},
"/api/tokens/workouts": {
post: {
tags: ["Auth"],
summary: "Mint a workout-ingest token",
description:
"Mints a Bearer scoped to exactly `workouts:write` and audits the mint. THE RESPONSE CARRIES THE RAW TOKEN, once; it is stored as an HMAC and no path re-reveals it.\n\n" +
"**What it can do.** `POST /api/workouts/batch`, on its owner's own record, and nothing else. It is the door a workout bridge pushes through: a watch vendor's sync relayed by a small service, or a script replaying an exported history. Workouts it writes carry `source: EXTERNAL`, resolved from the credential rather than taken from the body, and a batch in which any entry names a source is refused 422 (`workout.batch.source_not_permitted`) rather than relabelled. `EXTERNAL` is on no default source ladder, so when the phone also recorded the same session the phone's copy is the one the workout list shows.\n\n" +
"**What it cannot do.** List or read a workout, reach the measurement or document ingest routes, act on a shared record, or mint another token. It is a separate scope from `measurements:write` on purpose: a token already handed to a scale does not start accepting workouts, and a `measurements:write` token is refused 403 on the workout batch. Every other route names no scope and refuses it 403.\n\n" +
"Minting requires a COOKIE SESSION with a fresh proof, as for the measurement mint, for the reason it gives: a day-lived native access token must not be able to leave behind a year-lived credential. Gated by the operator's instance-wide API switch. Body capped at 16 KiB; 10 mints per user per minute, and at most 10 live workout tokens held at once. Tokens appear in `GET /api/tokens` and are revoked at `DELETE /api/tokens/{id}`.",
requestBody: {
required: true,
content: {
"application/json": { schema: createWorkoutTokenRequest },
},
},
responses: {
"201": {
description:
"Token minted. The `token` field is the only copy of the secret.",
content: {
"application/json": {
schema: dataEnvelope(
createWorkoutTokenResponse,
"CreateWorkoutTokenEnvelope",
),
},
},
},
"400": {
description: "Body is not parseable JSON.",
content: { "application/json": { schema: errorEnvelope } },
},
"403": {
description:
"The operator has switched the API off instance-wide. Nothing was minted. A Bearer caller is refused 401 before the switch is read.",
content: { "application/json": { schema: errorEnvelope } },
},
"413": {
description: "Body exceeds 16 KiB.",
content: { "application/json": { schema: errorEnvelope } },
},
"415": {
description: "Content-Type is not `application/json`.",
content: { "application/json": { schema: errorEnvelope } },
},
...stdResponses,
"422": {
description:
"Validation failed. The envelope carries every offending issue.",
content: { "application/json": { schema: errorEnvelope } },
},
"429": {
description:
"More than 10 mints from this account in a minute. Nothing was minted.",
content: { "application/json": { schema: errorEnvelope } },
},
"409": {
description:
"The account already holds 10 live workout tokens (`meta.errorCode` = `tokens.workouts.ceiling_reached`). Nothing was minted. Revoke one at `DELETE /api/tokens/{id}`; only live tokens carrying `workouts:write` count.",
content: { "application/json": { schema: errorEnvelope } },
},
},
},
},
"/api/tokens/documents": {
post: {
tags: ["Auth"],
src/lib/openapi/routes/measurements.ts
+2 -2
View File
@@ -696,7 +696,7 @@ const seriesPointSchema = z.object({
id: z
.string()
.describe(
"Measurement row id; `sleep:<wake-day>` for a sleep night; `day:<local-day>` for a day bucket (`pulse` / `glucose` beyond 90 days); `hour:<ISO start>` for an hour bucket (`pulse` within 90 days when the window holds more than 10 000 readings).",
"Measurement row id; `sleep:<wake-day>` for a sleep night; `day:<local-day>` for a day bucket (`pulse` / `glucose` beyond 90 days); `hour:<ISO start>` for an hour bucket (`pulse` / `glucose` within 90 days when the window holds more than 10 000 readings).",
),
at: z.iso.datetime({ offset: true }).describe("Point timestamp (ISO-8601)."),
value: z.number().describe("Primary value in the top-level `unit`."),
@@ -751,7 +751,7 @@ const seriesResponse = z
.meta({
id: "MeasurementsSeriesResponse",
description:
"iOS-friendly per-kind time series: one point per reading (or per reconstructed night for `sleep`), an explicit `unit` token, and a summary `stats` block. Sample-dense kinds are bucketed server-side: `pulse` and `glucose` beyond 90 days per local day, and `pulse` within 90 days per local hour when the window holds more than 10 000 readings (a bucket carries the mean as `value` and its range as `valueMin` / `valueMax`). `stats` is always computed over the underlying readings, never over the buckets.",
"iOS-friendly per-kind time series: one point per reading (or per reconstructed night for `sleep`), an explicit `unit` token, and a summary `stats` block. Sample-dense kinds are bucketed server-side: `pulse` and `glucose` beyond 90 days per local day, and `pulse` / `glucose` within 90 days per local hour when the window holds more than 10 000 readings (a continuous glucose sensor, a watch recording heart rate every minute; a cuff or fingerstick meter stays raw). A bucket carries the mean as `value`; a `pulse` bucket also carries its range as `valueMin` / `valueMax`, and a `glucose` bucket is converted to the user's unit like a raw reading. `stats` is always computed over the underlying readings, never over the buckets.",
});
// ── Batched daily series (v1.18.6 dashboard fetch coalescing) ────────
src/lib/openapi/routes/shared.ts
+1 -1
View File
@@ -144,7 +144,7 @@ export const measurementTypeEnum = measurementTypeEnumBase.meta({
export const measurementSourceEnum = measurementSourceEnumBase.meta({
id: "MeasurementSource",
description:
"Origin of the measurement. v1.4.23 added APPLE_HEALTH for the iOS HealthKit batch ingest path. v1.38.x added EXTERNAL for rows pushed in under a narrow `measurements:write` Bearer — a scale, a watch bridge, a home-automation rule. No client may name that value on a write; the server resolves it from the credential.",
"Origin of the measurement. v1.4.23 added APPLE_HEALTH for the iOS HealthKit batch ingest path. v1.38.x added EXTERNAL for rows pushed in under a narrow `measurements:write` Bearer — a scale, a watch bridge, a home-automation rule — and workouts pushed in under a narrow `workouts:write` Bearer carry it too. No client may name that value on a write; the server resolves it from the credential.",
});
export const loginPasswordSchema = loginPasswordSchemaBase.meta({
src/lib/openapi/routes/workouts.ts
+9 -1
View File
@@ -319,7 +319,8 @@ export const workoutPaths: NonNullable<ZodOpenApiObject["paths"]> = {
tags: ["Measurements"],
summary: "Typed workout batch ingest (v1.4.25 W16b)",
description:
'Server-side ingest endpoint for HKWorkout records (iOS) and Withings activity rows (server-to-server). Up to 100 workouts per call; nested route geometry (GeoJSON LineString) is capped at 20 000 points and stored in a 1:1 `WorkoutRoute` row keyed by `workoutId`. Idempotent via the `Idempotency-Key` header (replay window 24h). Per-entry status (`inserted | duplicate | enriched | skipped`) lets the iOS sync cursor checkpoint accurately. An entry matching a stored workout is first-write-wins for the workout row; when it carries a `samples` array and the stored workout has no heart-rate series, the series is attached and the entry reports `enriched` (repeatable: a workout that already has a series is a no-op). An entry whose `samples` array fails validation is refused on its own with `reason: "invalid_samples"` — the rest of the batch still lands. The request body ceiling is 5 MB enforced at the HTTP layer — clients above the ceiling receive a 413 with `workout.batch.payload_too_large`.',
'Server-side ingest endpoint for HKWorkout records (iOS) and Withings activity rows (server-to-server). Up to 100 workouts per call; nested route geometry (GeoJSON LineString) is capped at 20 000 points and stored in a 1:1 `WorkoutRoute` row keyed by `workoutId`. Idempotent via the `Idempotency-Key` header (replay window 24h). Per-entry status (`inserted | duplicate | enriched | skipped`) lets the iOS sync cursor checkpoint accurately. An entry matching a stored workout is first-write-wins for the workout row; when it carries a `samples` array and the stored workout has no heart-rate series, the series is attached and the entry reports `enriched` (repeatable: a workout that already has a series is a no-op). An entry whose `samples` array fails validation is refused on its own with `reason: "invalid_samples"` — the rest of the batch still lands. The request body ceiling is 5 MB enforced at the HTTP layer — clients above the ceiling receive a 413 with `workout.batch.payload_too_large`.\n\n' +
"**Narrow token.** Also reachable with a `workouts:write` Bearer (minted at `POST /api/tokens/workouts`), the supported way to push workouts from a bridge that is not the phone. This route is the scope's only one. Such a credential writes only on its owner's own record; its workouts are attributed `EXTERNAL`, resolved from the credential, and it may not name a source itself, so a batch in which any entry carries one, `MANUAL` and `APPLE_HEALTH` alike, is refused 422 (`workout.batch.source_not_permitted`) and nothing is written. `EXTERNAL` gives those writes their own `(userId, source, externalId)` namespace, so a bridge's ids can never collide with the phone's. Every other narrow token, `measurements:write` included, is refused 403.",
requestBody: {
required: true,
content: { "application/json": { schema: createBatchWorkoutSchema } },
@@ -347,6 +348,13 @@ export const workoutPaths: NonNullable<ZodOpenApiObject["paths"]> = {
content: { "application/json": { schema: errorEnvelope } },
},
...stdResponses,
// After the spread: `stdResponses` carries a generic 422 that would
// otherwise overwrite this wording.
"422": {
description:
"A narrow `workouts:write` credential sent a batch in which an entry names a source, any source (`workout.batch.source_not_permitted`). Nothing was written; resending the batch unchanged is refused the same way. Omit `source` and the server attributes the rows `EXTERNAL`.",
content: { "application/json": { schema: errorEnvelope } },
},
},
},
},
src/lib/sources/__tests__/pick-canonical-workout.test.ts
+31
View File
@@ -445,3 +445,34 @@ describe("pickCanonicalWorkout — Strava brick-workout regression", () => {
expect(sports).toEqual(["cycling", "running"]);
});
});
/**
* #1054 — workouts a `workouts:write` bridge pushes in carry `EXTERNAL`, which
* is on no default ladder. Pinned because the settings copy promises what
* follows from that: a bridged session the phone also recorded shows once, as
* the phone's copy, and a bridged session nothing else recorded still shows.
*/
describe("pickCanonicalWorkout — the unranked EXTERNAL source", () => {
it("is on no default ladder", () => {
expect(DEFAULT_WORKOUT_SOURCE_PRIORITY).not.toContain("EXTERNAL");
});
it("yields to the phone's copy of the same session", () => {
const result = pickCanonicalWorkout([
row("bridge-1", "EXTERNAL", "2026-09-20T06:30:00.000Z"),
row("apple-1", "APPLE_HEALTH", "2026-09-20T06:31:00.000Z"),
]);
expect(result.canonical.map((w) => w.id)).toEqual(["apple-1"]);
});
it("survives when nothing else recorded the session", () => {
const result = pickCanonicalWorkout([
row("bridge-1", "EXTERNAL", "2026-09-20T06:30:00.000Z"),
row("apple-1", "APPLE_HEALTH", "2026-09-20T09:00:00.000Z"),
]);
expect(result.canonical.map((w) => w.id).sort()).toEqual([
"apple-1",
"bridge-1",
]);
});
});
src/lib/validations/tokens.ts
+10
View File
@@ -34,3 +34,13 @@ export const createDocumentTokenSchema = z.object({
name: z.string().min(1, "Name required").max(100),
expiresInDays: z.number().int().min(1).max(365).optional(),
});
/**
* Body of `POST /api/tokens/workouts` — mint a Bearer that can push workouts
* and nothing else (#1054). The same shape as the other two mints and for the
* same reasons: no scope field, a bounded lifetime.
*/
export const createWorkoutTokenSchema = z.object({
name: z.string().min(1, "Name required").max(100),
expiresInDays: z.number().int().min(1).max(365).optional(),
});
src/lib/workouts/scopes.ts
+21
View File
@@ -0,0 +1,21 @@
/**
* The Bearer scope for pushing workouts in from another system (#1054): a
* watch vendor's bridge, a script replaying an export.
*
* A zero-import leaf for the reason `MEASUREMENTS_WRITE_SCOPE` is one (see
* `@/lib/measurements/scopes`): the mint route, the batch route, the settings
* card and the guards all read it, and none of them should inherit an import
* graph for a string.
*
* Its own scope rather than a second door for `measurements:write`. A token
* someone already pasted into a scale's uploader should not quietly start
* accepting workouts because a later release widened what the name means; a
* scope grants what it names, and a bridge that sends both mints both.
*
* Deliberately alone. There is no read counterpart and must not be one: the
* workout list and detail stay cookie-equivalent, so a credential pasted into
* a bridge can add a session and learn nothing about the ones already stored.
*/
/** The scope a narrow token must carry to write workouts. */
export const WORKOUTS_WRITE_SCOPE = "workouts:write";
tests/integration/backup-preview.test.ts
+352
View File
@@ -0,0 +1,352 @@
/**
* The restore preview is worked out while a copy is written and answered from
* the row, without opening the copy (#1031).
*
* Both ends and the pipe: the copies are written by the real weekly job and
* the real upload route, the preview route reads them back, and what it
* answers is held against a full read of the stored copy, which is how the
* preview used to be taken. A copy from before previews is read whole once
* and its preview kept; a key dropped after the copy was written is still
* named, because the verdict is taken again at every read.
*
* Mutations that must turn this red: drop the preview scanner from
* `storeBackupBlob` (no preview on the row, the route opens the copy), or
* answer from the stored verdict instead of taking it again (the dropped key
* is not named).
*/
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { _resetCryptoCacheForTests, encrypt } from "@/lib/crypto";
import { Prisma } from "@/generated/prisma/client";
import { cookieJar, headerJar } from "./mock-next-headers";
import { getPrismaClient, truncateAllTables } from "./setup";
import { readStoredBackup } from "./stored-backup-read";
vi.mock("next/headers", async () => {
const { cookieJar, headerJar } = await import("./mock-next-headers");
return {
headers: vi.fn(async () => ({
get: (name: string) => headerJar.get(name.toLowerCase()) ?? null,
})),
cookies: vi.fn(async () => ({
get: (name: string) => {
const value = cookieJar.get(name);
return value ? { name, value } : undefined;
},
set: (name: string, value: string) => {
cookieJar.set(name, value);
},
delete: (name: string) => {
cookieJar.delete(name);
},
})),
};
});
vi.mock("@/lib/db-compat", () => ({
ensureDbCompatibility: vi.fn().mockResolvedValue(undefined),
}));
const opened = vi.hoisted(() => ({ count: 0 }));
vi.mock("@/lib/export/stored-backup", async (importOriginal) => {
const actual =
await importOriginal<typeof import("@/lib/export/stored-backup")>();
return {
...actual,
openStoredBackup: (...a: Parameters<typeof actual.openStoredBackup>) => {
opened.count += 1;
return actual.openStoredBackup(...a);
},
};
});
const saved = {
key: process.env.ENCRYPTION_KEY,
keys: process.env.ENCRYPTION_KEYS,
active: process.env.ENCRYPTION_ACTIVE_KEY_ID,
};
function useKeys(keys: Record<string, string>, active: string) {
process.env.ENCRYPTION_KEY = "";
process.env.ENCRYPTION_KEYS = JSON.stringify(keys);
process.env.ENCRYPTION_ACTIVE_KEY_ID = active;
_resetCryptoCacheForTests();
}
const OLD = "11".repeat(32);
const CUR = "22".repeat(32);
beforeEach(async () => {
await truncateAllTables(getPrismaClient());
cookieJar.clear();
headerJar.clear();
opened.count = 0;
useKeys({ old: OLD, cur: CUR }, "cur");
});
afterEach(() => {
process.env.ENCRYPTION_KEY = saved.key ?? "";
process.env.ENCRYPTION_KEYS = saved.keys ?? "";
process.env.ENCRYPTION_ACTIVE_KEY_ID = saved.active ?? "";
_resetCryptoCacheForTests();
});
async function seedAdmin() {
const prisma = getPrismaClient();
const admin = await prisma.user.create({
data: {
username: "preview-admin",
email: "preview-admin@example.test",
role: "ADMIN",
},
});
const session = await prisma.session.create({
data: { userId: admin.id, expiresAt: new Date(Date.now() + 60_000) },
});
cookieJar.set("healthlog_session", session.id);
return admin;
}
async function preview(id: string) {
const { GET } = await import("@/app/api/admin/backups/[id]/summary/route");
const res = await GET(
new Request(`http://localhost/api/admin/backups/${id}/summary`) as never,
{ params: Promise.resolve({ id }) },
);
return {
status: res.status,
body: (await res.json()) as {
data: { summary: Record<string, unknown> } | null;
error: string | null;
meta?: { errorCode?: string; keyIds?: string[] };
},
};
}
/** The counts a full read of the stored copy gives: the old preview. */
async function fullReadSummary(id: string) {
const { parseBackupPayload, summarizeBackup } =
await import("@/lib/validations/backup");
const opensBefore = opened.count;
const payload = parseBackupPayload(
JSON.parse(await readStoredBackup(getPrismaClient(), id)),
);
opened.count = opensBefore;
return summarizeBackup(payload);
}
describe("restore preview kept with the copy", () => {
it("the weekly backup writes it, and the preview answers from it", async () => {
const prisma = getPrismaClient();
const admin = await seedAdmin();
await prisma.measurement.createMany({
data: Array.from({ length: 25 }, (_, i) => ({
userId: admin.id,
type: "PULSE" as const,
value: 60 + i,
unit: "bpm",
measuredAt: new Date(Date.UTC(2026, 8, 1, 0, i)),
source: "MANUAL" as const,
})),
});
const { handleDataBackup } =
await import("@/lib/jobs/reminder/backup-handlers");
await handleDataBackup([]);
const row = await prisma.dataBackup.findFirstOrThrow({
where: { userId: admin.id, type: "WEEKLY_AUTO" },
});
expect(row.preview).toMatchObject({
version: 1,
copy: `chunks:${row.chunkStreamId}:${row.chunkCount}`,
summary: { measurements: 25 },
});
const res = await preview(row.id);
expect(res.status).toBe(200);
expect(opened.count).toBe(0);
expect(res.body.data!.summary).toEqual(await fullReadSummary(row.id));
expect(res.body.data!.summary.measurements).toBe(25);
});
it("an upload writes it, and a key dropped later is still named", async () => {
const prisma = getPrismaClient();
const admin = await seedAdmin();
// A note written under `old`, as a copy taken before a rotation carries.
useKeys({ old: OLD, cur: CUR }, "old");
const note = Buffer.from(encrypt("before"), "utf8").toString("base64");
useKeys({ old: OLD, cur: CUR }, "cur");
const file = JSON.stringify({
schemaVersion: "2",
exportedAt: "2026-09-20T00:00:00.000Z",
userId: admin.id,
measurements: Array.from({ length: 30 }, (_, i) => ({
id: `m-${i}`,
type: "PULSE",
value: 60 + (i % 30),
unit: "bpm",
measuredAt: new Date(Date.UTC(2026, 8, 1, 0, i)).toISOString(),
source: "MANUAL",
...(i === 3 ? { notesEncrypted: note } : {}),
})),
});
const { POST } = await import("@/app/api/admin/backups/upload/route");
const uploaded = await POST(
new Request("http://localhost/api/admin/backups/upload", {
method: "POST",
headers: { "content-type": "application/json" },
body: file,
}) as never,
);
expect(uploaded.status).toBe(201);
const { data: upload } = (await uploaded.json()) as {
data: { id: string; summary: Record<string, unknown> };
};
const row = await prisma.dataBackup.findUniqueOrThrow({
where: { id: upload.id },
});
expect(row.preview).toMatchObject({ summary: { measurements: 30 } });
const res = await preview(upload.id);
expect(res.status).toBe(200);
expect(opened.count).toBe(0);
expect(res.body.data!.summary).toEqual(upload.summary);
expect(res.body.data!.summary).toEqual(await fullReadSummary(upload.id));
// The operator drops `old` after the copy was written.
useKeys({ cur: CUR }, "cur");
const refused = await preview(upload.id);
expect(refused.status).toBe(422);
expect(refused.body.meta).toMatchObject({
errorCode: "backup.key.missing",
keyIds: ["old"],
});
expect(opened.count).toBe(0);
});
it("a copy from before previews is read whole once, then answered from the row", async () => {
const prisma = getPrismaClient();
const admin = await seedAdmin();
await prisma.measurement.createMany({
data: Array.from({ length: 5 }, (_, i) => ({
userId: admin.id,
type: "PULSE" as const,
value: 60 + i,
unit: "bpm",
measuredAt: new Date(Date.UTC(2026, 8, 1, 0, i)),
source: "MANUAL" as const,
})),
});
const { handleDataBackup } =
await import("@/lib/jobs/reminder/backup-handlers");
await handleDataBackup([]);
const row = await prisma.dataBackup.findFirstOrThrow({
where: { userId: admin.id },
});
// As an earlier release left it.
await prisma.dataBackup.update({
where: { id: row.id },
data: { preview: Prisma.DbNull },
});
const first = await preview(row.id);
expect(first.status).toBe(200);
expect(opened.count).toBe(1);
expect(first.body.data!.summary.measurements).toBe(5);
await vi.waitFor(async () => {
const kept = await prisma.dataBackup.findUniqueOrThrow({
where: { id: row.id },
});
expect(kept.preview).toMatchObject({ summary: { measurements: 5 } });
});
const second = await preview(row.id);
expect(second.status).toBe(200);
expect(opened.count).toBe(1);
expect(second.body.data!.summary).toEqual(first.body.data!.summary);
});
it("a replaced copy is described by its own preview", async () => {
const prisma = getPrismaClient();
const admin = await seedAdmin();
const { handleDataBackup } =
await import("@/lib/jobs/reminder/backup-handlers");
await handleDataBackup([]);
await prisma.measurement.create({
data: {
userId: admin.id,
type: "PULSE",
value: 61,
unit: "bpm",
measuredAt: new Date(Date.UTC(2026, 8, 2)),
source: "MANUAL",
},
});
await handleDataBackup([]);
const row = await prisma.dataBackup.findFirstOrThrow({
where: { userId: admin.id },
});
expect(row.preview).toMatchObject({
copy: `chunks:${row.chunkStreamId}:${row.chunkCount}`,
summary: { measurements: 1 },
});
const res = await preview(row.id);
expect(res.body.data!.summary.measurements).toBe(1);
expect(opened.count).toBe(0);
});
it("a single stored value keeps its preview when it is converted to pieces", async () => {
const prisma = getPrismaClient();
const admin = await seedAdmin();
// The oldest stored form: the whole file as one encrypted value.
const row = await prisma.dataBackup.create({
data: {
userId: admin.id,
type: "MANUAL_UPLOAD_1",
data: encrypt(
JSON.stringify({
schemaVersion: "1",
exportedAt: "2026-05-09T10:00:00.000Z",
userId: admin.id,
measurements: [
{
type: "WEIGHT",
value: 80,
unit: "kg",
measuredAt: "2026-05-08T07:00:00.000Z",
source: "MANUAL",
},
],
}),
),
},
});
const first = await preview(row.id);
expect(first.status).toBe(200);
expect(opened.count).toBe(1);
await vi.waitFor(async () => {
const kept = await prisma.dataBackup.findUniqueOrThrow({
where: { id: row.id },
});
expect(kept.preview).toMatchObject({ summary: { measurements: 1 } });
});
const { convertSingleValueBackup } =
await import("@/lib/export/store-backup-blob");
expect(await convertSingleValueBackup(prisma, row.id)).toBe("converted");
const converted = await prisma.dataBackup.findUniqueOrThrow({
where: { id: row.id },
});
expect(converted.preview).toMatchObject({
copy: `chunks:${converted.chunkStreamId}:${converted.chunkCount}`,
summary: { measurements: 1 },
});
const second = await preview(row.id);
expect(second.status).toBe(200);
expect(opened.count).toBe(1);
expect(second.body.data!.summary).toEqual(first.body.data!.summary);
});
});
tests/integration/bearer-scope-enforcement.test.ts
+170
View File
@@ -951,3 +951,173 @@ describe("B8 — the measurement-ingest scope reaches its two routes and no othe
expect(row.userId).toBe(ownerId);
});
});
describe("B9 — the workout-ingest scope reaches the workout batch and nothing else", () => {
function postWorkouts(entries: Array<Record<string, unknown>>): NextRequest {
return new NextRequest("https://health.example/api/workouts/batch", {
method: "POST",
headers: {
authorization: headerJar.get("authorization")!,
"content-type": "application/json",
},
body: JSON.stringify({ workouts: entries }),
} as never);
}
function bridgedRun(
externalId: string,
extra: Record<string, unknown> = {},
): Record<string, unknown> {
return {
sportType: "running",
startedAt: "2026-09-20T06:30:00.000Z",
endedAt: "2026-09-20T07:15:00.000Z",
externalId,
...extra,
};
}
it("admits the scope on POST /api/workouts/batch, attributed EXTERNAL", async () => {
await armToken(["workouts:write"], "wwrite1");
const { POST } = await import("@/app/api/workouts/batch/route");
const res = await POST(postWorkouts([bridgedRun("bridge-run-1")]));
expect(res.status).toBe(200);
const row = await getPrismaClient().workout.findFirstOrThrow({
where: { userId: USER_ID, externalId: "bridge-run-1" },
});
// The body names no source; the schema would default it to MANUAL, and
// the route resolves EXTERNAL from the credential instead.
expect(row.source).toBe("EXTERNAL");
});
it("re-posting the same bridged workout is a duplicate, not a second row", async () => {
// The dedup probe has to look in the EXTERNAL namespace the insert wrote
// to. Probing under the body's defaulted MANUAL would miss the stored row
// and the unique index would be the only thing standing in the way.
await armToken(["workouts:write"], "wwrite1b");
const { POST } = await import("@/app/api/workouts/batch/route");
await POST(postWorkouts([bridgedRun("bridge-run-2")]));
const again = await POST(postWorkouts([bridgedRun("bridge-run-2")]));
expect(again.status).toBe(200);
expect((await again.json()).data.entries[0].status).toBe("duplicate");
expect(await getPrismaClient().workout.count()).toBe(1);
});
it.each(["APPLE_HEALTH", "MANUAL"])(
"refuses a batch in which an entry names %s",
async (source) => {
await armToken(["workouts:write"], `wwrite2-${source}`);
const { POST } = await import("@/app/api/workouts/batch/route");
const res = await POST(
postWorkouts([
bridgedRun("bridge-run-3"),
bridgedRun("bridge-run-4", { source }),
]),
);
expect(res.status).toBe(422);
expect((await res.json()).meta?.errorCode).toBe(
"workout.batch.source_not_permitted",
);
expect(await getPrismaClient().workout.count()).toBe(0);
},
);
it("a measurements:write token is refused on the workout batch", async () => {
// Separate scopes on purpose: a token handed to a scale does not start
// accepting workouts.
await armToken(["measurements:write"], "wwrite3");
const { POST } = await import("@/app/api/workouts/batch/route");
const res = await POST(postWorkouts([bridgedRun("bridge-run-5")]));
expect(res.status).toBe(403);
expect(await getPrismaClient().workout.count()).toBe(0);
});
it("and a workouts:write token is refused on the measurement batch", async () => {
await armToken(["workouts:write"], "wwrite4");
const { POST } = await import("@/app/api/measurements/batch/route");
const res = await POST(
new NextRequest("https://health.example/api/measurements/batch", {
method: "POST",
headers: {
authorization: headerJar.get("authorization")!,
"content-type": "application/json",
},
body: JSON.stringify({
entries: [
{
hkIdentifier: "HKQuantityTypeIdentifierBodyMass",
value: 70,
unit: "kg",
startDate: new Date().toISOString(),
endDate: new Date().toISOString(),
externalId: "uuid-workouts-write-1",
},
],
}),
} as never),
);
expect(res.status).toBe(403);
expect(await getPrismaClient().measurement.count()).toBe(0);
});
it("is refused on the workout read legs", async () => {
await armToken(["workouts:write"], "wwrite5");
const { GET } = await import("@/app/api/workouts/route");
expect((await GET(req("/api/workouts"))).status).toBe(403);
await expect
.poll(() => lastBearerFailureReason(), { timeout: 5_000, interval: 100 })
.toBe("undeclared_scope");
});
it("cannot mint another token", async () => {
await armToken(["workouts:write"], "wwrite6");
const { POST } = await import("@/app/api/tokens/workouts/route");
const res = await POST(
new NextRequest("https://health.example/api/tokens/workouts", {
method: "POST",
headers: {
authorization: headerJar.get("authorization")!,
"content-type": "application/json",
},
body: JSON.stringify({ name: "second" }),
} as never),
);
expect(res.status).toBe(401);
});
it("the mint issues exactly this scope, and the token it issues writes", async () => {
// Mint on a cookie session with a fresh proof, then use the raw token the
// response carries: the round trip a bridge operator makes.
await useCookieSession();
const { POST: mint } = await import("@/app/api/tokens/workouts/route");
const minted = await mint(
new NextRequest("https://health.example/api/tokens/workouts", {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify({ name: "watch bridge" }),
} as never),
);
expect(minted.status).toBe(201);
const raw = (await minted.json()).data.token as string;
const row = await getPrismaClient().apiToken.findFirstOrThrow({
where: { name: "watch bridge" },
});
expect(row.permissions).toEqual(["workouts:write"]);
cookieJar.clear();
headerJar.set("authorization", `Bearer ${raw}`);
const { POST } = await import("@/app/api/workouts/batch/route");
const res = await POST(postWorkouts([bridgedRun("bridge-run-6")]));
expect(res.status).toBe(200);
const written = await getPrismaClient().workout.findFirstOrThrow({
where: { externalId: "bridge-run-6" },
});
expect(written.source).toBe("EXTERNAL");
});
});
tests/integration/dense-glucose-raw-window-series.test.ts
+238
View File
@@ -0,0 +1,238 @@
/**
* The per-kind glucose series inside the 90-day raw window, on a continuous
* sensor stream.
*
* Past 90 days glucose already folded per day in SQL. Inside it, a sensor
* reading every five minutes sent every row raw: 26 000 points in 90 days,
* the same class as the per-minute heart-rate stream in
* `dense-pulse-raw-window-reads.test.ts`. Over the row cap the series now
* folds per local hour; a fingerstick meter stays far under it and keeps its
* exact old shape.
*
* The doctor report deliberately keeps glucose raw inside 90 days: its
* clinical panel (time in range, GMI, CV and the risk indices) is computed
* from the raw readings of every source, and the day path computes it from
* canonical-source day buckets, which is not the same figure when a meter and
* a sensor overlap. The last case pins that the panel is still the raw
* engine's answer.
*/
import { NextRequest } from "next/server";
import { beforeAll, describe, expect, it, vi } from "vitest";
import { cookieJar } from "./mock-next-headers";
import { getPrismaClient, truncateAllTables } from "./setup";
import { computeGlucoseClinicalMetrics } from "@/lib/analytics/glucose-metrics";
import { collectDoctorReportData } from "@/lib/doctor-report-data";
import { convertGlucose } from "@/lib/glucose";
import { selectionFromLeaves } from "@/lib/report-selection/selection";
import { invalidateUserTimezone } from "@/lib/tz/resolver";
vi.mock("next/headers", async () => {
const { cookieJar, headerJar } = await import("./mock-next-headers");
return {
headers: vi.fn(async () => ({
get: (name: string) => headerJar.get(name.toLowerCase()) ?? null,
})),
cookies: vi.fn(async () => ({
get: (name: string) => {
const value = cookieJar.get(name);
return value ? { name, value } : undefined;
},
set: (name: string, value: string) => {
cookieJar.set(name, value);
},
delete: (name: string) => {
cookieJar.delete(name);
},
})),
};
});
vi.mock("@/lib/db-compat", () => ({
ensureDbCompatibility: vi.fn().mockResolvedValue(undefined),
}));
const prisma = getPrismaClient();
const TZ = "America/New_York";
const DENSE = "dense-glucose-owner";
const SPARSE = "sparse-glucose-owner";
/** Forty days of one reading every five minutes: over the raw-window cap. */
const DENSE_ROWS = 40 * 288;
const NOW = new Date(Math.floor(Date.now() / 300_000) * 300_000 - 300_000);
/** A fingerstick meter: a handful of readings a day at irregular times. */
function sparseReadings(): Array<{ at: Date; value: number }> {
const out: Array<{ at: Date; value: number }> = [];
for (let day = 1; day <= 30; day++) {
const perDay = 2 + (day % 4);
for (let i = 0; i < perDay; i++) {
out.push({
at: new Date(
NOW.getTime() - day * 86_400_000 + (i * 211 + day * 17) * 60_000,
),
value: 78 + ((day * 13 + i * 29) % 120),
});
}
}
return out;
}
async function session(userId: string): Promise<void> {
const s = await prisma.session.create({
data: { userId, expiresAt: new Date(Date.now() + 3_600_000) },
});
cookieJar.clear();
cookieJar.set("healthlog_session", s.id);
}
beforeAll(async () => {
await truncateAllTables(prisma);
await prisma.user.create({
data: { id: DENSE, username: DENSE, timezone: TZ, glucoseUnit: "mmol/L" },
});
await prisma.user.create({
data: { id: SPARSE, username: SPARSE, timezone: TZ },
});
invalidateUserTimezone(DENSE);
invalidateUserTimezone(SPARSE);
await prisma.$executeRawUnsafe(
`INSERT INTO measurements (
id, user_id, type, value, unit, source, measured_at,
created_at, updated_at, sync_version)
SELECT 'cg' || lpad(g::text, 7, '0'), $1, 'BLOOD_GLUCOSE'::measurement_type,
62 + ((g * 37) % 170), 'mg/dL', 'NIGHTSCOUT'::measurement_source,
($2::timestamptz AT TIME ZONE 'UTC') - (g * interval '5 minutes'),
now(), now(), 1
FROM generate_series(0, $3::int - 1) g`,
DENSE,
NOW.toISOString(),
DENSE_ROWS,
);
await prisma.measurement.createMany({
data: sparseReadings().map((r, i) => ({
id: `fs${String(i).padStart(4, "0")}`,
userId: SPARSE,
type: "BLOOD_GLUCOSE" as const,
value: r.value,
unit: "mg/dL",
source: "MANUAL" as const,
measuredAt: r.at,
})),
});
await prisma.$executeRawUnsafe(`ANALYZE measurements`);
}, 120_000);
async function series(userId: string, days: number) {
await session(userId);
const { GET } = await import("@/app/api/measurements/series/route");
const res = await GET(
new NextRequest(
`http://localhost/api/measurements/series?kind=glucose&days=${days}`,
),
);
expect(res.status).toBe(200);
return (
(await res.json()) as {
data: {
unit: string;
points: Array<Record<string, unknown> & { id: string; at: string }>;
stats: {
count: number;
mean: number;
min: number;
max: number;
stdDev: number;
};
};
}
).data;
}
describe("per-kind glucose series inside the raw window", () => {
it("sends a sensor stream as hour buckets in the user's unit, statistics over every reading", async () => {
const data = await series(DENSE, 90);
const rows = await prisma.measurement.findMany({
where: { userId: DENSE, type: "BLOOD_GLUCOSE" },
select: { value: true, measuredAt: true },
});
expect(rows).toHaveLength(DENSE_ROWS);
// Forty days of hours, give or take the partial hours at either end.
expect(data.points.length).toBeLessThanOrEqual(40 * 24 + 2);
expect(data.points.length).toBeGreaterThanOrEqual(40 * 24 - 2);
expect(data.points.every((p) => p.id.startsWith("hour:"))).toBe(true);
expect(data.unit).toBe("mmol/L");
// Every bucket is the mean of its hour's readings, converted once. New
// York sits a whole number of hours off UTC, so its local hours are UTC
// hours.
const byHour = new Map<number, number[]>();
for (const r of rows) {
const hour = Math.floor(r.measuredAt.getTime() / 3_600_000) * 3_600_000;
(byHour.get(hour) ?? byHour.set(hour, []).get(hour)!).push(r.value);
}
for (const p of data.points) {
const values = byHour.get(new Date(p.at).getTime());
expect(values, p.at).toBeDefined();
const mean = values!.reduce((a, b) => a + b, 0) / values!.length;
expect(p).toEqual({
id: `hour:${p.at}`,
at: p.at,
value: convertGlucose(mean, "mmol/L"),
secondary: null,
});
}
const values = rows.map((r) => r.value);
const mean = values.reduce((a, b) => a + b, 0) / values.length;
expect(data.stats.count).toBe(DENSE_ROWS);
expect(data.stats.min).toBe(convertGlucose(Math.min(...values), "mmol/L"));
expect(data.stats.max).toBe(convertGlucose(Math.max(...values), "mmol/L"));
expect(data.stats.mean).toBe(
convertGlucose(Math.round(mean * 100) / 100, "mmol/L"),
);
});
it("keeps fingerstick readings raw, one point per reading", async () => {
const data = await series(SPARSE, 90);
const expected = sparseReadings()
.map((r, i) => ({
id: `fs${String(i).padStart(4, "0")}`,
at: r.at.toISOString(),
value: r.value,
secondary: null,
}))
.sort((a, b) => a.at.localeCompare(b.at));
expect(data.unit).toBe("mg/dL");
expect(data.points).toEqual(expected);
expect(data.stats.count).toBe(expected.length);
});
});
describe("doctor report keeps glucose raw inside the raw window", () => {
it("computes the clinical panel from every raw reading", async () => {
const end = new Date();
const start = new Date(end.getTime() - 60 * 86_400_000);
const data = await collectDoctorReportData(
DENSE,
{ start, end, days: 60 },
selectionFromLeaves(["BLOOD_GLUCOSE", "GLUCOSE_PANEL"]),
);
const rows = await prisma.measurement.findMany({
where: {
userId: DENSE,
type: "BLOOD_GLUCOSE",
measuredAt: { gte: start, lte: end },
},
orderBy: { measuredAt: "asc" },
select: { value: true, measuredAt: true },
});
expect(data.measurements.BLOOD_GLUCOSE).toHaveLength(rows.length);
expect(data.glucoseClinical).toEqual(
computeGlucoseClinicalMetrics(
rows.map((r) => ({ measuredAt: r.measuredAt, mgdl: r.value })),
{ windowDays: 60, now: end },
),
);
});
});
tests/integration/integration-backfill-lane-expiry.test.ts
+169
View File
@@ -0,0 +1,169 @@
/**
* The full-history admission lane against a real pg-boss on a real Postgres:
* what happens to an import that runs past its job's expiry.
*
* pg-boss 12.34, measured with the lane's worker shape (localConcurrency 1,
* groupConcurrency 1, exclusive policy, retries with backoff), a two-second
* expiry, a one-second retry delay and a handler that ignores the abort
* signal and runs for eight seconds:
*
* +5ms start #1 (job 1709c8a6…) running=1
* +2008ms abort signal #1
* +4051ms start #2 (same job id) running=2
* +6054ms abort signal #2
* +8007ms end #1 running=1
* +9593ms start #3 (same job id) running=2
* +12053ms end #2
* +17595ms end #3
* +19166ms start #4 (same job id) running=1
*
* At the expiry pg-boss fails the job, fires the signal, frees the worker and
* group slots, and retries the job after its delay; the handler keeps going.
* The retry then starts in the same worker beside the run it replaced, and
* the lane's one-at-a-time holds for everything except the import too long
* for its expiry. The first test keeps that as an executable statement about
* pg-boss; the second runs the lane's own handler under the same conditions.
*/
import { afterAll, beforeAll, describe, expect, it, vi } from "vitest";
import { PgBoss, type Job } from "pg-boss";
const runner = vi.hoisted(() => ({
running: 0,
maxRunning: 0,
calls: 0,
holdMs: 8_000,
}));
vi.mock("@/lib/jobs/google-health-backfill", () => ({
// Ignores the budget and the signal on purpose: the case the lock covers
// is a run that does not stop when pg-boss gives up on it.
runGoogleHealthBackfillForUser: async () => {
runner.calls += 1;
runner.running += 1;
runner.maxRunning = Math.max(runner.maxRunning, runner.running);
await new Promise((resolve) => setTimeout(resolve, runner.holdMs));
runner.running -= 1;
return { imported: 1 };
},
}));
import {
integrationBackfillAdmissionSendOptions,
type IntegrationBackfillAdmissionPayload,
} from "@/lib/jobs/integration-backfill-admission";
import { drainIntegrationBackfillAdmission } from "@/lib/jobs/integration-backfill-drain";
import { withIntegrationBackfillLock } from "@/lib/jobs/integration-backfill-lock";
import { runJob } from "@/lib/jobs/run-job";
let boss: PgBoss;
const suffix = `${Date.now()}-${Math.floor(Math.random() * 1e6)}`;
/** Short enough to cross in a test, the lane's shape otherwise. */
const EXPIRY_OVERRIDE = { expireInSeconds: 2, retryDelay: 1 } as const;
const WORK_OPTIONS = {
localConcurrency: 1,
groupConcurrency: 1,
pollingIntervalSeconds: 0.5,
} as const;
const payload: IntegrationBackfillAdmissionPayload = {
kind: "google-health-backfill",
data: { userId: `lane-user-${suffix}`, enqueuedAt: new Date().toISOString() },
};
const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));
beforeAll(async () => {
boss = new PgBoss({ connectionString: process.env.DATABASE_URL });
await boss.start();
}, 120_000);
afterAll(async () => {
await boss?.stop({ graceful: false });
});
describe("an import that outlives its expiry", () => {
it("pg-boss starts the retry of the same job beside the run it expired", async () => {
const queue = `lane-expiry-raw-${suffix}`;
await boss.createQueue(queue, { policy: "exclusive" });
const ids: string[] = [];
let running = 0;
let maxRunning = 0;
await boss.work(queue, WORK_OPTIONS, async ([job]: Job<object>[]) => {
ids.push(job!.id);
running += 1;
maxRunning = Math.max(maxRunning, running);
await sleep(8_000);
running -= 1;
});
await boss.send(queue, payload, {
...integrationBackfillAdmissionSendOptions(payload),
...EXPIRY_OVERRIDE,
});
await sleep(12_000);
await boss.offWork(queue);
expect(ids.length).toBeGreaterThanOrEqual(2);
expect(new Set(ids).size).toBe(1);
expect(maxRunning).toBeGreaterThanOrEqual(2);
}, 60_000);
it("the lane's handler runs the import once while its retries find it running", async () => {
const queue = `lane-expiry-drain-${suffix}`;
await boss.createQueue(queue, { policy: "exclusive" });
const handler = runJob(queue, drainIntegrationBackfillAdmission);
const deliveries: string[] = [];
await boss.work(
queue,
WORK_OPTIONS,
async (jobs: Job<IntegrationBackfillAdmissionPayload>[]) => {
deliveries.push(jobs[0]!.id);
return handler(jobs);
},
);
await boss.send(queue, payload, {
...integrationBackfillAdmissionSendOptions(payload),
...EXPIRY_OVERRIDE,
});
// The run holds for eight seconds; its retries arrive while it does.
await sleep(7_000);
await boss.offWork(queue);
await sleep(2_000);
expect(deliveries.length).toBeGreaterThanOrEqual(2);
expect(runner.calls).toBe(1);
expect(runner.maxRunning).toBe(1);
}, 60_000);
});
describe("withIntegrationBackfillLock", () => {
it("admits one run per identity, others alongside, and the next once it ends", async () => {
let release!: () => void;
const held = new Promise<void>((resolve) => (release = resolve));
const first = withIntegrationBackfillLock("kind|a", () => held);
await sleep(300);
const same = await withIntegrationBackfillLock("kind|a", async () => 1);
const other = await withIntegrationBackfillLock("kind|b", async () => 2);
expect(same).toEqual({ ran: false });
expect(other).toEqual({ ran: true, result: 2 });
release();
await first;
const after = await withIntegrationBackfillLock("kind|a", async () => 3);
expect(after).toEqual({ ran: true, result: 3 });
});
it("releases the lock when the run throws", async () => {
await expect(
withIntegrationBackfillLock("kind|c", async () => {
throw new Error("boom");
}),
).rejects.toThrow("boom");
const after = await withIntegrationBackfillLock("kind|c", async () => 4);
expect(after).toEqual({ ran: true, result: 4 });
});
});
tests/integration/sync-changes.test.ts
+1 -1
View File
@@ -399,7 +399,7 @@ describe("GET /api/sync/changes (real Postgres)", () => {
// pruned; the recent reachable tombstone survives. `drained` says the
// batched walk reached the end of the backlog rather than stopping at
// its per-run cap.
expect(pruned).toEqual({ deleted: 1, drained: true });
expect(pruned).toEqual({ deleted: 1, drained: true, deferredAccounts: 0 });
const survivors = await prisma.measurement.findMany({
where: { userId: TEST_USER_ID },
select: { externalId: true },
tests/integration/tombstone-cleanup-restore-lock.test.ts
+152
View File
@@ -0,0 +1,152 @@
/**
* #1031 — the nightly tombstone purge against a running restore.
*
* A restore clears every row of the account, tombstones included, inside one
* long transaction that holds the account's restore lock exclusively. The
* purge used to delete expired tombstones across accounts in one statement
* per batch, without the lock: on an account under restore it waited on the
* restore's row locks for as long as the restore ran, or deadlocked with it.
*
* Here a transaction stands in for the restore: it takes the lock, deletes the
* account's rows, and holds for three seconds before rolling back. The purge
* runs while it holds. What must hold: the purge does not wait, leaves the
* account under restore alone and says so, and still purges every other
* account.
*
* Mutation check: drop the `holdAccountAgainstRestore` call from
* `purgeTombstonesByAccount` and the purge waits out the hold, then deletes
* the restoring account's tombstones once the stand-in rolls back.
*/
import { afterAll, beforeEach, describe, expect, it, vi } from "vitest";
import { getPrismaClient, truncateAllTables } from "./setup";
vi.mock("@/lib/db-compat", () => ({
ensureDbCompatibility: vi.fn().mockResolvedValue(undefined),
}));
const RESTORING = "tombstone-restoring";
const OTHER = "tombstone-other";
const HOLD_MS = 3_000;
const LONG_AGO = new Date("2020-01-01T00:00:00.000Z");
async function seed() {
const prisma = getPrismaClient();
for (const id of [RESTORING, OTHER]) {
await prisma.user.create({
data: { id, username: id, role: "USER", timezone: "UTC" },
});
for (let i = 0; i < 3; i++) {
await prisma.measurement.create({
data: {
id: `${id}-tomb-${i}`,
userId: id,
type: "WEIGHT",
value: 70 + i,
unit: "kg",
source: "MANUAL",
measuredAt: new Date(`2020-01-0${i + 1}T08:00:00.000Z`),
deletedAt: LONG_AGO,
},
});
}
await prisma.moodEntry.create({
data: {
id: `${id}-mood-tomb`,
userId: id,
date: "2020-01-01",
mood: "OKAY",
score: 3,
moodLoggedAt: new Date("2020-01-01T08:00:00.000Z"),
deletedAt: LONG_AGO,
},
});
}
}
beforeEach(async () => {
await truncateAllTables(getPrismaClient());
});
afterAll(async () => {
await truncateAllTables(getPrismaClient());
});
describe("tombstone purge against a running restore (#1031)", () => {
it("leaves the account under restore for the next run without waiting on it", async () => {
const prisma = getPrismaClient();
await seed();
const { takeRestoreLock } = await import("@/lib/export/restore-lock");
const {
cleanupExpiredMeasurementTombstones,
cleanupExpiredMoodTombstones,
} = await import("@/lib/jobs/measurement-tombstone-cleanup");
let holding!: () => void;
const isHolding = new Promise<void>((resolve) => (holding = resolve));
const restore = prisma
.$transaction(
async (tx) => {
await takeRestoreLock(tx, RESTORING);
await tx.measurement.deleteMany({ where: { userId: RESTORING } });
await tx.moodEntry.deleteMany({ where: { userId: RESTORING } });
holding();
await new Promise((resolve) => setTimeout(resolve, HOLD_MS));
throw new Error("rolled back");
},
{ timeout: 30_000 },
)
.catch(() => undefined);
await isHolding;
const started = Date.now();
const measurements = await cleanupExpiredMeasurementTombstones(prisma);
const moods = await cleanupExpiredMoodTombstones(prisma);
const elapsed = Date.now() - started;
await restore;
expect(elapsed).toBeLessThan(HOLD_MS / 2);
expect(measurements).toEqual({
deleted: 3,
drained: true,
deferredAccounts: 1,
});
expect(moods).toEqual({ deleted: 1, drained: true, deferredAccounts: 1 });
// The stand-in rolled back, so the restoring account still has its
// tombstones; the next run purges them.
expect(
await prisma.measurement.count({ where: { userId: RESTORING } }),
).toBe(3);
expect(await prisma.measurement.count({ where: { userId: OTHER } })).toBe(
0,
);
expect(await prisma.moodEntry.count({ where: { userId: OTHER } })).toBe(0);
const next = await cleanupExpiredMeasurementTombstones(prisma);
expect(next).toEqual({ deleted: 3, drained: true, deferredAccounts: 0 });
}, 60_000);
it("a restore that starts during the purge waits only for the account's batch", async () => {
const prisma = getPrismaClient();
await seed();
const { takeRestoreLock } = await import("@/lib/export/restore-lock");
const { cleanupExpiredMeasurementTombstones } =
await import("@/lib/jobs/measurement-tombstone-cleanup");
const purge = cleanupExpiredMeasurementTombstones(prisma);
const restore = prisma.$transaction(async (tx) => {
await takeRestoreLock(tx, RESTORING);
return tx.measurement.deleteMany({ where: { userId: RESTORING } });
});
const [purged] = await Promise.all([purge, restore]);
// Whichever went first, both finished and nothing is left.
expect(purged.deferredAccounts).toBeLessThanOrEqual(1);
expect(await prisma.measurement.count({ where: { userId: OTHER } })).toBe(
0,
);
expect(
await prisma.measurement.count({ where: { userId: RESTORING } }),
).toBe(0);
}, 60_000);
});