Merge branch 'feat/apple-clinical-labs' into release/v1.47.7

# Conflicts:
#	CHANGELOG.md
#	src/__tests__/baselines/source-file-lines.json
#	src/__tests__/baselines/source-function-lines.json
#	src/__tests__/write-rate-limit-guard.test.ts
This commit is contained in:
Marc-André Bombeck committed 2026-11-17 19:42:40 +01:00
commit 5bb77b975a
84 files changed
+4668 -252

No files matched your search

+207 -9
View File
@@ -17419,6 +17419,51 @@ paths:
schema:
$ref: "#/components/schemas/ErrorEnvelope"
"429": *a12
/api/labs/fhir-batch:
post:
tags:
- Labs
summary: Ingest lab results from Apple Health's Health Records
description: "Maps each FHIR lab result (DSTU2 or R4 Observation) to a lab reading and writes it: final results only
(`final`, `amended`, `corrected`), the laboratory category, numeric values in their unit (UCUM codes are read),
a value with a comparator (`<0.5 mg/L`) kept as text as printed, string and coded values as text, the report's
own reference range, and the panel's name for its members. The name resolves to the caller's marker by its own
name, an alias, or the catalog analyte its LOINC code names; a new analyte gets a new marker; a reading in
another unit is converted when the conversion is known and refused otherwise. The identity is a digest of
`sourceURL` and `identifier`, the same one an Apple Health export import builds, so a result already imported
from an export is a `duplicate` here, and sending a batch again is safe. First write wins: an amended result
under a held identity is a duplicate. Owner-only: a cookie session or a full-scope login token. The body is
capped at 1 MiB (413 past it): a client splits a large sync by size as well as by count. Rate limit 60 batches a
minute; Idempotency-Key supported. A response carrying `write_failed` is `Cache-Control: no-store` and is not
replayed for its key. Audits as `labs.fhir.batch` with counts only."
parameters:
- *a6
requestBody:
required: true
content:
application/json:
schema:
$ref: "#/components/schemas/FhirLabBatchRequest"
responses:
"200":
description: Batch processed; one result per record. `outcome` reads `inserted` against `failed` (skips and duplicates
are information, not failure).
content:
application/json:
schema:
$ref: "#/components/schemas/FhirLabBatchEnvelope"
"401": *a1
"403": *a9
"409": *a7
"422":
description: "Nothing was written. `meta.errorCode` = `labs.fhir_batch.invalid`: no records, more than 200, a record
without a `resource` object, an over-long HealthKit field, or a field the request does not define (every
issue under `details.issues`)."
content:
application/json:
schema:
$ref: "#/components/schemas/ErrorEnvelope"
"429": *a5
/api/labs/overview:
get:
tags:
@@ -35222,6 +35267,47 @@ components:
optional `source` records provenance: `"OCR"` for an on-device-OCR capture (the raw image never reaches the
server on this path), defaulting to `"MANUAL"` when omitted. The optional `encounterId` files the reading
against a visit the caller owns — always optional, and a link that could not be made never fails the reading.'
FhirLabBatchRequest:
type: object
properties:
records:
minItems: 1
maxItems: 200
type: array
items:
type: object
properties:
resource:
type: object
propertyNames:
type: string
additionalProperties: {}
description: The FHIR resource as JSON (`HKFHIRResource.data`), DSTU2 or R4. Only Observations of the laboratory
category are kept.
fhirVersion:
description: "`HKFHIRResource.fhirVersion` (e.g. `4.0.1`)."
type: string
maxLength: 16
identifier:
description: "`HKFHIRResource.identifier`, the resource's id at its source."
type: string
maxLength: 256
sourceURL:
description: "`HKFHIRResource.sourceURL`, the issuing institution's FHIR base."
type: string
maxLength: 2048
sourceName:
description: The institution as the Health app names it (`sourceRevision.source.name`).
type: string
maxLength: 200
required:
- resource
required:
- records
additionalProperties: false
description: "1 to 200 Health Records lab results (`HKClinicalTypeIdentifierLabResultRecord`), each the FHIR resource as
JSON plus the HealthKit fields its identity is built from. Send a panel and its members in the same batch so the
members carry the panel's name. No `userId`: the owner is the caller."
UpdateLabResultRequest:
type: object
properties:
@@ -40595,12 +40681,12 @@ components:
insights.generate.scopeChanged; labs — labs.biomarker.alias.taken, labs.biomarker.create.invalid,
labs.biomarker.duplicate, labs.biomarker.merge.invalid, labs.biomarker.merge.unitMismatch,
labs.biomarker.notFound, labs.biomarker.update.invalid, labs.biomarker.update.referenceRangeInvalid,
labs.create.invalid, labs.series.invalid, labs.link.invalid, labs.ocr.budgetExceeded,
labs.ocr.extractFailed, labs.ocr.fileTooLarge, labs.ocr.fileType, labs.ocr.localOcrDisabled,
labs.ocr.outputLimit, labs.ocr.pdfRendererUnavailable, labs.ocr.providerUnsupported,
labs.ocr.rateLimited, labs.ocr.tooLarge, labs.restore.invalid, labs.result.notFound, labs.unit.mismatch,
labs.update.invalid, labs.update.linkedFieldsImmutable, labs.update.numericExpected,
labs.update.qualitativeExpected, labs.update.referenceRangeInvalid,
labs.create.invalid, labs.fhir_batch.invalid, labs.series.invalid, labs.link.invalid,
labs.ocr.budgetExceeded, labs.ocr.extractFailed, labs.ocr.fileTooLarge, labs.ocr.fileType,
labs.ocr.localOcrDisabled, labs.ocr.outputLimit, labs.ocr.pdfRendererUnavailable,
labs.ocr.providerUnsupported, labs.ocr.rateLimited, labs.ocr.tooLarge, labs.restore.invalid,
labs.result.notFound, labs.unit.mismatch, labs.update.invalid, labs.update.linkedFieldsImmutable,
labs.update.numericExpected, labs.update.qualitativeExpected, labs.update.referenceRangeInvalid,
labs.update.sourceReferenceRangeInvalid; life_event — life_event.invalid, life_event.not_found;
managed_profile — managed_profile.guardian.duplicate, managed_profile.guardian.managed_invitee,
managed_profile.guardian.required, managed_profile.handover.no_pending,
@@ -45440,6 +45526,12 @@ components:
source:
type: string
minLength: 1
externalId:
anyOf:
- type: string
minLength: 1
maxLength: 128
- type: "null"
biomarkerName:
type:
- string
@@ -77453,9 +77545,10 @@ components:
type: string
additionalProperties: {}
- type: "null"
description: The worker's outcome summary, null until it finishes. Free-form except for `ecg`, whose five counters
(`discovered` / `imported` / `updated` / `skipped` / `failed`) are normalised to bounded non-negative
integers before serving.
description: "The worker's outcome summary, null until it finishes. Free-form except for `ecg` and `labs`, whose five
counters each (`ecg`: `discovered` / `imported` / `updated` / `skipped` / `failed`; `labs`, the lab
results from Health Records: `discovered` / `imported` / `duplicate` / `skipped` / `failed`) are
normalised to bounded non-negative integers before serving."
failureReason:
description: Free text on `failed`; null otherwise.
type:
@@ -78001,6 +78094,9 @@ components:
type: string
source:
type: string
description: "Where the reading came from: `MANUAL` (typed), `OCR` (a scanned report), `DOCUMENT` (a document import),
`IMPORT` (a lab CSV) or `APPLE_HEALTH` (Apple Health's Health Records, from an export or `POST
/api/labs/fhir-batch`). Open-ended: render an unknown value as-is."
hasNote:
type: boolean
rangeStatus:
@@ -78095,6 +78191,105 @@ components:
- data
- error
additionalProperties: false
FhirLabBatchEnvelope:
type: object
properties:
data:
$ref: "#/components/schemas/FhirLabBatchResponse"
error:
type: "null"
meta:
type: object
properties:
requestId:
type: string
additionalProperties: false
required:
- data
- error
additionalProperties: false
FhirLabBatchResponse:
type: object
properties:
results:
type: array
items:
$ref: "#/components/schemas/FhirLabRecordResult"
inserted:
type: integer
minimum: -9007199254740991
maximum: 9007199254740991
duplicate:
type: integer
minimum: -9007199254740991
maximum: 9007199254740991
skipped:
type: integer
minimum: -9007199254740991
maximum: 9007199254740991
failed:
type: integer
minimum: -9007199254740991
maximum: 9007199254740991
outcome:
type: string
enum:
- empty
- failed
- partial
- success
required:
- results
- inserted
- duplicate
- skipped
- failed
- outcome
additionalProperties: false
FhirLabRecordResult:
type: object
properties:
index:
type: integer
minimum: -9007199254740991
maximum: 9007199254740991
status:
type: string
enum:
- inserted
- duplicate
- skipped
- failed
reason:
type: string
enum:
- not_observation
- not_lab
- not_final
- withdrawn
- panel_header
- no_value
- unsupported_value
- no_date
- future_date
- invalid_date
- too_large
- missing_unit
- unit_mismatch
- text_too_long
- write_failed
required:
- index
- status
additionalProperties: false
description: "One record's outcome, by its position in the request. `inserted`: written. `duplicate`: the record already
holds it, under the same identity (deleted readings included, so a deleted result stays deleted) or as the same
reading on the same day from another source. `skipped`: not a reading HealthLog keeps, with `reason`
`not_observation` | `not_lab` | `not_final` (preliminary; it is taken once final) | `withdrawn` | `panel_header`
| `no_value` | `unsupported_value` | `no_date` | `future_date` | `invalid_date` | `too_large` (a resource over
256 KiB). `failed`: a reading that could not be stored, with `reason` `missing_unit` | `unit_mismatch` (the
account's marker is in a unit this one does not convert into) | `text_too_long` | `write_failed` (transient:
send it again)."
LabOverviewEnvelope:
type: object
properties:
@@ -78331,6 +78526,9 @@ components:
type: string
source:
type: string
description: "Where the reading came from: `MANUAL` (typed), `OCR` (a scanned report), `DOCUMENT` (a document import),
`IMPORT` (a lab CSV) or `APPLE_HEALTH` (Apple Health's Health Records, from an export or `POST
/api/labs/fhir-batch`). Open-ended: render an unknown value as-is."
rangeStatus:
$ref: "#/components/schemas/LabReferenceRangeStatus"
createdAt:
+46 -13
View File
@@ -30,10 +30,12 @@ re-uploading the same archive merges instead of duplicating.
The archive contains `apple_health_export/export.xml` (the canonical
record stream) plus a handful of auxiliary files (workout routes,
clinical records, electrocardiograms). The importer reads `export.xml`
and supported ECG CSVs under
`apple_health_export/electrocardiograms/`; unsupported auxiliary files
remain ignored.
clinical records, electrocardiograms). The importer reads `export.xml`,
supported ECG CSVs under `apple_health_export/electrocardiograms/`,
workout routes (when the account keeps them) and the lab results among
the Health Records files under `apple_health_export/clinical-records/`;
other auxiliary files remain ignored. Large exports written as ZIP64 are
read in full.
## 2. Upload to HealthLog
@@ -107,13 +109,13 @@ they triggered.
The parser folds four XML element types and supported auxiliary ECG
CSVs into the existing HealthLog schema (CHANGELOG v1.4.34):
| XML element | Maps to | Notes |
| ------------------ | -------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `<Record>` | `Measurement` rows | One row per spot sample. Keyed by `HKMetadataKeyExternalUUID` when present, otherwise a deterministic `sample:<sha256-of-attributes>` fallback so re-imports stay idempotent. |
| `<Workout>` | `Workout` rows | Activity type + duration + energy + distance. |
| `<Correlation>` | `Measurement` rows | Compound samples like blood pressure (systolic + diastolic) explode into their per-metric children. |
| `<ClinicalRecord>` | `Measurement` rows where the type maps | Health-record FHIR snapshots; only the metric-typed fields are ingested. |
| ECG CSV | `EcgRecording` rows | The source-provided waveform is encrypted before persistence; descriptors and the device's classification are stored without diagnostic inference. |
| XML element | Maps to | Notes |
| ------------------ | ----------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `<Record>` | `Measurement` rows | One row per spot sample. Keyed by `HKMetadataKeyExternalUUID` when present, otherwise a deterministic `sample:<sha256-of-attributes>` fallback so re-imports stay idempotent. |
| `<Workout>` | `Workout` rows | Activity type + duration + energy + distance. |
| `<Correlation>` | `Measurement` rows | Compound samples like blood pressure (systolic + diastolic) explode into their per-metric children. |
| `<ClinicalRecord>` | `LabResult` rows (lab results only) | `HKClinicalTypeIdentifierLabResultRecord` entries: the referenced FHIR Observation (DSTU2 or R4) becomes a lab reading under source `APPLE_HEALTH`. Other clinical types are counted as skipped. |
| ECG CSV | `EcgRecording` rows | The source-provided waveform is encrypted before persistence; descriptors and the device's classification are stored without diagnostic inference. |
The HK quantity types that map to HealthLog metric types live in
`src/lib/measurements/apple-health-mapping.ts`. Anything outside the
@@ -161,6 +163,36 @@ device classifications are preserved as source-reported facts;
unknown or localized classifications are stored as `null`. HealthLog
does not inspect the waveform to infer AFib or make a diagnosis.
### Lab results from Health Records
When the Health app holds lab results downloaded from a provider
(Health Records), `export.xml` lists each one as a
`<ClinicalRecord type="HKClinicalTypeIdentifierLabResultRecord">` whose
`resourceFilePath` names a FHIR JSON file under `clinical-records/`.
After the `export.xml` rows are written, the importer reads those files
one at a time (at most 50,000 files, 2 MiB each, 256 MiB together) and
turns each final laboratory Observation into a lab reading:
- the name is the result's own text; a LOINC code matches it to the
account's existing marker for the same analyte, and a reading in
another unit is converted into the marker's unit with the printed
value kept beside it;
- a result with a comparator (`<0.5 mg/L`) is kept as text, as printed;
- the report's own reference range is kept with the reading;
- a panel header is not a reading, but its members carry its name;
- preliminary results, withdrawn results and other categories (vital
signs, for instance) are skipped.
Each reading keeps an identity built from the issuing institution and
the record identifier (a digest, `lab_results.external_id`). Uploading
the archive again writes nothing new, a result deleted in HealthLog
stays deleted, and the iOS app sending the same result through
`POST /api/labs/fhir-batch` is recognised as the same reading. A
reading already entered by hand, scanned or imported from a CSV on the
same day with the same value is not written twice. The pass is
fail-soft like the ECG pass, and the job result carries only
`discovered/imported/duplicate/skipped/failed` lab counts.
### Cumulative HK type → daily aggregate
Cumulative quantity types — steps, active energy, walking/running
@@ -201,8 +233,9 @@ The two-axis idempotency story:
1. **File-level** — re-uploading the same `export.zip` (same SHA-256
of bytes) short-circuits to the previous job only when that job
used the current parser revision. Existing jobs may use revisions 1
or 2; new jobs use revision 3, so an earlier successful archive can be
deliberately processed again under the corrected aggregation.
to 3; new jobs use revision 4, so an earlier successful archive can be
deliberately processed again: revision 3 corrected the aggregation,
revision 4 reads the lab results among the Health Records.
2. **Record-level** — when an older `export.zip` is re-exported from
iOS (same history, slightly newer device timestamps), every
record's external ID stays stable. The serialized reconciliation
+3 -2
View File
@@ -40,8 +40,9 @@ the browser — and a background job parses it while a progress indicator
polls for the imported / skipped counts.
Re-uploading the same archive resolves to the same job by content hash,
so it merges instead of creating duplicates. ECG CSVs and metric-typed
clinical records are imported too; see
so it merges instead of creating duplicates. ECG CSVs and the lab
results among the Health Records are imported too (lab results land in
Labs, matched to your markers); see
[`apple-health.md`](./apple-health.md) for the details and the full
type mapping.