mirror of
https://github.com/MBombeck/HealthLog.git
synced 2026-11-19 08:12:52 +00:00
Merge branch 'feat/apple-clinical-labs' into release/v1.47.7
# Conflicts: # CHANGELOG.md # src/__tests__/baselines/source-file-lines.json # src/__tests__/baselines/source-function-lines.json # src/__tests__/write-rate-limit-guard.test.ts
This commit is contained in:
commit
5bb77b975a
84 files changed
+4668
-252
No files matched your search
+207
-9
@@ -17419,6 +17419,51 @@ paths:
|
||||
schema:
|
||||
$ref: "#/components/schemas/ErrorEnvelope"
|
||||
"429": *a12
|
||||
/api/labs/fhir-batch:
|
||||
post:
|
||||
tags:
|
||||
- Labs
|
||||
summary: Ingest lab results from Apple Health's Health Records
|
||||
description: "Maps each FHIR lab result (DSTU2 or R4 Observation) to a lab reading and writes it: final results only
|
||||
(`final`, `amended`, `corrected`), the laboratory category, numeric values in their unit (UCUM codes are read),
|
||||
a value with a comparator (`<0.5 mg/L`) kept as text as printed, string and coded values as text, the report's
|
||||
own reference range, and the panel's name for its members. The name resolves to the caller's marker by its own
|
||||
name, an alias, or the catalog analyte its LOINC code names; a new analyte gets a new marker; a reading in
|
||||
another unit is converted when the conversion is known and refused otherwise. The identity is a digest of
|
||||
`sourceURL` and `identifier`, the same one an Apple Health export import builds, so a result already imported
|
||||
from an export is a `duplicate` here, and sending a batch again is safe. First write wins: an amended result
|
||||
under a held identity is a duplicate. Owner-only: a cookie session or a full-scope login token. The body is
|
||||
capped at 1 MiB (413 past it): a client splits a large sync by size as well as by count. Rate limit 60 batches a
|
||||
minute; Idempotency-Key supported. A response carrying `write_failed` is `Cache-Control: no-store` and is not
|
||||
replayed for its key. Audits as `labs.fhir.batch` with counts only."
|
||||
parameters:
|
||||
- *a6
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/FhirLabBatchRequest"
|
||||
responses:
|
||||
"200":
|
||||
description: Batch processed; one result per record. `outcome` reads `inserted` against `failed` (skips and duplicates
|
||||
are information, not failure).
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/FhirLabBatchEnvelope"
|
||||
"401": *a1
|
||||
"403": *a9
|
||||
"409": *a7
|
||||
"422":
|
||||
description: "Nothing was written. `meta.errorCode` = `labs.fhir_batch.invalid`: no records, more than 200, a record
|
||||
without a `resource` object, an over-long HealthKit field, or a field the request does not define (every
|
||||
issue under `details.issues`)."
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/ErrorEnvelope"
|
||||
"429": *a5
|
||||
/api/labs/overview:
|
||||
get:
|
||||
tags:
|
||||
@@ -35222,6 +35267,47 @@ components:
|
||||
optional `source` records provenance: `"OCR"` for an on-device-OCR capture (the raw image never reaches the
|
||||
server on this path), defaulting to `"MANUAL"` when omitted. The optional `encounterId` files the reading
|
||||
against a visit the caller owns — always optional, and a link that could not be made never fails the reading.'
|
||||
FhirLabBatchRequest:
|
||||
type: object
|
||||
properties:
|
||||
records:
|
||||
minItems: 1
|
||||
maxItems: 200
|
||||
type: array
|
||||
items:
|
||||
type: object
|
||||
properties:
|
||||
resource:
|
||||
type: object
|
||||
propertyNames:
|
||||
type: string
|
||||
additionalProperties: {}
|
||||
description: The FHIR resource as JSON (`HKFHIRResource.data`), DSTU2 or R4. Only Observations of the laboratory
|
||||
category are kept.
|
||||
fhirVersion:
|
||||
description: "`HKFHIRResource.fhirVersion` (e.g. `4.0.1`)."
|
||||
type: string
|
||||
maxLength: 16
|
||||
identifier:
|
||||
description: "`HKFHIRResource.identifier`, the resource's id at its source."
|
||||
type: string
|
||||
maxLength: 256
|
||||
sourceURL:
|
||||
description: "`HKFHIRResource.sourceURL`, the issuing institution's FHIR base."
|
||||
type: string
|
||||
maxLength: 2048
|
||||
sourceName:
|
||||
description: The institution as the Health app names it (`sourceRevision.source.name`).
|
||||
type: string
|
||||
maxLength: 200
|
||||
required:
|
||||
- resource
|
||||
required:
|
||||
- records
|
||||
additionalProperties: false
|
||||
description: "1 to 200 Health Records lab results (`HKClinicalTypeIdentifierLabResultRecord`), each the FHIR resource as
|
||||
JSON plus the HealthKit fields its identity is built from. Send a panel and its members in the same batch so the
|
||||
members carry the panel's name. No `userId`: the owner is the caller."
|
||||
UpdateLabResultRequest:
|
||||
type: object
|
||||
properties:
|
||||
@@ -40595,12 +40681,12 @@ components:
|
||||
insights.generate.scopeChanged; labs — labs.biomarker.alias.taken, labs.biomarker.create.invalid,
|
||||
labs.biomarker.duplicate, labs.biomarker.merge.invalid, labs.biomarker.merge.unitMismatch,
|
||||
labs.biomarker.notFound, labs.biomarker.update.invalid, labs.biomarker.update.referenceRangeInvalid,
|
||||
labs.create.invalid, labs.series.invalid, labs.link.invalid, labs.ocr.budgetExceeded,
|
||||
labs.ocr.extractFailed, labs.ocr.fileTooLarge, labs.ocr.fileType, labs.ocr.localOcrDisabled,
|
||||
labs.ocr.outputLimit, labs.ocr.pdfRendererUnavailable, labs.ocr.providerUnsupported,
|
||||
labs.ocr.rateLimited, labs.ocr.tooLarge, labs.restore.invalid, labs.result.notFound, labs.unit.mismatch,
|
||||
labs.update.invalid, labs.update.linkedFieldsImmutable, labs.update.numericExpected,
|
||||
labs.update.qualitativeExpected, labs.update.referenceRangeInvalid,
|
||||
labs.create.invalid, labs.fhir_batch.invalid, labs.series.invalid, labs.link.invalid,
|
||||
labs.ocr.budgetExceeded, labs.ocr.extractFailed, labs.ocr.fileTooLarge, labs.ocr.fileType,
|
||||
labs.ocr.localOcrDisabled, labs.ocr.outputLimit, labs.ocr.pdfRendererUnavailable,
|
||||
labs.ocr.providerUnsupported, labs.ocr.rateLimited, labs.ocr.tooLarge, labs.restore.invalid,
|
||||
labs.result.notFound, labs.unit.mismatch, labs.update.invalid, labs.update.linkedFieldsImmutable,
|
||||
labs.update.numericExpected, labs.update.qualitativeExpected, labs.update.referenceRangeInvalid,
|
||||
labs.update.sourceReferenceRangeInvalid; life_event — life_event.invalid, life_event.not_found;
|
||||
managed_profile — managed_profile.guardian.duplicate, managed_profile.guardian.managed_invitee,
|
||||
managed_profile.guardian.required, managed_profile.handover.no_pending,
|
||||
@@ -45440,6 +45526,12 @@ components:
|
||||
source:
|
||||
type: string
|
||||
minLength: 1
|
||||
externalId:
|
||||
anyOf:
|
||||
- type: string
|
||||
minLength: 1
|
||||
maxLength: 128
|
||||
- type: "null"
|
||||
biomarkerName:
|
||||
type:
|
||||
- string
|
||||
@@ -77453,9 +77545,10 @@ components:
|
||||
type: string
|
||||
additionalProperties: {}
|
||||
- type: "null"
|
||||
description: The worker's outcome summary, null until it finishes. Free-form except for `ecg`, whose five counters
|
||||
(`discovered` / `imported` / `updated` / `skipped` / `failed`) are normalised to bounded non-negative
|
||||
integers before serving.
|
||||
description: "The worker's outcome summary, null until it finishes. Free-form except for `ecg` and `labs`, whose five
|
||||
counters each (`ecg`: `discovered` / `imported` / `updated` / `skipped` / `failed`; `labs`, the lab
|
||||
results from Health Records: `discovered` / `imported` / `duplicate` / `skipped` / `failed`) are
|
||||
normalised to bounded non-negative integers before serving."
|
||||
failureReason:
|
||||
description: Free text on `failed`; null otherwise.
|
||||
type:
|
||||
@@ -78001,6 +78094,9 @@ components:
|
||||
type: string
|
||||
source:
|
||||
type: string
|
||||
description: "Where the reading came from: `MANUAL` (typed), `OCR` (a scanned report), `DOCUMENT` (a document import),
|
||||
`IMPORT` (a lab CSV) or `APPLE_HEALTH` (Apple Health's Health Records, from an export or `POST
|
||||
/api/labs/fhir-batch`). Open-ended: render an unknown value as-is."
|
||||
hasNote:
|
||||
type: boolean
|
||||
rangeStatus:
|
||||
@@ -78095,6 +78191,105 @@ components:
|
||||
- data
|
||||
- error
|
||||
additionalProperties: false
|
||||
FhirLabBatchEnvelope:
|
||||
type: object
|
||||
properties:
|
||||
data:
|
||||
$ref: "#/components/schemas/FhirLabBatchResponse"
|
||||
error:
|
||||
type: "null"
|
||||
meta:
|
||||
type: object
|
||||
properties:
|
||||
requestId:
|
||||
type: string
|
||||
additionalProperties: false
|
||||
required:
|
||||
- data
|
||||
- error
|
||||
additionalProperties: false
|
||||
FhirLabBatchResponse:
|
||||
type: object
|
||||
properties:
|
||||
results:
|
||||
type: array
|
||||
items:
|
||||
$ref: "#/components/schemas/FhirLabRecordResult"
|
||||
inserted:
|
||||
type: integer
|
||||
minimum: -9007199254740991
|
||||
maximum: 9007199254740991
|
||||
duplicate:
|
||||
type: integer
|
||||
minimum: -9007199254740991
|
||||
maximum: 9007199254740991
|
||||
skipped:
|
||||
type: integer
|
||||
minimum: -9007199254740991
|
||||
maximum: 9007199254740991
|
||||
failed:
|
||||
type: integer
|
||||
minimum: -9007199254740991
|
||||
maximum: 9007199254740991
|
||||
outcome:
|
||||
type: string
|
||||
enum:
|
||||
- empty
|
||||
- failed
|
||||
- partial
|
||||
- success
|
||||
required:
|
||||
- results
|
||||
- inserted
|
||||
- duplicate
|
||||
- skipped
|
||||
- failed
|
||||
- outcome
|
||||
additionalProperties: false
|
||||
FhirLabRecordResult:
|
||||
type: object
|
||||
properties:
|
||||
index:
|
||||
type: integer
|
||||
minimum: -9007199254740991
|
||||
maximum: 9007199254740991
|
||||
status:
|
||||
type: string
|
||||
enum:
|
||||
- inserted
|
||||
- duplicate
|
||||
- skipped
|
||||
- failed
|
||||
reason:
|
||||
type: string
|
||||
enum:
|
||||
- not_observation
|
||||
- not_lab
|
||||
- not_final
|
||||
- withdrawn
|
||||
- panel_header
|
||||
- no_value
|
||||
- unsupported_value
|
||||
- no_date
|
||||
- future_date
|
||||
- invalid_date
|
||||
- too_large
|
||||
- missing_unit
|
||||
- unit_mismatch
|
||||
- text_too_long
|
||||
- write_failed
|
||||
required:
|
||||
- index
|
||||
- status
|
||||
additionalProperties: false
|
||||
description: "One record's outcome, by its position in the request. `inserted`: written. `duplicate`: the record already
|
||||
holds it, under the same identity (deleted readings included, so a deleted result stays deleted) or as the same
|
||||
reading on the same day from another source. `skipped`: not a reading HealthLog keeps, with `reason`
|
||||
`not_observation` | `not_lab` | `not_final` (preliminary; it is taken once final) | `withdrawn` | `panel_header`
|
||||
| `no_value` | `unsupported_value` | `no_date` | `future_date` | `invalid_date` | `too_large` (a resource over
|
||||
256 KiB). `failed`: a reading that could not be stored, with `reason` `missing_unit` | `unit_mismatch` (the
|
||||
account's marker is in a unit this one does not convert into) | `text_too_long` | `write_failed` (transient:
|
||||
send it again)."
|
||||
LabOverviewEnvelope:
|
||||
type: object
|
||||
properties:
|
||||
@@ -78331,6 +78526,9 @@ components:
|
||||
type: string
|
||||
source:
|
||||
type: string
|
||||
description: "Where the reading came from: `MANUAL` (typed), `OCR` (a scanned report), `DOCUMENT` (a document import),
|
||||
`IMPORT` (a lab CSV) or `APPLE_HEALTH` (Apple Health's Health Records, from an export or `POST
|
||||
/api/labs/fhir-batch`). Open-ended: render an unknown value as-is."
|
||||
rangeStatus:
|
||||
$ref: "#/components/schemas/LabReferenceRangeStatus"
|
||||
createdAt:
|
||||
|
||||
@@ -30,10 +30,12 @@ re-uploading the same archive merges instead of duplicating.
|
||||
|
||||
The archive contains `apple_health_export/export.xml` (the canonical
|
||||
record stream) plus a handful of auxiliary files (workout routes,
|
||||
clinical records, electrocardiograms). The importer reads `export.xml`
|
||||
and supported ECG CSVs under
|
||||
`apple_health_export/electrocardiograms/`; unsupported auxiliary files
|
||||
remain ignored.
|
||||
clinical records, electrocardiograms). The importer reads `export.xml`,
|
||||
supported ECG CSVs under `apple_health_export/electrocardiograms/`,
|
||||
workout routes (when the account keeps them) and the lab results among
|
||||
the Health Records files under `apple_health_export/clinical-records/`;
|
||||
other auxiliary files remain ignored. Large exports written as ZIP64 are
|
||||
read in full.
|
||||
|
||||
## 2. Upload to HealthLog
|
||||
|
||||
@@ -107,13 +109,13 @@ they triggered.
|
||||
The parser folds four XML element types and supported auxiliary ECG
|
||||
CSVs into the existing HealthLog schema (CHANGELOG v1.4.34):
|
||||
|
||||
| XML element | Maps to | Notes |
|
||||
| ------------------ | -------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `<Record>` | `Measurement` rows | One row per spot sample. Keyed by `HKMetadataKeyExternalUUID` when present, otherwise a deterministic `sample:<sha256-of-attributes>` fallback so re-imports stay idempotent. |
|
||||
| `<Workout>` | `Workout` rows | Activity type + duration + energy + distance. |
|
||||
| `<Correlation>` | `Measurement` rows | Compound samples like blood pressure (systolic + diastolic) explode into their per-metric children. |
|
||||
| `<ClinicalRecord>` | `Measurement` rows where the type maps | Health-record FHIR snapshots; only the metric-typed fields are ingested. |
|
||||
| ECG CSV | `EcgRecording` rows | The source-provided waveform is encrypted before persistence; descriptors and the device's classification are stored without diagnostic inference. |
|
||||
| XML element | Maps to | Notes |
|
||||
| ------------------ | ----------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| `<Record>` | `Measurement` rows | One row per spot sample. Keyed by `HKMetadataKeyExternalUUID` when present, otherwise a deterministic `sample:<sha256-of-attributes>` fallback so re-imports stay idempotent. |
|
||||
| `<Workout>` | `Workout` rows | Activity type + duration + energy + distance. |
|
||||
| `<Correlation>` | `Measurement` rows | Compound samples like blood pressure (systolic + diastolic) explode into their per-metric children. |
|
||||
| `<ClinicalRecord>` | `LabResult` rows (lab results only) | `HKClinicalTypeIdentifierLabResultRecord` entries: the referenced FHIR Observation (DSTU2 or R4) becomes a lab reading under source `APPLE_HEALTH`. Other clinical types are counted as skipped. |
|
||||
| ECG CSV | `EcgRecording` rows | The source-provided waveform is encrypted before persistence; descriptors and the device's classification are stored without diagnostic inference. |
|
||||
|
||||
The HK quantity types that map to HealthLog metric types live in
|
||||
`src/lib/measurements/apple-health-mapping.ts`. Anything outside the
|
||||
@@ -161,6 +163,36 @@ device classifications are preserved as source-reported facts;
|
||||
unknown or localized classifications are stored as `null`. HealthLog
|
||||
does not inspect the waveform to infer AFib or make a diagnosis.
|
||||
|
||||
### Lab results from Health Records
|
||||
|
||||
When the Health app holds lab results downloaded from a provider
|
||||
(Health Records), `export.xml` lists each one as a
|
||||
`<ClinicalRecord type="HKClinicalTypeIdentifierLabResultRecord">` whose
|
||||
`resourceFilePath` names a FHIR JSON file under `clinical-records/`.
|
||||
After the `export.xml` rows are written, the importer reads those files
|
||||
one at a time (at most 50,000 files, 2 MiB each, 256 MiB together) and
|
||||
turns each final laboratory Observation into a lab reading:
|
||||
|
||||
- the name is the result's own text; a LOINC code matches it to the
|
||||
account's existing marker for the same analyte, and a reading in
|
||||
another unit is converted into the marker's unit with the printed
|
||||
value kept beside it;
|
||||
- a result with a comparator (`<0.5 mg/L`) is kept as text, as printed;
|
||||
- the report's own reference range is kept with the reading;
|
||||
- a panel header is not a reading, but its members carry its name;
|
||||
- preliminary results, withdrawn results and other categories (vital
|
||||
signs, for instance) are skipped.
|
||||
|
||||
Each reading keeps an identity built from the issuing institution and
|
||||
the record identifier (a digest, `lab_results.external_id`). Uploading
|
||||
the archive again writes nothing new, a result deleted in HealthLog
|
||||
stays deleted, and the iOS app sending the same result through
|
||||
`POST /api/labs/fhir-batch` is recognised as the same reading. A
|
||||
reading already entered by hand, scanned or imported from a CSV on the
|
||||
same day with the same value is not written twice. The pass is
|
||||
fail-soft like the ECG pass, and the job result carries only
|
||||
`discovered/imported/duplicate/skipped/failed` lab counts.
|
||||
|
||||
### Cumulative HK type → daily aggregate
|
||||
|
||||
Cumulative quantity types — steps, active energy, walking/running
|
||||
@@ -201,8 +233,9 @@ The two-axis idempotency story:
|
||||
1. **File-level** — re-uploading the same `export.zip` (same SHA-256
|
||||
of bytes) short-circuits to the previous job only when that job
|
||||
used the current parser revision. Existing jobs may use revisions 1
|
||||
or 2; new jobs use revision 3, so an earlier successful archive can be
|
||||
deliberately processed again under the corrected aggregation.
|
||||
to 3; new jobs use revision 4, so an earlier successful archive can be
|
||||
deliberately processed again: revision 3 corrected the aggregation,
|
||||
revision 4 reads the lab results among the Health Records.
|
||||
2. **Record-level** — when an older `export.zip` is re-exported from
|
||||
iOS (same history, slightly newer device timestamps), every
|
||||
record's external ID stays stable. The serialized reconciliation
|
||||
|
||||
@@ -40,8 +40,9 @@ the browser — and a background job parses it while a progress indicator
|
||||
polls for the imported / skipped counts.
|
||||
|
||||
Re-uploading the same archive resolves to the same job by content hash,
|
||||
so it merges instead of creating duplicates. ECG CSVs and metric-typed
|
||||
clinical records are imported too; see
|
||||
so it merges instead of creating duplicates. ECG CSVs and the lab
|
||||
results among the Health Records are imported too (lab results land in
|
||||
Labs, matched to your markers); see
|
||||
[`apple-health.md`](./apple-health.md) for the details and the full
|
||||
type mapping.
|
||||
|
||||
|
||||
Reference in new issue
Block a user