diff --git a/CHANGELOG.md b/CHANGELOG.md index 085276698..1bc6658b1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,10 @@ ## [Unreleased] +### Added + +- Lab results from Apple Health's Health Records now land in Labs. An Apple Health export used to skip every clinical record; the lab results among them are now read, matched to your existing markers (by name, alias or LOINC code, with a reading in another unit converted and the printed value kept), and filed under their panel. A result like "<0.5 mg/L" is kept as printed, the report's own reference range comes along, and preliminary or withdrawn results are left out. Importing the same results again writes nothing, a result you delete stays deleted, and a reading you already typed or scanned is not doubled. If you imported an archive before, upload it once more to pick up its lab results. For the iOS app, `POST /api/labs/fhir-batch` takes the same results straight from HealthKit. Thanks to @camiracode for asking for this. (#1400) + ### Changed - API responses and CSV or JSON exports now go out gzip-compressed when the @@ -13,6 +17,7 @@ - A lab marker created from a text result, such as "negative" on a scanned report or a CSV row without a unit, can be edited again. It opens with "No unit" ticked, a rename saves without touching the unit, and the unit can be set while the marker holds only text results. The unit stays locked once numbers are stored under it, and writing the same unit another way (`mg/dl` for `mg/dL`) no longer counts as a change. (#1393) - On an account with a watch sending heart rate every few seconds, weight, blood pressure and other occasional readings went missing from the MCP metric series, the Coach and the insights, because the heart rate filled every read before they were reached. Each kind of reading now gets its own share, so they all show up again. (#1398) +- Large Apple Health exports, which the Health app writes as ZIP64 archives, no longer lose their ECGs and workout routes: the reader for those files refused ZIP64 and counted every one as failed. ### Security diff --git a/docs/integrations/apple-health.md b/docs/integrations/apple-health.md index 7a74f821e..1df0e6ede 100644 --- a/docs/integrations/apple-health.md +++ b/docs/integrations/apple-health.md @@ -30,10 +30,12 @@ re-uploading the same archive merges instead of duplicating. The archive contains `apple_health_export/export.xml` (the canonical record stream) plus a handful of auxiliary files (workout routes, -clinical records, electrocardiograms). The importer reads `export.xml` -and supported ECG CSVs under -`apple_health_export/electrocardiograms/`; unsupported auxiliary files -remain ignored. +clinical records, electrocardiograms). The importer reads `export.xml`, +supported ECG CSVs under `apple_health_export/electrocardiograms/`, +workout routes (when the account keeps them) and the lab results among +the Health Records files under `apple_health_export/clinical-records/`; +other auxiliary files remain ignored. Large exports written as ZIP64 are +read in full. ## 2. Upload to HealthLog @@ -107,13 +109,13 @@ they triggered. The parser folds four XML element types and supported auxiliary ECG CSVs into the existing HealthLog schema (CHANGELOG v1.4.34): -| XML element | Maps to | Notes | -| ------------------ | -------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `` | `Measurement` rows | One row per spot sample. Keyed by `HKMetadataKeyExternalUUID` when present, otherwise a deterministic `sample:` fallback so re-imports stay idempotent. | -| `` | `Workout` rows | Activity type + duration + energy + distance. | -| `` | `Measurement` rows | Compound samples like blood pressure (systolic + diastolic) explode into their per-metric children. | -| `` | `Measurement` rows where the type maps | Health-record FHIR snapshots; only the metric-typed fields are ingested. | -| ECG CSV | `EcgRecording` rows | The source-provided waveform is encrypted before persistence; descriptors and the device's classification are stored without diagnostic inference. | +| XML element | Maps to | Notes | +| ------------------ | ----------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `` | `Measurement` rows | One row per spot sample. Keyed by `HKMetadataKeyExternalUUID` when present, otherwise a deterministic `sample:` fallback so re-imports stay idempotent. | +| `` | `Workout` rows | Activity type + duration + energy + distance. | +| `` | `Measurement` rows | Compound samples like blood pressure (systolic + diastolic) explode into their per-metric children. | +| `` | `LabResult` rows (lab results only) | `HKClinicalTypeIdentifierLabResultRecord` entries: the referenced FHIR Observation (DSTU2 or R4) becomes a lab reading under source `APPLE_HEALTH`. Other clinical types are counted as skipped. | +| ECG CSV | `EcgRecording` rows | The source-provided waveform is encrypted before persistence; descriptors and the device's classification are stored without diagnostic inference. | The HK quantity types that map to HealthLog metric types live in `src/lib/measurements/apple-health-mapping.ts`. Anything outside the @@ -161,6 +163,36 @@ device classifications are preserved as source-reported facts; unknown or localized classifications are stored as `null`. HealthLog does not inspect the waveform to infer AFib or make a diagnosis. +### Lab results from Health Records + +When the Health app holds lab results downloaded from a provider +(Health Records), `export.xml` lists each one as a +`` whose +`resourceFilePath` names a FHIR JSON file under `clinical-records/`. +After the `export.xml` rows are written, the importer reads those files +one at a time (at most 50,000 files, 2 MiB each, 256 MiB together) and +turns each final laboratory Observation into a lab reading: + +- the name is the result's own text; a LOINC code matches it to the + account's existing marker for the same analyte, and a reading in + another unit is converted into the marker's unit with the printed + value kept beside it; +- a result with a comparator (`<0.5 mg/L`) is kept as text, as printed; +- the report's own reference range is kept with the reading; +- a panel header is not a reading, but its members carry its name; +- preliminary results, withdrawn results and other categories (vital + signs, for instance) are skipped. + +Each reading keeps an identity built from the issuing institution and +the record identifier (a digest, `lab_results.external_id`). Uploading +the archive again writes nothing new, a result deleted in HealthLog +stays deleted, and the iOS app sending the same result through +`POST /api/labs/fhir-batch` is recognised as the same reading. A +reading already entered by hand, scanned or imported from a CSV on the +same day with the same value is not written twice. The pass is +fail-soft like the ECG pass, and the job result carries only +`discovered/imported/duplicate/skipped/failed` lab counts. + ### Cumulative HK type → daily aggregate Cumulative quantity types — steps, active energy, walking/running @@ -201,8 +233,9 @@ The two-axis idempotency story: 1. **File-level** — re-uploading the same `export.zip` (same SHA-256 of bytes) short-circuits to the previous job only when that job used the current parser revision. Existing jobs may use revisions 1 - or 2; new jobs use revision 3, so an earlier successful archive can be - deliberately processed again under the corrected aggregation. + to 3; new jobs use revision 4, so an earlier successful archive can be + deliberately processed again: revision 3 corrected the aggregation, + revision 4 reads the lab results among the Health Records. 2. **Record-level** — when an older `export.zip` is re-exported from iOS (same history, slightly newer device timestamps), every record's external ID stays stable. The serialized reconciliation diff --git a/docs/integrations/data-import.md b/docs/integrations/data-import.md index c59065e39..c4438ed83 100644 --- a/docs/integrations/data-import.md +++ b/docs/integrations/data-import.md @@ -40,8 +40,9 @@ the browser — and a background job parses it while a progress indicator polls for the imported / skipped counts. Re-uploading the same archive resolves to the same job by content hash, -so it merges instead of creating duplicates. ECG CSVs and metric-typed -clinical records are imported too; see +so it merges instead of creating duplicates. ECG CSVs and the lab +results among the Health Records are imported too (lab results land in +Labs, matched to your markers); see [`apple-health.md`](./apple-health.md) for the details and the full type mapping. diff --git a/src/lib/measurements/apple-health-mapping.ts b/src/lib/measurements/apple-health-mapping.ts index 02a7931a0..23084402c 100644 --- a/src/lib/measurements/apple-health-mapping.ts +++ b/src/lib/measurements/apple-health-mapping.ts @@ -1263,7 +1263,10 @@ export const HK_QUANTITY_TYPE_DEFERRED = new Set([ // branch runs. "HKCategoryTypeIdentifierBleedingAfterPregnancy", "HKCategoryTypeIdentifierBleedingDuringPregnancy", - // Clinical (FHIR) — v1.6+ + // Clinical (FHIR). These never arrive as a ``; the export writes + // `` elements, which the parser handles itself: lab results + // are imported as lab readings (`src/lib/apple-health/clinical-labs-import.ts`), + // every other clinical type is counted in `clinical.skipped`. "HKClinicalTypeIdentifierAllergyRecord", "HKClinicalTypeIdentifierConditionRecord", "HKClinicalTypeIdentifierImmunizationRecord",