fix(ingest): refuse an external id that cannot be a stable identity

An external id is half of every ingest dedup key — (userId, source,
externalId), (userId, externalSource, externalId), (userId, type,
source, externalId). Re-posting the same triple is idempotent and
returns the existing row, which holds only while the id is stable
across client launches. A client that serialised an opaque platform
object with its default object description sent a per-allocation
memory address instead, so no sweep ever matched its own earlier row
and each one minted a fresh record: a live instance grew 23 phantom
medications in a day, none of which could collect a dose, because the
dose side resolved through the same rotating string.

Add one shared classifier for identifiers that provably cannot be an
identity — an angle-bracketed object description carrying a hex
pointer, a bare 0x address, and a blank value — and wire it at every
surface that accepts a client-supplied externalId.

Single-entry creates refuse through the schema, so the client gets the
standard 422 multi-issue envelope with a message naming the shape it
sent and what to send instead. Batch and bulk surfaces classify per
entry and report the offending row as skipped with reason
unstable_external_id, leaving the rest of the payload to land: one bad
row must not stop a sync. Every refusal carries pinned wide-event meta
(count, shape, surface) and never the id itself.

A structural guard freezes both routes into the floor: a request-side
externalId declaration either applies the field check or carries a
marker naming the file that checks it per entry, and the guard verifies
that file really does. The backup-restore payload is the one exemption,
with its reason recorded — it replays ids the server itself wrote.
This commit is contained in:
Marc-André Bombeck committed 2026-07-25 17:21:00 +02:00
1 parent ef9891fec5
commit 2ab66ab2e3
26 files changed
+1232 -64

No files matched your search

+2 -1
View File
@@ -81,7 +81,8 @@ const batchEntrySchema = z
"Dedup key for the `(userId, type, source, externalId)` composite index. Three shapes:\n" +
"• `HKSample.uuid` — a per-sample reading. First-write-wins: a re-post returns `duplicate` and the stored row is immutable.\n" +
"• `stats:<HKQuantityTypeIdentifier>:<YYYY-MM-DD>` — a per-day cumulative total (Steps, Active Energy, Sleep Duration, Walking/Running Distance, Flights Climbed). A re-post OVERWRITES the row and returns `updated`.\n" +
"• `stats:HKQuantityTypeIdentifierHeartRate:<bucket-start>` — v1.19.0 (iOS #34), refined to 10-minute granularity in v1.30.7, heart-rate bucket carrying the bucket's AVERAGE bpm as one PULSE row. `<bucket-start>` is the ISO-8601 UTC 10-minute boundary — zeroed seconds/millis, minutes ∈ {00,10,20,30,40,50}, trailing `Z` (e.g. `2026-06-21T14:10:00.000Z`). A re-post OVERWRITES the row and returns `updated`, so iOS uploads ~144 rows/day instead of one per raw HR sample. A row that targets this prefix with a malformed bucket-start suffix is `skipped` with reason `malformed_hr_bucket_id`.",
"• `stats:HKQuantityTypeIdentifierHeartRate:<bucket-start>` — v1.19.0 (iOS #34), refined to 10-minute granularity in v1.30.7, heart-rate bucket carrying the bucket's AVERAGE bpm as one PULSE row. `<bucket-start>` is the ISO-8601 UTC 10-minute boundary — zeroed seconds/millis, minutes ∈ {00,10,20,30,40,50}, trailing `Z` (e.g. `2026-06-21T14:10:00.000Z`). A re-post OVERWRITES the row and returns `updated`, so iOS uploads ~144 rows/day instead of one per raw HR sample. A row that targets this prefix with a malformed bucket-start suffix is `skipped` with reason `malformed_hr_bucket_id`.\n" +
"The id MUST be stable across app restarts. An object description or memory address (`<Class: 0x…>`, `0x12ab34cd`) is a new value on every launch, so it can never match its own earlier row; such an entry is `skipped` with reason `unstable_external_id` while the rest of the batch still lands.",
),
externalSourceVersion: z.string().min(1).max(120).optional(),
// v1.8.6 W6 — optional per-entry source tag. Defaults to
+1 -1
View File
@@ -104,7 +104,7 @@ const bulkIntakeEntry = z
.max(128)
.optional()
.describe(
"The HealthKit dose-event UUID. Drives the idempotent re-sync dedup (first-write-wins per Apple dose).",
"The HealthKit dose-event UUID. Drives the idempotent re-sync dedup (first-write-wins per Apple dose). Must be stable across app restarts — an object description or memory address (`<Class: 0x…>`, `0x12ab34cd`) is a new value on every launch; such an entry is `skipped` with reason `unstable_external_id` while the rest of the batch still lands.",
),
})
.meta({
+1 -1
View File
@@ -221,7 +221,7 @@ const bulkMoodEntry = z
.max(120)
.optional()
.describe(
"iOS-side source-stable id for idempotent dedup on the `(userId, source, externalId)` key.",
"iOS-side source-stable id for idempotent dedup on the `(userId, source, externalId)` key. Must be stable across app restarts — an object description or memory address (`<Class: 0x…>`, `0x12ab34cd`) is a new value on every launch; such an entry is `skipped` with reason `unstable_external_id` while the rest of the batch still lands.",
),
})
.meta({