fix(ingest): refuse an external id that cannot be a stable identity

An external id is half of every ingest dedup key — (userId, source,
externalId), (userId, externalSource, externalId), (userId, type,
source, externalId). Re-posting the same triple is idempotent and
returns the existing row, which holds only while the id is stable
across client launches. A client that serialised an opaque platform
object with its default object description sent a per-allocation
memory address instead, so no sweep ever matched its own earlier row
and each one minted a fresh record: a live instance grew 23 phantom
medications in a day, none of which could collect a dose, because the
dose side resolved through the same rotating string.

Add one shared classifier for identifiers that provably cannot be an
identity — an angle-bracketed object description carrying a hex
pointer, a bare 0x address, and a blank value — and wire it at every
surface that accepts a client-supplied externalId.

Single-entry creates refuse through the schema, so the client gets the
standard 422 multi-issue envelope with a message naming the shape it
sent and what to send instead. Batch and bulk surfaces classify per
entry and report the offending row as skipped with reason
unstable_external_id, leaving the rest of the payload to land: one bad
row must not stop a sync. Every refusal carries pinned wide-event meta
(count, shape, surface) and never the id itself.

A structural guard freezes both routes into the floor: a request-side
externalId declaration either applies the field check or carries a
marker naming the file that checks it per entry, and the guard verifies
that file really does. The backup-restore payload is the one exemption,
with its reason recorded — it replays ids the server itself wrote.
This commit is contained in:
Marc-André Bombeck committed 2026-07-25 17:21:00 +02:00
1 parent ef9891fec5
commit 2ab66ab2e3
26 files changed
+1232 -64

No files matched your search

+43 -10
View File
@@ -70,6 +70,12 @@ import {
createBatchWorkoutSchema,
MAX_WORKOUTS_PER_BATCH,
} from "@/lib/validations/workout";
import {
classifyExternalId,
unstableExternalIdMeta,
UNSTABLE_EXTERNAL_ID_REASON,
type UnstableExternalIdShape,
} from "@/lib/validations/external-id";
import { dedupeWorkoutBatch } from "@/lib/workouts/canonical-rows";
import { Prisma, type MeasurementSource } from "@/generated/prisma/client";
@@ -281,7 +287,26 @@ async function postBatch(request: NextRequest): Promise<Response> {
};
const results: EntryResult[] = new Array(workouts.length);
const prepared: Prepared[] = workouts.map((w, index) => {
const unstableShapes: UnstableExternalIdShape[] = [];
const prepared: Prepared[] = workouts.flatMap((w, index) => {
// The `(userId, source, externalId)` dedup key is only idempotent
// while the id is STABLE across client launches. An id that rotates
// per process (an object description carrying a memory address)
// never matches its own earlier row, so every re-sync mints another
// workout. Refused per entry — one bad row must not stop the rest of
// the batch landing.
const unstable =
w.externalId === undefined ? null : classifyExternalId(w.externalId);
if (unstable) {
unstableShapes.push(unstable);
results[index] = {
index,
status: "skipped",
reason: UNSTABLE_EXTERNAL_ID_REASON,
};
return [];
}
const durationSec = Math.max(
0,
Math.round((w.endedAt.getTime() - w.startedAt.getTime()) / 1000),
@@ -319,17 +344,25 @@ async function postBatch(request: NextRequest): Promise<Response> {
? (w.metadata as Prisma.InputJsonValue)
: Prisma.JsonNull,
};
return {
index,
row,
route,
samples,
dedupKey: w.externalId
? { source: w.source, externalId: w.externalId }
: null,
};
return [
{
index,
row,
route,
samples,
dedupKey: w.externalId
? { source: w.source, externalId: w.externalId }
: null,
},
];
});
if (unstableShapes.length > 0) {
annotate({
meta: unstableExternalIdMeta("workout.batch", unstableShapes),
});
}
// v1.4.42 W5 — write-time cross-source dedup. The v1.5 iOS app
// drains a HealthKit observer queue that frequently carries the
// SAME logical workout from two paired sensors (Apple Watch +