fix(ingest): hold the stability floor for identities no request schema sees

The dedup key on the medication intake surfaces is not only `externalId`.
`MedicationIntakeEvent.idempotency_key` is a `@unique` column persisted
with the dose row forever, with no TTL and no sweep, matched by plain
equality on the per-medication POST, the bulk drain and the external
ingest bridge. That makes it an identity, not a replay token, and it
carries the same failure mode: a key that rotates per client launch never
matches its own earlier post, so every retry logs the dose again. The
blank case is sharper still — the column is unique table-wide, so an
empty key resolves to whichever unrelated row stored it first. Refuse
both shapes at the schema on the single-entry paths and per entry on the
bulk drain.

The MCP write path hashes a host-supplied key into the `externalId` half
of its dedup key, so the shape is laundered before anything downstream
could see it and the check has to run on the input. That closes a live
one: every tool handler falls back to `""` when the host omits the
argument, and `""` digests to a constant, so each keyless write after the
first reported `already_logged` and was silently dropped.

The HTTP `Idempotency-Key` header stays out, and now says why in the
guard: `IdempotencyKey.expiresAt` bounds it and the cleanup job sweeps
it, so it is a windowed replay token and rotation is its design.

Extend the structural guard with a provenance rule for identities that
never pass a Zod request schema — a spreadsheet column, an MCP tool
argument. Inside the import and MCP trees, a module handling an identity
must apply the floor itself or import a module that does. That is a real
chain rather than a list of known offenders: a new importer that mints an
`externalId` from file input and imports nothing checked fails it, with
no allowlist to escape into.
This commit is contained in:
Marc-André Bombeck committed 2026-07-25 18:07:13 +02:00
1 parent 6e4bcffadf
commit 15b5f6f92f
19 files changed
+1574 -43

No files matched your search

@@ -202,3 +202,59 @@ describe("parseCsvMeasurements — optional columns", () => {
expect(out.rows[0].row?.externalId).toBeUndefined();
});
});
describe("parseCsvMeasurements — external-id stability floor", () => {
/**
* The CSV route upserts on `(userId, type, source=IMPORT, externalId)`,
* so an id that cannot be stable between two exports re-imports as a
* fresh row every time instead of converging on the one it already
* wrote. This importer has no Zod field to hang a refine on — its
* contract is a per-row `reason`, so the refusal lands there and every
* other row still parses.
*/
it("skips a row whose externalId is an object description", () => {
const out = parse([
"WEIGHT,80,kg,2026-05-01T08:00:00Z,,,<HKQuantitySample: 0x12568db80>",
]);
expect(out.rows[0]).toMatchObject({
status: "skipped",
reason: "unstable_external_id",
});
expect(out.rows[0].row).toBeUndefined();
});
it("skips a row whose externalId is a bare memory address", () => {
expect(
parse(["WEIGHT,80,kg,2026-05-01T08:00:00Z,,,0x126b25160"]).rows[0],
).toMatchObject({ status: "skipped", reason: "unstable_external_id" });
});
it("skips only the offending row — the rest of the file still imports", () => {
const out = parse([
"WEIGHT,80,kg,2026-05-01T08:00:00Z,,,scale-row-42",
"WEIGHT,81,kg,2026-05-02T08:00:00Z,,,<HKQuantitySample: 0x12568db80>",
"WEIGHT,82,kg,2026-05-03T08:00:00Z,,,scale-row-43",
]);
expect(out.rows.map((r) => r.status)).toEqual(["ok", "skipped", "ok"]);
expect(out.rows[1].reason).toBe("unstable_external_id");
expect(out.rows[0].row?.externalId).toBe("scale-row-42");
expect(out.rows[2].row?.externalId).toBe("scale-row-43");
expect(out.fatal).toBeUndefined();
});
it("accepts the spreadsheet / meter-export id shapes a real file carries", () => {
const out = parse([
"WEIGHT,80,kg,2026-05-01T08:00:00Z,,,scale-row-42",
"WEIGHT,81,kg,2026-05-02T08:00:00Z,,,8AD2A9CB-3F0C-4E4D-9C1E-4B7E2A1D6F30",
"WEIGHT,82,kg,2026-05-03T08:00:00Z,,,stats:HKQuantityTypeIdentifierBodyMass:2026-05-03",
"WEIGHT,83,kg,2026-05-04T08:00:00Z,,,Morning Weigh-In 4",
]);
expect(out.rows.every((r) => r.status === "ok")).toBe(true);
});
it("leaves a row with no externalId alone — an absent id is not a blank one", () => {
const out = parse(["WEIGHT,80,kg,2026-05-01T08:00:00Z,,,"]);
expect(out.rows[0].status).toBe("ok");
expect(out.rows[0].row?.externalId).toBeUndefined();
});
});
+76
View File
@@ -389,3 +389,79 @@ describe("logMcpBloodPressure", () => {
}
});
});
describe("MCP writes — idempotency-key stability floor", () => {
/**
* The MCP key is hashed into the `externalId` half of
* `(userId, type, MCP, externalId)` and matched forever, so it is an
* identity. The hash launders the shape — a pointer address and a real
* key both digest to clean hex — so the check has to run on the INPUT,
* before `mcpExternalId`.
*
* The blank case is the live one: every tool handler falls back to `""`
* when the host omits the argument, and `""` digests to a CONSTANT, so
* without this floor every keyless write after the first would report
* `already_logged` and be silently dropped.
*/
it.each([
["blank", ""],
["whitespace", " "],
["pointer address", "0x12568db80"],
["object description", "<NSObject: 0x600000c1a2b0>"],
])("refuses a %s measurement key before it is hashed", async (_l, key) => {
const result = await logMcpMeasurement({
userId: "u-1",
type: "WEIGHT",
value: 80,
idempotencyKey: key,
});
expect(result.status).toBe("unstable_idempotency_key");
expect(measurement.findUnique).not.toHaveBeenCalled();
expect(measurement.create).not.toHaveBeenCalled();
});
it("refuses a blank mood key instead of collapsing onto one row", async () => {
const result = await logMcpMood({
userId: "u-1",
score: 4,
idempotencyKey: "",
});
expect(result.status).toBe("unstable_idempotency_key");
expect(moodEntry.create).not.toHaveBeenCalled();
});
it("refuses a blank blood-pressure key instead of collapsing onto one row", async () => {
const result = await logMcpBloodPressure({
userId: "u-1",
systolic: 120,
diastolic: 80,
idempotencyKey: "",
});
expect(result.status).toBe("unstable_idempotency_key");
expect($transaction).not.toHaveBeenCalled();
});
it("carries a message naming the field a host has to fix", async () => {
const result = await logMcpMeasurement({
userId: "u-1",
type: "WEIGHT",
value: 80,
idempotencyKey: "<NSObject: 0x600000c1a2b0>",
});
expect(result).toMatchObject({ status: "unstable_idempotency_key" });
if (result.status !== "unstable_idempotency_key") return;
expect(result.reason).toContain("idempotency key");
expect(result.reason).toContain("stable across app restarts");
});
it("still accepts a normal host-chosen key", async () => {
const result = await logMcpMeasurement({
userId: "u-1",
type: "WEIGHT",
value: 80,
idempotencyKey: "weight-2026-07-25-morning",
});
expect(result.status).toBe("written");
expect(measurement.create).toHaveBeenCalledTimes(1);
});
});
+21
View File
@@ -268,6 +268,13 @@ const WRITE_TOOL_DEFINITIONS: McpToolDefinition[] = [
idempotencyKey,
});
if (result.status === "unstable_idempotency_key") {
return {
written: false,
error: "unstable_idempotency_key",
reason: result.reason,
};
}
if (result.status === "unsupported_type") {
return { written: false, error: "unsupported_type" };
}
@@ -344,6 +351,13 @@ const WRITE_TOOL_DEFINITIONS: McpToolDefinition[] = [
idempotencyKey,
});
if (result.status === "unstable_idempotency_key") {
return {
written: false,
error: "unstable_idempotency_key",
reason: result.reason,
};
}
if (result.status === "invalid_score") {
return { written: false, error: "invalid_score" };
}
@@ -429,6 +443,13 @@ const WRITE_TOOL_DEFINITIONS: McpToolDefinition[] = [
idempotencyKey,
});
if (result.status === "unstable_idempotency_key") {
return {
written: false,
error: "unstable_idempotency_key",
reason: result.reason,
};
}
if (result.status === "out_of_range") {
return { written: false, error: "out_of_range", reason: result.reason };
}
+59
View File
@@ -41,6 +41,12 @@ import { recomputeMoodBucketsForEntry } from "@/lib/rollups/mood-rollups";
import { MOOD_ENUM_BY_SCORE } from "@/lib/mood/labels";
import { getScoreForMood } from "@/lib/validations/mood";
import { moodDateKey, DEFAULT_TIMEZONE } from "@/lib/mood/date-key";
import {
classifyExternalId,
unstableExternalIdMessage,
unstableExternalIdMeta,
IDEMPOTENCY_KEY_LABEL,
} from "@/lib/validations/external-id";
/** The MCP write provenance — distinct from MANUAL / TELEGRAM. */
const MCP_SOURCE = "MCP" as const;
@@ -52,6 +58,41 @@ const MCP_SOURCE = "MCP" as const;
* `(userId, type, MCP, externalId)` so a key can never collide with a
* Telegram or manual row.
*/
/**
* The MCP write key is an IDENTITY, not a windowed replay token: it is
* hashed into the `externalId` half of `(userId, type, MCP, externalId)`
* and matched forever. A key that changes between the preview call and
* the commit call — or between two commits of the same logical write —
* mints a second row instead of converging.
*
* The hash launders the shape, so the floor has to run on the INPUT: a
* pointer-shaped key and a stable one both digest to clean hex, and the
* `externalId` guard downstream could never tell them apart. The blank
* case is the live one — every tool handler falls back to `""` when the
* host omits the argument, and `""` digests to a CONSTANT, so every
* keyless write after the first would report `already_logged` and be
* silently dropped.
*/
function checkMcpIdempotencyKey(
tool: string,
idempotencyKey: string,
): { status: "unstable_idempotency_key"; reason: string } | null {
const shape = classifyExternalId(idempotencyKey);
if (shape === null) return null;
annotate({
action: { name: "mcp.tool.write" },
meta: {
tool,
status: "unstable_idempotency_key",
...unstableExternalIdMeta(`mcp.${tool}`, [shape]),
},
});
return {
status: "unstable_idempotency_key",
reason: unstableExternalIdMessage(shape, IDEMPOTENCY_KEY_LABEL),
};
}
function mcpExternalId(
prefix: "measure" | "mood" | "bp",
idempotencyKey: string,
@@ -186,6 +227,7 @@ export type McpMeasurementResult =
| { status: "unsupported_type" }
| { status: "unsupported_unit" }
| { status: "out_of_range"; reason: string }
| { status: "unstable_idempotency_key"; reason: string }
| { status: "written"; measurement: McpMeasurementRecord }
| { status: "already_logged"; measurement: McpMeasurementRecord };
@@ -205,6 +247,12 @@ export async function logMcpMeasurement(input: {
measuredAt?: Date;
idempotencyKey: string;
}): Promise<McpMeasurementResult> {
const unstableKey = checkMcpIdempotencyKey(
"log_measurement",
input.idempotencyKey,
);
if (unstableKey) return unstableKey;
// Normalise the caller's unit hint to canonical (converting the value) or
// refuse it — a non-canonical unit must never be persisted verbatim. Runs
// BEFORE the range check so the plausibility gate sees the canonical value
@@ -337,6 +385,7 @@ export interface McpBloodPressureRecord {
export type McpBloodPressureResult =
| { status: "out_of_range"; reason: string }
| { status: "unstable_idempotency_key"; reason: string }
| { status: "written"; bloodPressure: McpBloodPressureRecord }
| { status: "already_logged"; bloodPressure: McpBloodPressureRecord };
@@ -359,6 +408,12 @@ export async function logMcpBloodPressure(input: {
measuredAt?: Date;
idempotencyKey: string;
}): Promise<McpBloodPressureResult> {
const unstableKey = checkMcpIdempotencyKey(
"log_blood_pressure",
input.idempotencyKey,
);
if (unstableKey) return unstableKey;
// Same validation the confirm-gate preview runs (both ranges, the systolic >
// diastolic guard, and the measuredAt instant bound) — one verdict for both.
const check = checkMcpBloodPressure(
@@ -514,6 +569,7 @@ export interface McpMoodRecord {
export type McpMoodResult =
| { status: "invalid_score" }
| { status: "unstable_idempotency_key"; reason: string }
| { status: "written"; moodEntry: McpMoodRecord }
| { status: "already_logged"; moodEntry: McpMoodRecord };
@@ -531,6 +587,9 @@ export async function logMcpMood(input: {
tz?: string | null;
idempotencyKey: string;
}): Promise<McpMoodResult> {
const unstableKey = checkMcpIdempotencyKey("log_mood", input.idempotencyKey);
if (unstableKey) return unstableKey;
const mood = MOOD_ENUM_BY_SCORE[input.score];
if (!mood) {
annotate({
+70 -18
View File
@@ -96,19 +96,43 @@ export function isStableExternalId(value: string): boolean {
return classifyExternalId(value) === null;
}
const SHAPE_MESSAGES: Record<UnstableExternalIdShape, string> = {
blank: "external identifier must not be empty",
pointer_address:
"external identifier must be stable across app restarts; received a memory address (0x…), which is a new value on every launch and mints a duplicate record on every sync",
object_description:
"external identifier must be stable across app restarts; received an object description (<Class: 0x…>), which embeds a memory address and is a new value on every launch — serialise the identity itself, not the object's description",
};
/**
* How the offending field is named back to the caller. The rule is about
* the VALUE, so it applies to every client-supplied string this system
* later matches on as an identity — not only `externalId`.
*/
export const EXTERNAL_ID_LABEL = "external identifier";
export const IDEMPOTENCY_KEY_LABEL = "idempotency key";
/** The 422 message a client developer reads. Names the shape received. */
export function unstableExternalIdMessage(
shape: UnstableExternalIdShape,
label: string = EXTERNAL_ID_LABEL,
): string {
return SHAPE_MESSAGES[shape];
switch (shape) {
case "blank":
return `${label} must not be empty`;
case "pointer_address":
return `${label} must be stable across app restarts; received a memory address (0x…), which is a new value on every launch and mints a duplicate record on every sync`;
case "object_description":
return `${label} must be stable across app restarts; received an object description (<Class: 0x…>), which embeds a memory address and is a new value on every launch — serialise the identity itself, not the object's description`;
}
}
function assertStableIdentityField(
raw: unknown,
ctx: z.RefinementCtx,
field: string,
label: string,
): void {
if (typeof raw !== "string") return;
const shape = classifyExternalId(raw);
if (shape === null) return;
ctx.addIssue({
code: "custom",
path: [field],
message: unstableExternalIdMessage(shape, label),
});
}
/**
@@ -127,15 +151,42 @@ export function assertStableExternalId(
value: { externalId?: string | null },
ctx: z.RefinementCtx,
): void {
const raw = value.externalId;
if (typeof raw !== "string") return;
const shape = classifyExternalId(raw);
if (shape === null) return;
ctx.addIssue({
code: "custom",
path: ["externalId"],
message: unstableExternalIdMessage(shape),
});
assertStableIdentityField(
value.externalId,
ctx,
"externalId",
EXTERNAL_ID_LABEL,
);
}
/**
* The same floor for a client-supplied `idempotencyKey`.
*
* The HTTP `Idempotency-Key` header is a replay token inside a TTL and is
* deliberately out of scope — `IdempotencyKey.expiresAt` bounds it and a
* cron sweeps it. `MedicationIntakeEvent.idempotency_key` is NOT that: it
* is a `@unique` column persisted with the dose row forever, with no TTL
* and no sweep, matched by plain equality on the per-medication intake
* POST, the bulk intake ingest and the external ingest surface. That makes
* it an identity, and it carries the same failure mode — a value that
* rotates per client launch never matches its own earlier row, so every
* re-sync logs the dose again.
*
* The blank arm matters more here than anywhere else: the column is
* `@unique` across the table, so a second row carrying "" cannot exist,
* and an empty key would otherwise resolve to whichever unrelated row got
* there first.
*/
export function assertStableIdempotencyKey(
value: { idempotencyKey?: string | null },
ctx: z.RefinementCtx,
): void {
assertStableIdentityField(
value.idempotencyKey,
ctx,
"idempotencyKey",
IDEMPOTENCY_KEY_LABEL,
);
}
/**
@@ -146,9 +197,10 @@ export function assertStableExternalId(
*/
export function unstableExternalIdShape(
body: unknown,
field: "externalId" | "idempotencyKey" = "externalId",
): UnstableExternalIdShape | null {
if (typeof body !== "object" || body === null) return null;
const raw = (body as { externalId?: unknown }).externalId;
const raw = (body as Record<string, unknown>)[field];
if (typeof raw !== "string") return null;
return classifyExternalId(raw);
}
+18 -6
View File
@@ -1,6 +1,7 @@
import { z } from "zod/v4";
import { boundedTakenAtSchema, injectionSiteEnum } from "./base";
import { assertStableIdempotencyKey } from "@/lib/validations/external-id";
export const intakeSchema = z
.object({
@@ -82,18 +83,29 @@ export const intakeSchema = z
"Dose actually consumed for THIS intake when it deviates from (or documents) the medication's configured dose, e.g. a half tablet or a titration step. Free text, max 50 characters. Omit to record the take under the medication's configured dose.",
),
})
// `MedicationIntakeEvent.idempotency_key` is a `@unique` column persisted
// with the dose row forever and matched by equality — an identity, not a
// windowed replay token. A key that rotates per client launch never
// matches its own earlier row, so every re-sync logs the dose again.
.superRefine(assertStableIdempotencyKey)
.meta({
id: "MedicationIntakeRequest",
description:
"Per-medication intake log body. Idempotent via `idempotencyKey`; the server also dedupes by a 60-second sliding window when the key is absent. Non-skipped intakes auto-decrement pen inventory (best-effort), refresh the per-day compliance rollup, and — for one-shot medications — flip `active` to false. The optional `injectionSite` is persisted only for an INJECTION medication with site-tracking enabled and is validated against the medication's effective allowed set (422 on a disallowed value).",
});
export const externalIntakeSchema = z.object({
medicationName: z.string().min(1).max(200),
// v1.16.9 — same plausibility bounds as the interactive create paths.
takenAt: boundedTakenAtSchema.optional(),
idempotencyKey: z.string().max(128),
});
export const externalIntakeSchema = z
.object({
medicationName: z.string().min(1).max(200),
// v1.16.9 — same plausibility bounds as the interactive create paths.
takenAt: boundedTakenAtSchema.optional(),
idempotencyKey: z.string().max(128),
})
// Same identity floor as the interactive twin. The blank arm carries real
// weight on this surface: the column is `@unique` table-wide, so an empty
// key resolves to whichever unrelated row happened to store "" first
// instead of to this caller's own earlier post.
.superRefine(assertStableIdempotencyKey);
export const listIntakeEventsSchema = z.object({
limit: z.coerce.number().int().min(1).max(500).optional().default(25),