mirror of
https://github.com/MBombeck/HealthLog.git
synced 2026-11-18 16:12:11 +00:00
fix(ingest): hold the stability floor for identities no request schema sees
The dedup key on the medication intake surfaces is not only `externalId`. `MedicationIntakeEvent.idempotency_key` is a `@unique` column persisted with the dose row forever, with no TTL and no sweep, matched by plain equality on the per-medication POST, the bulk drain and the external ingest bridge. That makes it an identity, not a replay token, and it carries the same failure mode: a key that rotates per client launch never matches its own earlier post, so every retry logs the dose again. The blank case is sharper still — the column is unique table-wide, so an empty key resolves to whichever unrelated row stored it first. Refuse both shapes at the schema on the single-entry paths and per entry on the bulk drain. The MCP write path hashes a host-supplied key into the `externalId` half of its dedup key, so the shape is laundered before anything downstream could see it and the check has to run on the input. That closes a live one: every tool handler falls back to `""` when the host omits the argument, and `""` digests to a constant, so each keyless write after the first reported `already_logged` and was silently dropped. The HTTP `Idempotency-Key` header stays out, and now says why in the guard: `IdempotencyKey.expiresAt` bounds it and the cleanup job sweeps it, so it is a windowed replay token and rotation is its design. Extend the structural guard with a provenance rule for identities that never pass a Zod request schema — a spreadsheet column, an MCP tool argument. Inside the import and MCP trees, a module handling an identity must apply the floor itself or import a module that does. That is a real chain rather than a list of known offenders: a new importer that mints an `externalId` from file input and imports nothing checked fails it, with no allowlist to escape into.
This commit is contained in:
1 parent
6e4bcffadf
commit
15b5f6f92f
19 files changed
+1574
-43
No files matched your search
@@ -202,3 +202,59 @@ describe("parseCsvMeasurements — optional columns", () => {
|
||||
expect(out.rows[0].row?.externalId).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
describe("parseCsvMeasurements — external-id stability floor", () => {
|
||||
/**
|
||||
* The CSV route upserts on `(userId, type, source=IMPORT, externalId)`,
|
||||
* so an id that cannot be stable between two exports re-imports as a
|
||||
* fresh row every time instead of converging on the one it already
|
||||
* wrote. This importer has no Zod field to hang a refine on — its
|
||||
* contract is a per-row `reason`, so the refusal lands there and every
|
||||
* other row still parses.
|
||||
*/
|
||||
it("skips a row whose externalId is an object description", () => {
|
||||
const out = parse([
|
||||
"WEIGHT,80,kg,2026-05-01T08:00:00Z,,,<HKQuantitySample: 0x12568db80>",
|
||||
]);
|
||||
expect(out.rows[0]).toMatchObject({
|
||||
status: "skipped",
|
||||
reason: "unstable_external_id",
|
||||
});
|
||||
expect(out.rows[0].row).toBeUndefined();
|
||||
});
|
||||
|
||||
it("skips a row whose externalId is a bare memory address", () => {
|
||||
expect(
|
||||
parse(["WEIGHT,80,kg,2026-05-01T08:00:00Z,,,0x126b25160"]).rows[0],
|
||||
).toMatchObject({ status: "skipped", reason: "unstable_external_id" });
|
||||
});
|
||||
|
||||
it("skips only the offending row — the rest of the file still imports", () => {
|
||||
const out = parse([
|
||||
"WEIGHT,80,kg,2026-05-01T08:00:00Z,,,scale-row-42",
|
||||
"WEIGHT,81,kg,2026-05-02T08:00:00Z,,,<HKQuantitySample: 0x12568db80>",
|
||||
"WEIGHT,82,kg,2026-05-03T08:00:00Z,,,scale-row-43",
|
||||
]);
|
||||
expect(out.rows.map((r) => r.status)).toEqual(["ok", "skipped", "ok"]);
|
||||
expect(out.rows[1].reason).toBe("unstable_external_id");
|
||||
expect(out.rows[0].row?.externalId).toBe("scale-row-42");
|
||||
expect(out.rows[2].row?.externalId).toBe("scale-row-43");
|
||||
expect(out.fatal).toBeUndefined();
|
||||
});
|
||||
|
||||
it("accepts the spreadsheet / meter-export id shapes a real file carries", () => {
|
||||
const out = parse([
|
||||
"WEIGHT,80,kg,2026-05-01T08:00:00Z,,,scale-row-42",
|
||||
"WEIGHT,81,kg,2026-05-02T08:00:00Z,,,8AD2A9CB-3F0C-4E4D-9C1E-4B7E2A1D6F30",
|
||||
"WEIGHT,82,kg,2026-05-03T08:00:00Z,,,stats:HKQuantityTypeIdentifierBodyMass:2026-05-03",
|
||||
"WEIGHT,83,kg,2026-05-04T08:00:00Z,,,Morning Weigh-In 4",
|
||||
]);
|
||||
expect(out.rows.every((r) => r.status === "ok")).toBe(true);
|
||||
});
|
||||
|
||||
it("leaves a row with no externalId alone — an absent id is not a blank one", () => {
|
||||
const out = parse(["WEIGHT,80,kg,2026-05-01T08:00:00Z,,,"]);
|
||||
expect(out.rows[0].status).toBe("ok");
|
||||
expect(out.rows[0].row?.externalId).toBeUndefined();
|
||||
});
|
||||
});
|
||||
@@ -389,3 +389,79 @@ describe("logMcpBloodPressure", () => {
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe("MCP writes — idempotency-key stability floor", () => {
|
||||
/**
|
||||
* The MCP key is hashed into the `externalId` half of
|
||||
* `(userId, type, MCP, externalId)` and matched forever, so it is an
|
||||
* identity. The hash launders the shape — a pointer address and a real
|
||||
* key both digest to clean hex — so the check has to run on the INPUT,
|
||||
* before `mcpExternalId`.
|
||||
*
|
||||
* The blank case is the live one: every tool handler falls back to `""`
|
||||
* when the host omits the argument, and `""` digests to a CONSTANT, so
|
||||
* without this floor every keyless write after the first would report
|
||||
* `already_logged` and be silently dropped.
|
||||
*/
|
||||
it.each([
|
||||
["blank", ""],
|
||||
["whitespace", " "],
|
||||
["pointer address", "0x12568db80"],
|
||||
["object description", "<NSObject: 0x600000c1a2b0>"],
|
||||
])("refuses a %s measurement key before it is hashed", async (_l, key) => {
|
||||
const result = await logMcpMeasurement({
|
||||
userId: "u-1",
|
||||
type: "WEIGHT",
|
||||
value: 80,
|
||||
idempotencyKey: key,
|
||||
});
|
||||
expect(result.status).toBe("unstable_idempotency_key");
|
||||
expect(measurement.findUnique).not.toHaveBeenCalled();
|
||||
expect(measurement.create).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("refuses a blank mood key instead of collapsing onto one row", async () => {
|
||||
const result = await logMcpMood({
|
||||
userId: "u-1",
|
||||
score: 4,
|
||||
idempotencyKey: "",
|
||||
});
|
||||
expect(result.status).toBe("unstable_idempotency_key");
|
||||
expect(moodEntry.create).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("refuses a blank blood-pressure key instead of collapsing onto one row", async () => {
|
||||
const result = await logMcpBloodPressure({
|
||||
userId: "u-1",
|
||||
systolic: 120,
|
||||
diastolic: 80,
|
||||
idempotencyKey: "",
|
||||
});
|
||||
expect(result.status).toBe("unstable_idempotency_key");
|
||||
expect($transaction).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("carries a message naming the field a host has to fix", async () => {
|
||||
const result = await logMcpMeasurement({
|
||||
userId: "u-1",
|
||||
type: "WEIGHT",
|
||||
value: 80,
|
||||
idempotencyKey: "<NSObject: 0x600000c1a2b0>",
|
||||
});
|
||||
expect(result).toMatchObject({ status: "unstable_idempotency_key" });
|
||||
if (result.status !== "unstable_idempotency_key") return;
|
||||
expect(result.reason).toContain("idempotency key");
|
||||
expect(result.reason).toContain("stable across app restarts");
|
||||
});
|
||||
|
||||
it("still accepts a normal host-chosen key", async () => {
|
||||
const result = await logMcpMeasurement({
|
||||
userId: "u-1",
|
||||
type: "WEIGHT",
|
||||
value: 80,
|
||||
idempotencyKey: "weight-2026-07-25-morning",
|
||||
});
|
||||
expect(result.status).toBe("written");
|
||||
expect(measurement.create).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
});
|
||||
@@ -268,6 +268,13 @@ const WRITE_TOOL_DEFINITIONS: McpToolDefinition[] = [
|
||||
idempotencyKey,
|
||||
});
|
||||
|
||||
if (result.status === "unstable_idempotency_key") {
|
||||
return {
|
||||
written: false,
|
||||
error: "unstable_idempotency_key",
|
||||
reason: result.reason,
|
||||
};
|
||||
}
|
||||
if (result.status === "unsupported_type") {
|
||||
return { written: false, error: "unsupported_type" };
|
||||
}
|
||||
@@ -344,6 +351,13 @@ const WRITE_TOOL_DEFINITIONS: McpToolDefinition[] = [
|
||||
idempotencyKey,
|
||||
});
|
||||
|
||||
if (result.status === "unstable_idempotency_key") {
|
||||
return {
|
||||
written: false,
|
||||
error: "unstable_idempotency_key",
|
||||
reason: result.reason,
|
||||
};
|
||||
}
|
||||
if (result.status === "invalid_score") {
|
||||
return { written: false, error: "invalid_score" };
|
||||
}
|
||||
@@ -429,6 +443,13 @@ const WRITE_TOOL_DEFINITIONS: McpToolDefinition[] = [
|
||||
idempotencyKey,
|
||||
});
|
||||
|
||||
if (result.status === "unstable_idempotency_key") {
|
||||
return {
|
||||
written: false,
|
||||
error: "unstable_idempotency_key",
|
||||
reason: result.reason,
|
||||
};
|
||||
}
|
||||
if (result.status === "out_of_range") {
|
||||
return { written: false, error: "out_of_range", reason: result.reason };
|
||||
}
|
||||
|
||||
@@ -41,6 +41,12 @@ import { recomputeMoodBucketsForEntry } from "@/lib/rollups/mood-rollups";
|
||||
import { MOOD_ENUM_BY_SCORE } from "@/lib/mood/labels";
|
||||
import { getScoreForMood } from "@/lib/validations/mood";
|
||||
import { moodDateKey, DEFAULT_TIMEZONE } from "@/lib/mood/date-key";
|
||||
import {
|
||||
classifyExternalId,
|
||||
unstableExternalIdMessage,
|
||||
unstableExternalIdMeta,
|
||||
IDEMPOTENCY_KEY_LABEL,
|
||||
} from "@/lib/validations/external-id";
|
||||
|
||||
/** The MCP write provenance — distinct from MANUAL / TELEGRAM. */
|
||||
const MCP_SOURCE = "MCP" as const;
|
||||
@@ -52,6 +58,41 @@ const MCP_SOURCE = "MCP" as const;
|
||||
* `(userId, type, MCP, externalId)` so a key can never collide with a
|
||||
* Telegram or manual row.
|
||||
*/
|
||||
/**
|
||||
* The MCP write key is an IDENTITY, not a windowed replay token: it is
|
||||
* hashed into the `externalId` half of `(userId, type, MCP, externalId)`
|
||||
* and matched forever. A key that changes between the preview call and
|
||||
* the commit call — or between two commits of the same logical write —
|
||||
* mints a second row instead of converging.
|
||||
*
|
||||
* The hash launders the shape, so the floor has to run on the INPUT: a
|
||||
* pointer-shaped key and a stable one both digest to clean hex, and the
|
||||
* `externalId` guard downstream could never tell them apart. The blank
|
||||
* case is the live one — every tool handler falls back to `""` when the
|
||||
* host omits the argument, and `""` digests to a CONSTANT, so every
|
||||
* keyless write after the first would report `already_logged` and be
|
||||
* silently dropped.
|
||||
*/
|
||||
function checkMcpIdempotencyKey(
|
||||
tool: string,
|
||||
idempotencyKey: string,
|
||||
): { status: "unstable_idempotency_key"; reason: string } | null {
|
||||
const shape = classifyExternalId(idempotencyKey);
|
||||
if (shape === null) return null;
|
||||
annotate({
|
||||
action: { name: "mcp.tool.write" },
|
||||
meta: {
|
||||
tool,
|
||||
status: "unstable_idempotency_key",
|
||||
...unstableExternalIdMeta(`mcp.${tool}`, [shape]),
|
||||
},
|
||||
});
|
||||
return {
|
||||
status: "unstable_idempotency_key",
|
||||
reason: unstableExternalIdMessage(shape, IDEMPOTENCY_KEY_LABEL),
|
||||
};
|
||||
}
|
||||
|
||||
function mcpExternalId(
|
||||
prefix: "measure" | "mood" | "bp",
|
||||
idempotencyKey: string,
|
||||
@@ -186,6 +227,7 @@ export type McpMeasurementResult =
|
||||
| { status: "unsupported_type" }
|
||||
| { status: "unsupported_unit" }
|
||||
| { status: "out_of_range"; reason: string }
|
||||
| { status: "unstable_idempotency_key"; reason: string }
|
||||
| { status: "written"; measurement: McpMeasurementRecord }
|
||||
| { status: "already_logged"; measurement: McpMeasurementRecord };
|
||||
|
||||
@@ -205,6 +247,12 @@ export async function logMcpMeasurement(input: {
|
||||
measuredAt?: Date;
|
||||
idempotencyKey: string;
|
||||
}): Promise<McpMeasurementResult> {
|
||||
const unstableKey = checkMcpIdempotencyKey(
|
||||
"log_measurement",
|
||||
input.idempotencyKey,
|
||||
);
|
||||
if (unstableKey) return unstableKey;
|
||||
|
||||
// Normalise the caller's unit hint to canonical (converting the value) or
|
||||
// refuse it — a non-canonical unit must never be persisted verbatim. Runs
|
||||
// BEFORE the range check so the plausibility gate sees the canonical value
|
||||
@@ -337,6 +385,7 @@ export interface McpBloodPressureRecord {
|
||||
|
||||
export type McpBloodPressureResult =
|
||||
| { status: "out_of_range"; reason: string }
|
||||
| { status: "unstable_idempotency_key"; reason: string }
|
||||
| { status: "written"; bloodPressure: McpBloodPressureRecord }
|
||||
| { status: "already_logged"; bloodPressure: McpBloodPressureRecord };
|
||||
|
||||
@@ -359,6 +408,12 @@ export async function logMcpBloodPressure(input: {
|
||||
measuredAt?: Date;
|
||||
idempotencyKey: string;
|
||||
}): Promise<McpBloodPressureResult> {
|
||||
const unstableKey = checkMcpIdempotencyKey(
|
||||
"log_blood_pressure",
|
||||
input.idempotencyKey,
|
||||
);
|
||||
if (unstableKey) return unstableKey;
|
||||
|
||||
// Same validation the confirm-gate preview runs (both ranges, the systolic >
|
||||
// diastolic guard, and the measuredAt instant bound) — one verdict for both.
|
||||
const check = checkMcpBloodPressure(
|
||||
@@ -514,6 +569,7 @@ export interface McpMoodRecord {
|
||||
|
||||
export type McpMoodResult =
|
||||
| { status: "invalid_score" }
|
||||
| { status: "unstable_idempotency_key"; reason: string }
|
||||
| { status: "written"; moodEntry: McpMoodRecord }
|
||||
| { status: "already_logged"; moodEntry: McpMoodRecord };
|
||||
|
||||
@@ -531,6 +587,9 @@ export async function logMcpMood(input: {
|
||||
tz?: string | null;
|
||||
idempotencyKey: string;
|
||||
}): Promise<McpMoodResult> {
|
||||
const unstableKey = checkMcpIdempotencyKey("log_mood", input.idempotencyKey);
|
||||
if (unstableKey) return unstableKey;
|
||||
|
||||
const mood = MOOD_ENUM_BY_SCORE[input.score];
|
||||
if (!mood) {
|
||||
annotate({
|
||||
|
||||
@@ -96,19 +96,43 @@ export function isStableExternalId(value: string): boolean {
|
||||
return classifyExternalId(value) === null;
|
||||
}
|
||||
|
||||
const SHAPE_MESSAGES: Record<UnstableExternalIdShape, string> = {
|
||||
blank: "external identifier must not be empty",
|
||||
pointer_address:
|
||||
"external identifier must be stable across app restarts; received a memory address (0x…), which is a new value on every launch and mints a duplicate record on every sync",
|
||||
object_description:
|
||||
"external identifier must be stable across app restarts; received an object description (<Class: 0x…>), which embeds a memory address and is a new value on every launch — serialise the identity itself, not the object's description",
|
||||
};
|
||||
/**
|
||||
* How the offending field is named back to the caller. The rule is about
|
||||
* the VALUE, so it applies to every client-supplied string this system
|
||||
* later matches on as an identity — not only `externalId`.
|
||||
*/
|
||||
export const EXTERNAL_ID_LABEL = "external identifier";
|
||||
export const IDEMPOTENCY_KEY_LABEL = "idempotency key";
|
||||
|
||||
/** The 422 message a client developer reads. Names the shape received. */
|
||||
export function unstableExternalIdMessage(
|
||||
shape: UnstableExternalIdShape,
|
||||
label: string = EXTERNAL_ID_LABEL,
|
||||
): string {
|
||||
return SHAPE_MESSAGES[shape];
|
||||
switch (shape) {
|
||||
case "blank":
|
||||
return `${label} must not be empty`;
|
||||
case "pointer_address":
|
||||
return `${label} must be stable across app restarts; received a memory address (0x…), which is a new value on every launch and mints a duplicate record on every sync`;
|
||||
case "object_description":
|
||||
return `${label} must be stable across app restarts; received an object description (<Class: 0x…>), which embeds a memory address and is a new value on every launch — serialise the identity itself, not the object's description`;
|
||||
}
|
||||
}
|
||||
|
||||
function assertStableIdentityField(
|
||||
raw: unknown,
|
||||
ctx: z.RefinementCtx,
|
||||
field: string,
|
||||
label: string,
|
||||
): void {
|
||||
if (typeof raw !== "string") return;
|
||||
const shape = classifyExternalId(raw);
|
||||
if (shape === null) return;
|
||||
ctx.addIssue({
|
||||
code: "custom",
|
||||
path: [field],
|
||||
message: unstableExternalIdMessage(shape, label),
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -127,15 +151,42 @@ export function assertStableExternalId(
|
||||
value: { externalId?: string | null },
|
||||
ctx: z.RefinementCtx,
|
||||
): void {
|
||||
const raw = value.externalId;
|
||||
if (typeof raw !== "string") return;
|
||||
const shape = classifyExternalId(raw);
|
||||
if (shape === null) return;
|
||||
ctx.addIssue({
|
||||
code: "custom",
|
||||
path: ["externalId"],
|
||||
message: unstableExternalIdMessage(shape),
|
||||
});
|
||||
assertStableIdentityField(
|
||||
value.externalId,
|
||||
ctx,
|
||||
"externalId",
|
||||
EXTERNAL_ID_LABEL,
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* The same floor for a client-supplied `idempotencyKey`.
|
||||
*
|
||||
* The HTTP `Idempotency-Key` header is a replay token inside a TTL and is
|
||||
* deliberately out of scope — `IdempotencyKey.expiresAt` bounds it and a
|
||||
* cron sweeps it. `MedicationIntakeEvent.idempotency_key` is NOT that: it
|
||||
* is a `@unique` column persisted with the dose row forever, with no TTL
|
||||
* and no sweep, matched by plain equality on the per-medication intake
|
||||
* POST, the bulk intake ingest and the external ingest surface. That makes
|
||||
* it an identity, and it carries the same failure mode — a value that
|
||||
* rotates per client launch never matches its own earlier row, so every
|
||||
* re-sync logs the dose again.
|
||||
*
|
||||
* The blank arm matters more here than anywhere else: the column is
|
||||
* `@unique` across the table, so a second row carrying "" cannot exist,
|
||||
* and an empty key would otherwise resolve to whichever unrelated row got
|
||||
* there first.
|
||||
*/
|
||||
export function assertStableIdempotencyKey(
|
||||
value: { idempotencyKey?: string | null },
|
||||
ctx: z.RefinementCtx,
|
||||
): void {
|
||||
assertStableIdentityField(
|
||||
value.idempotencyKey,
|
||||
ctx,
|
||||
"idempotencyKey",
|
||||
IDEMPOTENCY_KEY_LABEL,
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -146,9 +197,10 @@ export function assertStableExternalId(
|
||||
*/
|
||||
export function unstableExternalIdShape(
|
||||
body: unknown,
|
||||
field: "externalId" | "idempotencyKey" = "externalId",
|
||||
): UnstableExternalIdShape | null {
|
||||
if (typeof body !== "object" || body === null) return null;
|
||||
const raw = (body as { externalId?: unknown }).externalId;
|
||||
const raw = (body as Record<string, unknown>)[field];
|
||||
if (typeof raw !== "string") return null;
|
||||
return classifyExternalId(raw);
|
||||
}
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { z } from "zod/v4";
|
||||
|
||||
import { boundedTakenAtSchema, injectionSiteEnum } from "./base";
|
||||
import { assertStableIdempotencyKey } from "@/lib/validations/external-id";
|
||||
|
||||
export const intakeSchema = z
|
||||
.object({
|
||||
@@ -82,18 +83,29 @@ export const intakeSchema = z
|
||||
"Dose actually consumed for THIS intake when it deviates from (or documents) the medication's configured dose, e.g. a half tablet or a titration step. Free text, max 50 characters. Omit to record the take under the medication's configured dose.",
|
||||
),
|
||||
})
|
||||
// `MedicationIntakeEvent.idempotency_key` is a `@unique` column persisted
|
||||
// with the dose row forever and matched by equality — an identity, not a
|
||||
// windowed replay token. A key that rotates per client launch never
|
||||
// matches its own earlier row, so every re-sync logs the dose again.
|
||||
.superRefine(assertStableIdempotencyKey)
|
||||
.meta({
|
||||
id: "MedicationIntakeRequest",
|
||||
description:
|
||||
"Per-medication intake log body. Idempotent via `idempotencyKey`; the server also dedupes by a 60-second sliding window when the key is absent. Non-skipped intakes auto-decrement pen inventory (best-effort), refresh the per-day compliance rollup, and — for one-shot medications — flip `active` to false. The optional `injectionSite` is persisted only for an INJECTION medication with site-tracking enabled and is validated against the medication's effective allowed set (422 on a disallowed value).",
|
||||
});
|
||||
|
||||
export const externalIntakeSchema = z.object({
|
||||
medicationName: z.string().min(1).max(200),
|
||||
// v1.16.9 — same plausibility bounds as the interactive create paths.
|
||||
takenAt: boundedTakenAtSchema.optional(),
|
||||
idempotencyKey: z.string().max(128),
|
||||
});
|
||||
export const externalIntakeSchema = z
|
||||
.object({
|
||||
medicationName: z.string().min(1).max(200),
|
||||
// v1.16.9 — same plausibility bounds as the interactive create paths.
|
||||
takenAt: boundedTakenAtSchema.optional(),
|
||||
idempotencyKey: z.string().max(128),
|
||||
})
|
||||
// Same identity floor as the interactive twin. The blank arm carries real
|
||||
// weight on this surface: the column is `@unique` table-wide, so an empty
|
||||
// key resolves to whichever unrelated row happened to store "" first
|
||||
// instead of to this caller's own earlier post.
|
||||
.superRefine(assertStableIdempotencyKey);
|
||||
|
||||
export const listIntakeEventsSchema = z.object({
|
||||
limit: z.coerce.number().int().min(1).max(500).optional().default(25),
|
||||
|
||||
Reference in new issue
Block a user