mirror of
https://github.com/MBombeck/HealthLog.git
synced 2026-11-12 14:16:12 +00:00
Marc-André Bombeck
5d62236412
Every ingest path in the product can now emit a typed, salience-classified event into pg-boss, consumed by one worker that claims a durable per-day reaction marker and fans out to the reaction surfaces. The recency classifier runs at the seam, before anything is enqueued, so a mass backfill — a multi-year Apple Health export, a provider re-sync, a 30-day catch-up — emits nothing at all. Non-salient writes are annotated and dropped. The worker makes no provider call by construction: it imports no provider module, writes only markers, and enqueues nothing that is not budget-gated downstream. Business refusals return a status rather than throwing, so a refusal is never retried against a ceiling that will not move until the local day rolls over. Seams: sleep (one hook, inherited by all seven transports), workouts (the batch route, the four provider syncs, the export importer), weight and blood pressure (both measurement routes), labs (manual, OCR commit, document fact commit). The provider workout syncs could not previously tell a created row from an updated one, so a re-sync of a recent session would have re-fired on every poll; they now recover the distinction from the row the upsert already returns. Reaction lines are AES-256-GCM at rest, following the InsightNarrative precedent, and the column is registered for key rotation. Markers are pruned after 14 days by a daily maintenance job.
760 lines
30 KiB
TypeScript
760 lines
30 KiB
TypeScript
/**
|
||
* `POST /api/workouts/batch` — typed workout ingest with nested route.
|
||
*
|
||
* Consumers (v1.5):
|
||
* 1. The native iOS app draining its HealthKit observer queue —
|
||
* maps `HKWorkout` + `HKWorkoutRoute` samples into the request
|
||
* shape locked by `createBatchWorkoutSchema` in
|
||
* `src/lib/validations/workout.ts`.
|
||
* 2. The Withings activity sync (W17b, deferred) — server-to-server
|
||
* ingest with `externalId` formatted as
|
||
* `"${withings.id}:${withings.model_id ?? "0"}:${withings.startdate}"`.
|
||
* Withings ships no route geometry, so the nested `route` is
|
||
* always absent for WITHINGS-sourced entries. NOTE: the request
|
||
* schema's `source` is narrowed to the client-writable set
|
||
* (MANUAL / APPLE_HEALTH), so this server path must write its
|
||
* `WITHINGS`-sourced rows directly via `prisma.workout.upsert`
|
||
* (as the WHOOP / Fitbit syncs already do), not through this route.
|
||
*
|
||
* Cross-cutting concerns mirror `/api/measurements/batch` so the iOS
|
||
* sync engine can re-use the same retry / cursor plumbing:
|
||
* - `requireAuth()` — cookie + Bearer (narrow-scope-token safe per
|
||
* v1.4.25 W10 fix-C: when the route declares no scope, any
|
||
* authenticated token passes).
|
||
* - `withIdempotency` — `Idempotency-Key` header replays the cached
|
||
* response on retry (24h window).
|
||
* - `checkRateLimit("workouts:batch:${user.id}", 60, 60s)` — same
|
||
* window as the measurements batch. 60 batches/min × 100
|
||
* workouts/batch = 6 000 workouts/min headroom (orders of
|
||
* magnitude past any healthy iOS sync).
|
||
* - `Content-Length` ceiling at 5 MB before parsing — anything
|
||
* larger returns 413 so the iOS client falls back to one workout
|
||
* per call.
|
||
* - Per-entry status (`inserted | duplicate | skipped`) so the iOS
|
||
* sync cursor can checkpoint accurately.
|
||
*
|
||
* Idempotency contract:
|
||
* - HTTP-level via `Idempotency-Key` (replays cached envelope).
|
||
* - Per-entry via the `@@unique([userId, source, externalId])`
|
||
* composite index. Re-posting the same batch surfaces duplicates
|
||
* rather than failing the call.
|
||
*
|
||
* Race reconciliation (mirrors the measurements batch fix from
|
||
* v1.4.25 W10): under contention, `createMany` with `skipDuplicates`
|
||
* absorbs duplicate-key conflicts but cannot tell us WHICH rows it
|
||
* absorbed. We trust the `createMany.count` return value and downgrade
|
||
* enough per-entry "inserted" statuses to "duplicate" so the envelope
|
||
* sums stay consistent with the aggregate counts. The iOS sync cursor
|
||
* advances past both statuses identically.
|
||
*
|
||
* Prompt-injection surface: `Workout.metadata` is opaque — the Coach
|
||
* pipeline reads typed columns only (`sportType`, `durationSec`,
|
||
* distance, HR). A future change that pulls metadata into the Coach
|
||
* prompt must add deliberate escaping; the ingest path stores it raw.
|
||
*/
|
||
import { NextRequest } from "next/server";
|
||
|
||
import { prisma } from "@/lib/db";
|
||
import { apiHandler, requireAuth } from "@/lib/api-handler";
|
||
import { annotate } from "@/lib/logging/context";
|
||
import { auditLog } from "@/lib/auth/audit";
|
||
import {
|
||
apiError,
|
||
apiSuccess,
|
||
getClientIp,
|
||
safeJson,
|
||
} from "@/lib/api-response";
|
||
import { withIdempotency } from "@/lib/idempotency";
|
||
import { checkRateLimit } from "@/lib/rate-limit";
|
||
import { enqueuePrDetection } from "@/lib/jobs/pr-detection";
|
||
import { invalidateUserMeasurements } from "@/lib/cache/invalidate";
|
||
import { emitDataArrival } from "@/lib/arrivals/emit-shared";
|
||
import {
|
||
createBatchWorkoutSchema,
|
||
MAX_WORKOUTS_PER_BATCH,
|
||
} from "@/lib/validations/workout";
|
||
import { dedupeWorkoutBatch } from "@/lib/workouts/canonical-rows";
|
||
import { Prisma, type MeasurementSource } from "@/generated/prisma/client";
|
||
|
||
// v1.4.25 W16c — push-suppression threshold for workout PRs. A batch
|
||
// larger than this fires the detection job with `silent: true` so a
|
||
// multi-year HKWorkout backfill doesn't spam the user with hundreds of
|
||
// pushes. Matches the measurements-batch threshold so behaviour stays
|
||
// uniform across both ingest paths.
|
||
const PR_DETECTION_SILENT_THRESHOLD = 50;
|
||
|
||
// v1.4.25 W10 reconcile (security H-2 parity): cap batch ingest at 60
|
||
// batches per user per minute. Healthy iOS sync drains its HealthKit
|
||
// observer queue in well under one batch per minute (the observer
|
||
// pattern coalesces); 60/min × 100 workouts/batch = 6 000 workouts/
|
||
// min headroom — generous for legitimate use, and a hard stop for a
|
||
// leaked wildcard token trying to saturate the write pipeline.
|
||
const BATCH_RATE_LIMIT_MAX = 60;
|
||
const BATCH_RATE_LIMIT_WINDOW_MS = 60 * 1000;
|
||
|
||
// 5 MB request-body ceiling (pre-existing). The route geometry is the
|
||
// largest tail in a workout payload; a typical batch sits comfortably
|
||
// under this, but the 20 000-point LineString cap combined with the
|
||
// 100-workout batch cap allows a theoretical tens-of-MB body for an
|
||
// extreme GPS-heavy backlog. Such a body returns 413 so the iOS client
|
||
// falls back to smaller / one-workout-per-call batches — which is also
|
||
// the correct response to a misbehaving client or an attempted DoS.
|
||
const MAX_BODY_BYTES = 5 * 1024 * 1024;
|
||
|
||
/**
|
||
* Per-entry outcome the iOS client uses to advance its sync cursor.
|
||
* `inserted` and `duplicate` both indicate the row landed (or was
|
||
* already present); the client can checkpoint past them identically.
|
||
* `skipped` covers entries the server cannot durably store — currently
|
||
* unused by this endpoint (the Zod schema rejects malformed entries
|
||
* with a 422 before we reach the per-entry pass), but reserved on the
|
||
* envelope so the response shape parallels the measurements batch and
|
||
* future server-side validation can surface fine-grained skips without
|
||
* breaking the iOS DTO.
|
||
*/
|
||
type EntryStatus = "inserted" | "duplicate" | "skipped";
|
||
interface EntryResult {
|
||
index: number;
|
||
status: EntryStatus;
|
||
reason?: string;
|
||
}
|
||
|
||
/**
|
||
* v1.31.0 — emit one data arrival per workout this batch actually inserted.
|
||
*
|
||
* `createMany` does not return ids, so they are recovered with a single indexed
|
||
* lookup on the `(userId, source, externalId)` dedup key. Entries without an
|
||
* external id cannot be looked up and are emitted without a `refId`; the spine
|
||
* then falls back to a day-scoped key for them, which is an honest degradation
|
||
* (no external id means no stable per-workout referent to react to anyway).
|
||
*
|
||
* Fully best-effort — every failure path returns quietly. A reaction is never
|
||
* worth failing an ingest that already succeeded.
|
||
*/
|
||
async function emitWorkoutArrivals(
|
||
userId: string,
|
||
prepared: ReadonlyArray<{
|
||
index: number;
|
||
row: Prisma.WorkoutCreateManyInput;
|
||
dedupKey: { source: string; externalId: string } | null;
|
||
}>,
|
||
results: ReadonlyArray<EntryResult | undefined>,
|
||
): Promise<void> {
|
||
const inserted = prepared.filter(
|
||
(p) => results[p.index]?.status === "inserted",
|
||
);
|
||
if (inserted.length === 0) return;
|
||
|
||
const externalIds = inserted
|
||
.map((p) => p.dedupKey?.externalId)
|
||
.filter((id): id is string => typeof id === "string");
|
||
|
||
const idByExternalId = new Map<string, string>();
|
||
if (externalIds.length > 0) {
|
||
const rows = await prisma.workout.findMany({
|
||
where: { userId, externalId: { in: externalIds } },
|
||
select: { id: true, externalId: true },
|
||
});
|
||
for (const row of rows) {
|
||
if (row.externalId) idByExternalId.set(row.externalId, row.id);
|
||
}
|
||
}
|
||
|
||
for (const p of inserted) {
|
||
const startedAt =
|
||
p.row.startedAt instanceof Date
|
||
? p.row.startedAt
|
||
: new Date(p.row.startedAt as string);
|
||
if (Number.isNaN(startedAt.getTime())) continue;
|
||
const externalId = p.dedupKey?.externalId;
|
||
await emitDataArrival({
|
||
userId,
|
||
kind: "workout",
|
||
newestSampleAt: startedAt,
|
||
insertedCount: 1,
|
||
refId: externalId ? idByExternalId.get(externalId) : undefined,
|
||
source: "batch",
|
||
});
|
||
}
|
||
}
|
||
|
||
export const POST = apiHandler(withIdempotency<[NextRequest]>(postBatch));
|
||
|
||
async function postBatch(request: NextRequest): Promise<Response> {
|
||
const { user } = await requireAuth();
|
||
|
||
// v1.4.43 W9 — resolve the per-user source-priority blob once at
|
||
// the start of the handler so the write-time canonical-row picker
|
||
// walks the SAME ladder the read-time picker walks. Without this
|
||
// lookup a user who customised Settings → Sources (e.g. MANUAL >
|
||
// APPLE_HEALTH) would see their preferred row dropped at write-
|
||
// time before the read-time picker ever ran. One indexed
|
||
// findUnique per batch — well under the cost of letting the
|
||
// duplicate persist and carrying it forward on every read.
|
||
const userPriority = await prisma.user.findUnique({
|
||
where: { id: user.id },
|
||
select: { sourcePriorityJson: true },
|
||
});
|
||
|
||
// Content-Length pre-flight. The body cap exists to bound the
|
||
// request-pipeline cost BEFORE Node has to read the full payload —
|
||
// a misbehaving client shipping a 50 MB body shouldn't tie up the
|
||
// worker for a parse it'd reject anyway. We trust the header here
|
||
// because Next.js / undici populates it for any client that sets a
|
||
// body length; for chunked uploads (no Content-Length) we fall
|
||
// through and rely on the schema-level point / workout caps to keep
|
||
// memory bounded.
|
||
const contentLength = request.headers.get("content-length");
|
||
if (contentLength !== null) {
|
||
const parsedLength = Number.parseInt(contentLength, 10);
|
||
if (Number.isFinite(parsedLength) && parsedLength > MAX_BODY_BYTES) {
|
||
annotate({
|
||
action: { name: "workout.batch.ingest" },
|
||
meta: { outcome: "payload_too_large", bytes: parsedLength },
|
||
});
|
||
return apiError(
|
||
`Request body exceeds the ${MAX_BODY_BYTES}-byte limit`,
|
||
413,
|
||
{ errorCode: "workout.batch.payload_too_large" },
|
||
);
|
||
}
|
||
}
|
||
|
||
// v1.4.25 W10 reconcile (security H-2 parity): per-user rate limit.
|
||
// Without it, a leaked wildcard iOS token can sustain unbounded
|
||
// writes (100 workouts/batch × N batches/sec) and degrade the
|
||
// database for every other user on the host. 60 batches/min/user is
|
||
// generous for healthy iOS sync and tight enough that a misbehaving
|
||
// client bottoms out within a minute.
|
||
const rl = await checkRateLimit(
|
||
`workouts:batch:${user.id}`,
|
||
BATCH_RATE_LIMIT_MAX,
|
||
BATCH_RATE_LIMIT_WINDOW_MS,
|
||
);
|
||
if (!rl.allowed) {
|
||
annotate({
|
||
action: { name: "workout.batch.ingest" },
|
||
meta: { outcome: "rate_limited" },
|
||
});
|
||
return apiError("Too many batch submissions, try again later", 429);
|
||
}
|
||
|
||
// The Content-Length pre-flight above bounds the common case, but a
|
||
// chunked upload carries no Content-Length. The `safeJson` body cap
|
||
// backstops that gap: the raw text is measured before `JSON.parse`,
|
||
// so an over-limit chunked body returns a clean 413 before the parse
|
||
// cost. Reuses the same 5 MB ceiling — a typical iOS batch sits under
|
||
// it; an extreme GPS-heavy backlog that exceeds it falls back to
|
||
// smaller batches client-side.
|
||
const { data: rawBody, error: jsonError } = await safeJson<unknown>(request, {
|
||
maxBytes: MAX_BODY_BYTES,
|
||
});
|
||
if (jsonError) return jsonError;
|
||
|
||
// Distinguish "too many entries" from "validation failed" so the
|
||
// client surfaces a useful diagnostic. The cap stops a buggy client
|
||
// from saturating the request pipeline; the iOS app paginates above
|
||
// this threshold.
|
||
if (
|
||
typeof rawBody === "object" &&
|
||
rawBody !== null &&
|
||
"workouts" in rawBody &&
|
||
Array.isArray((rawBody as { workouts: unknown }).workouts) &&
|
||
(rawBody as { workouts: unknown[] }).workouts.length >
|
||
MAX_WORKOUTS_PER_BATCH
|
||
) {
|
||
return apiError(
|
||
`Batch exceeds the ${MAX_WORKOUTS_PER_BATCH}-workout limit`,
|
||
400,
|
||
{ errorCode: "workout.batch.too_large" },
|
||
);
|
||
}
|
||
|
||
const parsed = createBatchWorkoutSchema.safeParse(rawBody);
|
||
if (!parsed.success) {
|
||
const issue = parsed.error.issues[0];
|
||
return apiError(issue?.message ?? "Invalid batch", 400, {
|
||
errorCode: "workout.batch.invalid",
|
||
});
|
||
}
|
||
|
||
const { workouts } = parsed.data;
|
||
|
||
// Pre-flight duplicate detection so we can return per-entry status.
|
||
// The composite unique index is `(userId, source, externalId)` —
|
||
// entries without an `externalId` cannot collide on the key (NULL-
|
||
// distinct semantics), so they always insert as new rows.
|
||
type Prepared = {
|
||
index: number;
|
||
row: Prisma.WorkoutCreateManyInput;
|
||
route: {
|
||
geometry: unknown;
|
||
sampleTimestamps: unknown;
|
||
} | null;
|
||
/** Route-independent per-workout HR series. NULL when the entry
|
||
* ships no `samples` array. Persisted to the `WorkoutSamples`
|
||
* child by FK after the workout row lands. */
|
||
samples: {
|
||
samples: unknown;
|
||
sampleCount: number;
|
||
} | null;
|
||
/** Composite-key probe for the pre-flight findMany. NULL when
|
||
* the entry has no externalId — those rows always pass through
|
||
* as "inserted" because the NULL-distinct unique index can never
|
||
* collide them. */
|
||
dedupKey: { source: string; externalId: string } | null;
|
||
};
|
||
|
||
const results: EntryResult[] = new Array(workouts.length);
|
||
const prepared: Prepared[] = workouts.map((w, index) => {
|
||
const durationSec = Math.max(
|
||
0,
|
||
Math.round((w.endedAt.getTime() - w.startedAt.getTime()) / 1000),
|
||
);
|
||
const route = w.route
|
||
? {
|
||
geometry: w.route.geometry as unknown,
|
||
sampleTimestamps: (w.route.sampleTimestamps ?? null) as unknown,
|
||
}
|
||
: null;
|
||
const samples = w.samples
|
||
? {
|
||
samples: w.samples as unknown,
|
||
sampleCount: w.samples.length,
|
||
}
|
||
: null;
|
||
const row: Prisma.WorkoutCreateManyInput = {
|
||
userId: user.id,
|
||
sportType: w.sportType,
|
||
startedAt: w.startedAt,
|
||
endedAt: w.endedAt,
|
||
durationSec,
|
||
totalEnergyKcal: w.totalEnergyKcal ?? null,
|
||
totalDistanceM: w.totalDistanceM ?? null,
|
||
avgHeartRate: w.avgHeartRate ?? null,
|
||
maxHeartRate: w.maxHeartRate ?? null,
|
||
minHeartRate: w.minHeartRate ?? null,
|
||
stepCount: w.stepCount ?? null,
|
||
elevationM: w.elevationM ?? null,
|
||
pauseDurationSec: w.pauseDurationSec ?? null,
|
||
source: w.source,
|
||
externalId: w.externalId ?? null,
|
||
externalSourceVersion: w.externalSourceVersion ?? null,
|
||
metadata: w.metadata
|
||
? (w.metadata as Prisma.InputJsonValue)
|
||
: Prisma.JsonNull,
|
||
};
|
||
return {
|
||
index,
|
||
row,
|
||
route,
|
||
samples,
|
||
dedupKey: w.externalId
|
||
? { source: w.source, externalId: w.externalId }
|
||
: null,
|
||
};
|
||
});
|
||
|
||
// v1.4.42 W5 — write-time cross-source dedup. The v1.5 iOS app
|
||
// drains a HealthKit observer queue that frequently carries the
|
||
// SAME logical workout from two paired sensors (Apple Watch +
|
||
// Withings ScanWatch). Without this pass, both rows persist (their
|
||
// `externalId`s differ → the `(userId, source, externalId)` unique
|
||
// index never fires) and the read-time canonical-picker has to
|
||
// carry the duplicate forward on every dashboard render.
|
||
//
|
||
// The helper is pure (no Prisma, no user lookups). It groups by
|
||
// `(userId, activityType, startedAt ± 90 s)`, prefers the canonical
|
||
// source ladder (APPLE_HEALTH > WITHINGS > MANUAL > IMPORT), and
|
||
// breaks ties on calories > earliest createdAt > input order. Rows
|
||
// dropped here surface to the iOS client as `duplicate` so the
|
||
// sync cursor advances past them identically to the externalId
|
||
// case below.
|
||
const canonicalPrepared = dedupeWorkoutBatch(
|
||
prepared.map((p) => ({
|
||
userId: p.row.userId,
|
||
activityType: p.row.sportType,
|
||
startedAt: p.row.startedAt as Date,
|
||
// Zod's `measurementSourceEnum.optional().default("MANUAL")`
|
||
// guarantees `source` is set post-parse, but Prisma's
|
||
// `WorkoutCreateManyInput` types it optional. Default through
|
||
// the same fallback the validator uses so the picker's source
|
||
// ladder still resolves on the rare row where TypeScript
|
||
// can't prove the union.
|
||
source: (p.row.source ?? "MANUAL") as MeasurementSource,
|
||
caloriesKcal: p.row.totalEnergyKcal ?? null,
|
||
// No createdAt yet — the row is pre-insert; the tie-breaker
|
||
// falls through to input order via the `index` field below.
|
||
index: p.index,
|
||
})),
|
||
userPriority?.sourcePriorityJson ?? null,
|
||
);
|
||
const survivingIndices = new Set(canonicalPrepared.map((r) => r.index ?? -1));
|
||
const droppedByWriteDedup: number[] = [];
|
||
for (const p of prepared) {
|
||
if (!survivingIndices.has(p.index)) {
|
||
results[p.index] = { index: p.index, status: "duplicate" };
|
||
droppedByWriteDedup.push(p.index);
|
||
}
|
||
}
|
||
const survivors = prepared.filter((p) => survivingIndices.has(p.index));
|
||
let duplicateCount = droppedByWriteDedup.length;
|
||
let insertedCount = 0;
|
||
|
||
if (survivors.length > 0) {
|
||
// v1.4.42 W5 — only the write-time dedup survivors are probed
|
||
// against existing rows; the dropped twins already carry a
|
||
// `duplicate` status and never reach the DB.
|
||
const dedupCandidates = survivors
|
||
.map((p) => p.dedupKey)
|
||
.filter((k): k is { source: string; externalId: string } => k !== null);
|
||
|
||
const existing = dedupCandidates.length
|
||
? await prisma.workout.findMany({
|
||
where: {
|
||
userId: user.id,
|
||
OR: dedupCandidates.map((k) => ({
|
||
source: k.source as Prisma.WorkoutCreateManyInput["source"],
|
||
externalId: k.externalId,
|
||
})),
|
||
},
|
||
select: { source: true, externalId: true },
|
||
})
|
||
: [];
|
||
|
||
const existingSet = new Set(
|
||
existing.map((row) => `${row.source}::${row.externalId}`),
|
||
);
|
||
|
||
const toInsert: Prepared[] = [];
|
||
for (const p of survivors) {
|
||
const keyTuple = p.dedupKey
|
||
? `${p.dedupKey.source}::${p.dedupKey.externalId}`
|
||
: null;
|
||
if (keyTuple !== null && existingSet.has(keyTuple)) {
|
||
results[p.index] = { index: p.index, status: "duplicate" };
|
||
duplicateCount += 1;
|
||
} else {
|
||
results[p.index] = { index: p.index, status: "inserted" };
|
||
toInsert.push(p);
|
||
}
|
||
}
|
||
|
||
if (toInsert.length > 0) {
|
||
// Two-step write: createMany the workout rows, then findMany the
|
||
// freshly-inserted rows to recover their ids so the route table
|
||
// can be populated by FK. We chunk both steps for parity with the
|
||
// measurements path and to keep individual SQL statements bounded
|
||
// under Postgres's 65k-parameter cap.
|
||
const CHUNK = 100;
|
||
|
||
await prisma.$transaction(async (tx) => {
|
||
for (let i = 0; i < toInsert.length; i += CHUNK) {
|
||
const slice = toInsert.slice(i, i + CHUNK);
|
||
const result = await tx.workout.createMany({
|
||
data: slice.map((p) => p.row),
|
||
skipDuplicates: true,
|
||
});
|
||
insertedCount += result.count;
|
||
}
|
||
|
||
// Look up the workouts we just wrote (or that the racing batch
|
||
// wrote) so we can attach their child rows (route geometry +
|
||
// the route-independent HR series) by FK. We probe by the
|
||
// composite unique key for entries that have one, and by the
|
||
// `(userId, source, startedAt, sportType)` tuple as a fallback
|
||
// for manual entries (where externalId is null). The fallback
|
||
// is best-effort — manual entries from the iOS client always
|
||
// carry an HK uuid, so the null-externalId case is exercised
|
||
// only by future surface (manual workout entry).
|
||
//
|
||
// v1.10.0 — `samples` (the per-workout HR series) rides the
|
||
// SAME id lookup as `route`: an indoor workout carries samples
|
||
// with no route, so the lookup runs whenever EITHER child is
|
||
// present on a to-insert entry.
|
||
const childAttachable = toInsert.filter(
|
||
(p) => p.route !== null || p.samples !== null,
|
||
);
|
||
if (childAttachable.length > 0) {
|
||
const withExternal = childAttachable.filter(
|
||
(p) => p.dedupKey !== null,
|
||
);
|
||
const withoutExternal = childAttachable.filter(
|
||
(p) => p.dedupKey === null,
|
||
);
|
||
|
||
const lookedUp = withExternal.length
|
||
? await tx.workout.findMany({
|
||
where: {
|
||
userId: user.id,
|
||
OR: withExternal.map((p) => ({
|
||
source: p.dedupKey!
|
||
.source as Prisma.WorkoutCreateManyInput["source"],
|
||
externalId: p.dedupKey!.externalId,
|
||
})),
|
||
},
|
||
select: {
|
||
id: true,
|
||
source: true,
|
||
externalId: true,
|
||
},
|
||
})
|
||
: [];
|
||
|
||
const idBySourceExt = new Map<string, string>();
|
||
for (const row of lookedUp) {
|
||
if (row.externalId !== null) {
|
||
idBySourceExt.set(`${row.source}::${row.externalId}`, row.id);
|
||
}
|
||
}
|
||
|
||
// Manual entries — `externalId` is null and the unique index
|
||
// is NULL-distinct, so two manual entries on the same instant
|
||
// would both insert. We look these up by the deterministic
|
||
// (userId, source, startedAt, sportType) tuple for each row.
|
||
//
|
||
// v1.4.27 B7 / BL-code-M3 — single batched findMany with one
|
||
// OR-row per entry instead of N serial findFirst probes. For
|
||
// a 100-row batch this cuts the per-batch round-trip count
|
||
// from 1 + N to 2; the per-entry "most-recent createdAt"
|
||
// tie-break is preserved by sorting the returned rows in
|
||
// memory and keying on the deterministic tuple.
|
||
type ManualMatch = { id: string; createdAt: Date };
|
||
const manualMatchesById = new Map<string, ManualMatch>();
|
||
if (withoutExternal.length > 0) {
|
||
const manualRows = await tx.workout.findMany({
|
||
where: {
|
||
userId: user.id,
|
||
route: null,
|
||
OR: withoutExternal.map((p) => ({
|
||
source: p.row.source,
|
||
startedAt: p.row.startedAt as Date,
|
||
sportType: p.row.sportType,
|
||
})),
|
||
},
|
||
select: {
|
||
id: true,
|
||
source: true,
|
||
startedAt: true,
|
||
sportType: true,
|
||
createdAt: true,
|
||
},
|
||
orderBy: { createdAt: "desc" },
|
||
});
|
||
for (const row of manualRows) {
|
||
const key = `${row.source}::${row.startedAt.getTime()}::${row.sportType}`;
|
||
const existing = manualMatchesById.get(key);
|
||
if (!existing || existing.createdAt < row.createdAt) {
|
||
manualMatchesById.set(key, {
|
||
id: row.id,
|
||
createdAt: row.createdAt,
|
||
});
|
||
}
|
||
}
|
||
}
|
||
const manualMatches = withoutExternal.map((p) => {
|
||
const key = `${p.row.source}::${(p.row.startedAt as Date).getTime()}::${p.row.sportType}`;
|
||
const match = manualMatchesById.get(key);
|
||
return match ? { id: match.id } : null;
|
||
});
|
||
|
||
// Resolve the freshly-written workout id for a prepared
|
||
// entry — by composite key for externalId-bearing rows, by
|
||
// the deterministic (source, startedAt, sportType) tuple for
|
||
// manual rows. Shared by both child tables (route + samples)
|
||
// so the HR series and the route attach to the same row.
|
||
const resolveId = (
|
||
p: Prepared,
|
||
manualIndex: number,
|
||
): string | undefined => {
|
||
if (p.dedupKey !== null) {
|
||
return idBySourceExt.get(
|
||
`${p.dedupKey.source}::${p.dedupKey.externalId}`,
|
||
);
|
||
}
|
||
return manualMatches[manualIndex]?.id ?? undefined;
|
||
};
|
||
|
||
const routesToInsert: Prisma.WorkoutRouteCreateManyInput[] = [];
|
||
const samplesToInsert: Prisma.WorkoutSamplesCreateManyInput[] = [];
|
||
for (let i = 0; i < withExternal.length; i++) {
|
||
const p = withExternal[i];
|
||
const id = resolveId(p, -1);
|
||
if (!id) continue;
|
||
if (p.route) {
|
||
routesToInsert.push({
|
||
workoutId: id,
|
||
geometry: p.route.geometry as Prisma.InputJsonValue,
|
||
sampleTimestamps:
|
||
p.route.sampleTimestamps === null
|
||
? Prisma.JsonNull
|
||
: (p.route.sampleTimestamps as Prisma.InputJsonValue),
|
||
});
|
||
}
|
||
if (p.samples) {
|
||
samplesToInsert.push({
|
||
workoutId: id,
|
||
samples: p.samples.samples as Prisma.InputJsonValue,
|
||
sampleCount: p.samples.sampleCount,
|
||
});
|
||
}
|
||
}
|
||
for (let i = 0; i < withoutExternal.length; i++) {
|
||
const p = withoutExternal[i];
|
||
const id = resolveId(p, i);
|
||
if (!id) continue;
|
||
if (p.route) {
|
||
routesToInsert.push({
|
||
workoutId: id,
|
||
geometry: p.route.geometry as Prisma.InputJsonValue,
|
||
sampleTimestamps:
|
||
p.route.sampleTimestamps === null
|
||
? Prisma.JsonNull
|
||
: (p.route.sampleTimestamps as Prisma.InputJsonValue),
|
||
});
|
||
}
|
||
if (p.samples) {
|
||
samplesToInsert.push({
|
||
workoutId: id,
|
||
samples: p.samples.samples as Prisma.InputJsonValue,
|
||
sampleCount: p.samples.sampleCount,
|
||
});
|
||
}
|
||
}
|
||
|
||
if (routesToInsert.length > 0) {
|
||
for (let i = 0; i < routesToInsert.length; i += CHUNK) {
|
||
await tx.workoutRoute.createMany({
|
||
data: routesToInsert.slice(i, i + CHUNK),
|
||
// 1:1 FK — `workoutId` has a UNIQUE index, so the
|
||
// composite unique guards routes too. `skipDuplicates`
|
||
// means a re-submitted batch that wins the race on the
|
||
// workout row but loses on the route is a no-op rather
|
||
// than a hard error.
|
||
skipDuplicates: true,
|
||
});
|
||
}
|
||
}
|
||
|
||
if (samplesToInsert.length > 0) {
|
||
for (let i = 0; i < samplesToInsert.length; i += CHUNK) {
|
||
await tx.workoutSamples.createMany({
|
||
data: samplesToInsert.slice(i, i + CHUNK),
|
||
// 1:1 FK on `workoutId` (UNIQUE), same idempotency as
|
||
// the route child: a re-submitted batch that wins the
|
||
// workout-row race but loses the sample-row race is a
|
||
// no-op, not a hard error.
|
||
skipDuplicates: true,
|
||
});
|
||
}
|
||
}
|
||
}
|
||
});
|
||
|
||
// v1.4.25 W10 reconcile (senior-dev H-1 parity): when two batches
|
||
// race on overlapping externalIds, only one row lands per key.
|
||
// `createMany.count` is the source of truth for how many rows
|
||
// THIS request wrote; downgrade enough per-entry "inserted"
|
||
// statuses to "duplicate" so the per-entry envelope sums match
|
||
// the aggregate counts. Order doesn't matter — the iOS sync
|
||
// cursor advances past both statuses identically.
|
||
const racedDuplicates = toInsert.length - insertedCount;
|
||
if (racedDuplicates > 0) {
|
||
let downgraded = 0;
|
||
// v1.4.42 W5 — walk `survivors` (not `prepared`) so the
|
||
// already-downgraded-by-write-dedup losers don't get
|
||
// double-counted toward the race-reconcile budget.
|
||
for (const p of survivors) {
|
||
if (downgraded >= racedDuplicates) break;
|
||
if (results[p.index]?.status === "inserted") {
|
||
results[p.index] = { index: p.index, status: "duplicate" };
|
||
duplicateCount += 1;
|
||
downgraded += 1;
|
||
}
|
||
}
|
||
}
|
||
}
|
||
}
|
||
|
||
const skipped = results
|
||
.filter((r) => r.status === "skipped")
|
||
.map((r) => ({ index: r.index, reason: r.reason ?? "unknown" }));
|
||
|
||
await auditLog("workout.batch.ingest", {
|
||
userId: user.id,
|
||
ipAddress: getClientIp(request),
|
||
details: {
|
||
processed: workouts.length,
|
||
inserted: insertedCount,
|
||
duplicates: duplicateCount,
|
||
skipped: skipped.length,
|
||
},
|
||
});
|
||
|
||
// v1.4.25 W16c — kick off PR detection for this user. Suppress push
|
||
// notifications for historical-backfill batches so a multi-year
|
||
// HKWorkout import doesn't fire hundreds of pushes during initial
|
||
// sync. The detector also scans Measurement history on the same
|
||
// pass, so a workout-only batch still surfaces measurement-side PRs
|
||
// the user may have racked up since the last detection.
|
||
if (insertedCount > 0 || duplicateCount > 0) {
|
||
const silent = workouts.length > PR_DETECTION_SILENT_THRESHOLD;
|
||
try {
|
||
await enqueuePrDetection(user.id, { silent });
|
||
await auditLog("personal_records.detection_enqueued", {
|
||
userId: user.id,
|
||
details: {
|
||
source: "workout.batch",
|
||
batchSize: workouts.length,
|
||
silent,
|
||
},
|
||
});
|
||
} catch (err) {
|
||
annotate({
|
||
action: { name: "personal_records.detection_enqueue_failed" },
|
||
meta: { error: err instanceof Error ? err.message : String(err) },
|
||
});
|
||
}
|
||
}
|
||
|
||
annotate({
|
||
action: { name: "workout.batch.ingest" },
|
||
meta: {
|
||
processed: workouts.length,
|
||
inserted: insertedCount,
|
||
duplicates: duplicateCount,
|
||
skipped: skipped.length,
|
||
},
|
||
});
|
||
|
||
// v1.4.34 IW-G — bust per-user analytics + achievements + workouts
|
||
// caches when at least one row landed. Workouts ride on the
|
||
// measurements bucket because achievements / analytics also touch
|
||
// workout-derived metrics.
|
||
if (insertedCount > 0) {
|
||
invalidateUserMeasurements(user.id);
|
||
|
||
// v1.31.0 — the workout arm of the data-arrival spine.
|
||
//
|
||
// Unlike every other kind, workouts emit ONE arrival PER workout, keyed by
|
||
// the workout's own id. Two sessions in a day are two events and each
|
||
// deserves its own reaction; a day-scoped key would silently collapse the
|
||
// second one.
|
||
//
|
||
// That needs the ids, and `createMany` does not return them, so this costs
|
||
// one extra indexed lookup — but only on a batch that actually inserted
|
||
// something, and only for entries carrying an external id. The classifier
|
||
// still runs per emit, so a historical import walks straight into
|
||
// `backfill` and enqueues nothing regardless of how many rows it wrote.
|
||
void emitWorkoutArrivals(user.id, prepared, results).catch(() => {});
|
||
}
|
||
|
||
return apiSuccess({
|
||
processed: workouts.length,
|
||
inserted: insertedCount,
|
||
duplicates: duplicateCount,
|
||
skipped,
|
||
entries: results,
|
||
});
|
||
}
|