fix(medications): identify an imported dose by when it was taken

The intake import built its replay key from the row's optional `zaehler`
field. That reads as a per-row counter — the documented example showed one —
but nothing required it to be unique and nothing checked. A reported export
carried the dose quantity there instead: 1 tablet, on all 28 rows. Every entry
queued under the same key, one dose reached the ledger, and the other 27 came
back reported as duplicates of a dose they had nothing to do with.

The key is the instant now, and `zaehler` is out of the contract rather than
accepted-and-ignored. Nothing is lost by dropping it: the live-row unique on
`(user_id, medication_id, scheduled_for, source)` already makes an imported
dose unique per instant, so a re-import of a file first imported under the old
key still dedups.

The telling is the other half. Two different facts were counted as one number:
entries of a submission that collide with each other, and doses the ledger
already holds. They are separate reasons now, grouped with a count, and the
dialog renders them through the same classifier and the same outcome line as
the CSV and JSON import cards — so an import that wrote nothing can no longer
answer in success tone. That rule shipped for the settings import panel and
this surface was not part of it; the guard now lists every import surface by
path, so a fourth one has to be added deliberately. `skippedInvalid` is gone:
it was hardcoded to zero on every path and reported a capability the importer
never had.

Alongside it, the other half of the environment report — a scheduled job that
fails every run has to say so. `recordError()` increments one unnamed counter,
so the admin status page could show a number and not which queue produced it,
and the module's own screen showed a zero day count with nothing to
distinguish "nothing stored yet" from "every run failed". pg-boss has held the
answer the whole time: a job that exhausts its retries keeps its queue name,
its error and its timestamp. The admin status now names the failing queues
with counts and the last error, the environment overview reports its own
account's failed runs, and neither needed a change at any of the thirty
`recordError()` call sites — a visibility fix that has to be remembered per
call site is the same failure one level up.

Refs #650, #646
This commit is contained in:
Marc-André Bombeck committed 2026-07-26 14:37:57 +02:00
1 parent a8d798442f
commit f0c5b5ae94
27 files changed
+1626 -154

No files matched your search

+14
View File
@@ -3,6 +3,10 @@ import { apiHandler, requireAdmin } from "@/lib/api-handler";
import { apiSuccess } from "@/lib/api-response";
import { annotate } from "@/lib/logging/context";
import { getWorkerStatus } from "@/lib/jobs/worker-status";
import {
JOB_FAILURE_WINDOW_HOURS,
readFailingQueues,
} from "@/lib/jobs/job-failures";
import { readFileSync } from "node:fs";
// Build timestamp (set at build time via next.config)
@@ -57,6 +61,12 @@ export const GET = apiHandler(async () => {
appSettings?.webPushVapidSubject,
);
const workerStatus = getWorkerStatus();
// `worker.errors` is one unnamed counter, which is how a queue that failed on
// every run for months could sit behind a number nobody could act on. The
// queue itself knows which job broke and why, so that is what the status
// reports. `null` = no queue schema to ask (web-only deployment); an empty
// array = asked, nothing is failing.
const failingJobs = await readFailingQueues();
return apiSuccess({
version: process.env.npm_package_version ?? "0.1.0",
@@ -66,6 +76,10 @@ export const GET = apiHandler(async () => {
startTime: START_TIME,
database: "connected",
worker: workerStatus,
failingJobs:
failingJobs === null
? null
: { windowHours: JOB_FAILURE_WINDOW_HOURS, queues: failingJobs },
counts: {
users: userCount,
measurements: measurementCount,
+44 -31
View File
@@ -12,6 +12,8 @@ import { annotate } from "@/lib/logging/context";
import { requireModuleEnabled } from "@/lib/modules/gate";
import { prisma } from "@/lib/db";
import { OPEN_METEO_ATTRIBUTION } from "@/lib/environment/open-meteo";
import { ENVIRONMENT_FETCH_QUEUE } from "@/lib/jobs/environment-fetch";
import { readQueueFailureForUser } from "@/lib/jobs/job-failures";
export const dynamic = "force-dynamic";
@@ -21,37 +23,43 @@ export const GET = apiHandler(async () => {
const gate = await requireModuleEnabled(user.id, "environment");
if (!gate.enabled) return gate.response;
const [profile, travel, contextCount, latest] = await Promise.all([
prisma.user.findUnique({
where: { id: user.id },
select: {
homeLat: true,
homeLon: true,
homeLabel: true,
homeTimezone: true,
homeSince: true,
timezone: true,
},
}),
prisma.environmentTravelLocation.findMany({
where: { userId: user.id },
orderBy: { startDate: "desc" },
select: {
id: true,
startDate: true,
endDate: true,
lat: true,
lon: true,
label: true,
},
}),
prisma.environmentContext.count({ where: { userId: user.id } }),
prisma.environmentContext.findFirst({
where: { userId: user.id },
orderBy: { date: "desc" },
select: { date: true, fetchedAt: true },
}),
]);
const [profile, travel, contextCount, latest, lastFetchFailure] =
await Promise.all([
prisma.user.findUnique({
where: { id: user.id },
select: {
homeLat: true,
homeLon: true,
homeLabel: true,
homeTimezone: true,
homeSince: true,
timezone: true,
},
}),
prisma.environmentTravelLocation.findMany({
where: { userId: user.id },
orderBy: { startDate: "desc" },
select: {
id: true,
startDate: true,
endDate: true,
lat: true,
lon: true,
label: true,
},
}),
prisma.environmentContext.count({ where: { userId: user.id } }),
prisma.environmentContext.findFirst({
where: { userId: user.id },
orderBy: { date: "desc" },
select: { date: true, fetchedAt: true },
}),
// A day count on its own cannot tell an empty module from a broken one —
// which is exactly how a job that failed on every run since the module
// shipped could render as "0 days recorded" and nothing else. The queue
// knows; the overview asks it.
readQueueFailureForUser(ENVIRONMENT_FETCH_QUEUE, user.id),
]);
const home =
profile?.homeLat != null && profile?.homeLon != null
@@ -72,6 +80,7 @@ export const GET = apiHandler(async () => {
has_home: home !== null,
travel_count: travel.length,
context_days: contextCount,
last_fetch_failed: lastFetchFailure !== null,
},
});
@@ -83,6 +92,10 @@ export const GET = apiHandler(async () => {
latestDate: latest?.date ?? null,
latestFetchedAt: latest?.fetchedAt?.toISOString() ?? null,
},
// null = the last background runs did not fail (or there is no queue to
// ask). Non-null names the failure so a zero day count is never left to
// stand alone. No error text: that message is written for an operator.
lastFetchFailure,
attribution: OPEN_METEO_ATTRIBUTION,
});
});
@@ -0,0 +1,30 @@
[
{ "datum": "2026-05-31", "uhrzeit": "08:30:00", "zaehler": 1.0 },
{ "datum": "2026-06-01", "uhrzeit": "08:30:00", "zaehler": 1.0 },
{ "datum": "2026-06-02", "uhrzeit": "08:30:00", "zaehler": 1.0 },
{ "datum": "2026-06-03", "uhrzeit": "08:30:00", "zaehler": 1.0 },
{ "datum": "2026-06-04", "uhrzeit": "08:30:00", "zaehler": 1.0 },
{ "datum": "2026-06-05", "uhrzeit": "08:30:00", "zaehler": 1.0 },
{ "datum": "2026-06-06", "uhrzeit": "08:30:00", "zaehler": 1.0 },
{ "datum": "2026-06-07", "uhrzeit": "08:30:00", "zaehler": 1.0 },
{ "datum": "2026-06-08", "uhrzeit": "08:30:00", "zaehler": 1.0 },
{ "datum": "2026-06-09", "uhrzeit": "08:30:00", "zaehler": 1.0 },
{ "datum": "2026-06-10", "uhrzeit": "08:30:00", "zaehler": 1.0 },
{ "datum": "2026-06-11", "uhrzeit": "08:30:00", "zaehler": 1.0 },
{ "datum": "2026-06-12", "uhrzeit": "08:30:00", "zaehler": 1.0 },
{ "datum": "2026-06-13", "uhrzeit": "08:30:00", "zaehler": 1.0 },
{ "datum": "2026-06-14", "uhrzeit": "08:30:00", "zaehler": 1.0 },
{ "datum": "2026-06-15", "uhrzeit": "08:30:00", "zaehler": 1.0 },
{ "datum": "2026-06-16", "uhrzeit": "08:30:00", "zaehler": 1.0 },
{ "datum": "2026-06-17", "uhrzeit": "08:30:00", "zaehler": 1.0 },
{ "datum": "2026-06-18", "uhrzeit": "08:30:00", "zaehler": 1.0 },
{ "datum": "2026-06-19", "uhrzeit": "08:30:00", "zaehler": 1.0 },
{ "datum": "2026-06-20", "uhrzeit": "08:30:00", "zaehler": 1.0 },
{ "datum": "2026-06-21", "uhrzeit": "08:30:00", "zaehler": 1.0 },
{ "datum": "2026-06-22", "uhrzeit": "08:30:00", "zaehler": 1.0 },
{ "datum": "2026-06-23", "uhrzeit": "08:30:00", "zaehler": 1.0 },
{ "datum": "2026-06-24", "uhrzeit": "08:30:00", "zaehler": 1.0 },
{ "datum": "2026-06-25", "uhrzeit": "08:30:00", "zaehler": 1.0 },
{ "datum": "2026-06-26", "uhrzeit": "08:30:00", "zaehler": 1.0 },
{ "datum": "2026-06-27", "uhrzeit": "08:30:00", "zaehler": 1.0 }
]
@@ -4,6 +4,9 @@
* The request process validates and persists the complete payload, then
* returns a polling handle without performing intake writes itself.
*/
import { readFileSync } from "node:fs";
import { join } from "node:path";
import { describe, it, expect, vi, beforeEach } from "vitest";
import { NextRequest } from "next/server";
@@ -233,14 +236,17 @@ describe("POST /api/medications/[id]/intake/import — durable kickoff", () => {
userId: "user-1",
medicationId: "m1",
status: "queued",
// A legacy `zaehler` on the first row rides along and decides nothing:
// both keys come from the instant, so two different times can never
// collapse onto one key again.
payload: {
entries: [
{
idempotencyKey: "import-m1-41",
idempotencyKey: `import-m1-${Date.parse("2026-01-01T07:00:00")}`,
takenAt: expect.any(String),
},
{
idempotencyKey: expect.stringMatching(/^import-m1-\d+$/),
idempotencyKey: `import-m1-${Date.parse("2026-01-01T19:00:00")}`,
takenAt: expect.any(String),
},
],
@@ -248,7 +254,7 @@ describe("POST /api/medications/[id]/intake/import — durable kickoff", () => {
progress: expect.objectContaining({
processed: 0,
imported: 0,
skippedDuplicates: 0,
skippedByReason: {},
total: 2,
}),
}),
@@ -412,3 +418,69 @@ describe("POST /api/medications/[id]/intake/import — future bound and limiter"
);
});
});
describe("POST /api/medications/[id]/intake/import — the reported export (#650)", () => {
// The file attached to the report: 28 doses, one per day, all at 08:30, and
// `"zaehler": 1.0` on every single row — a dose quantity, not a row counter.
// The old key derivation used that field verbatim, so all 28 entries queued
// under `import-m1-1`, one row was written and 27 were reported back as
// duplicates of each other.
const REPORTED_EXPORT = JSON.parse(
readFileSync(
join(__dirname, "fixtures", "reported-intake-export.json"),
"utf8",
),
) as Array<{ datum: string; uhrzeit: string; zaehler: number }>;
it("is the reported shape (a fixture that drifted would prove nothing)", () => {
expect(REPORTED_EXPORT).toHaveLength(28);
expect(new Set(REPORTED_EXPORT.map((row) => row.zaehler)).size).toBe(1);
expect(
new Set(REPORTED_EXPORT.map((row) => `${row.datum}T${row.uhrzeit}`)).size,
).toBe(28);
});
it("queues 28 distinct doses instead of collapsing them onto one key", async () => {
const res = await POST(postReq(REPORTED_EXPORT), ROUTE_CTX);
expect(res.status).toBe(202);
const created = vi.mocked(prisma.medicationIntakeImportJob.create).mock
.calls[0][0] as unknown as {
data: {
payload: {
entries: Array<{ takenAt: string; idempotencyKey: string }>;
};
};
};
const entries = created.data.payload.entries;
expect(entries).toHaveLength(28);
expect(new Set(entries.map((entry) => entry.idempotencyKey)).size).toBe(28);
expect(new Set(entries.map((entry) => entry.takenAt)).size).toBe(28);
// The key is a function of the instant and nothing else.
for (const [index, entry] of entries.entries()) {
const row = REPORTED_EXPORT[index];
const instant = Date.parse(`${row.datum}T${row.uhrzeit}`);
expect(entry.idempotencyKey).toBe(`import-m1-${instant}`);
expect(entry.takenAt).toBe(new Date(instant).toISOString());
}
});
it("keys two rows at the same instant identically, whatever else they carry", async () => {
await POST(
postReq([
{ datum: "2026-03-01", uhrzeit: "08:00:00", zaehler: 7 },
{ datum: "2026-03-01", uhrzeit: "08:00:00", zaehler: 9 },
]),
ROUTE_CTX,
);
const created = vi.mocked(prisma.medicationIntakeImportJob.create).mock
.calls[0][0] as unknown as {
data: { payload: { entries: Array<{ idempotencyKey: string }> } };
};
const keys = created.data.payload.entries.map(
(entry) => entry.idempotencyKey,
);
expect(keys[0]).toBe(keys[1]);
});
});
@@ -21,14 +21,31 @@ import {
type MedicationImportProgress,
} from "@/lib/jobs/medication-intake-import";
import { annotate } from "@/lib/logging/context";
import { buildMedicationImportEntries } from "@/lib/medications/intake-import-payload";
import { assertMedicationOwnership } from "@/lib/medications/route-guards";
import { checkRateLimit } from "@/lib/rate-limit";
import { isPlausibleEntryInstant } from "@/lib/validations/entry-instant";
// The entry shape carried a third field, `zaehler`, and the replay key was
// built from it whenever it was present. That reads as a per-row counter,
// which is what the documented example showed — but the field was optional,
// its uniqueness was never stated and never checked, and an export that puts
// the dose quantity there (1 tablet, on every row) collapsed a month of
// history onto one key: the first row imported, every later one lost the
// `(userId, idempotencyKey)` unique and came back reported as a duplicate of
// a dose it had nothing to do with.
//
// `zaehler` is gone from the contract rather than accepted-and-ignored. A
// payload that still carries it imports unchanged — `z.object` drops
// unrecognised keys — it simply no longer decides anything. The key now comes
// from the instant (`buildMedicationImportEntries`), which is also the grain
// the database already enforces: the live-row unique on
// `(user_id, medication_id, scheduled_for, source)` makes an imported dose
// unique per instant whatever the key says, so a re-import of a file first
// imported under the old key still dedups.
const importEntrySchema = z.object({
datum: z.string().regex(/^\d{4}-\d{2}-\d{2}$/),
uhrzeit: z.string().regex(/^\d{2}:\d{2}:\d{2}$/),
zaehler: z.union([z.number().int(), z.string()]).optional(),
});
function isValidImportTimestamp(datum: string, uhrzeit: string): boolean {
@@ -149,19 +166,13 @@ export const POST = apiHandler(
const entries = parsed.data;
const normalized: MedicationImportPayload = {
entries: entries.map((entry) => {
const takenAt = new Date(`${entry.datum}T${entry.uhrzeit}`);
const idempotencyKey = entry.zaehler
? `import-${id}-${String(entry.zaehler)}`
: `import-${id}-${takenAt.getTime()}`;
return { takenAt: takenAt.toISOString(), idempotencyKey };
}),
entries: buildMedicationImportEntries(id, entries),
};
const progress: MedicationImportProgress = {
processed: 0,
total: normalized.entries.length,
imported: 0,
skippedDuplicates: 0,
skippedByReason: {},
touchedDays: [],
rollupProcessed: 0,
};