fix(import): accept a blood-glucose reading that carries no context

A CSV import of a continuous-sensor export wrote nothing and rendered a
green success state. Three faults compounded, and the reporter took all
three apart before filing.

The parser refused every BLOOD_GLUCOSE row with a blank glucoseContext.
Behind it, the CHECK constraint from migration 0021 read both ways at
once: a glucose row MUST carry a context, every other type MUST NOT. The
second half is right and stays. The first was written when the only
glucose writer was the entry form, where a person picks fasting or
post-meal at the moment of the reading. Every machine writer that
arrived since has no such answer per sample, so the Nightscout pull, the
MCP log tool, the Telegram capture and both bulk importers were all
building rows the database then refused.

A missing context now stores as NULL, never as a fifth enum member.
Every per-context reader selects by equality against the four members,
so the reading stays out of the doctor report's fasting rows, the
matching FHIR observations and the per-context targets, while the chart,
the all-time summary and the clinical panel keep it in full. Hand entry
still asks for a context, because at that moment you know it.

The import cards now read the counts rather than the status code.
Nothing written is a failure, a mixed file is a warning, and the refused
rows are grouped by reason with a count instead of fifty identical
lines, with the per-line detail one disclosure away. Both cards render
through one outcome table, so a tick cannot reach a result that wrote
nothing.

Refs #640
This commit is contained in:
Marc-André Bombeck committed 2026-07-25 21:23:26 +02:00
1 parent 019ad52c0d
commit 0c9832eb8d
19 files changed
+639 -114

No files matched your search

@@ -4,7 +4,6 @@ import { useId, useRef, useState } from "react";
import Link from "next/link";
import {
AlertCircle,
CheckCircle2,
Download,
FileSpreadsheet,
Loader2,
@@ -19,21 +18,10 @@ import { apiFetchRaw } from "@/lib/api/api-fetch";
import { ImportCardShell } from "./import-card-shell";
import { MAX_PASTE_CHARS } from "./constants";
import { EXAMPLE_CSV } from "./import-examples";
interface CsvRowResult {
line: number;
status: "inserted" | "updated" | "skipped";
reason?: string;
}
interface CsvImportResult {
inserted: number;
updated: number;
skipped: number;
total: number;
dryRun: boolean;
rows: CsvRowResult[];
}
import {
CsvImportResultView,
type CsvImportResult,
} from "./import-result-view";
export function CsvImportCard() {
const { t } = useTranslations();
@@ -117,8 +105,6 @@ export function CsvImportCard() {
}
}
const errorRows = result?.rows.filter((r) => r.status === "skipped") ?? [];
return (
<ImportCardShell
testId="import-card-csv"
@@ -169,40 +155,7 @@ export function CsvImportCard() {
</p>
<div aria-live="polite" className="space-y-2">
{result && (
<div data-testid="import-csv-result" className="space-y-1.5">
<p className="text-foreground flex items-start gap-2 text-xs">
<CheckCircle2
className="text-success mt-0.5 h-3.5 w-3.5 shrink-0"
aria-hidden="true"
/>
<span>
{result.dryRun
? t("settings.sections.export.import.csv.previewSummary", {
inserted: result.inserted,
skipped: result.skipped,
})
: t("settings.sections.export.import.csv.resultSummary", {
inserted: result.inserted,
updated: result.updated,
skipped: result.skipped,
})}
</span>
</p>
{errorRows.length > 0 && (
<ul className="text-muted-foreground max-h-32 space-y-0.5 overflow-auto text-xs">
{errorRows.slice(0, 50).map((r) => (
<li key={r.line}>
{t("settings.sections.export.import.csv.rowError", {
line: r.line,
reason: r.reason ?? "skipped",
})}
</li>
))}
</ul>
)}
</div>
)}
{result && <CsvImportResultView result={result} />}
{error && (
<p
role="alert"
@@ -0,0 +1,72 @@
/**
* Import result classification — the one place that decides whether an
* import reads as a success.
*
* A per-row importer answers HTTP 200 for a file where every single row was
* refused; the transport succeeded, the import did not. Rendering that as a
* green tick is how a self-hoster's 1 597-reading sensor export could write
* nothing and still look finished. So the counts, not the status code, pick
* the state:
*
* - nothing accepted, nothing skipped → `empty` (the file carried no data)
* - nothing accepted, rows skipped → `failed` (never a success tone)
* - some accepted, some skipped → `partial` (a warning, not a tick)
* - all accepted → `success`
*
* Both import cards render from this, and `import-result-state.test.ts` pins
* the invariant that `success` is unreachable while `written === 0`.
*/
export type ImportOutcome = "empty" | "failed" | "partial" | "success";
export interface ImportCounts {
/** Rows that reached the database — inserted + updated. */
written: number;
/** Rows the importer refused, for any reason. */
skipped: number;
}
export function classifyImportOutcome({
written,
skipped,
}: ImportCounts): ImportOutcome {
if (written <= 0) return skipped > 0 ? "failed" : "empty";
return skipped > 0 ? "partial" : "success";
}
export interface SkipReasonGroup {
/** The machine-readable reason as the route reported it. */
reason: string;
count: number;
}
/** A row of the per-row envelope, narrowed to what the grouping needs. */
export interface ImportRowResult {
line: number;
status: string;
reason?: string;
}
const UNKNOWN_REASON = "unknown";
/**
* Collapse the per-row envelope into one entry per skip reason.
*
* A sensor export that trips one validation trips it on every row, so the
* raw list is the same sentence a few thousand times. Grouping turns "scroll
* fifty identical lines" into one fact with a count. Ordered by count
* descending, then by reason, so the same result always renders the same way.
*/
export function groupSkipReasons(
rows: readonly ImportRowResult[],
): SkipReasonGroup[] {
const counts = new Map<string, number>();
for (const row of rows) {
if (row.status !== "skipped") continue;
const reason = row.reason ?? UNKNOWN_REASON;
counts.set(reason, (counts.get(reason) ?? 0) + 1);
}
return [...counts.entries()]
.map(([reason, count]) => ({ reason, count }))
.sort((a, b) => b.count - a.count || a.reason.localeCompare(b.reason));
}
@@ -0,0 +1,205 @@
"use client";
import { AlertCircle, CheckCircle2, Info, TriangleAlert } from "lucide-react";
import type { LucideIcon } from "lucide-react";
import { useTranslations } from "@/lib/i18n/context";
import {
classifyImportOutcome,
groupSkipReasons,
type ImportOutcome,
type ImportRowResult,
} from "./import-result-state";
/**
* The one mapping from an import outcome to how it looks.
*
* `CheckCircle2` and `text-success` appear exactly once in the import panel,
* on the `success` key, so no future edit can attach a tick to a result that
* wrote nothing. `import-result-guard.test.ts` holds that line.
*/
const OUTCOME_PRESENTATION: Record<
ImportOutcome,
{ Icon: LucideIcon; className: string }
> = {
success: { Icon: CheckCircle2, className: "text-success" },
partial: { Icon: TriangleAlert, className: "text-warning" },
failed: { Icon: AlertCircle, className: "text-destructive" },
empty: { Icon: Info, className: "text-muted-foreground" },
};
/**
* The headline line of an import result. Sits inside each card's existing
* `aria-live="polite"` region, so the outcome is announced as it lands.
*/
export function ImportOutcomeLine({
outcome,
message,
testId,
}: {
outcome: ImportOutcome;
message: string;
testId: string;
}) {
const { Icon, className } = OUTCOME_PRESENTATION[outcome];
return (
<p
data-testid={testId}
data-outcome={outcome}
className="text-foreground flex items-start gap-2 text-xs"
>
<Icon
className={`${className} mt-0.5 h-3.5 w-3.5 shrink-0`}
aria-hidden="true"
/>
<span>{message}</span>
</p>
);
}
export interface CsvImportResult {
inserted: number;
updated: number;
skipped: number;
total: number;
dryRun: boolean;
rows: ImportRowResult[];
}
/**
* Plain-language label per machine-readable skip reason.
*
* Written as literal `t(...)` calls rather than a key lookup so the i18n
* call-site coverage guard sees every one of them.
*/
function useSkipReasonLabel(): (reason: string) => string {
const { t } = useTranslations();
const labels: Record<string, string> = {
unknown_type: t("settings.sections.export.import.csv.reasonUnknownType"),
invalid_value: t("settings.sections.export.import.csv.reasonInvalidValue"),
value_out_of_range: t(
"settings.sections.export.import.csv.reasonValueOutOfRange",
),
unknown_unit: t("settings.sections.export.import.csv.reasonUnknownUnit"),
missing_timezone_offset: t(
"settings.sections.export.import.csv.reasonMissingTimezoneOffset",
),
invalid_timestamp: t(
"settings.sections.export.import.csv.reasonInvalidTimestamp",
),
implausible_timestamp: t(
"settings.sections.export.import.csv.reasonImplausibleTimestamp",
),
unexpected_glucose_context: t(
"settings.sections.export.import.csv.reasonUnexpectedGlucoseContext",
),
invalid_glucose_context: t(
"settings.sections.export.import.csv.reasonInvalidGlucoseContext",
),
notes_too_long: t("settings.sections.export.import.csv.reasonNotesTooLong"),
external_id_too_long: t(
"settings.sections.export.import.csv.reasonExternalIdTooLong",
),
unstable_external_id: t(
"settings.sections.export.import.csv.reasonUnstableExternalId",
),
duplicate: t("settings.sections.export.import.csv.reasonDuplicate"),
};
const fallback = t("settings.sections.export.import.csv.reasonUnknown");
return (reason: string) => labels[reason] ?? fallback;
}
/**
* The CSV import result: an honest headline, then the skipped rows grouped by
* reason with a count, with the per-line detail kept one disclosure away.
*/
export function CsvImportResultView({ result }: { result: CsvImportResult }) {
const { t, tCount } = useTranslations();
const labelFor = useSkipReasonLabel();
const written = result.inserted + result.updated;
const outcome = classifyImportOutcome({ written, skipped: result.skipped });
const groups = groupSkipReasons(result.rows);
const skippedRows = result.rows.filter((row) => row.status === "skipped");
const message = (() => {
if (outcome === "empty") {
return t("settings.sections.export.import.csv.resultEmpty");
}
if (result.dryRun) {
if (outcome === "failed") {
return t("settings.sections.export.import.csv.previewNothing", {
skipped: result.skipped,
});
}
if (outcome === "partial") {
return t("settings.sections.export.import.csv.previewSummary", {
inserted: result.inserted,
skipped: result.skipped,
});
}
return t("settings.sections.export.import.csv.previewSuccess", {
inserted: result.inserted,
});
}
if (outcome === "failed") {
return t("settings.sections.export.import.csv.resultNothing", {
skipped: result.skipped,
});
}
if (outcome === "partial") {
return t("settings.sections.export.import.csv.resultSummary", {
inserted: result.inserted,
updated: result.updated,
skipped: result.skipped,
});
}
return t("settings.sections.export.import.csv.resultSuccess", {
inserted: result.inserted,
updated: result.updated,
});
})();
return (
<div data-testid="import-csv-result" className="space-y-1.5">
<ImportOutcomeLine
outcome={outcome}
message={message}
testId="import-csv-outcome"
/>
{groups.length > 0 && (
<ul
data-testid="import-csv-skip-groups"
className="text-muted-foreground space-y-0.5 text-xs"
>
{groups.map((group) => (
<li key={group.reason}>
{tCount(
"settings.sections.export.import.csv.skipGroup",
group.count,
{ count: group.count, reason: labelFor(group.reason) },
)}
</li>
))}
</ul>
)}
{skippedRows.length > 0 && (
<details className="text-muted-foreground text-xs">
<summary className="cursor-pointer">
{t("settings.sections.export.import.csv.skipDetails")}
</summary>
<ul className="mt-1 max-h-32 space-y-0.5 overflow-auto">
{skippedRows.map((row) => (
<li key={row.line}>
{t("settings.sections.export.import.csv.rowError", {
line: row.line,
reason: labelFor(row.reason ?? "unknown"),
})}
</li>
))}
</ul>
</details>
)}
</div>
);
}
@@ -2,14 +2,7 @@
import { useId, useRef, useState } from "react";
import Link from "next/link";
import {
AlertCircle,
CheckCircle2,
Download,
FileJson,
Loader2,
Upload,
} from "lucide-react";
import { AlertCircle, Download, FileJson, Loader2, Upload } from "lucide-react";
import { Button } from "@/components/ui/button";
import { Label } from "@/components/ui/label";
@@ -19,6 +12,8 @@ import { apiFetchRaw } from "@/lib/api/api-fetch";
import { ImportCardShell } from "./import-card-shell";
import { MAX_PASTE_CHARS } from "./constants";
import { EXAMPLE_IMPORT, parseImportJson } from "./import-examples";
import { classifyImportOutcome } from "./import-result-state";
import { ImportOutcomeLine } from "./import-result-view";
interface JsonImportResult {
measurements: number;
@@ -28,6 +23,38 @@ interface JsonImportResult {
export function JsonImportCard() {
const { t } = useTranslations();
// Same reasoning as the CSV card: the JSON route answers 200 with a per-
// concept count envelope, so a payload where every entry was refused used
// to render the success tick. The counts decide the tone.
const jsonOutcome = (r: JsonImportResult) =>
classifyImportOutcome({
written: r.measurements + r.moodEntries,
skipped: r.skipped,
});
const jsonMessage = (r: JsonImportResult) => {
const outcome = jsonOutcome(r);
if (outcome === "empty") {
return t("settings.sections.export.import.json.resultEmpty");
}
if (outcome === "failed") {
return t("settings.sections.export.import.json.resultNothing", {
skipped: r.skipped,
});
}
if (outcome === "partial") {
return t("settings.sections.export.import.json.resultSummary", {
measurements: r.measurements,
moods: r.moodEntries,
skipped: r.skipped,
});
}
return t("settings.sections.export.import.json.resultSuccess", {
measurements: r.measurements,
moods: r.moodEntries,
});
};
const fileInputRef = useRef<HTMLInputElement>(null);
const [text, setText] = useState("");
const [busy, setBusy] = useState(false);
@@ -164,22 +191,11 @@ export function JsonImportCard() {
<div aria-live="polite" className="space-y-2">
{result && (
<p
data-testid="import-json-result"
className="text-foreground flex items-start gap-2 text-xs"
>
<CheckCircle2
className="text-success mt-0.5 h-3.5 w-3.5 shrink-0"
aria-hidden="true"
/>
<span>
{t("settings.sections.export.import.json.resultSummary", {
measurements: result.measurements,
moods: result.moodEntries,
skipped: result.skipped,
})}
</span>
</p>
<ImportOutcomeLine
outcome={jsonOutcome(result)}
message={jsonMessage(result)}
testId="import-json-result"
/>
)}
{error && (
<p
+11 -11
View File
@@ -27,7 +27,11 @@
* without an offset is skipped — the importer never guesses
* a timezone. The instant is bounded by `validateEntryInstant`
* (no future beyond a 5-min skew, no instant before 1900).
* - `glucoseContext` — required for BLOOD_GLUCOSE, forbidden otherwise.
* - `glucoseContext` — optional on BLOOD_GLUCOSE, forbidden otherwise. A
* blank cell stores NULL: a continuous sensor export
* classifies nothing per reading, and an absent context
* must read as absent rather than as a guessed one. A
* non-empty cell is still checked against the enum.
* - `notes` — free text, ≤ 200 chars.
* - `externalId` — optional source-stable id. When present, the write loop
* upserts on `(userId, type, source=IMPORT, externalId)` so a
@@ -64,13 +68,11 @@ export type CsvRowReason =
| "missing_timezone_offset"
| "invalid_timestamp"
| "implausible_timestamp"
| "missing_glucose_context"
| "unexpected_glucose_context"
| "invalid_glucose_context"
| "notes_too_long"
| "external_id_too_long"
| "unstable_external_id"
| "wrong_column_count";
| "unstable_external_id";
export type CsvRowStatus = "ok" | "skipped";
@@ -278,18 +280,16 @@ export function parseCsvMeasurements(
continue;
}
// Glucose context: required for BLOOD_GLUCOSE, forbidden otherwise.
// Glucose context: optional on BLOOD_GLUCOSE, forbidden otherwise. A
// blank cell leaves the column NULL — the row is a reading either way,
// and a sensor export carries no fasting / post-meal classification.
let glucoseContext: string | undefined;
if (type === "BLOOD_GLUCOSE") {
if (!rawContext) {
skip("missing_glucose_context");
continue;
}
if (!KNOWN_GLUCOSE_CONTEXTS.has(rawContext)) {
if (rawContext && !KNOWN_GLUCOSE_CONTEXTS.has(rawContext)) {
skip("invalid_glucose_context");
continue;
}
glucoseContext = rawContext;
glucoseContext = rawContext || undefined;
} else if (rawContext) {
skip("unexpected_glucose_context");
continue;
+2 -2
View File
@@ -53,7 +53,7 @@ export const importPaths: NonNullable<ZodOpenApiObject["paths"]> = {
tags: ["Import"],
summary: "Import measurements from CSV",
description:
"Bulk-import measurements from a CSV file (web cold-start escape hatch). Body is raw `text/csv` (≤ 16 MB, ≤ 10 000 valid rows). Header (order-independent): `type,value,unit,measuredAt[,glucoseContext,notes,externalId]`. `measuredAt` must carry an explicit ISO-8601 offset and is bounded (no future beyond a 5-min skew, no instant before 1900). Glucose accepts `mmol/L` (converted to canonical `mg/dL`); weight accepts `lb` (converted to `kg`). An `externalId` column makes re-upload idempotent (upsert on `(userId, type, source=IMPORT, externalId)`); without it a re-upload duplicates. `?dryRun=1` validates + previews without writing. Shares the 5/hour `import:` rate bucket.",
"Bulk-import measurements from a CSV file (web cold-start escape hatch). Body is raw `text/csv` (≤ 16 MB, ≤ 10 000 valid rows). Header (order-independent): `type,value,unit,measuredAt[,glucoseContext,notes,externalId]`. `measuredAt` must carry an explicit ISO-8601 offset and is bounded (no future beyond a 5-min skew, no instant before 1900). Glucose accepts `mmol/L` (converted to canonical `mg/dL`); weight accepts `lb` (converted to `kg`). `glucoseContext` is optional on a `BLOOD_GLUCOSE` row — a blank cell stores no context, which is the normal shape of a continuous-sensor export; a non-empty value is still checked against the enum, and the column is refused on any other type. An `externalId` column makes re-upload idempotent (upsert on `(userId, type, source=IMPORT, externalId)`); without it a re-upload duplicates. `?dryRun=1` validates + previews without writing. Shares the 5/hour `import:` rate bucket.",
parameters: [
{
name: "dryRun",
@@ -71,7 +71,7 @@ export const importPaths: NonNullable<ZodOpenApiObject["paths"]> = {
schema: {
type: "string",
example:
"type,value,unit,measuredAt,glucoseContext,notes,externalId\nWEIGHT,80.5,kg,2026-05-01T08:00:00Z,,morning,\nBLOOD_GLUCOSE,5.3,mmol/L,2026-05-01T08:05:00+02:00,FASTING,,meter-001",
"type,value,unit,measuredAt,glucoseContext,notes,externalId\nWEIGHT,80.5,kg,2026-05-01T08:00:00Z,,morning,\nBLOOD_GLUCOSE,5.3,mmol/L,2026-05-01T08:05:00+02:00,FASTING,,meter-001\nBLOOD_GLUCOSE,5.9,mmol/L,2026-05-01T08:20:00+02:00,,,sensor-001",
},
},
},
+3 -1
View File
@@ -256,7 +256,9 @@ export const measurementResource = z
.string()
.nullable()
.optional()
.describe("Set only on BLOOD_GLUCOSE rows."),
.describe(
"Set only on BLOOD_GLUCOSE rows, and null there too when the writer recorded no classification (a sensor export, a device sync). Never invent one.",
),
sleepStage: z
.string()
.nullable()
+1 -1
View File
@@ -70,7 +70,7 @@ loginPasswordSchema.meta({
createMeasurementSchema.meta({
id: "CreateMeasurementRequest",
description:
"Single-measurement ingest body. Plausibility-range guard runs server-side; out-of-range values fail 422.",
"Single-measurement ingest body. Plausibility-range guard runs server-side; out-of-range values fail 422. `glucoseContext` stays REQUIRED on a `BLOOD_GLUCOSE` row here: this is the hand-entry surface, where the person taking the reading knows whether it was fasting or after a meal. The bulk ingest paths (CSV import, JSON import, device sync) accept a contextless reading, because a sensor export classifies nothing per sample.",
});
listMeasurementsSchema.meta({
+12 -2
View File
@@ -648,8 +648,10 @@ export const createMeasurementSchema = z
// `APPLE_HEALTH`; `WITHINGS` / `IMPORT` / `COMPUTED` are server-owned and
// forging them would pollute the per-source canonical picker.
source: writableMeasurementSourceEnum.optional().default("MANUAL"),
// Only applies when type === BLOOD_GLUCOSE. Mirrored by a CHECK
// constraint in Postgres (see migration 0021).
// Only applies when type === BLOOD_GLUCOSE. The "not on any other type"
// half is mirrored by a CHECK constraint in Postgres (migrations 0021 +
// 0274). The "required on a glucose row" half below is a rule of THIS
// schema only — see the refine.
glucoseContext: glucoseContextEnum.optional(),
// v1.4.25 W10 reconcile (code-review M4): the single-entry POST
// dropped `deviceType` silently because the column existed on
@@ -669,6 +671,14 @@ export const createMeasurementSchema = z
.refine((data) => validateMeasurementRange(data.type, data.value) === null, {
message: "Value out of plausible range",
})
// Deliberately asymmetric with the bulk ingest paths. This schema backs the
// hand-entry surface, where a person is present at the reading and knows
// whether it was fasting, after a meal, random or at bedtime — the form
// always offers the choice, so requiring it costs nothing and keeps the
// per-context breakdowns meaningful. A sensor or meter export has no such
// answer per sample, so the CSV import, the JSON import and the device sync
// paths accept a contextless reading and store NULL rather than guessing.
// Absence stays absence; it is never widened into a classification.
.refine(
(data) =>
(data.type === "BLOOD_GLUCOSE" && data.glucoseContext !== undefined) ||