feat(security): add second-factor schema and encrypt free-text health notes at rest

Lay the account-security and data-sovereignty substrate for v1.23. No
user-facing behaviour yet — schema, crypto registry, redaction, and the
note-encryption data path only.

Schema + migrations:
- 0199 — User totp_secret_encrypted / totp_confirmed_at / mfa_enforced /
  passkey_upgrade_nudge_dismissed; Session mfa_verified_at / last_active_at;
  Passkey last_used_at. All additive, nullable / defaulted.
- 0200 — mfa_challenges (single-use step-up ticket; only the ticket hash is
  stored, so no session or token is ever minted before the second factor),
  mfa_recovery_codes (Argon2id at rest), webauthn_mfa_credentials (second-factor
  keys, kept separate from passkeys, which remain the passwordless-primary set).
- 0201 — mood_entries.note_encrypted + measurements.notes_encrypted (BYTEA),
  matching the existing free-text encrypted-note convention.

Free-text health notes:
- New writes encrypt the mood / measurement note into the *_encrypted column and
  null the legacy plaintext column; reads prefer the ciphertext and fall back to
  plaintext only for not-yet-migrated rows, via a single read/write helper.
- A boot-time job migrates existing plaintext rows in an idempotent,
  transactional, fail-closed pass (encrypt, then null the plaintext, per row in
  one transaction; safe to re-run; a row with content is never left both-null).
- The legacy plaintext columns are kept this release and dropped in a follow-up
  once the backfill reports zero remaining rows — the same boundary the
  key-rotation playbook uses before dropping a legacy key.

Crypto coverage:
- Register the three new encrypted columns and wire them into the key-rotation
  script so a rotation never orphans them.
- Extend the observability and log redaction denylists so TOTP secrets, MFA /
  recovery / backup codes, and export passphrases never reach a wide event.
This commit is contained in:
Marc-André Bombeck committed 2026-06-27 19:33:10 +02:00
1 parent bed879500f
commit 4d3a02cc1a
33 files changed
+1583 -378

No files matched your search

@@ -0,0 +1,24 @@
-- v1.23 — account-security columns (second-factor substrate).
--
-- Additive, nullable / defaulted columns only — existing rows need no
-- backfill. The TOTP shared secret (`totp_secret_encrypted`) is AES-256-GCM
-- ciphertext like every other `*_encrypted` column (registered in the
-- encrypted-columns registry + the rotation script). `totp_confirmed_at`
-- marks the pending->active promotion; `mfa_enforced` /
-- `passkey_upgrade_nudge_dismissed` are per-account flags later waves read.
--
-- `sessions.mfa_verified_at` is the step-up freshness stamp; `last_active_at`
-- backs the user-facing active-session list. `passkeys.last_used_at` is set on
-- each successful assertion so the management UI can surface "last used".
ALTER TABLE "users"
ADD COLUMN IF NOT EXISTS "totp_secret_encrypted" TEXT,
ADD COLUMN IF NOT EXISTS "totp_confirmed_at" TIMESTAMP(3),
ADD COLUMN IF NOT EXISTS "mfa_enforced" BOOLEAN NOT NULL DEFAULT false,
ADD COLUMN IF NOT EXISTS "passkey_upgrade_nudge_dismissed" BOOLEAN NOT NULL DEFAULT false;
ALTER TABLE "sessions"
ADD COLUMN IF NOT EXISTS "mfa_verified_at" TIMESTAMP(3),
ADD COLUMN IF NOT EXISTS "last_active_at" TIMESTAMP(3);
ALTER TABLE "passkeys"
ADD COLUMN IF NOT EXISTS "last_used_at" TIMESTAMP(3);
@@ -0,0 +1,69 @@
-- v1.23 — second-factor (MFA) tables.
--
-- Three new stores backing the TOTP / WebAuthn-second-factor / recovery-code
-- flows later waves build on. Phase F lands the empty tables only — no
-- behaviour yet.
--
-- mfa_challenges single-use step-up ticket minted after a valid
-- password when the account has a second factor.
-- The "password OK, awaiting factor 2" state lives
-- here, never in a half-built session, so no
-- session/token is issued before factor 2 passes.
-- Only the ticket HASH is stored (ticket_hash).
-- mfa_recovery_codes one row per single-use recovery code, Argon2id-
-- hashed at rest; used_at burns it.
-- webauthn_mfa_credentials second-factor security keys, kept SEPARATE from
-- `passkeys` (the passwordless-primary store) —
-- different semantics, ceremony, management UI.
-- Mirrors the passkeys field encoding.
CREATE TABLE "mfa_challenges" (
"id" TEXT NOT NULL,
"user_id" TEXT NOT NULL,
"kind" TEXT NOT NULL,
"ticket_hash" TEXT NOT NULL,
"attempts" INTEGER NOT NULL DEFAULT 0,
"expires_at" TIMESTAMP(3) NOT NULL,
"consumed_at" TIMESTAMP(3),
"created_at" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
CONSTRAINT "mfa_challenges_pkey" PRIMARY KEY ("id")
);
CREATE UNIQUE INDEX "mfa_challenges_ticket_hash_key" ON "mfa_challenges"("ticket_hash");
CREATE INDEX "mfa_challenges_user_id_idx" ON "mfa_challenges"("user_id");
-- The verify path and the expiry sweep both filter on expires_at.
CREATE INDEX "mfa_challenges_expires_at_idx" ON "mfa_challenges"("expires_at");
CREATE TABLE "mfa_recovery_codes" (
"id" TEXT NOT NULL,
"user_id" TEXT NOT NULL,
"code_hash" TEXT NOT NULL,
"used_at" TIMESTAMP(3),
"created_at" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
CONSTRAINT "mfa_recovery_codes_pkey" PRIMARY KEY ("id")
);
CREATE INDEX "mfa_recovery_codes_user_id_idx" ON "mfa_recovery_codes"("user_id");
CREATE TABLE "webauthn_mfa_credentials" (
"id" TEXT NOT NULL,
"user_id" TEXT NOT NULL,
"name" TEXT NOT NULL DEFAULT 'Security key',
"credential_id" TEXT NOT NULL,
"credential_public_key" BYTEA NOT NULL,
"counter" BIGINT NOT NULL DEFAULT 0,
"transports" TEXT[],
"created_at" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
"last_used_at" TIMESTAMP(3),
CONSTRAINT "webauthn_mfa_credentials_pkey" PRIMARY KEY ("id")
);
CREATE UNIQUE INDEX "webauthn_mfa_credentials_credential_id_key" ON "webauthn_mfa_credentials"("credential_id");
CREATE INDEX "webauthn_mfa_credentials_user_id_idx" ON "webauthn_mfa_credentials"("user_id");
ALTER TABLE "mfa_challenges" ADD CONSTRAINT "mfa_challenges_user_id_fkey" FOREIGN KEY ("user_id") REFERENCES "users"("id") ON DELETE CASCADE ON UPDATE CASCADE;
ALTER TABLE "mfa_recovery_codes" ADD CONSTRAINT "mfa_recovery_codes_user_id_fkey" FOREIGN KEY ("user_id") REFERENCES "users"("id") ON DELETE CASCADE ON UPDATE CASCADE;
ALTER TABLE "webauthn_mfa_credentials" ADD CONSTRAINT "webauthn_mfa_credentials_user_id_fkey" FOREIGN KEY ("user_id") REFERENCES "users"("id") ON DELETE CASCADE ON UPDATE CASCADE;
@@ -0,0 +1,28 @@
-- v1.23 — encrypt free-text health notes at rest.
--
-- Adds the AES-256-GCM ciphertext columns for the two free-text health-note
-- fields that were still plaintext (`mood_entries.note`, `measurements.notes`).
--
-- STORAGE CONVENTION: Bytes (`bytea`), matching the dominant free-text
-- encrypted-note convention already in the schema — illness_day_logs.note_
-- encrypted, cycle_day_logs.notes_encrypted (string-kind), labs.note_encrypted,
-- coach_*.{...}_encrypted are all the shared-codec shape (the `encrypt()`
-- ciphertext string stored UTF-8 as `bytea`). `bytea` is chosen here because
-- the comparable note columns (IllnessDayLog / LabResult) use it and the
-- rotation script's `rotateBytesColumn` already covers that shape with least
-- friction. Both columns are registered in ENCRYPTED_COLUMNS + the rotation
-- script in the same change.
--
-- This migration is ADDITIVE ONLY. The legacy plaintext columns
-- (`mood_entries.note`, `measurements.notes`) are intentionally NOT dropped
-- here: an idempotent, transactional, fail-closed boot-time job encrypts the
-- existing rows (plaintext -> ciphertext, then nulls the plaintext) so the
-- read path can fall back to plaintext for not-yet-migrated rows. Dropping the
-- plaintext columns is a deliberate FOLLOW-UP release, once the backfill
-- reports zero remaining plaintext-but-no-ciphertext rows — the same boundary
-- the encryption-key-rotation playbook uses before dropping a legacy key.
ALTER TABLE "mood_entries"
ADD COLUMN IF NOT EXISTS "note_encrypted" BYTEA;
ALTER TABLE "measurements"
ADD COLUMN IF NOT EXISTS "notes_encrypted" BYTEA;
+430 -327
View File
File diff suppressed because it is too large. Load diff
+18
View File
@@ -193,6 +193,8 @@ async function main() {
"aiOpenaiKeyEncrypted",
"aiOcrKeyEncrypted",
"insuranceNumberEncrypted",
// v1.23 — TOTP shared secret (second factor).
"totpSecretEncrypted",
];
for (const field of userFields) {
results.push(await rotateStringColumn("User", field, prisma.user));
@@ -405,6 +407,22 @@ async function main() {
),
);
// ───── v1.23 free-text health notes (Bytes columns) ─────
// "noteEncrypted" (MoodEntry) + "notesEncrypted" (Measurement). Same shared-
// codec shape as the clinical-spine note columns. NULL on rows whose note is
// still in the legacy plaintext column (pre-backfill) — `rotateBytesColumn`
// skips those.
results.push(
await rotateBytesColumn("MoodEntry", "noteEncrypted", prisma.moodEntry),
);
results.push(
await rotateBytesColumn(
"Measurement",
"notesEncrypted",
prisma.measurement,
),
);
console.log("\n=== Rotation summary ===");
let totalRotated = 0;
let totalErrors = 0;
@@ -36,6 +36,7 @@ import { apiHandler, HttpError, requireAdmin } from "@/lib/api-handler";
import { apiError, apiSuccess, getClientIp } from "@/lib/api-response";
import { auditLog } from "@/lib/auth/audit";
import { decrypt } from "@/lib/crypto";
import { encryptNote } from "@/lib/crypto/note-cipher";
import { defaultUserIdResolver, withIdempotency } from "@/lib/idempotency";
import { annotate } from "@/lib/logging/context";
import {
@@ -330,7 +331,10 @@ const handler = apiHandler(
unit: m.unit,
source: (m.source ?? "MANUAL") as never,
measuredAt: new Date(m.measuredAt),
notes: m.notes ?? null,
// v1.23 — re-encrypt the note from the (decrypted) backup payload
// into the at-rest column; legacy plaintext column nulled.
notes: null,
notesEncrypted: encryptNote(m.notes ?? null),
})),
// Backups can replay the same data — the unique
// (userId, type, measuredAt, source) constraint catches
+4 -1
View File
@@ -21,6 +21,7 @@ import { annotate } from "@/lib/logging/context";
import { auditLog } from "@/lib/auth/audit";
import { apiError, getClientIp } from "@/lib/api-response";
import { checkRateLimit } from "@/lib/rate-limit";
import { readNote } from "@/lib/crypto/note-cipher";
import { BACKUP_SCHEMA_VERSION } from "@/lib/validations/backup";
import { buildCycleBackupSection } from "@/lib/cycle/backup";
import { NextRequest, NextResponse } from "next/server";
@@ -94,7 +95,9 @@ export const GET = apiHandler(async (request: NextRequest) => {
unit: m.unit,
measuredAt: m.measuredAt.toISOString(),
source: m.source,
notes: m.notes,
// v1.23 — decrypt the note for the human-readable backup; an admin
// restore re-encrypts it on re-insert.
notes: readNote(m.notesEncrypted, m.notes),
})),
medications: medications.map((m) => ({
name: m.name,
+11 -6
View File
@@ -23,6 +23,7 @@ import { auditLog } from "@/lib/auth/audit";
import { apiError, getClientIp } from "@/lib/api-response";
import { checkRateLimit } from "@/lib/rate-limit";
import { toCSV, formatMeasurementsForExport } from "@/lib/export";
import { shapeMeasurementNotes } from "@/lib/crypto/note-cipher";
import { resolveUserTimezone } from "@/lib/tz/resolver";
import { loadUserSourcePriority } from "@/lib/rollups/measurement-read";
import { resolveGlucoseUnit } from "@/lib/glucose";
@@ -65,12 +66,16 @@ export const GET = apiHandler(async (request: NextRequest) => {
const glucoseUnit = resolveGlucoseUnit(profile?.glucoseUnit ?? null);
const csv = toCSV(
formatMeasurementsForExport(measurements, userTz, {
granularity,
sleepTz: userTz,
sourcePriorityJson,
glucoseUnit,
}),
formatMeasurementsForExport(
measurements.map(shapeMeasurementNotes),
userTz,
{
granularity,
sleepTz: userTz,
sourcePriorityJson,
glucoseUnit,
},
),
);
await auditLog("user.export.measurements", {
+4 -1
View File
@@ -19,6 +19,7 @@ import { auditLog } from "@/lib/auth/audit";
import { apiError, getClientIp } from "@/lib/api-response";
import { checkRateLimit } from "@/lib/rate-limit";
import { toCSV, formatMoodEntriesForExport } from "@/lib/export";
import { shapeMoodNote } from "@/lib/crypto/note-cipher";
import { resolveUserTimezone } from "@/lib/tz/resolver";
import { NextRequest, NextResponse } from "next/server";
@@ -62,7 +63,9 @@ export const GET = apiHandler(async (request: NextRequest) => {
resolveUserTimezone(user.id),
]);
const csv = toCSV(formatMoodEntriesForExport(entries, userTz));
const csv = toCSV(
formatMoodEntriesForExport(entries.map(shapeMoodNote), userTz),
);
await auditLog("user.export.mood", {
userId: user.id,
+6 -3
View File
@@ -11,6 +11,7 @@ import {
formatIntakeEventsForExport,
formatMoodEntriesForExport,
} from "@/lib/export";
import { shapeMeasurementNotes, shapeMoodNote } from "@/lib/crypto/note-cipher";
import { resolveUserTimezone } from "@/lib/tz/resolver";
import { loadUserSourcePriority } from "@/lib/rollups/measurement-read";
import { NextRequest, NextResponse } from "next/server";
@@ -70,7 +71,8 @@ export const GET = apiHandler(async (request: NextRequest) => {
// Thread the user's source-priority so a multi-source night dedups to
// the same canonical source the UI shows.
const measurementRecords = formatMeasurementsForExport(
measurements,
// v1.23 — decrypt notes before formatting; strip the ciphertext column.
measurements.map(shapeMeasurementNotes),
userTz,
{ sleepTz: userTz, sourcePriorityJson },
);
@@ -110,10 +112,11 @@ export const GET = apiHandler(async (request: NextRequest) => {
where: { userId, deletedAt: null },
orderBy: { moodLoggedAt: "desc" },
});
const decryptedMood = moodEntries.map(shapeMoodNote);
data.moodEntries =
format === "csv"
? toCSV(formatMoodEntriesForExport(moodEntries, userTz))
: formatMoodEntriesForExport(moodEntries, userTz);
? toCSV(formatMoodEntriesForExport(decryptedMood, userTz))
: formatMoodEntriesForExport(decryptedMood, userTz);
}
await auditLog("export.download", {
@@ -18,6 +18,15 @@ vi.mock("@/lib/rate-limit", () => ({
checkRateLimit: vi.fn(),
}));
// v1.23 — deterministic note-cipher so the route test stays isolated from the
// encryption-key env. `enc:<text>` stands in for the AES-256-GCM ciphertext.
vi.mock("@/lib/crypto/note-cipher", () => ({
encryptNote: (s: string | null | undefined) =>
s === null || s === undefined || s.length === 0
? null
: new Uint8Array(Buffer.from(`enc:${s}`, "utf8")),
}));
vi.mock("@/lib/auth/audit", () => ({
auditLog: vi.fn().mockResolvedValue(undefined),
}));
+7 -3
View File
@@ -3,6 +3,7 @@ import { apiHandler, requireAuth } from "@/lib/api-handler";
import { annotate } from "@/lib/logging/context";
import { auditLog } from "@/lib/auth/audit";
import { apiSuccess, apiError, getClientIp } from "@/lib/api-response";
import { encryptNote } from "@/lib/crypto/note-cipher";
import { checkRateLimit } from "@/lib/rate-limit";
import { NextRequest } from "next/server";
@@ -124,7 +125,8 @@ export const POST = apiHandler(async (request: NextRequest) => {
value: m.value,
unit: m.unit,
measuredAt: m.measuredAt,
notes: m.notes ?? null,
notes: null,
notesEncrypted: encryptNote(m.notes ?? null),
glucoseContext:
(m.glucoseContext as GlucoseContext | undefined) ?? null,
},
@@ -136,7 +138,8 @@ export const POST = apiHandler(async (request: NextRequest) => {
source: "IMPORT",
externalId: m.externalId,
measuredAt: m.measuredAt,
notes: m.notes ?? null,
notes: null,
notesEncrypted: encryptNote(m.notes ?? null),
glucoseContext:
(m.glucoseContext as GlucoseContext | undefined) ?? null,
},
@@ -160,7 +163,8 @@ export const POST = apiHandler(async (request: NextRequest) => {
unit: m.unit,
source: "IMPORT",
measuredAt: m.measuredAt,
notes: m.notes ?? null,
notes: null,
notesEncrypted: encryptNote(m.notes ?? null),
glucoseContext:
(m.glucoseContext as GlucoseContext | undefined) ?? null,
},
+7 -3
View File
@@ -18,6 +18,7 @@ import {
glucoseContextEnum,
} from "@/lib/validations/measurement";
import { validateEntryInstant } from "@/lib/validations/entry-instant";
import { encryptNote } from "@/lib/crypto/note-cipher";
import { recomputeUserMoodRollups } from "@/lib/rollups/mood-rollups";
import {
collapseToTypeDayKeys,
@@ -144,7 +145,8 @@ export const POST = apiHandler(async (request: NextRequest) => {
value: m.value,
unit: m.unit,
measuredAt,
notes: m.notes || null,
notes: null,
notesEncrypted: encryptNote(m.notes || null),
glucoseContext: m.glucoseContext ?? null,
},
create: {
@@ -155,7 +157,8 @@ export const POST = apiHandler(async (request: NextRequest) => {
source: "IMPORT",
externalId: m.externalId,
measuredAt,
notes: m.notes || null,
notes: null,
notesEncrypted: encryptNote(m.notes || null),
glucoseContext: m.glucoseContext ?? null,
},
});
@@ -168,7 +171,8 @@ export const POST = apiHandler(async (request: NextRequest) => {
unit: m.unit,
source: "IMPORT",
measuredAt,
notes: m.notes || null,
notes: null,
notesEncrypted: encryptNote(m.notes || null),
glucoseContext: m.glucoseContext ?? null,
},
});
+8 -2
View File
@@ -11,6 +11,7 @@ import {
sanitiseZodIssues,
} from "@/lib/api-response";
import { updateMeasurementSchema } from "@/lib/validations/measurement";
import { encryptNote, shapeMeasurementNotes } from "@/lib/crypto/note-cipher";
import { invalidateUserMeasurements } from "@/lib/cache/invalidate";
import { invalidateStatusInsightsForTypes } from "@/lib/insights/comprehensive-generate";
import { recomputeBucketsForMeasurement } from "@/lib/rollups/measurement-rollups";
@@ -110,7 +111,12 @@ export const PUT = apiHandler(
data: {
...(data.value !== undefined && { value: data.value }),
...(data.measuredAt !== undefined && { measuredAt: data.measuredAt }),
...(data.notes !== undefined && { notes: data.notes }),
// v1.23 — write the note to the encrypted column; null the legacy
// plaintext column. An explicit `null` clears the note.
...(data.notes !== undefined && {
notes: null,
notesEncrypted: encryptNote(data.notes),
}),
},
});
} catch (err) {
@@ -188,7 +194,7 @@ export const PUT = apiHandler(
console.warn("[measurements] status-insight invalidate failed", err);
});
return apiSuccess(measurement);
return apiSuccess(shapeMeasurementNotes(measurement));
},
);
+15 -5
View File
@@ -27,6 +27,7 @@ import {
localDayWindow,
} from "@/lib/measurements/drain-per-sample-cumulative";
import { withIdempotency } from "@/lib/idempotency";
import { encryptNote, shapeMeasurementNotes } from "@/lib/crypto/note-cipher";
import { invalidateUserMeasurements } from "@/lib/cache/invalidate";
import { invalidateStatusInsightsForTypes } from "@/lib/insights/comprehensive-generate";
import { enqueueReminderSatisfy } from "@/lib/jobs/reminder-satisfy";
@@ -491,7 +492,9 @@ export const GET = apiHandler(async (request: NextRequest) => {
annotate({ action: { name: "measurement.list" }, meta: { total, type } });
return apiSuccess({
measurements,
// v1.23 — decrypt `notesEncrypted` onto `notes` and strip the ciphertext
// before it leaves the server (iOS MeasurementWireDTO contract).
measurements: measurements.map(shapeMeasurementNotes),
meta: { total, limit, offset },
});
});
@@ -738,7 +741,10 @@ async function postMeasurement(request: NextRequest) {
unit: getUnitForType(m.type),
source: (m.source ?? "MANUAL") as MeasurementSource,
measuredAt: m.measuredAt,
notes: m.notes ?? null,
// v1.23 — encrypt the note at rest; the legacy plaintext column is
// written null for new rows.
notes: null,
notesEncrypted: encryptNote(m.notes ?? null),
glucoseContext:
(m.glucoseContext as GlucoseContext | undefined) ?? null,
// v1.4.25 W10 reconcile (code-review M4): mirror the
@@ -750,6 +756,8 @@ async function postMeasurement(request: NextRequest) {
),
);
const shapedResults = results.map(shapeMeasurementNotes);
await auditLog("measurement.create.batch", {
userId: user.id,
ipAddress: getClientIp(request),
@@ -823,7 +831,7 @@ async function postMeasurement(request: NextRequest) {
console.warn("[measurements] rollup recompute failed", err);
}
return apiSuccess(results, 201);
return apiSuccess(shapedResults, 201);
}
// Single mode (existing behavior)
@@ -878,7 +886,9 @@ async function postMeasurement(request: NextRequest) {
unit: getUnitForType(type),
source: (source ?? "MANUAL") as MeasurementSource,
measuredAt,
notes: notes ?? null,
// v1.23 — encrypt the note at rest; legacy plaintext column nulled.
notes: null,
notesEncrypted: encryptNote(notes ?? null),
glucoseContext: (glucoseContext as GlucoseContext | undefined) ?? null,
// v1.4.25 W10 reconcile (code-review M4): the deviceType column
// already accepts client metadata from the batch route. Mirror
@@ -963,5 +973,5 @@ async function postMeasurement(request: NextRequest) {
console.warn("[measurements] status-insight invalidate failed", err);
});
return apiSuccess(measurement, 201);
return apiSuccess(shapeMeasurementNotes(measurement), 201);
}
+6 -2
View File
@@ -16,6 +16,7 @@ import { NextRequest } from "next/server";
import { apiHandler, requireAuth } from "@/lib/api-handler";
import { annotate } from "@/lib/logging/context";
import { moodDateKey, DEFAULT_TIMEZONE } from "@/lib/mood/date-key";
import { encryptNote, shapeMoodNote } from "@/lib/crypto/note-cipher";
import { invalidateUserMood } from "@/lib/cache/invalidate";
import { recomputeMoodBucketsForEntry } from "@/lib/rollups/mood-rollups";
import { replaceTagLinks } from "@/lib/mood/tag-links";
@@ -53,7 +54,7 @@ export const GET = apiHandler(
meta: { moodEntryId: id },
});
return apiSuccess({ ...entry, tags: parseTags(entry.tags) });
return apiSuccess({ ...shapeMoodNote(entry), tags: parseTags(entry.tags) });
},
);
@@ -128,7 +129,10 @@ export const PUT = apiHandler(
updateData.tags = data.tags ? JSON.stringify(data.tags) : null;
}
if (data.note !== undefined) {
updateData.note = data.note;
// v1.23 — write to the encrypted column; null the legacy plaintext. An
// explicit `null` clears the note.
updateData.note = null;
updateData.noteEncrypted = encryptNote(data.note);
}
// v1.7.0 sync — mood is last-writer-wins by syncVersion; bump it on
+5 -2
View File
@@ -44,6 +44,7 @@ import {
moodSourceEnum,
} from "@/lib/validations/moodlog";
import { moodDateKey, DEFAULT_TIMEZONE } from "@/lib/mood/date-key";
import { encryptNote } from "@/lib/crypto/note-cipher";
import { invalidateUserMood } from "@/lib/cache/invalidate";
import { recomputeMoodBucketsForEntry } from "@/lib/rollups/mood-rollups";
import { pushMoodEntriesToMoodLog } from "@/lib/moodlog/push";
@@ -238,7 +239,8 @@ async function postBulk(request: NextRequest): Promise<Response> {
mood: entry.mood,
score,
tags: entry.tags ? JSON.stringify(entry.tags) : null,
note: entry.note ?? null,
note: null,
noteEncrypted: encryptNote(entry.note ?? null),
source: resolvedSource,
externalId: entry.externalId ?? null,
moodLoggedAt: entry.moodLoggedAt,
@@ -253,7 +255,8 @@ async function postBulk(request: NextRequest): Promise<Response> {
mood: entry.mood,
score,
tags: entry.tags ? JSON.stringify(entry.tags) : null,
note: entry.note ?? null,
note: null,
noteEncrypted: encryptNote(entry.note ?? null),
...(entry.externalId
? { date, moodLoggedAt: entry.moodLoggedAt }
: {}),
+12 -6
View File
@@ -18,6 +18,7 @@ import { Prisma } from "@/generated/prisma/client";
import { apiHandler, requireAuth } from "@/lib/api-handler";
import { annotate } from "@/lib/logging/context";
import { withIdempotency } from "@/lib/idempotency";
import { encryptNote, readNote, shapeMoodNote } from "@/lib/crypto/note-cipher";
import { moodDateKey, DEFAULT_TIMEZONE } from "@/lib/mood/date-key";
import { invalidateUserMood } from "@/lib/cache/invalidate";
import { recomputeMoodBucketsForEntry } from "@/lib/rollups/mood-rollups";
@@ -119,7 +120,8 @@ export const GET = apiHandler(async (request: NextRequest) => {
});
const entriesWithParsedTags = entries.map(({ tagLinks, ...e }) => ({
...e,
// v1.23 — decrypt `noteEncrypted` onto `note`, strip the ciphertext.
...shapeMoodNote(e),
tags: parseTags(e.tags),
// v1.8.5 — flat list of binary structured-tag keys attached to the
// entry (rated factors are surfaced separately below).
@@ -234,7 +236,8 @@ async function postMoodEntry(request: NextRequest) {
mood,
score,
tags: tags ? JSON.stringify(tags) : null,
note: note ?? null,
note: null,
noteEncrypted: encryptNote(note ?? null),
source: resolvedSource,
externalId,
moodLoggedAt,
@@ -245,7 +248,8 @@ async function postMoodEntry(request: NextRequest) {
mood,
score,
tags: tags ? JSON.stringify(tags) : null,
note: note ?? null,
note: null,
noteEncrypted: encryptNote(note ?? null),
moodLoggedAt,
},
})
@@ -257,7 +261,8 @@ async function postMoodEntry(request: NextRequest) {
mood,
score,
tags: tags ? JSON.stringify(tags) : null,
note: note ?? null,
note: null,
noteEncrypted: encryptNote(note ?? null),
source: resolvedSource,
moodLoggedAt,
},
@@ -347,7 +352,7 @@ async function postMoodEntry(request: NextRequest) {
date: entry.date,
moodLoggedAt: entry.moodLoggedAt,
mood: entry.mood,
note: entry.note ?? null,
note: readNote(entry.noteEncrypted, entry.note),
tags: entry.tags,
source: entry.source,
},
@@ -362,7 +367,8 @@ async function postMoodEntry(request: NextRequest) {
return apiSuccess(
{
...entry,
// v1.23 — decrypt `noteEncrypted` onto `note`, strip the ciphertext.
...shapeMoodNote(entry),
tags: parseTags(entry.tags),
// v1.8.5 — surface the persisted structured-tag keys so a client
// hydrating from the create response renders the tag set without a
+5 -2
View File
@@ -50,6 +50,7 @@ import { apiHandler, requireAuth } from "@/lib/api-handler";
import { apiSuccess, apiError } from "@/lib/api-response";
import { annotate } from "@/lib/logging/context";
import { checkRateLimit } from "@/lib/rate-limit";
import { readNote } from "@/lib/crypto/note-cipher";
import { TOMBSTONE_RETENTION_DAYS } from "@/lib/auth/native-client";
import {
decodeCursor,
@@ -249,6 +250,7 @@ export const GET = apiHandler(async (request: NextRequest) => {
measuredAt: true,
source: true,
notes: true,
notesEncrypted: true,
syncVersion: true,
deletedAt: true,
updatedAt: true,
@@ -265,6 +267,7 @@ export const GET = apiHandler(async (request: NextRequest) => {
score: true,
tags: true,
note: true,
noteEncrypted: true,
moodLoggedAt: true,
source: true,
syncVersion: true,
@@ -336,7 +339,7 @@ export const GET = apiHandler(async (request: NextRequest) => {
unit: row.unit,
measuredAt: row.measuredAt.toISOString(),
source: row.source,
notes: row.notes,
notes: readNote(row.notesEncrypted, row.notes),
syncVersion: row.syncVersion,
updatedAt: row.updatedAt.toISOString(),
});
@@ -365,7 +368,7 @@ export const GET = apiHandler(async (request: NextRequest) => {
mood: row.mood,
score: row.score,
tags: row.tags,
note: row.note,
note: readNote(row.noteEncrypted, row.note),
moodLoggedAt: row.moodLoggedAt.toISOString(),
source: row.source,
syncVersion: row.syncVersion,
@@ -0,0 +1,104 @@
import { describe, it, expect, beforeEach, afterEach, vi } from "vitest";
import {
encryptNote,
readNote,
shapeMeasurementNotes,
shapeMoodNote,
} from "../note-cipher";
// A 32-byte key as 64 hex chars (the legacy single-key path).
const KEY_A = "a".repeat(64);
const KEY_B = "b".repeat(64);
beforeEach(() => {
// Isolate from any ambient ENCRYPTION_KEYS map; drive the legacy single key.
vi.stubEnv("ENCRYPTION_KEYS", "");
vi.stubEnv("ENCRYPTION_ACTIVE_KEY_ID", "");
vi.stubEnv("ENCRYPTION_KEY", KEY_A);
});
afterEach(() => {
vi.unstubAllEnvs();
});
describe("encryptNote / readNote", () => {
it("round-trips a note through encrypt -> decrypt", () => {
const cipher = encryptNote("felt dizzy after the second dose");
expect(cipher).toBeInstanceOf(Uint8Array);
expect(readNote(cipher, null)).toBe("felt dizzy after the second dose");
});
it("preserves Unicode (umlauts / emoji) verbatim", () => {
const cipher = encryptNote("Schwindel nach 2. Dosis 🤕 — Nürnberg");
expect(readNote(cipher, null)).toBe(
"Schwindel nach 2. Dosis 🤕 — Nürnberg",
);
});
it("stores nothing for an absent / empty note", () => {
expect(encryptNote(null)).toBeNull();
expect(encryptNote(undefined)).toBeNull();
expect(encryptNote("")).toBeNull();
});
it("falls back to the legacy plaintext column ONLY when there is no ciphertext", () => {
// A not-yet-backfilled row: plaintext present, ciphertext null.
expect(readNote(null, "legacy plaintext note")).toBe(
"legacy plaintext note",
);
expect(readNote(new Uint8Array(0), "legacy plaintext note")).toBe(
"legacy plaintext note",
);
expect(readNote(null, null)).toBeNull();
});
it("prefers the ciphertext over a stale plaintext value", () => {
const cipher = encryptNote("the real note");
// Even if a stale plaintext lingers, the ciphertext wins.
expect(readNote(cipher, "stale plaintext")).toBe("the real note");
});
it("is fail-closed: a ciphertext that cannot be decrypted throws (never leaks the fallback)", () => {
const cipher = encryptNote("secret note");
// Rotate the key out from under the ciphertext: same key id, wrong bytes.
vi.stubEnv("ENCRYPTION_KEY", KEY_B);
expect(() =>
readNote(cipher, "fallback that must NOT be returned"),
).toThrow();
});
});
describe("shapeMeasurementNotes / shapeMoodNote", () => {
it("decrypts onto `notes` and strips the ciphertext column", () => {
const notesEncrypted = encryptNote("after lunch");
const row = {
id: "m1",
value: 5.4,
notes: null,
notesEncrypted,
};
const shaped = shapeMeasurementNotes(row);
expect(shaped.notes).toBe("after lunch");
expect("notesEncrypted" in shaped).toBe(false);
expect(shaped.id).toBe("m1");
expect(shaped.value).toBe(5.4);
});
it("decrypts onto `note` and strips the ciphertext column", () => {
const noteEncrypted = encryptNote("rough night");
const row = { id: "x1", mood: "OKAY", note: null, noteEncrypted };
const shaped = shapeMoodNote(row);
expect(shaped.note).toBe("rough night");
expect("noteEncrypted" in shaped).toBe(false);
expect(shaped.mood).toBe("OKAY");
});
it("surfaces the legacy plaintext for a not-yet-backfilled measurement row", () => {
const shaped = shapeMeasurementNotes({
id: "m2",
notes: "legacy plaintext",
notesEncrypted: null,
});
expect(shaped.notes).toBe("legacy plaintext");
});
});
+9
View File
@@ -71,6 +71,9 @@ export const ENCRYPTED_COLUMNS: readonly EncryptedColumn[] = [
// v1.22 (#90) — dedicated document-scan (Lab-OCR) provider key.
{ model: "User", field: "aiOcrKeyEncrypted", kind: "string" },
{ model: "User", field: "insuranceNumberEncrypted", kind: "string" },
// v1.23 — TOTP shared secret (second factor). String ciphertext like the
// other User credential columns.
{ model: "User", field: "totpSecretEncrypted", kind: "string" },
// ───── OAuth token tables (`*_token`-convention columns) ─────
{ model: "WithingsConnection", field: "accessToken", kind: "string" },
@@ -141,6 +144,12 @@ export const ENCRYPTED_COLUMNS: readonly EncryptedColumn[] = [
// ───── v1.19.0 ECG waveform (Bytes column) ─────
{ model: "EcgRecording", field: "waveformEncrypted", kind: "bytes" },
// ───── v1.23 free-text health notes (Bytes columns) ─────
// Free-text mood diary + per-measurement notes. Same class as the
// IllnessDayLog / LabResult note columns above; Bytes via the shared codec.
{ model: "MoodEntry", field: "noteEncrypted", kind: "bytes" },
{ model: "Measurement", field: "notesEncrypted", kind: "bytes" },
] as const;
/** Stable `Model.field` key for a registry entry. */
+75
View File
@@ -0,0 +1,75 @@
/**
* Free-text health-note ↔ encrypted-column helpers (v1.23).
*
* `MoodEntry.note` and `Measurement.notes` are migrating from plaintext to the
* AES-256-GCM `Bytes` columns `noteEncrypted` / `notesEncrypted`. These helpers
* are the single read/write boundary every call site goes through so the
* encrypt-on-write + prefer-ciphertext-fallback-plaintext contract stays
* consistent.
*
* Storage shape is the shared Bytes codec (`@/lib/ai/coach/bytes-codec`) used
* by every other free-text encrypted-note column (IllnessDayLog, LabResult,
* Coach memory) — the `encrypt()` ciphertext string stored UTF-8 as `bytea`.
*
* FAIL-CLOSED: `readNote` decrypts a present ciphertext and throws on a bad key
* id / malformed payload rather than silently returning plaintext — the same
* posture as the rest of the crypto layer. The plaintext fallback only fires
* when there is NO ciphertext (a legacy row not yet covered by the backfill).
*/
import { decryptFromBytes, encryptToBytes } from "@/lib/ai/coach/bytes-codec";
/**
* Encrypt a free-text note for the `Bytes` ciphertext column. Returns `null`
* for a null / undefined / empty value so an absent note stores nothing (and
* the row reads back as "no note"). Whitespace is preserved verbatim — callers
* that want trimming trim before calling.
*/
export function encryptNote(
plaintext: string | null | undefined,
): Uint8Array<ArrayBuffer> | null {
if (plaintext === null || plaintext === undefined || plaintext.length === 0) {
return null;
}
return encryptToBytes(plaintext);
}
/**
* Read a free-text note, preferring the encrypted column. Falls back to the
* legacy plaintext column ONLY when there is no ciphertext (a row the backfill
* has not yet migrated). A present ciphertext is always decrypted; a bad key
* id / malformed payload throws (fail-closed) rather than leaking the plaintext
* fallback.
*/
export function readNote(
ciphertext: Uint8Array | null | undefined,
plaintextFallback: string | null | undefined,
): string | null {
if (ciphertext && ciphertext.byteLength > 0) {
return decryptFromBytes(ciphertext);
}
return plaintextFallback ?? null;
}
/**
* Shape a measurement row for a response/export: surface the decrypted note on
* `notes` and strip the raw `notesEncrypted` ciphertext so it never leaves the
* server. Keeps every other column untouched.
*/
export function shapeMeasurementNotes<
T extends { notes: string | null; notesEncrypted: Uint8Array | null },
>(row: T): Omit<T, "notesEncrypted"> & { notes: string | null } {
const { notesEncrypted, ...rest } = row;
return { ...rest, notes: readNote(notesEncrypted, rest.notes) };
}
/**
* Shape a mood row for a response/export: surface the decrypted note on `note`
* and strip the raw `noteEncrypted` ciphertext. Keeps every other column
* untouched.
*/
export function shapeMoodNote<
T extends { note: string | null; noteEncrypted: Uint8Array | null },
>(row: T): Omit<T, "noteEncrypted"> & { note: string | null } {
const { noteEncrypted, ...rest } = row;
return { ...rest, note: readNote(noteEncrypted, rest.note) };
}
@@ -0,0 +1,181 @@
import { describe, it, expect, beforeEach, afterEach, vi } from "vitest";
import { readNote } from "@/lib/crypto/note-cipher";
// ── in-memory store backing a minimal prisma mock ──────────────────────────
interface MeasRow {
id: string;
userId: string;
notes: string | null;
notesEncrypted: Uint8Array | null;
}
interface MoodRow {
id: string;
userId: string;
note: string | null;
noteEncrypted: Uint8Array | null;
}
const store = vi.hoisted(() => ({
measurements: [] as MeasRow[],
mood: [] as MoodRow[],
}));
// A `notes`/`note` write that is meant to throw on encrypt (simulates a
// fail-closed key error) so we can assert no data loss.
const fail = vi.hoisted(() => ({ encryptThrows: false }));
vi.mock("@/lib/jobs/boss-instance", () => ({ getGlobalBoss: () => null }));
vi.mock("@/lib/logging/context", () => ({ annotate: vi.fn() }));
vi.mock("@/lib/db", () => {
// `delegates` is fully typed before `$transaction` closes over it, so the
// interactive-transaction mock has no self-referential `typeof prisma`.
const delegates = {
measurement: {
findMany: async (args: { where: { userId: string }; take: number }) =>
store.measurements
.filter(
(r) =>
r.userId === args.where.userId &&
r.notes !== null &&
r.notesEncrypted === null,
)
.slice(0, args.take)
.map((r) => ({ id: r.id })),
findUnique: async (args: { where: { id: string } }) => {
const r = store.measurements.find((x) => x.id === args.where.id);
return r ? { notes: r.notes, notesEncrypted: r.notesEncrypted } : null;
},
update: async (args: {
where: { id: string };
data: { notes: string | null; notesEncrypted: Uint8Array | null };
}) => {
const r = store.measurements.find((x) => x.id === args.where.id)!;
r.notes = args.data.notes;
r.notesEncrypted = args.data.notesEncrypted;
return r;
},
},
moodEntry: {
findMany: async (args: { where: { userId: string }; take: number }) =>
store.mood
.filter(
(r) =>
r.userId === args.where.userId &&
r.note !== null &&
r.noteEncrypted === null,
)
.slice(0, args.take)
.map((r) => ({ id: r.id })),
findUnique: async (args: { where: { id: string } }) => {
const r = store.mood.find((x) => x.id === args.where.id);
return r ? { note: r.note, noteEncrypted: r.noteEncrypted } : null;
},
update: async (args: {
where: { id: string };
data: { note: string | null; noteEncrypted: Uint8Array | null };
}) => {
const r = store.mood.find((x) => x.id === args.where.id)!;
r.note = args.data.note;
r.noteEncrypted = args.data.noteEncrypted;
return r;
},
},
};
const prisma = {
...delegates,
$transaction: async (fn: (tx: typeof delegates) => unknown) =>
fn(delegates),
};
return { prisma };
});
import { runNoteEncryptionBackfillForUser } from "@/lib/jobs/note-encryption-backfill";
const KEY = "a".repeat(64);
beforeEach(() => {
vi.stubEnv("ENCRYPTION_KEYS", "");
vi.stubEnv("ENCRYPTION_ACTIVE_KEY_ID", "");
vi.stubEnv("ENCRYPTION_KEY", KEY);
store.measurements = [
{ id: "m1", userId: "u1", notes: "felt dizzy", notesEncrypted: null },
{ id: "m2", userId: "u1", notes: null, notesEncrypted: null }, // no note
{
id: "m3",
userId: "u1",
notes: null,
notesEncrypted: new Uint8Array(0), // already migrated-ish; left alone
},
];
store.mood = [
{ id: "x1", userId: "u1", note: "rough night", noteEncrypted: null },
{ id: "x2", userId: "u2", note: "other user", noteEncrypted: null },
];
fail.encryptThrows = false;
});
afterEach(() => {
vi.unstubAllEnvs();
});
describe("runNoteEncryptionBackfillForUser", () => {
it("encrypts the plaintext note, nulls the plaintext, and round-trips", async () => {
const summary = await runNoteEncryptionBackfillForUser("u1");
expect(summary.measurementsMigrated).toBe(1);
expect(summary.moodEntriesMigrated).toBe(1);
const m1 = store.measurements.find((r) => r.id === "m1")!;
expect(m1.notes).toBeNull();
expect(m1.notesEncrypted).not.toBeNull();
// No data loss: the ciphertext decrypts back to the original.
expect(readNote(m1.notesEncrypted, null)).toBe("felt dizzy");
const x1 = store.mood.find((r) => r.id === "x1")!;
expect(x1.note).toBeNull();
expect(readNote(x1.noteEncrypted, null)).toBe("rough night");
});
it("leaves note-less rows untouched and never writes a both-null content row", async () => {
await runNoteEncryptionBackfillForUser("u1");
const m2 = store.measurements.find((r) => r.id === "m2")!;
expect(m2.notes).toBeNull();
expect(m2.notesEncrypted).toBeNull(); // never had content -> nothing written
});
it("does not touch another user's rows", async () => {
await runNoteEncryptionBackfillForUser("u1");
const x2 = store.mood.find((r) => r.id === "x2")!;
expect(x2.note).toBe("other user");
expect(x2.noteEncrypted).toBeNull();
});
it("is idempotent: a second run migrates zero rows and keeps one ciphertext", async () => {
await runNoteEncryptionBackfillForUser("u1");
const m1First = store.measurements.find(
(r) => r.id === "m1",
)!.notesEncrypted;
const second = await runNoteEncryptionBackfillForUser("u1");
expect(second.measurementsMigrated).toBe(0);
expect(second.moodEntriesMigrated).toBe(0);
const m1 = store.measurements.find((r) => r.id === "m1")!;
// Unchanged ciphertext, plaintext still null, value still recoverable.
expect(m1.notesEncrypted).toBe(m1First);
expect(m1.notes).toBeNull();
expect(readNote(m1.notesEncrypted, null)).toBe("felt dizzy");
});
it("is fail-closed: a key error leaves the plaintext row intact (no data loss)", async () => {
// Unset every key so encryptNote throws inside the per-row transaction.
vi.stubEnv("ENCRYPTION_KEY", "");
vi.stubEnv("ENCRYPTION_KEYS", "");
await expect(runNoteEncryptionBackfillForUser("u1")).rejects.toThrow();
const m1 = store.measurements.find((r) => r.id === "m1")!;
// The transaction rolled back: plaintext preserved, no ciphertext written.
expect(m1.notes).toBe("felt dizzy");
expect(m1.notesEncrypted).toBeNull();
});
});
+237
View File
@@ -0,0 +1,237 @@
/**
* v1.23 — boot-time converging backfill that migrates the two free-text
* health-note columns from plaintext to AES-256-GCM at rest:
* `MoodEntry.note` -> `MoodEntry.noteEncrypted` and
* `Measurement.notes` -> `Measurement.notesEncrypted`.
*
* Modelled on `mean-consolidation.ts` / `rollup-full-backfill`: a discovery
* query enqueues one job per user still holding an un-migrated row, the
* per-user handler walks that user's rows, and the pass is idempotent across
* reboots — once a row is migrated it drops off the discovery + candidate sets.
*
* DATA-LOSS SAFETY (the property the security review must confirm):
* - Per row, the encrypt-then-null happens in a SINGLE interactive
* transaction guarded by a re-read inside the tx, so the plaintext column is
* only nulled AFTER the ciphertext is committed in the same atomic unit. A
* row that had content is never left both-null.
* - FAIL-CLOSED: `encryptNote` throws on a missing / malformed key. The throw
* aborts the transaction (the plaintext row is untouched) and propagates so
* pg-boss retries — the backfill never silently drops a note or writes a
* plaintext-shaped value into the ciphertext column.
* - IDEMPOTENT: the per-row guard (`notesEncrypted IS NULL AND notes != null`)
* is re-checked inside the tx, so a re-run — or two workers racing — migrates
* a row at most once; a second pass migrates zero rows.
*
* The legacy plaintext columns are intentionally NOT dropped in this release.
* That is a deliberate FOLLOW-UP release boundary, once this backfill reports
* zero remaining un-migrated rows on every instance — the same discipline the
* encryption-key-rotation playbook uses before dropping a legacy key.
*
* The queue name MUST be registered in the maintenance registrar
* (`src/lib/jobs/reminder/register-maintenance.ts`) so pg-boss provisions it at
* boot; an unregistered queue silently never drains.
*/
import { prisma } from "@/lib/db";
import { annotate } from "@/lib/logging/context";
import { getGlobalBoss } from "@/lib/jobs/boss-instance";
import { encryptNote } from "@/lib/crypto/note-cipher";
export const NOTE_ENCRYPTION_BACKFILL_QUEUE = "note-encryption-backfill";
/**
* Serial concurrency — the populator walks a user's plaintext-note rows and
* writes one transaction per row; concurrency-1 keeps it off the request pool,
* matching the consolidation / rollup-backfill convention.
*/
export const NOTE_ENCRYPTION_BACKFILL_CONCURRENCY = 1;
/** How many candidate rows to pull per page within a user's pass. */
const PAGE_SIZE = 200;
export interface NoteEncryptionBackfillPayload {
/** Absent on the daily discovery tick; present for a per-user job. */
userId?: string;
enqueuedAt?: string;
}
export interface NoteEncryptionBackfillSummary {
measurementsMigrated: number;
moodEntriesMigrated: number;
}
/**
* Migrate one measurement row's plaintext `notes` into `notesEncrypted` and
* null the plaintext — atomically and idempotently. Returns true if it
* migrated the row, false if a concurrent pass already did.
*/
async function migrateMeasurementRow(id: string): Promise<boolean> {
return prisma.$transaction(async (tx) => {
const fresh = await tx.measurement.findUnique({
where: { id },
select: { notes: true, notesEncrypted: true },
});
// Already migrated (ciphertext present) or nothing to migrate.
if (!fresh || fresh.notesEncrypted || fresh.notes === null) return false;
// FAIL-CLOSED: a key error throws here and rolls the tx back, leaving the
// plaintext row untouched.
const encrypted = encryptNote(fresh.notes);
await tx.measurement.update({
where: { id },
data: { notesEncrypted: encrypted, notes: null },
});
return true;
});
}
/**
* Migrate one mood row's plaintext `note` into `noteEncrypted` and null the
* plaintext — atomically and idempotently.
*/
async function migrateMoodRow(id: string): Promise<boolean> {
return prisma.$transaction(async (tx) => {
const fresh = await tx.moodEntry.findUnique({
where: { id },
select: { note: true, noteEncrypted: true },
});
if (!fresh || fresh.noteEncrypted || fresh.note === null) return false;
const encrypted = encryptNote(fresh.note);
await tx.moodEntry.update({
where: { id },
data: { noteEncrypted: encrypted, note: null },
});
return true;
});
}
/**
* Per-user queue handler. Walks every un-migrated measurement + mood note for
* one account and migrates each in its own transaction. Safe to re-run.
*/
export async function runNoteEncryptionBackfillForUser(
userId: string,
): Promise<NoteEncryptionBackfillSummary> {
let measurementsMigrated = 0;
let moodEntriesMigrated = 0;
// Measurements — page through candidates until the predicate is empty.
// Each migration nulls `notes`, so a migrated row drops out of the next
// `notes: { not: null }, notesEncrypted: null` page.
for (;;) {
const rows = await prisma.measurement.findMany({
where: { userId, notes: { not: null }, notesEncrypted: null },
select: { id: true },
take: PAGE_SIZE,
});
if (rows.length === 0) break;
let migratedThisPage = 0;
for (const { id } of rows) {
if (await migrateMeasurementRow(id)) {
measurementsMigrated += 1;
migratedThisPage += 1;
}
}
// Guard against an unexpected non-converging page (every row already
// migrated by a racing worker) so the loop always terminates.
if (migratedThisPage === 0) break;
}
for (;;) {
const rows = await prisma.moodEntry.findMany({
where: { userId, note: { not: null }, noteEncrypted: null },
select: { id: true },
take: PAGE_SIZE,
});
if (rows.length === 0) break;
let migratedThisPage = 0;
for (const { id } of rows) {
if (await migrateMoodRow(id)) {
moodEntriesMigrated += 1;
migratedThisPage += 1;
}
}
if (migratedThisPage === 0) break;
}
annotate({
action: {
name: "note.encryption.backfill",
details: {
measurements_migrated: measurementsMigrated,
mood_entries_migrated: moodEntriesMigrated,
},
},
});
return { measurementsMigrated, moodEntriesMigrated };
}
/**
* Boot-time discovery. Finds every user holding at least one row whose note is
* still in the legacy plaintext column (no ciphertext yet) across either table
* and enqueues one backfill job per account.
*
* Idempotent across reboots: once a user's rows are migrated, the
* `*Encrypted IS NULL AND * != null` predicate drops them from the discovery
* set. pg-boss `singletonKey` coalesces duplicate sends. Best-effort: errors
* come back through the result value so worker boot never fails on a miss.
*/
export async function enqueueBootTimeNoteEncryptionBackfill(): Promise<{
enqueued: number;
skipped: number;
error: string | null;
}> {
const boss = getGlobalBoss();
if (!boss) {
return { enqueued: 0, skipped: 0, error: null };
}
try {
const [measurementUsers, moodUsers] = await Promise.all([
prisma.measurement.findMany({
where: { notes: { not: null }, notesEncrypted: null },
select: { userId: true },
distinct: ["userId"],
}),
prisma.moodEntry.findMany({
where: { note: { not: null }, noteEncrypted: null },
select: { userId: true },
distinct: ["userId"],
}),
]);
const userIds = new Set<string>();
for (const { userId } of measurementUsers) userIds.add(userId);
for (const { userId } of moodUsers) userIds.add(userId);
if (userIds.size === 0) {
return { enqueued: 0, skipped: 0, error: null };
}
let enqueued = 0;
let skipped = 0;
for (const userId of userIds) {
const payload: NoteEncryptionBackfillPayload = {
userId,
enqueuedAt: new Date().toISOString(),
};
const jobId = await boss.send(NOTE_ENCRYPTION_BACKFILL_QUEUE, payload, {
retryLimit: 5,
retryDelay: 60,
retryBackoff: true,
singletonKey: `note-encryption-backfill|${userId}`,
});
if (jobId) {
enqueued += 1;
} else {
skipped += 1;
}
}
return { enqueued, skipped, error: null };
} catch (err) {
return {
enqueued: 0,
skipped: 0,
error: err instanceof Error ? err.message : String(err),
};
}
}
+4 -1
View File
@@ -7,6 +7,7 @@
import { type Job } from "pg-boss";
import { recordError } from "@/lib/jobs/worker-status";
import { encrypt } from "@/lib/crypto";
import { readNote } from "@/lib/crypto/note-cipher";
import { withBackgroundEvent } from "@/lib/logging/background";
import { buildCycleBackupSection } from "@/lib/cycle/backup";
import { runOffhostBackup } from "@/lib/jobs/offhost-backup";
@@ -94,7 +95,9 @@ export async function handleDataBackup(jobs: Job<DataBackupPayload>[]) {
unit: m.unit,
measuredAt: m.measuredAt.toISOString(),
source: m.source,
notes: m.notes,
// v1.23 — decrypt into the (whole-blob-encrypted) backup payload
// so an admin restore re-encrypts on re-insert.
notes: readNote(m.notesEncrypted, m.notes),
})),
medications: medications.map((m) => ({
name: m.name,
@@ -55,6 +55,13 @@ import {
RESTORE_DRILL_CRON,
RESTORE_DRILL_QUEUE,
} from "@/lib/jobs/restore-drill";
import {
NOTE_ENCRYPTION_BACKFILL_QUEUE,
NOTE_ENCRYPTION_BACKFILL_CONCURRENCY,
runNoteEncryptionBackfillForUser,
enqueueBootTimeNoteEncryptionBackfill,
type NoteEncryptionBackfillPayload,
} from "@/lib/jobs/note-encryption-backfill";
import { recordError } from "@/lib/jobs/worker-status";
import { workerLog } from "./shared";
import { createAndSchedule, type ScheduleEntry } from "./registrar-shared";
@@ -168,6 +175,13 @@ const MEASUREMENT_TOMBSTONE_CLEANUP_CRON = "40 3 * * *";
const COACH_MESSAGE_CLEANUP_QUEUE = "coach-message-cleanup";
const COACH_MESSAGE_CLEANUP_CRON = "50 3 * * *";
// v1.23 — converging backfill that migrates the free-text health-note columns
// (mood + measurement) from plaintext to AES-256-GCM at rest. Boot discovery
// enqueues one per-user job; a daily 03:55 Europe/Berlin discovery tick (empty
// payload) re-fans for any rows that landed between worker reboots, so the
// migration converges without a restart. Idempotent + fail-closed per row.
const NOTE_ENCRYPTION_BACKFILL_CRON = "55 3 * * *";
const allQueues = [
DATA_BACKUP_QUEUE,
@@ -209,6 +223,11 @@ const allQueues = [
// the daily schedule silently no-ops and the encrypted coach_messages
// table grows unbounded.
COACH_MESSAGE_CLEANUP_QUEUE,
// v1.23 — free-text health-note encryption backfill. Boot discovery + a
// daily discovery cron enqueue one per-user job per account still holding a
// plaintext note; without this entry pg-boss never provisions the queue and
// both the boot enqueue and the cron silently no-op.
NOTE_ENCRYPTION_BACKFILL_QUEUE,
];
const schedules: ScheduleEntry[] = [
@@ -260,6 +279,10 @@ const schedules: ScheduleEntry[] = [
[MEASUREMENT_TOMBSTONE_CLEANUP_QUEUE, MEASUREMENT_TOMBSTONE_CLEANUP_CRON],
// v1.18.7 — daily 03:50 Europe/Berlin prune for stale Coach history.
[COACH_MESSAGE_CLEANUP_QUEUE, COACH_MESSAGE_CLEANUP_CRON],
// v1.23 — daily 03:55 Europe/Berlin note-encryption backfill discovery.
// The empty cron payload (no `userId`) is the handler's signal to fan out
// one per-user job per account still holding a plaintext note.
[NOTE_ENCRYPTION_BACKFILL_QUEUE, NOTE_ENCRYPTION_BACKFILL_CRON],
];
/**
@@ -427,6 +450,46 @@ export async function registerMaintenanceQueues(
},
);
// v1.23 — note-encryption backfill worker. The boot enqueue helper (and the
// daily 03:55 discovery tick) send one job per user still holding a plaintext
// note; this handler migrates that user's rows to the encrypted columns. An
// empty-userId payload is the daily discovery tick and re-fans the per-user
// jobs (singletonKey-coalesced, identical to the boot pass). Serial
// concurrency so the migration never crowds the request pool.
await boss.work<Partial<NoteEncryptionBackfillPayload>>(
NOTE_ENCRYPTION_BACKFILL_QUEUE,
{ localConcurrency: NOTE_ENCRYPTION_BACKFILL_CONCURRENCY },
async (jobs) => {
for (const job of jobs) {
const { userId } = job.data;
if (!userId) {
const result = await enqueueBootTimeNoteEncryptionBackfill();
workerLog(
"info",
`[note-encryption-backfill] daily discovery enqueued=${result.enqueued} skipped=${result.skipped}${result.error ? ` error=${result.error}` : ""}`,
);
continue;
}
try {
const { measurementsMigrated, moodEntriesMigrated } =
await runNoteEncryptionBackfillForUser(userId);
workerLog(
"info",
`[note-encryption-backfill] user=${userId} measurements=${measurementsMigrated} moodEntries=${moodEntriesMigrated}`,
);
} catch (err) {
recordError();
workerLog(
"error",
`[note-encryption-backfill] user=${userId} failed`,
err,
);
throw err;
}
}
},
);
return allQueues;
}
@@ -456,4 +519,30 @@ export async function enqueueMaintenanceBootDiscovery(): Promise<void> {
err,
);
}
// v1.23 — note-encryption backfill. Finds every user still holding a
// plaintext mood/measurement note (no ciphertext yet) and enqueues one
// migration job per account. Idempotent across reboots: a migrated row nulls
// its plaintext column and drops off the discovery predicate.
try {
const { enqueued, skipped, error } =
await enqueueBootTimeNoteEncryptionBackfill();
if (error) {
workerLog(
"error",
`[note-encryption-backfill] boot discovery failed: ${error}`,
);
} else {
workerLog(
"info",
`[note-encryption-backfill] boot discovery: enqueued=${enqueued} skipped=${skipped}`,
);
}
} catch (err) {
workerLog(
"error",
"[note-encryption-backfill] boot discovery threw an unexpected error",
err,
);
}
}
+5 -1
View File
@@ -96,8 +96,12 @@ export function redactSecrets(input: string): string {
// v1.12.2 — `ticket` (the one-time WHOOP connect ticket rides in the
// `GET /api/whoop/connect?ticket=…` URL): scrub it so the opaque value
// never lands in `http.path`/error strings reaching Loki/Glitchtip.
// v1.23 — second-factor / account-recovery / encrypted-export material.
// `totp`/`mfa` tickets-or-codes, `backup_code` / `recovery_code`, and the
// `passphrase` that derives the encrypted-export key must never land in
// `http.path` / error strings reaching Loki / Glitchtip.
.replace(
/([?&])(secret|code|token|ticket|api[_-]?key|insurance(?:number)?|kvnr)=[^&\s]+/gi,
/([?&])(secret|code|token|ticket|api[_-]?key|insurance(?:number)?|kvnr|totp|mfa|backup[_-]?code|recovery[_-]?code|passphrase)=[^&\s]+/gi,
"$1$2=[REDACTED]",
)
);
@@ -28,6 +28,23 @@ vi.mock("@/lib/logging/context", () => ({
getEvent: () => ({ addMeta: vi.fn() }),
}));
// v1.23 — deterministic note-cipher so the unit test stays isolated from the
// encryption-key env (mirrors the illness day-log-write test's bytes-codec
// mock). `enc:<text>` stands in for the AES-256-GCM ciphertext.
vi.mock("@/lib/crypto/note-cipher", () => ({
encryptNote: (s: string | null | undefined) =>
s === null || s === undefined || s.length === 0
? null
: new Uint8Array(Buffer.from(`enc:${s}`, "utf8")),
readNote: (
cipher: Uint8Array | null | undefined,
plain: string | null | undefined,
) =>
cipher && cipher.byteLength > 0
? Buffer.from(cipher).toString("utf8").replace(/^enc:/, "")
: (plain ?? null),
}));
import {
logTelegramMood,
attachTelegramMoodNote,
@@ -106,7 +123,11 @@ describe("attachTelegramMoodNote", () => {
userId: "user-1",
deletedAt: null,
});
expect(arg.data.note).toBe("rough night");
// v1.23 — the note now rides the encrypted column; the plaintext is nulled.
expect(arg.data.note).toBeNull();
expect(Buffer.from(arg.data.noteEncrypted).toString("utf8")).toBe(
"enc:rough night",
);
expect(invalidateUserMood).toHaveBeenCalledWith("user-1");
});
+15 -4
View File
@@ -18,6 +18,7 @@ import { prisma as defaultPrisma } from "@/lib/db";
import { MOOD_ENUM_BY_SCORE } from "@/lib/mood/labels";
import { getScoreForMood } from "@/lib/validations/moodlog";
import { moodDateKey, DEFAULT_TIMEZONE } from "@/lib/mood/date-key";
import { encryptNote, readNote } from "@/lib/crypto/note-cipher";
import { invalidateUserMood } from "@/lib/cache/invalidate";
import { recomputeMoodBucketsForEntry } from "@/lib/rollups/mood-rollups";
import { pushMoodEntriesToMoodLog } from "@/lib/moodlog/push";
@@ -79,12 +80,21 @@ export async function logTelegramMood(input: {
tz,
mood,
score,
note: input.note ?? null,
// v1.23 — encrypt the note at rest; legacy plaintext column nulled.
note: null,
noteEncrypted: encryptNote(input.note ?? null),
source: "TELEGRAM",
externalId: input.externalId,
moodLoggedAt,
},
select: { id: true, date: true, mood: true, note: true, tags: true },
select: {
id: true,
date: true,
mood: true,
note: true,
noteEncrypted: true,
tags: true,
},
});
invalidateUserMood(input.userId);
@@ -105,7 +115,7 @@ export async function logTelegramMood(input: {
date: entry.date,
moodLoggedAt,
mood: entry.mood,
note: entry.note ?? null,
note: readNote(entry.noteEncrypted, entry.note),
tags: entry.tags,
source: "TELEGRAM",
},
@@ -134,7 +144,8 @@ export async function attachTelegramMoodNote(input: {
userId: input.userId,
deletedAt: null,
},
data: { note },
// v1.23 — encrypt the attached note at rest; legacy plaintext column nulled.
data: { note: null, noteEncrypted: encryptNote(note) },
});
if (updated.count === 0) return false;
invalidateUserMood(input.userId);
+10
View File
@@ -43,6 +43,16 @@ export const SENSITIVE_KEY_PATTERNS: readonly RegExp[] = [
/otp/i,
/recovery/i,
/dsn/i,
// v1.23 — second-factor / account-recovery material. `/otp/i` already
// catches `totp`/`otpauth` and `/recovery/i` already catches
// `recoveryCode`, but the explicit `/totp/i`, `/mfa/i`, `/backup.?code/i`,
// `/recovery.?code/i` keep MFA secrets / recovery codes / the TOTP
// `otpauth://` URI off every wide-event excerpt regardless of field
// spelling. `passphrase` (the encrypted-export key) is covered above.
/totp/i,
/mfa/i,
/backup.?code/i,
/recovery.?code/i,
// v1.7.0 — health-insurance identity. `insuranceNumber` / `kvnr`
// (the German statutory-insurance number) is encrypted at rest and is
// absent from every current wide-event excerpt, but the field name
@@ -1,15 +1,19 @@
/**
* v1.4.30 — `MoodEntry.note` column round-trip integration.
* v1.4.30 / v1.23 — mood-note round-trip integration.
*
* v1.23 moved the free-text note to AES-256-GCM at rest: new writes land in
* `noteEncrypted` (Bytes) and the legacy plaintext `note` column is nulled.
*
* Asserts:
* - POST /api/mood-entries accepts the new `note` field
* - PUT /api/mood-entries/[id] updates `note` (and accepts null to clear)
* - POST /api/mood-entries/bulk persists `note` per entry
* - POST /api/mood-entries encrypts the `note` (plaintext column nulled)
* - PUT /api/mood-entries/[id] re-encrypts `note` (and accepts null to clear)
* - POST /api/mood-entries/bulk encrypts `note` per entry
* - The Zod cap (500 chars) rejects oversize prose
*/
import { NextRequest } from "next/server";
import { beforeEach, describe, expect, it, vi } from "vitest";
import { readNote } from "@/lib/crypto/note-cipher";
import { cookieJar } from "./mock-next-headers";
import { getPrismaClient, truncateAllTables } from "./setup";
@@ -87,10 +91,18 @@ describe("MoodEntry.note round-trip (real Postgres)", () => {
}),
);
expect(res.status).toBe(201);
// The response carries the decrypted note back to the client.
const body = (await res.json()) as { data: { note: string | null } };
expect(body.data.note).toBe("Long run before breakfast.");
const stored = await getPrismaClient().moodEntry.findFirst({
where: { userId: TEST_USER_ID },
});
expect(stored?.note).toBe("Long run before breakfast.");
// At rest: plaintext column nulled, ciphertext present + decrypts back.
expect(stored?.note).toBeNull();
expect(stored?.noteEncrypted).not.toBeNull();
expect(readNote(stored?.noteEncrypted ?? null, null)).toBe(
"Long run before breakfast.",
);
});
it("PUT /api/mood-entries/[id] updates `note` and accepts null to clear it", async () => {
@@ -115,7 +127,8 @@ describe("MoodEntry.note round-trip (real Postgres)", () => {
const after1 = await getPrismaClient().moodEntry.findUnique({
where: { id: created.id },
});
expect(after1?.note).toBe("revised note");
expect(after1?.note).toBeNull();
expect(readNote(after1?.noteEncrypted ?? null, null)).toBe("revised note");
const res2 = await PUT(
putRequest(`/api/mood-entries/${created.id}`, { note: null }),
@@ -125,7 +138,9 @@ describe("MoodEntry.note round-trip (real Postgres)", () => {
const after2 = await getPrismaClient().moodEntry.findUnique({
where: { id: created.id },
});
// Cleared: both the plaintext and the ciphertext column are null.
expect(after2?.note).toBeNull();
expect(after2?.noteEncrypted).toBeNull();
});
it("POST /api/mood-entries/bulk persists `note` per entry", async () => {
@@ -151,7 +166,12 @@ describe("MoodEntry.note round-trip (real Postgres)", () => {
where: { userId: TEST_USER_ID },
orderBy: { moodLoggedAt: "asc" },
});
expect(stored.map((r) => r.note)).toEqual(["Morning", "Afternoon dip"]);
// Plaintext column nulled; ciphertext decrypts back to each note.
expect(stored.map((r) => r.note)).toEqual([null, null]);
expect(stored.map((r) => readNote(r.noteEncrypted, null))).toEqual([
"Morning",
"Afternoon dip",
]);
});
it("rejects an oversize note (501 chars) with 422", async () => {
@@ -0,0 +1,124 @@
/**
* v1.23 — free-text health-note encryption backfill (real Postgres).
*
* Seeds legacy plaintext rows (note in the plaintext column, ciphertext null),
* runs the per-user backfill, and asserts:
* - the plaintext is encrypted into `*Encrypted` and the plaintext column is
* nulled, in one transaction (no row left both-null when it had content);
* - the ciphertext decrypts back to the original (no data loss);
* - a note-less row is untouched;
* - a second run is a no-op (idempotent).
*/
import { beforeEach, describe, expect, it } from "vitest";
import { readNote } from "@/lib/crypto/note-cipher";
import { runNoteEncryptionBackfillForUser } from "@/lib/jobs/note-encryption-backfill";
import { getPrismaClient, truncateAllTables } from "./setup";
const TEST_USER_ID = "user-note-backfill";
beforeEach(async () => {
await truncateAllTables(getPrismaClient());
await getPrismaClient().user.create({
data: {
id: TEST_USER_ID,
username: "note-backfill",
email: "note-backfill@example.test",
timezone: "Europe/Berlin",
},
});
});
describe("note-encryption backfill (real Postgres)", () => {
it("migrates legacy plaintext notes to ciphertext and nulls the plaintext", async () => {
const prisma = getPrismaClient();
// Legacy rows: plaintext present, ciphertext null (the pre-v1.23 shape).
const withNote = await prisma.measurement.create({
data: {
userId: TEST_USER_ID,
type: "WEIGHT",
value: 80,
unit: "kg",
measuredAt: new Date("2026-05-16T08:00:00.000Z"),
notes: "felt dizzy after",
},
});
const noNote = await prisma.measurement.create({
data: {
userId: TEST_USER_ID,
type: "WEIGHT",
value: 81,
unit: "kg",
measuredAt: new Date("2026-05-17T08:00:00.000Z"),
notes: null,
},
});
const mood = await prisma.moodEntry.create({
data: {
userId: TEST_USER_ID,
date: "2026-05-16",
tz: "Europe/Berlin",
mood: "OKAY",
score: 3,
moodLoggedAt: new Date("2026-05-16T08:00:00.000Z"),
note: "rough night",
},
});
const summary = await runNoteEncryptionBackfillForUser(TEST_USER_ID);
expect(summary.measurementsMigrated).toBe(1);
expect(summary.moodEntriesMigrated).toBe(1);
const m = await prisma.measurement.findUnique({
where: { id: withNote.id },
});
expect(m?.notes).toBeNull();
expect(m?.notesEncrypted).not.toBeNull();
expect(readNote(m?.notesEncrypted ?? null, null)).toBe("felt dizzy after");
const untouched = await prisma.measurement.findUnique({
where: { id: noNote.id },
});
expect(untouched?.notes).toBeNull();
expect(untouched?.notesEncrypted).toBeNull();
const x = await prisma.moodEntry.findUnique({ where: { id: mood.id } });
expect(x?.note).toBeNull();
expect(readNote(x?.noteEncrypted ?? null, null)).toBe("rough night");
});
it("is idempotent: a second run migrates zero rows and preserves the data", async () => {
const prisma = getPrismaClient();
const row = await prisma.measurement.create({
data: {
userId: TEST_USER_ID,
type: "WEIGHT",
value: 80,
unit: "kg",
measuredAt: new Date("2026-05-16T08:00:00.000Z"),
notes: "side effect note",
},
});
await runNoteEncryptionBackfillForUser(TEST_USER_ID);
const first = await prisma.measurement.findUnique({
where: { id: row.id },
});
const firstCipher = Buffer.from(first?.notesEncrypted ?? new Uint8Array());
const second = await runNoteEncryptionBackfillForUser(TEST_USER_ID);
expect(second.measurementsMigrated).toBe(0);
const after = await prisma.measurement.findUnique({
where: { id: row.id },
});
// Same ciphertext, plaintext still null, note still recoverable.
expect(Buffer.from(after?.notesEncrypted ?? new Uint8Array())).toEqual(
firstCipher,
);
expect(after?.notes).toBeNull();
expect(readNote(after?.notesEncrypted ?? null, null)).toBe(
"side effect note",
);
});
});
+8
View File
@@ -16,6 +16,14 @@
import { prisma } from "@/lib/db";
import type { PrismaClient } from "@/generated/prisma/client";
// v1.23 — every integration test imports this module. The free-text health-note
// columns (mood + measurement) are now AES-256-GCM at rest, so any test that
// writes/reads a note needs an encryption key. Crypto reads the key lazily on
// first `encrypt()`, so a `??=` default here covers every test without touching
// each file (individual tests that set their own key still win — `??=`).
process.env.ENCRYPTION_KEY ??=
"0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef";
/**
* The application's Prisma singleton. Tests use this so any code
* imported via `await import("@/lib/...")` shares the exact same client