mirror of
https://github.com/MBombeck/HealthLog.git
synced 2026-11-08 20:42:43 +00:00
fix(privacy): encrypt Coach conversation titles and custom-metric notes at rest
The title of a Coach conversation is the first 80 characters of the opening message, and the note on a custom-metric reading is free text; both were stored readable while every other note was encrypted. Migration 0357 adds coach_conversations.title_encrypted and custom_metric_entries.note_encrypted (the title loses its NOT NULL) plus partial indexes on the backfill's discovery predicate. Every writer now stores ciphertext only: conversation create, proactive nudge, reminder surfacing, rename, entry create and edit, and both restore formats. Readers prefer the ciphertext and fall back to the old column for rows not yet migrated; a ciphertext that does not open throws instead of falling back. The history title search decrypts in memory, since SQL can no longer match the title. free-text-encryption-backfill is a boot-time, idempotent pg-boss job (staggered past the startup storm, daily discovery tick) that seals each remaining readable value and nulls it in one transaction per row. A pass that sees a row twice stops with an error instead of looping. Both columns are registered for key rotation (registry, CLI script, and the in-app rotation that walks the registry). A disaster-recovery backup carries ciphertext only, sealing a not-yet-migrated row on the way out; a portable export carries readable text. The old columns are marked for removal in a later release.
This commit is contained in:
1 parent
4af6b87a2a
commit
f7b3ce6ee0
28 files changed
+1735
-83
No files matched your search
@@ -0,0 +1,32 @@
|
||||
-- Two free-text columns were still stored readable: the title of a Coach
|
||||
-- conversation (the first 80 characters of the opening message, or the name a
|
||||
-- person gave the thread) and the note on a custom-metric reading. Both move
|
||||
-- to AES-256-GCM at rest, in the same Bytes codec every other note uses.
|
||||
--
|
||||
-- SQL cannot encrypt, so this migration only adds the ciphertext columns and
|
||||
-- the discovery indexes. The boot-time `free-text-encryption-backfill` job
|
||||
-- encrypts the existing rows one at a time and nulls the readable value in the
|
||||
-- same transaction. Readers prefer the ciphertext and fall back to the old
|
||||
-- column until the backfill has reached a row. The old columns are dropped in
|
||||
-- a later release, once the backfill reports nothing left on every instance.
|
||||
--
|
||||
-- Additive only: no table rewrite, no data change. The title loses its NOT
|
||||
-- NULL because every new title is written to the ciphertext column and the
|
||||
-- readable one stays empty.
|
||||
|
||||
ALTER TABLE "coach_conversations" ADD COLUMN "title_encrypted" BYTEA;
|
||||
ALTER TABLE "coach_conversations" ALTER COLUMN "title" DROP NOT NULL;
|
||||
|
||||
ALTER TABLE "custom_metric_entries" ADD COLUMN "note_encrypted" BYTEA;
|
||||
|
||||
-- Partial indexes on exactly the backfill's discovery predicate, so the boot
|
||||
-- scan is index-only and shrinks to nothing as the backfill converges (the
|
||||
-- 0243 precedent). Non-concurrent: Prisma Migrate runs the file in a
|
||||
-- transaction, and the build lands in the pre-traffic window at boot.
|
||||
CREATE INDEX IF NOT EXISTS "coach_conversations_title_backfill_idx"
|
||||
ON "coach_conversations" ("user_id")
|
||||
WHERE "title" IS NOT NULL;
|
||||
|
||||
CREATE INDEX IF NOT EXISTS "custom_metric_entries_note_backfill_idx"
|
||||
ON "custom_metric_entries" ("user_id")
|
||||
WHERE "note" IS NOT NULL;
|
||||
+25
-1
@@ -5789,7 +5789,15 @@ model CustomMetricEntry {
|
||||
/// Unit snapshot from the metric at write time.
|
||||
unit String
|
||||
measuredAt DateTime @map("measured_at")
|
||||
/// @internal: legacy plaintext note, read only as a fallback for rows the
|
||||
/// free-text encryption backfill has not reached yet (v1.39.3). Every write
|
||||
/// since then goes to `noteEncrypted` and leaves this NULL; the backfill
|
||||
/// nulls it on the old rows. DROP this column in a later release, once the
|
||||
/// backfill reports zero remaining rows on every instance.
|
||||
note String?
|
||||
/// v1.39.3 — AES-256-GCM ciphertext of the note. Same Bytes codec as the
|
||||
/// measurement and mood notes.
|
||||
noteEncrypted Bytes? @map("note_encrypted")
|
||||
createdAt DateTime @default(now()) @map("created_at")
|
||||
/// v1.37.20 — soft-delete tombstone, peer parity with Measurement /
|
||||
/// MoodEntry (every other entry surface deletes recoverably with an undo
|
||||
@@ -5800,6 +5808,10 @@ model CustomMetricEntry {
|
||||
metric CustomMetric @relation(fields: [customMetricId], references: [id], onDelete: Cascade)
|
||||
|
||||
@@index([userId, customMetricId, measuredAt])
|
||||
// `custom_metric_entries_note_backfill_idx` is a partial index over
|
||||
// `(user_id) WHERE note IS NOT NULL` — see
|
||||
// migration 0357 for the free-text encryption boot-discovery scan. Raw SQL
|
||||
// only (partial predicate not expressible in Prisma).
|
||||
@@map("custom_metric_entries")
|
||||
}
|
||||
|
||||
@@ -6050,7 +6062,15 @@ model RecommendationFeedback {
|
||||
model CoachConversation {
|
||||
id String @id @default(cuid())
|
||||
userId String @map("user_id")
|
||||
title String
|
||||
/// @internal: legacy plaintext title, read only as a fallback for rows the
|
||||
/// free-text encryption backfill has not reached yet (v1.39.3). Every write
|
||||
/// since then goes to `titleEncrypted` and leaves this NULL; the backfill
|
||||
/// nulls it on the old rows. DROP this column in a later release, once the
|
||||
/// backfill reports zero remaining rows on every instance.
|
||||
title String?
|
||||
/// v1.39.3 — AES-256-GCM ciphertext of the title (the first 80 characters of
|
||||
/// the opening Coach message, or the rename). Same Bytes codec as the notes.
|
||||
titleEncrypted Bytes? @map("title_encrypted")
|
||||
createdAt DateTime @default(now()) @map("created_at")
|
||||
updatedAt DateTime @updatedAt @map("updated_at")
|
||||
|
||||
@@ -6084,6 +6104,10 @@ model CoachConversation {
|
||||
// and the tool-route send path filters on the flag. Replaces the retired
|
||||
// [userId, documentId, updatedAt] composite.
|
||||
@@index([userId, documentScoped, updatedAt])
|
||||
// `coach_conversations_title_backfill_idx` is a partial index over
|
||||
// `(user_id) WHERE title IS NOT NULL` — see
|
||||
// migration 0357 for the free-text encryption boot-discovery scan. Raw SQL
|
||||
// only (partial predicate not expressible in Prisma).
|
||||
@@map("coach_conversations")
|
||||
}
|
||||
|
||||
|
||||
@@ -408,7 +408,7 @@ async function main() {
|
||||
}
|
||||
|
||||
// ───── Coach (Bytes columns) ─────
|
||||
// "encryptedContent" "summaryEncrypted" "factEncrypted"
|
||||
// "encryptedContent" "summaryEncrypted" "titleEncrypted" "factEncrypted"
|
||||
results.push(
|
||||
await rotateBytesColumn(
|
||||
"CoachMessage",
|
||||
@@ -423,6 +423,16 @@ async function main() {
|
||||
prisma.coachConversation,
|
||||
),
|
||||
);
|
||||
// v1.39.3 — the conversation title. NULL on a row the free-text backfill
|
||||
// has not reached yet (its title is still in the readable column), which
|
||||
// `rotateBytesColumn` skips.
|
||||
results.push(
|
||||
await rotateBytesColumn(
|
||||
"CoachConversation",
|
||||
"titleEncrypted",
|
||||
prisma.coachConversation,
|
||||
),
|
||||
);
|
||||
results.push(
|
||||
await rotateBytesColumn("CoachFact", "factEncrypted", prisma.coachFact),
|
||||
);
|
||||
@@ -681,6 +691,17 @@ async function main() {
|
||||
),
|
||||
);
|
||||
|
||||
// ───── v1.39.3 custom-metric reading note (Bytes column) ─────
|
||||
// NULL on a reading without a note, and on a row the free-text backfill has
|
||||
// not reached yet; `rotateBytesColumn` skips both.
|
||||
results.push(
|
||||
await rotateBytesColumn(
|
||||
"CustomMetricEntry",
|
||||
"noteEncrypted",
|
||||
prisma.customMetricEntry,
|
||||
),
|
||||
);
|
||||
|
||||
// ───── v1.38.0 dose free text (Bytes column) ─────
|
||||
// The note on an administered dose — a reaction, a sore arm. Always
|
||||
// encrypted on write, so every non-null row rotates.
|
||||
|
||||
@@ -12,6 +12,7 @@ import {
|
||||
import { auditLog } from "@/lib/auth/audit";
|
||||
import { invalidateUserCorrelationPatterns } from "@/lib/cache/invalidate";
|
||||
import { serialiseCustomMetricEntry } from "@/lib/custom-metrics/custom-metric-store";
|
||||
import { encryptNote } from "@/lib/crypto/note-cipher";
|
||||
import { prisma } from "@/lib/db";
|
||||
import { annotate } from "@/lib/logging/context";
|
||||
import { updateCustomMetricEntrySchema } from "@/lib/validations/custom-metrics";
|
||||
@@ -83,7 +84,13 @@ export const PATCH = apiHandler(
|
||||
const data: Record<string, unknown> = {};
|
||||
if (d.value !== undefined) data.value = d.value;
|
||||
if (d.measuredAt !== undefined) data.measuredAt = d.measuredAt;
|
||||
if (d.note !== undefined) data.note = d.note;
|
||||
// v1.39.3 — the note is stored encrypted. Any note write (a new value or
|
||||
// an explicit clear) also nulls the legacy readable column, so an edited
|
||||
// row never keeps its old text beside the new ciphertext.
|
||||
if (d.note !== undefined) {
|
||||
data.noteEncrypted = encryptNote(d.note);
|
||||
data.note = null;
|
||||
}
|
||||
|
||||
const updated = await prisma.customMetricEntry.update({
|
||||
where: { id: entryId },
|
||||
|
||||
@@ -0,0 +1,238 @@
|
||||
import { describe, it, expect, vi, beforeEach } from "vitest";
|
||||
import { NextRequest } from "next/server";
|
||||
|
||||
vi.mock("@/lib/rate-limit", async (importOriginal) => ({
|
||||
...(await importOriginal<typeof import("@/lib/rate-limit")>()),
|
||||
// The shared single-record write ceiling. Allowed here: this file is about
|
||||
// what the handler does with a body it accepted, not about the bucket.
|
||||
checkRecordWriteRateLimit: vi.fn(async () => ({
|
||||
allowed: true,
|
||||
limit: 300,
|
||||
remaining: 299,
|
||||
resetAt: Date.now() + 60_000,
|
||||
})),
|
||||
}));
|
||||
vi.mock("@/lib/cache/invalidate", () => ({
|
||||
invalidateUserCorrelationPatterns: vi.fn(),
|
||||
}));
|
||||
vi.mock("@/lib/db", () => ({
|
||||
prisma: {
|
||||
customMetric: { findFirst: vi.fn() },
|
||||
customMetricEntry: {
|
||||
findMany: vi.fn(),
|
||||
count: vi.fn(),
|
||||
create: vi.fn(),
|
||||
findFirst: vi.fn(),
|
||||
update: vi.fn(),
|
||||
},
|
||||
auditLog: { create: vi.fn().mockResolvedValue({}) },
|
||||
},
|
||||
}));
|
||||
|
||||
vi.mock("@/lib/auth/session", () => ({ getSession: vi.fn() }));
|
||||
|
||||
vi.mock("@/lib/auth/audit", () => ({
|
||||
auditLog: vi.fn().mockResolvedValue(undefined),
|
||||
}));
|
||||
|
||||
vi.mock("@/lib/logging/transports", () => ({ emitIfSampled: vi.fn() }));
|
||||
|
||||
vi.mock("@/lib/db-compat", () => ({
|
||||
ensureDbCompatibility: vi.fn().mockResolvedValue(undefined),
|
||||
}));
|
||||
|
||||
vi.mock("@/lib/idempotency", () => ({
|
||||
withIdempotency: (fn: unknown) => fn,
|
||||
}));
|
||||
|
||||
vi.mock("next/headers", () => ({
|
||||
headers: vi.fn(async () => ({ get: () => null })),
|
||||
cookies: vi.fn(async () => ({
|
||||
get: () => undefined,
|
||||
set: () => {},
|
||||
delete: () => {},
|
||||
})),
|
||||
}));
|
||||
|
||||
import { GET, POST } from "../route";
|
||||
import { PATCH } from "../[entryId]/route";
|
||||
import { getSession } from "@/lib/auth/session";
|
||||
import { prisma } from "@/lib/db";
|
||||
import { decryptFromBytes, encryptToBytes } from "@/lib/ai/coach/bytes-codec";
|
||||
|
||||
/**
|
||||
* v1.39.3 — the note on a custom-metric reading is encrypted at rest. A write
|
||||
* stores ciphertext and leaves the readable column empty; a read opens the
|
||||
* ciphertext and falls back to the readable column only for a row the backfill
|
||||
* has not reached.
|
||||
*/
|
||||
|
||||
process.env.ENCRYPTION_KEY ??=
|
||||
"0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef";
|
||||
|
||||
const SESSION_OK = {
|
||||
session: { id: "sess-1", expiresAt: new Date(Date.now() + 3_600_000) },
|
||||
user: { id: "user-1", username: "u", role: "USER" as const, locale: "en" },
|
||||
};
|
||||
|
||||
const ENTRY = {
|
||||
id: "e-1",
|
||||
userId: "user-1",
|
||||
customMetricId: "cm-1",
|
||||
value: 42,
|
||||
unit: "kg",
|
||||
measuredAt: new Date("2026-06-10T00:00:00.000Z"),
|
||||
note: null as string | null,
|
||||
noteEncrypted: null as Uint8Array | null,
|
||||
createdAt: new Date("2026-06-10T00:00:00.000Z"),
|
||||
deletedAt: null,
|
||||
};
|
||||
|
||||
function jsonRequest(url: string, method: string, body: unknown) {
|
||||
return new NextRequest(url, {
|
||||
method,
|
||||
body: JSON.stringify(body),
|
||||
headers: { "content-type": "application/json" },
|
||||
});
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
vi.resetAllMocks();
|
||||
vi.mocked(prisma.auditLog.create).mockResolvedValue({} as never);
|
||||
vi.mocked(getSession).mockResolvedValue(SESSION_OK as never);
|
||||
});
|
||||
|
||||
describe("custom-metric reading note at rest", () => {
|
||||
it("POST stores the note as ciphertext and answers with the readable note", async () => {
|
||||
vi.mocked(prisma.customMetric.findFirst).mockResolvedValue({
|
||||
id: "cm-1",
|
||||
unit: "kg",
|
||||
} as never);
|
||||
vi.mocked(prisma.customMetricEntry.create).mockImplementation(
|
||||
(async (args: { data: Record<string, unknown> }) => ({
|
||||
...ENTRY,
|
||||
...args.data,
|
||||
})) as never,
|
||||
);
|
||||
|
||||
const res = await POST(
|
||||
jsonRequest("http://localhost/api/custom-metrics/cm-1/entries", "POST", {
|
||||
value: 42,
|
||||
measuredAt: "2026-06-10T00:00:00.000Z",
|
||||
note: "after the long ride",
|
||||
}),
|
||||
{ params: Promise.resolve({ id: "cm-1" }) },
|
||||
);
|
||||
expect(res.status).toBe(201);
|
||||
const data = vi.mocked(prisma.customMetricEntry.create).mock.calls[0][0]
|
||||
.data as Record<string, unknown>;
|
||||
expect(data).not.toHaveProperty("note");
|
||||
expect(decryptFromBytes(data.noteEncrypted as Uint8Array)).toBe(
|
||||
"after the long ride",
|
||||
);
|
||||
const body = await res.json();
|
||||
expect(body.data.note).toBe("after the long ride");
|
||||
expect(body.data).not.toHaveProperty("noteEncrypted");
|
||||
});
|
||||
|
||||
it("GET opens sealed notes and still shows a legacy readable one", async () => {
|
||||
vi.mocked(prisma.customMetric.findFirst).mockResolvedValue({
|
||||
id: "cm-1",
|
||||
} as never);
|
||||
vi.mocked(prisma.customMetricEntry.findMany).mockResolvedValue([
|
||||
{ ...ENTRY, id: "e-1", noteEncrypted: encryptToBytes("sealed note") },
|
||||
{ ...ENTRY, id: "e-2", note: "legacy note" },
|
||||
{ ...ENTRY, id: "e-3" },
|
||||
] as never);
|
||||
vi.mocked(prisma.customMetricEntry.count).mockResolvedValue(3 as never);
|
||||
|
||||
const res = await GET(
|
||||
new NextRequest("http://localhost/api/custom-metrics/cm-1/entries"),
|
||||
{ params: Promise.resolve({ id: "cm-1" }) },
|
||||
);
|
||||
expect(res.status).toBe(200);
|
||||
const body = await res.json();
|
||||
expect(
|
||||
body.data.entries.map((e: { note: string | null }) => e.note),
|
||||
).toEqual(["sealed note", "legacy note", null]);
|
||||
for (const entry of body.data.entries) {
|
||||
expect(entry).not.toHaveProperty("noteEncrypted");
|
||||
}
|
||||
});
|
||||
|
||||
it("PATCH seals a new note and clears the legacy readable column", async () => {
|
||||
vi.mocked(prisma.customMetricEntry.findFirst).mockResolvedValue({
|
||||
...ENTRY,
|
||||
note: "old readable note",
|
||||
} as never);
|
||||
vi.mocked(prisma.customMetricEntry.update).mockImplementation(
|
||||
(async (args: { data: Record<string, unknown> }) => ({
|
||||
...ENTRY,
|
||||
...args.data,
|
||||
})) as never,
|
||||
);
|
||||
|
||||
const res = await PATCH(
|
||||
jsonRequest(
|
||||
"http://localhost/api/custom-metrics/cm-1/entries/e-1",
|
||||
"PATCH",
|
||||
{ note: "edited" },
|
||||
),
|
||||
{ params: Promise.resolve({ id: "cm-1", entryId: "e-1" }) },
|
||||
);
|
||||
expect(res.status).toBe(200);
|
||||
const data = vi.mocked(prisma.customMetricEntry.update).mock.calls[0][0]
|
||||
.data as Record<string, unknown>;
|
||||
expect(data.note).toBeNull();
|
||||
expect(decryptFromBytes(data.noteEncrypted as Uint8Array)).toBe("edited");
|
||||
expect((await res.json()).data.note).toBe("edited");
|
||||
});
|
||||
|
||||
it("PATCH with an explicit null clears both columns", async () => {
|
||||
vi.mocked(prisma.customMetricEntry.findFirst).mockResolvedValue({
|
||||
...ENTRY,
|
||||
noteEncrypted: encryptToBytes("to be cleared"),
|
||||
} as never);
|
||||
vi.mocked(prisma.customMetricEntry.update).mockImplementation(
|
||||
(async (args: { data: Record<string, unknown> }) => ({
|
||||
...ENTRY,
|
||||
...args.data,
|
||||
})) as never,
|
||||
);
|
||||
|
||||
const res = await PATCH(
|
||||
jsonRequest(
|
||||
"http://localhost/api/custom-metrics/cm-1/entries/e-1",
|
||||
"PATCH",
|
||||
{ note: null },
|
||||
),
|
||||
{ params: Promise.resolve({ id: "cm-1", entryId: "e-1" }) },
|
||||
);
|
||||
expect(res.status).toBe(200);
|
||||
const data = vi.mocked(prisma.customMetricEntry.update).mock.calls[0][0]
|
||||
.data as Record<string, unknown>;
|
||||
expect(data).toMatchObject({ note: null, noteEncrypted: null });
|
||||
expect((await res.json()).data.note).toBeNull();
|
||||
});
|
||||
|
||||
it("PATCH without a note leaves both note columns untouched", async () => {
|
||||
vi.mocked(prisma.customMetricEntry.findFirst).mockResolvedValue(
|
||||
ENTRY as never,
|
||||
);
|
||||
vi.mocked(prisma.customMetricEntry.update).mockResolvedValue(
|
||||
ENTRY as never,
|
||||
);
|
||||
await PATCH(
|
||||
jsonRequest(
|
||||
"http://localhost/api/custom-metrics/cm-1/entries/e-1",
|
||||
"PATCH",
|
||||
{ value: 43 },
|
||||
),
|
||||
{ params: Promise.resolve({ id: "cm-1", entryId: "e-1" }) },
|
||||
);
|
||||
const data = vi.mocked(prisma.customMetricEntry.update).mock.calls[0][0]
|
||||
.data as Record<string, unknown>;
|
||||
expect(data).not.toHaveProperty("note");
|
||||
expect(data).not.toHaveProperty("noteEncrypted");
|
||||
});
|
||||
});
|
||||
@@ -13,6 +13,7 @@ import { checkRecordWriteRateLimit } from "@/lib/rate-limit";
|
||||
import { auditLog } from "@/lib/auth/audit";
|
||||
import { invalidateUserCorrelationPatterns } from "@/lib/cache/invalidate";
|
||||
import { serialiseCustomMetricEntry } from "@/lib/custom-metrics/custom-metric-store";
|
||||
import { encryptNote } from "@/lib/crypto/note-cipher";
|
||||
import { prisma } from "@/lib/db";
|
||||
import { withIdempotency } from "@/lib/idempotency";
|
||||
import { annotate } from "@/lib/logging/context";
|
||||
@@ -161,7 +162,8 @@ async function postCustomMetricEntry(
|
||||
value,
|
||||
unit: metric.unit,
|
||||
measuredAt,
|
||||
note: note ?? null,
|
||||
// v1.39.3 — the note is stored encrypted; the readable column stays NULL.
|
||||
noteEncrypted: encryptNote(note),
|
||||
},
|
||||
});
|
||||
|
||||
|
||||
@@ -104,15 +104,12 @@ describe("listConversations — filter + DTO", () => {
|
||||
});
|
||||
|
||||
// v1.30.2 (QoL H1) — server-side title search for the full-history rail.
|
||||
it("applies a case-insensitive title-contains filter when q is given", async () => {
|
||||
it("keeps the title out of SQL when q is given (it is encrypted) and stays owner-scoped", async () => {
|
||||
findMany.mockResolvedValue([]);
|
||||
await listConversations({ userId: "user-1", q: "blood pressure" });
|
||||
const where = findMany.mock.calls[0][0].where;
|
||||
expect(where.userId).toBe("user-1");
|
||||
expect(where.title).toEqual({
|
||||
contains: "blood pressure",
|
||||
mode: "insensitive",
|
||||
});
|
||||
expect("title" in where).toBe(false);
|
||||
});
|
||||
|
||||
it("trims q and omits the title filter for an empty/whitespace-only query", async () => {
|
||||
|
||||
@@ -1,5 +1,8 @@
|
||||
import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||
|
||||
process.env.ENCRYPTION_KEY ??=
|
||||
"0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef";
|
||||
|
||||
const { updateMany } = vi.hoisted(() => ({ updateMany: vi.fn() }));
|
||||
|
||||
vi.mock("@/lib/db", () => ({
|
||||
@@ -9,6 +12,7 @@ vi.mock("@/lib/db", () => ({
|
||||
}));
|
||||
|
||||
import { renameConversation } from "../persistence";
|
||||
import { decryptFromBytes } from "../bytes-codec";
|
||||
|
||||
beforeEach(() => {
|
||||
updateMany.mockReset();
|
||||
@@ -21,10 +25,12 @@ describe("renameConversation", () => {
|
||||
await expect(
|
||||
renameConversation("owner-1", "conversation-1", "Renamed"),
|
||||
).resolves.toEqual({ id: "conversation-1", title: "Renamed" });
|
||||
expect(updateMany).toHaveBeenCalledWith({
|
||||
where: { id: "conversation-1", userId: "owner-1" },
|
||||
data: { title: "Renamed" },
|
||||
});
|
||||
const arg = updateMany.mock.calls[0][0];
|
||||
expect(arg.where).toEqual({ id: "conversation-1", userId: "owner-1" });
|
||||
// v1.39.3 — the new title is stored as ciphertext and the readable column
|
||||
// is cleared, so a legacy row never keeps its old title beside the new one.
|
||||
expect(arg.data.title).toBeNull();
|
||||
expect(decryptFromBytes(arg.data.titleEncrypted)).toBe("Renamed");
|
||||
});
|
||||
|
||||
it("returns null when the owned update matched no row", async () => {
|
||||
|
||||
@@ -0,0 +1,199 @@
|
||||
/**
|
||||
* v1.39.3 — the Coach conversation title is encrypted at rest. These pin the
|
||||
* three things that change with it:
|
||||
* 1. every write stores the title as ciphertext and never in the readable
|
||||
* column;
|
||||
* 2. every read prefers the ciphertext, falls back to the readable column
|
||||
* only for a row the backfill has not reached, and never falls back past
|
||||
* a ciphertext that does not open;
|
||||
* 3. the history search, which used to be an SQL `ILIKE`, still finds titles
|
||||
* (sealed and legacy alike), case-insensitively, and pages with the same
|
||||
* cursor contract.
|
||||
*/
|
||||
import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||
|
||||
process.env.ENCRYPTION_KEY ??=
|
||||
"0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef";
|
||||
|
||||
const { findMany, create } = vi.hoisted(() => ({
|
||||
findMany: vi.fn(),
|
||||
create: vi.fn(),
|
||||
}));
|
||||
|
||||
vi.mock("@/lib/db", () => ({
|
||||
prisma: {
|
||||
coachConversation: { findMany },
|
||||
$transaction: async (
|
||||
fn: (tx: { coachConversation: { create: typeof create } }) => unknown,
|
||||
) => fn({ coachConversation: { create } }),
|
||||
},
|
||||
}));
|
||||
|
||||
import {
|
||||
createConversation,
|
||||
listConversations,
|
||||
readConversationTitle,
|
||||
} from "../persistence";
|
||||
import { decryptFromBytes, encryptToBytes } from "../bytes-codec";
|
||||
|
||||
const now = new Date("2026-09-20T10:00:00.000Z");
|
||||
|
||||
interface StoredRow {
|
||||
id: string;
|
||||
title: string | null;
|
||||
titleEncrypted: Uint8Array | null;
|
||||
}
|
||||
|
||||
function fullRow(row: StoredRow) {
|
||||
return {
|
||||
...row,
|
||||
createdAt: now,
|
||||
updatedAt: now,
|
||||
documentScoped: false,
|
||||
_count: { messages: 2 },
|
||||
attachments: [],
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* A findMany that answers both reads of the search path: the id + title scan
|
||||
* (select) and the page fetch (`id: { in }`).
|
||||
*/
|
||||
function serve(rows: StoredRow[]) {
|
||||
findMany.mockImplementation(
|
||||
async (args: {
|
||||
where: { id?: { in: string[] } };
|
||||
select?: unknown;
|
||||
take?: number;
|
||||
}) => {
|
||||
if (args.where.id) {
|
||||
// Returned out of order on purpose: the page must follow the scan order.
|
||||
return rows
|
||||
.filter((r) => args.where.id!.in.includes(r.id))
|
||||
.reverse()
|
||||
.map(fullRow);
|
||||
}
|
||||
if (args.select) return rows.slice(0, args.take);
|
||||
return rows.slice(0, args.take).map(fullRow);
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
findMany.mockReset();
|
||||
create.mockReset();
|
||||
});
|
||||
|
||||
describe("createConversation", () => {
|
||||
it("stores the summarised title as ciphertext only", async () => {
|
||||
create.mockResolvedValue({
|
||||
id: "c1",
|
||||
title: null,
|
||||
titleEncrypted: null,
|
||||
createdAt: now,
|
||||
updatedAt: now,
|
||||
documentScoped: false,
|
||||
});
|
||||
const dto = await createConversation({
|
||||
userId: "u1",
|
||||
title: "Is my resting pulse\nhigher since the new job?",
|
||||
});
|
||||
const data = create.mock.calls[0][0].data;
|
||||
expect(data).not.toHaveProperty("title");
|
||||
expect(decryptFromBytes(data.titleEncrypted)).toBe(
|
||||
"Is my resting pulse higher since the new job?",
|
||||
);
|
||||
expect(dto.title).toBe("Is my resting pulse higher since the new job?");
|
||||
});
|
||||
});
|
||||
|
||||
describe("readConversationTitle", () => {
|
||||
it("prefers the ciphertext over a readable value", () => {
|
||||
expect(
|
||||
readConversationTitle({
|
||||
title: "stale",
|
||||
titleEncrypted: encryptToBytes("sealed"),
|
||||
}),
|
||||
).toBe("sealed");
|
||||
});
|
||||
|
||||
it("falls back to the readable column only when there is no ciphertext", () => {
|
||||
expect(
|
||||
readConversationTitle({ title: "legacy", titleEncrypted: null }),
|
||||
).toBe("legacy");
|
||||
expect(readConversationTitle({ title: null, titleEncrypted: null })).toBe(
|
||||
"",
|
||||
);
|
||||
});
|
||||
|
||||
it("throws on a ciphertext that does not open instead of showing the readable value", () => {
|
||||
expect(() =>
|
||||
readConversationTitle({
|
||||
title: "must not leak",
|
||||
titleEncrypted: new TextEncoder().encode("not-a-ciphertext"),
|
||||
}),
|
||||
).toThrow();
|
||||
});
|
||||
});
|
||||
|
||||
describe("listConversations title search", () => {
|
||||
const rows: StoredRow[] = [
|
||||
{ id: "c5", title: null, titleEncrypted: encryptToBytes("Blood pressure") },
|
||||
{ id: "c4", title: null, titleEncrypted: encryptToBytes("Sleep debt") },
|
||||
// A row the backfill has not reached yet is still searchable.
|
||||
{ id: "c3", title: "Morning BLOOD pressure spikes", titleEncrypted: null },
|
||||
{ id: "c2", title: null, titleEncrypted: encryptToBytes("Knee pain") },
|
||||
{
|
||||
id: "c1",
|
||||
title: null,
|
||||
titleEncrypted: encryptToBytes("blood pressure and salt"),
|
||||
},
|
||||
];
|
||||
|
||||
it("matches decrypted and legacy titles case-insensitively, in rail order", async () => {
|
||||
serve(rows);
|
||||
const page = await listConversations({ userId: "u1", q: "Blood Pressure" });
|
||||
expect(page.conversations.map((c) => c.id)).toEqual(["c5", "c3", "c1"]);
|
||||
expect(page.conversations.map((c) => c.title)).toEqual([
|
||||
"Blood pressure",
|
||||
"Morning BLOOD pressure spikes",
|
||||
"blood pressure and salt",
|
||||
]);
|
||||
expect(page.nextCursor).toBeNull();
|
||||
// The scan stays owner-scoped and never matches in SQL.
|
||||
const scan = findMany.mock.calls[0][0];
|
||||
expect(scan.where.userId).toBe("u1");
|
||||
expect("title" in scan.where).toBe(false);
|
||||
});
|
||||
|
||||
it("pages the matches with the cursor contract of the unfiltered rail", async () => {
|
||||
serve(rows);
|
||||
const first = await listConversations({
|
||||
userId: "u1",
|
||||
q: "blood",
|
||||
limit: 2,
|
||||
});
|
||||
expect(first.conversations.map((c) => c.id)).toEqual(["c5", "c3"]);
|
||||
expect(first.nextCursor).toBe("c3");
|
||||
|
||||
const second = await listConversations({
|
||||
userId: "u1",
|
||||
q: "blood",
|
||||
limit: 2,
|
||||
cursor: first.nextCursor,
|
||||
});
|
||||
expect(second.conversations.map((c) => c.id)).toEqual(["c1"]);
|
||||
expect(second.nextCursor).toBeNull();
|
||||
});
|
||||
|
||||
it("answers an unknown cursor with an empty page", async () => {
|
||||
serve(rows);
|
||||
const page = await listConversations({
|
||||
userId: "u1",
|
||||
q: "blood",
|
||||
cursor: "gone",
|
||||
});
|
||||
expect(page.conversations).toEqual([]);
|
||||
expect(page.nextCursor).toBeNull();
|
||||
});
|
||||
});
|
||||
@@ -96,10 +96,14 @@ describe("recordProactiveNudge", () => {
|
||||
createdAt: now,
|
||||
});
|
||||
|
||||
// Conversation owned by the user, title summarised.
|
||||
expect(txCreate.coachConversation.create).toHaveBeenCalledWith({
|
||||
data: { userId: "user_1", title: "Time to weigh in" },
|
||||
});
|
||||
// Conversation owned by the user, title summarised and stored as
|
||||
// ciphertext only (v1.39.3); the readable column is never written.
|
||||
const convArg = txCreate.coachConversation.create.mock.calls[0][0];
|
||||
expect(convArg.data.userId).toBe("user_1");
|
||||
expect(convArg.data).not.toHaveProperty("title");
|
||||
expect(new TextDecoder().decode(convArg.data.titleEncrypted)).toBe(
|
||||
"enc:Time to weigh in",
|
||||
);
|
||||
|
||||
// The body is encrypted at rest (Bytes), role is assistant, and the
|
||||
// message hangs off the new conversation. No raw plaintext column.
|
||||
|
||||
+126
-41
@@ -31,9 +31,9 @@ import type {
|
||||
|
||||
/**
|
||||
* Title-from-message — first 80 chars trimmed, ellipsis on overflow.
|
||||
* Stays plain text (the history rail needs to render without paying
|
||||
* the per-message decrypt cost), so callers should pass already-
|
||||
* sanitised input.
|
||||
* Callers should pass already-sanitised input. The result is stored
|
||||
* encrypted (`titleEncrypted`, v1.39.3): it is the opening words of the
|
||||
* person's first message, which is as sensitive as the message itself.
|
||||
*/
|
||||
export function summariseTitle(input: string): string {
|
||||
const collapsed = input.replace(/\s+/g, " ").trim();
|
||||
@@ -301,15 +301,44 @@ export interface AppendMessageParams {
|
||||
* Create a brand-new conversation row owned by `userId`. Caller is
|
||||
* expected to immediately append the first user message.
|
||||
*/
|
||||
/**
|
||||
* v1.39.3 — the title's at-rest form. Always a ciphertext: `summariseTitle`
|
||||
* never returns an empty string and a rename is validated non-empty, so there
|
||||
* is no "no title" case to store as NULL.
|
||||
*/
|
||||
export function encryptConversationTitle(
|
||||
title: string,
|
||||
): Uint8Array<ArrayBuffer> {
|
||||
return encryptToBytes(title);
|
||||
}
|
||||
|
||||
/**
|
||||
* v1.39.3 — read a conversation title, ciphertext first. The readable column
|
||||
* is consulted only when there is no ciphertext, which is a row the free-text
|
||||
* encryption backfill has not reached yet. A present ciphertext that does not
|
||||
* decrypt throws, the same fail-closed posture as every note reader
|
||||
* (`readNote`); it never falls back to the readable column.
|
||||
*/
|
||||
export function readConversationTitle(row: {
|
||||
title: string | null;
|
||||
titleEncrypted: Uint8Array | null;
|
||||
}): string {
|
||||
if (row.titleEncrypted && row.titleEncrypted.byteLength > 0) {
|
||||
return decryptFromBytes(row.titleEncrypted);
|
||||
}
|
||||
return row.title ?? "";
|
||||
}
|
||||
|
||||
export async function createConversation(
|
||||
params: CreateConversationParams,
|
||||
): Promise<CoachConversationDTO> {
|
||||
const attachmentIds = params.attachmentIds ?? [];
|
||||
const title = summariseTitle(params.title);
|
||||
const row = await prisma.$transaction(async (tx) => {
|
||||
const conversation = await tx.coachConversation.create({
|
||||
data: {
|
||||
userId: params.userId,
|
||||
title: summariseTitle(params.title),
|
||||
titleEncrypted: encryptConversationTitle(title),
|
||||
documentScoped: params.documentScoped ?? false,
|
||||
},
|
||||
});
|
||||
@@ -327,7 +356,7 @@ export async function createConversation(
|
||||
});
|
||||
return {
|
||||
id: row.id,
|
||||
title: row.title,
|
||||
title,
|
||||
createdAt: row.createdAt.toISOString(),
|
||||
updatedAt: row.updatedAt.toISOString(),
|
||||
messageCount: 0,
|
||||
@@ -439,7 +468,7 @@ export async function recordProactiveNudge(
|
||||
const conversation = await tx.coachConversation.create({
|
||||
data: {
|
||||
userId: params.userId,
|
||||
title: summariseTitle(params.title),
|
||||
titleEncrypted: encryptConversationTitle(summariseTitle(params.title)),
|
||||
},
|
||||
});
|
||||
const message = await tx.coachMessage.create({
|
||||
@@ -559,7 +588,7 @@ export async function fetchConversationWithMessages(
|
||||
const attachments = mapAttachments(row.attachments);
|
||||
return {
|
||||
id: row.id,
|
||||
title: row.title,
|
||||
title: readConversationTitle(row),
|
||||
createdAt: row.createdAt.toISOString(),
|
||||
updatedAt: row.updatedAt.toISOString(),
|
||||
messageCount: messages.length,
|
||||
@@ -586,15 +615,31 @@ export interface ListConversationsParams {
|
||||
/**
|
||||
* v1.30.2 (QoL H1) — optional server-side title search for the history
|
||||
* rail + the standalone conversations page. Case-insensitive substring
|
||||
* match against `title` only — the only plaintext column on the row;
|
||||
* `CoachMessage.encryptedContent` cannot be searched without decrypting
|
||||
* every message, so message BODIES are out of scope for this pass (see
|
||||
* the route's doc comment). Trimmed empty string is treated as "no
|
||||
* filter", matching the pre-existing cursor/limit handling style.
|
||||
* match against the title only; message BODIES are out of scope (see the
|
||||
* route's doc comment). Since v1.39.3 the title is encrypted at rest, so
|
||||
* the match runs on the decrypted titles in memory rather than in SQL (see
|
||||
* `TITLE_SEARCH_SCAN_CAP`). Trimmed empty string is treated as "no filter",
|
||||
* matching the pre-existing cursor/limit handling style.
|
||||
*/
|
||||
q?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* v1.39.3 — how many of an account's conversations a title search decrypts,
|
||||
* newest first. The title is encrypted at rest, so SQL cannot match it; the
|
||||
* search reads id + title for the account, decrypts, and filters in memory.
|
||||
* A title is at most 80 characters, so a scan of the whole cap is a few
|
||||
* milliseconds of AES. An account past the cap has its oldest threads left out
|
||||
* of search results (never out of the unfiltered rail), which no real account
|
||||
* comes near: the rail pages 20 at a time and a person opens a few a day.
|
||||
*/
|
||||
export const TITLE_SEARCH_SCAN_CAP = 5000;
|
||||
|
||||
const CONVERSATION_LIST_ORDER = [
|
||||
{ updatedAt: "desc" as const },
|
||||
{ id: "desc" as const },
|
||||
];
|
||||
|
||||
/**
|
||||
* Cursor-paginated list of conversations for the rail. Default limit
|
||||
* 20, cap 50. Cursor is the id of the last item on the previous page;
|
||||
@@ -608,43 +653,81 @@ export async function listConversations(
|
||||
}> {
|
||||
const limit = Math.min(Math.max(params.limit ?? 20, 1), 50);
|
||||
const q = params.q?.trim();
|
||||
const rows = await prisma.coachConversation.findMany({
|
||||
where: {
|
||||
userId: params.userId,
|
||||
...(params.attachedDocumentId
|
||||
? { attachments: { some: { documentId: params.attachedDocumentId } } }
|
||||
: {}),
|
||||
...(q ? { title: { contains: q, mode: "insensitive" } } : {}),
|
||||
},
|
||||
orderBy: [{ updatedAt: "desc" }, { id: "desc" }],
|
||||
take: limit + 1,
|
||||
...(params.cursor
|
||||
? {
|
||||
cursor: { id: params.cursor },
|
||||
skip: 1,
|
||||
}
|
||||
const scope = {
|
||||
userId: params.userId,
|
||||
...(params.attachedDocumentId
|
||||
? { attachments: { some: { documentId: params.attachedDocumentId } } }
|
||||
: {}),
|
||||
include: {
|
||||
_count: { select: { messages: true } },
|
||||
// v1.29.x (S7) — the live attachment set (label columns only) so the rail
|
||||
// can badge a fenced thread with a paperclip + the first document's title.
|
||||
// Empty on a health thread.
|
||||
attachments: {
|
||||
orderBy: { addedAt: "asc" },
|
||||
include: { document: { select: { title: true, filename: true } } },
|
||||
},
|
||||
};
|
||||
const include = {
|
||||
_count: { select: { messages: true } },
|
||||
// v1.29.x (S7) — the live attachment set (label columns only) so the rail
|
||||
// can badge a fenced thread with a paperclip + the first document's title.
|
||||
// Empty on a health thread.
|
||||
attachments: {
|
||||
orderBy: { addedAt: "asc" as const },
|
||||
include: { document: { select: { title: true, filename: true } } },
|
||||
},
|
||||
});
|
||||
};
|
||||
|
||||
const page = rows.slice(0, limit);
|
||||
const nextCursor = rows.length > limit ? page[page.length - 1].id : null;
|
||||
let page;
|
||||
let nextCursor: string | null;
|
||||
if (q) {
|
||||
// Match on the decrypted titles, then page the matching ids with the same
|
||||
// cursor contract the SQL path has: the cursor is the last id of the
|
||||
// previous page, and an unknown cursor yields an empty page.
|
||||
const needle = q.toLocaleLowerCase();
|
||||
const candidates = await prisma.coachConversation.findMany({
|
||||
where: scope,
|
||||
orderBy: CONVERSATION_LIST_ORDER,
|
||||
take: TITLE_SEARCH_SCAN_CAP,
|
||||
select: { id: true, title: true, titleEncrypted: true },
|
||||
});
|
||||
const matching = candidates
|
||||
.filter((c) =>
|
||||
readConversationTitle(c).toLocaleLowerCase().includes(needle),
|
||||
)
|
||||
.map((c) => c.id);
|
||||
let start = 0;
|
||||
if (params.cursor) {
|
||||
const at = matching.indexOf(params.cursor);
|
||||
start = at === -1 ? matching.length : at + 1;
|
||||
}
|
||||
const pageIds = matching.slice(start, start + limit);
|
||||
nextCursor =
|
||||
matching.length > start + limit ? pageIds[pageIds.length - 1] : null;
|
||||
const rows = await prisma.coachConversation.findMany({
|
||||
where: { ...scope, id: { in: pageIds } },
|
||||
include,
|
||||
});
|
||||
const byId = new Map(rows.map((r) => [r.id, r]));
|
||||
page = pageIds.flatMap((id) => {
|
||||
const row = byId.get(id);
|
||||
return row ? [row] : [];
|
||||
});
|
||||
} else {
|
||||
const rows = await prisma.coachConversation.findMany({
|
||||
where: scope,
|
||||
orderBy: CONVERSATION_LIST_ORDER,
|
||||
take: limit + 1,
|
||||
...(params.cursor
|
||||
? {
|
||||
cursor: { id: params.cursor },
|
||||
skip: 1,
|
||||
}
|
||||
: {}),
|
||||
include,
|
||||
});
|
||||
page = rows.slice(0, limit);
|
||||
nextCursor = rows.length > limit ? page[page.length - 1].id : null;
|
||||
}
|
||||
|
||||
return {
|
||||
conversations: page.map((r) => {
|
||||
const attachments = mapAttachments(r.attachments);
|
||||
return {
|
||||
id: r.id,
|
||||
title: r.title,
|
||||
title: readConversationTitle(r),
|
||||
createdAt: r.createdAt.toISOString(),
|
||||
updatedAt: r.updatedAt.toISOString(),
|
||||
messageCount: r._count.messages,
|
||||
@@ -669,7 +752,9 @@ export async function renameConversation(
|
||||
): Promise<{ id: string; title: string } | null> {
|
||||
const { count } = await prisma.coachConversation.updateMany({
|
||||
where: { id: conversationId, userId },
|
||||
data: { title },
|
||||
// The readable column is cleared on a rename so a legacy row never keeps
|
||||
// its old title beside the new ciphertext.
|
||||
data: { title: null, titleEncrypted: encryptConversationTitle(title) },
|
||||
});
|
||||
return count === 1 ? { id: conversationId, title } : null;
|
||||
}
|
||||
|
||||
@@ -119,7 +119,10 @@ export async function surfaceCoachReminders(
|
||||
|
||||
await prisma.$transaction(async (tx) => {
|
||||
const conversation = await tx.coachConversation.create({
|
||||
data: { userId, title: title.slice(0, 80) },
|
||||
data: {
|
||||
userId,
|
||||
titleEncrypted: encryptToBytes(title.slice(0, 80)),
|
||||
},
|
||||
});
|
||||
await tx.coachMessage.create({
|
||||
data: {
|
||||
|
||||
@@ -208,6 +208,8 @@ export const ENCRYPTED_COLUMNS: readonly EncryptedColumn[] = [
|
||||
// ───── Coach (Bytes columns) ─────
|
||||
{ model: "CoachMessage", field: "encryptedContent", kind: "bytes" },
|
||||
{ model: "CoachConversation", field: "summaryEncrypted", kind: "bytes" },
|
||||
// v1.39.3 — the title: the opening words of the first message, or a rename.
|
||||
{ model: "CoachConversation", field: "titleEncrypted", kind: "bytes" },
|
||||
{ model: "CoachFact", field: "factEncrypted", kind: "bytes" },
|
||||
// v1.21.3 (B1) — Coach goal / if-then plan free-text columns.
|
||||
{ model: "CoachPlan", field: "ifCueEncrypted", kind: "bytes" },
|
||||
@@ -310,6 +312,12 @@ export const ENCRYPTED_COLUMNS: readonly EncryptedColumn[] = [
|
||||
{ model: "MedicationDoseChange", field: "noteEncrypted", kind: "bytes" },
|
||||
{ model: "MedicationInventoryItem", field: "notesEncrypted", kind: "bytes" },
|
||||
|
||||
// ───── v1.39.3 custom-metric reading note (Bytes column) ─────
|
||||
// The one free-text field on a reading of a metric the person defined. The
|
||||
// catalog columns (name, unit, description) stay plaintext: they are the
|
||||
// definition of a series, listed and sorted by.
|
||||
{ model: "CustomMetricEntry", field: "noteEncrypted", kind: "bytes" },
|
||||
|
||||
// ───── v1.25 mental-health screener item answers (Bytes column) ─────
|
||||
// The PHQ-9 / GAD-7 per-item responses (incl. the safety-critical item 9)
|
||||
// ride a single AES-256-GCM blob. The most sensitive payload in the wave;
|
||||
|
||||
@@ -1,15 +1,19 @@
|
||||
/**
|
||||
* v1.25.5 — server-side serialisers for the user-defined custom-metric store.
|
||||
*
|
||||
* Custom metrics are PLAINTEXT (name / unit / description), so — unlike the
|
||||
* Biomarker catalog with its AES-256-GCM `contextEncrypted` codec — there is no
|
||||
* encrypt/decrypt layer here. These helpers just map a Prisma row into the
|
||||
* stable wire DTO the web + iOS clients render.
|
||||
* The catalog columns (name / unit / description) are plaintext: they are the
|
||||
* definition of a series and are listed and sorted by. The one free-text field
|
||||
* on a reading, its note, is AES-256-GCM at rest since v1.39.3
|
||||
* (`noteEncrypted`), read here through the shared note boundary. These
|
||||
* helpers map a Prisma row into the stable wire DTO the web + iOS clients
|
||||
* render.
|
||||
*
|
||||
* The store is deliberately ISOLATED from the closed `MeasurementType` system:
|
||||
* no rollup, no sync, no FHIR, no insights. Charts read entries LIVE.
|
||||
*/
|
||||
|
||||
import { readNote } from "@/lib/crypto/note-cipher";
|
||||
|
||||
/** A custom-metric catalog row as the API serialises it. */
|
||||
export interface CustomMetricRow {
|
||||
id: string;
|
||||
@@ -37,7 +41,9 @@ export interface CustomMetricEntryRow {
|
||||
value: number;
|
||||
unit: string;
|
||||
measuredAt: Date;
|
||||
/** Legacy readable note, set only on rows the backfill has not reached. */
|
||||
note: string | null;
|
||||
noteEncrypted: Uint8Array | null;
|
||||
createdAt: Date;
|
||||
}
|
||||
|
||||
@@ -79,7 +85,7 @@ export function serialiseCustomMetricEntry(row: CustomMetricEntryRow) {
|
||||
value: row.value,
|
||||
unit: row.unit,
|
||||
measuredAt: row.measuredAt.toISOString(),
|
||||
note: row.note,
|
||||
note: readNote(row.noteEncrypted, row.note),
|
||||
createdAt: row.createdAt.toISOString(),
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,146 @@
|
||||
/**
|
||||
* v1.39.3 — the Coach conversation title in both backup formats.
|
||||
*
|
||||
* A disaster-recovery file carries the title as ciphertext only, including for
|
||||
* a row the free-text backfill has not reached yet (it is sealed on the way
|
||||
* out). A portable file carries it readable, for the person who owns it. A
|
||||
* restore of either writes the ciphertext column and never the readable one,
|
||||
* and a file written before this release (readable title only) still restores.
|
||||
*/
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
|
||||
process.env.ENCRYPTION_KEY ??=
|
||||
"0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef";
|
||||
|
||||
import { decryptFromBytes, encryptToBytes } from "@/lib/ai/coach/bytes-codec";
|
||||
import {
|
||||
buildCoachBackupSection,
|
||||
restoreCoachData,
|
||||
type RestoredCoachConversation,
|
||||
} from "../coach-backup";
|
||||
import { parseBackupPayload } from "@/lib/validations/backup";
|
||||
|
||||
const at = "2026-09-20T10:00:00.000Z";
|
||||
|
||||
function row(id: string, title: string | null, sealed: string | null) {
|
||||
return {
|
||||
id,
|
||||
title,
|
||||
titleEncrypted: sealed === null ? null : encryptToBytes(sealed),
|
||||
documentScoped: false,
|
||||
summaryEncrypted: null,
|
||||
summaryUpdatedAt: null,
|
||||
summaryTurnCount: 0,
|
||||
createdAt: new Date(at),
|
||||
updatedAt: new Date(at),
|
||||
messages: [],
|
||||
attachments: [],
|
||||
};
|
||||
}
|
||||
|
||||
function prismaWith(rows: ReturnType<typeof row>[]) {
|
||||
return {
|
||||
coachConversation: { findMany: vi.fn().mockResolvedValue(rows) },
|
||||
} as never;
|
||||
}
|
||||
|
||||
const rows = [
|
||||
row("c1", null, "Sealed title"),
|
||||
row("c2", "Legacy readable title", null),
|
||||
];
|
||||
|
||||
describe("coach backup title", () => {
|
||||
it("a disaster-recovery file carries only ciphertext, sealing legacy rows on the way out", async () => {
|
||||
const section = await buildCoachBackupSection(prismaWith(rows), "u1", {
|
||||
purpose: "disaster-recovery",
|
||||
});
|
||||
const json = JSON.stringify(section);
|
||||
expect(json).not.toContain("Sealed title");
|
||||
expect(json).not.toContain("Legacy readable title");
|
||||
expect(
|
||||
section.coachConversations.map((c) => {
|
||||
expect(c).not.toHaveProperty("title");
|
||||
return decryptFromBytes(Buffer.from(c.titleEncrypted!, "base64"));
|
||||
}),
|
||||
).toEqual(["Sealed title", "Legacy readable title"]);
|
||||
});
|
||||
|
||||
it("a portable file carries the readable title and no ciphertext", async () => {
|
||||
const section = await buildCoachBackupSection(prismaWith(rows), "u1", {
|
||||
purpose: "portable-export",
|
||||
});
|
||||
expect(section.coachConversations.map((c) => c.title)).toEqual([
|
||||
"Sealed title",
|
||||
"Legacy readable title",
|
||||
]);
|
||||
for (const c of section.coachConversations) {
|
||||
expect(c).not.toHaveProperty("titleEncrypted");
|
||||
}
|
||||
});
|
||||
|
||||
it("a restore of either format, and of an older file, writes only the ciphertext", async () => {
|
||||
const created: Array<Record<string, unknown>> = [];
|
||||
const tx = {
|
||||
coachMessage: { count: vi.fn().mockResolvedValue(0) },
|
||||
coachConversationDocument: {
|
||||
count: vi.fn().mockResolvedValue(0),
|
||||
createMany: vi.fn(),
|
||||
},
|
||||
coachConversation: {
|
||||
deleteMany: vi.fn().mockResolvedValue({ count: 0 }),
|
||||
create: vi.fn(async (args: { data: Record<string, unknown> }) => {
|
||||
created.push(args.data);
|
||||
return args.data;
|
||||
}),
|
||||
},
|
||||
};
|
||||
const base = {
|
||||
documentScoped: false,
|
||||
createdAt: at,
|
||||
updatedAt: at,
|
||||
messages: [],
|
||||
attachments: [],
|
||||
};
|
||||
const conversations: RestoredCoachConversation[] = [
|
||||
{
|
||||
...base,
|
||||
id: "dr",
|
||||
titleEncrypted: Buffer.from(encryptToBytes("From DR")).toString(
|
||||
"base64",
|
||||
),
|
||||
},
|
||||
{ ...base, id: "portable", title: "From a portable file" },
|
||||
];
|
||||
await restoreCoachData(
|
||||
tx as never,
|
||||
"u1",
|
||||
{ coachConversations: conversations },
|
||||
new Set(),
|
||||
{ entries: [] } as never,
|
||||
);
|
||||
expect(created.map((d) => d.title)).toEqual([undefined, undefined]);
|
||||
expect(
|
||||
created.map((d) => decryptFromBytes(d.titleEncrypted as Uint8Array)),
|
||||
).toEqual(["From DR", "From a portable file"]);
|
||||
});
|
||||
|
||||
it("the wire schema accepts a pre-release file with a readable title", () => {
|
||||
const parsed = parseBackupPayload({
|
||||
schemaVersion: "2",
|
||||
exportedAt: at,
|
||||
userId: "u1",
|
||||
coachConversations: [
|
||||
{
|
||||
id: "old",
|
||||
title: "Written before the title was sealed",
|
||||
documentScoped: false,
|
||||
createdAt: at,
|
||||
updatedAt: at,
|
||||
},
|
||||
],
|
||||
});
|
||||
expect(parsed.coachConversations[0].title).toBe(
|
||||
"Written before the title was sealed",
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -1,6 +1,6 @@
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import { Buffer } from "node:buffer";
|
||||
import { encryptToBytes } from "@/lib/ai/coach/bytes-codec";
|
||||
import { decryptFromBytes, encryptToBytes } from "@/lib/ai/coach/bytes-codec";
|
||||
import { parseBackupPayload } from "@/lib/validations/backup";
|
||||
|
||||
process.env.ENCRYPTION_KEY =
|
||||
@@ -1286,11 +1286,22 @@ describe("buildFullBackupPayload — profile and custom metrics", () => {
|
||||
value: 44.5,
|
||||
unit: "kg",
|
||||
measuredAt: "2026-07-19T06:30:00.000Z",
|
||||
note: "felt strong",
|
||||
// v1.39.3 — a DR file carries the note sealed, never readable,
|
||||
// including for a row the backfill has not reached yet.
|
||||
note: null,
|
||||
noteEncrypted: expect.any(String),
|
||||
}),
|
||||
],
|
||||
}),
|
||||
]);
|
||||
const [sealedMetric] = payload.customMetrics as Array<{
|
||||
entries: Array<{ noteEncrypted: string }>;
|
||||
}>;
|
||||
expect(
|
||||
decryptFromBytes(
|
||||
Buffer.from(sealedMetric.entries[0].noteEncrypted, "base64"),
|
||||
),
|
||||
).toBe("felt strong");
|
||||
expect(counts.customMetrics).toBe(1);
|
||||
expect(counts.customMetricEntries).toBe(1);
|
||||
expect(payload.correlationPatterns).toEqual([
|
||||
|
||||
@@ -572,8 +572,11 @@ describe("backup round trip — export, wire schema, restore", () => {
|
||||
expect(entries[0]).toMatchObject({
|
||||
value: 44.5,
|
||||
unit: "kg",
|
||||
note: "felt strong",
|
||||
});
|
||||
// v1.39.3 — the note comes back sealed; the readable column is never
|
||||
// written, even for a row that was still readable when it was exported.
|
||||
expect(entries[0]).not.toHaveProperty("note");
|
||||
expect(entries[0].noteEncrypted).toBeInstanceOf(Uint8Array);
|
||||
});
|
||||
|
||||
it("writes persisted pattern identity and dismissal evidence back", async () => {
|
||||
|
||||
@@ -92,7 +92,13 @@ export interface CoachConversationBackupEntry {
|
||||
/** Always carried: messages, attachments and the Coach's own facts, plans
|
||||
* and reminders all address a conversation by it. */
|
||||
id: string;
|
||||
title: string;
|
||||
/**
|
||||
* The readable title. Present on a portable payload only (and on every file
|
||||
* written before v1.39.3, when the title was stored readable).
|
||||
*/
|
||||
title?: string;
|
||||
/** v1.39.3 — the title's ciphertext as base64. Disaster-recovery only. */
|
||||
titleEncrypted?: string;
|
||||
/**
|
||||
* The permanent fence marker. Carried and restored verbatim; see the file
|
||||
* header for why letting this default is a security regression and not a
|
||||
@@ -130,6 +136,7 @@ export interface CoachBackupCounts {
|
||||
const COACH_CONVERSATION_BACKUP_SELECT = {
|
||||
id: true,
|
||||
title: true,
|
||||
titleEncrypted: true,
|
||||
documentScoped: true,
|
||||
summaryEncrypted: true,
|
||||
summaryUpdatedAt: true,
|
||||
@@ -174,6 +181,17 @@ function decryptTurnSoft(bytes: Uint8Array | null): string | null {
|
||||
}
|
||||
}
|
||||
|
||||
/** A title for a reader: ciphertext first, legacy readable column after. */
|
||||
function readTitleSoft(row: {
|
||||
title: string | null;
|
||||
titleEncrypted: Uint8Array | null;
|
||||
}): string {
|
||||
if (row.titleEncrypted && row.titleEncrypted.byteLength > 0) {
|
||||
return decryptTurnSoft(row.titleEncrypted) ?? "";
|
||||
}
|
||||
return row.title ?? "";
|
||||
}
|
||||
|
||||
export async function buildCoachBackupSection(
|
||||
prisma: Pick<PrismaClient, "coachConversation">,
|
||||
userId: string,
|
||||
@@ -190,7 +208,15 @@ export async function buildCoachBackupSection(
|
||||
return {
|
||||
coachConversations: rows.map((row) => ({
|
||||
id: row.id,
|
||||
title: row.title,
|
||||
...(disasterRecovery
|
||||
? {
|
||||
// A row the backfill has not reached yet is sealed here, so a DR
|
||||
// file never carries a title readable.
|
||||
titleEncrypted: Buffer.from(
|
||||
row.titleEncrypted ?? encryptToBytes(row.title ?? ""),
|
||||
).toString("base64"),
|
||||
}
|
||||
: { title: readTitleSoft(row) }),
|
||||
documentScoped: row.documentScoped,
|
||||
...(disasterRecovery
|
||||
? {
|
||||
@@ -293,12 +319,17 @@ export type RestoredCoachMessage = Pick<
|
||||
|
||||
export type RestoredCoachConversation = Pick<
|
||||
CoachConversationBackupEntry,
|
||||
"id" | "title" | "documentScoped" | "createdAt" | "updatedAt"
|
||||
"id" | "documentScoped" | "createdAt" | "updatedAt"
|
||||
> &
|
||||
OptionalNullable<
|
||||
Pick<
|
||||
CoachConversationBackupEntry,
|
||||
"summaryEncrypted" | "summary" | "summaryUpdatedAt" | "summaryTurnCount"
|
||||
| "title"
|
||||
| "titleEncrypted"
|
||||
| "summaryEncrypted"
|
||||
| "summary"
|
||||
| "summaryUpdatedAt"
|
||||
| "summaryTurnCount"
|
||||
>
|
||||
> & {
|
||||
messages: RestoredCoachMessage[];
|
||||
@@ -343,7 +374,9 @@ export async function restoreCoachData(
|
||||
data: {
|
||||
id: conversation.id,
|
||||
userId: ownerId,
|
||||
title: conversation.title,
|
||||
// v1.39.3 — the title is stored encrypted; the readable column is
|
||||
// never written on a restore.
|
||||
titleEncrypted: resolveTitleBytes(conversation),
|
||||
// Verbatim. See the file header: this is the one field whose loss is a
|
||||
// security regression rather than a gap in the history.
|
||||
documentScoped: conversation.documentScoped,
|
||||
@@ -420,6 +453,15 @@ function resolveTurnBytes(
|
||||
return encryptToBytes(message.content ?? "");
|
||||
}
|
||||
|
||||
function resolveTitleBytes(
|
||||
conversation: RestoredCoachConversation,
|
||||
): Uint8Array<ArrayBuffer> {
|
||||
if (conversation.titleEncrypted !== undefined) {
|
||||
return decodeBase64(conversation.titleEncrypted);
|
||||
}
|
||||
return encryptToBytes(conversation.title ?? "");
|
||||
}
|
||||
|
||||
function resolveSummaryBytes(
|
||||
conversation: RestoredCoachConversation,
|
||||
): Uint8Array<ArrayBuffer> | null {
|
||||
|
||||
@@ -25,6 +25,7 @@ import { Buffer } from "node:buffer";
|
||||
|
||||
import type { Prisma, PrismaClient } from "@/generated/prisma/client";
|
||||
import { decryptFromBytes, encryptToBytes } from "@/lib/ai/coach/bytes-codec";
|
||||
import { encryptNote } from "@/lib/crypto/note-cipher";
|
||||
import { getEvent } from "@/lib/logging/context";
|
||||
import {
|
||||
DEFAULT_HEALTH_PROFILE_AI_SECTIONS,
|
||||
@@ -116,7 +117,13 @@ export interface CustomMetricEntryBackupEntry {
|
||||
/** Unit snapshot taken from the metric at write time, not a live join. */
|
||||
unit: string;
|
||||
measuredAt: string;
|
||||
/** Portable exports carry the readable note; DR payloads leave this null. */
|
||||
note: string | null;
|
||||
/**
|
||||
* v1.39.3 — base64 ciphertext of the note, present only in disaster-recovery
|
||||
* payloads (null when the reading has no note).
|
||||
*/
|
||||
noteEncrypted?: string | null;
|
||||
createdAt?: string;
|
||||
/** v1.37.20 (A3-11) — soft-delete tombstone; DR payloads only. */
|
||||
deletedAt?: string | null;
|
||||
@@ -227,6 +234,32 @@ function decryptProfileFieldSoft(
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* v1.39.3 — a custom-metric reading's note for a portable export: ciphertext
|
||||
* first, the legacy readable column for a row the backfill has not reached.
|
||||
* An unreadable ciphertext is recorded in the manifest and exported as no
|
||||
* note, never replaced by the readable column.
|
||||
*/
|
||||
function readEntryNoteSoft(
|
||||
entry: { id: string; note: string | null; noteEncrypted: Uint8Array | null },
|
||||
failures: string[],
|
||||
): string | null {
|
||||
if (entry.noteEncrypted && entry.noteEncrypted.byteLength > 0) {
|
||||
try {
|
||||
return decryptFromBytes(entry.noteEncrypted);
|
||||
} catch (err) {
|
||||
getEvent()?.addWarning(
|
||||
`custom metric entry note decrypt failed: ${
|
||||
err instanceof Error ? err.message : String(err)
|
||||
}`,
|
||||
);
|
||||
failures.push(`customMetricEntry.${entry.id}.note`);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
return entry.note;
|
||||
}
|
||||
|
||||
function toBase64(buf: Uint8Array | null): string | null {
|
||||
return buf && buf.byteLength > 0 ? Buffer.from(buf).toString("base64") : null;
|
||||
}
|
||||
@@ -447,12 +480,18 @@ export async function buildProfileBackupSection(
|
||||
id: entry.id,
|
||||
createdAt: entry.createdAt.toISOString(),
|
||||
deletedAt: entry.deletedAt?.toISOString() ?? null,
|
||||
// v1.39.3 — the ciphertext rides verbatim. A row the backfill has
|
||||
// not reached yet is sealed here, so a DR file never carries the
|
||||
// note readable either way.
|
||||
noteEncrypted: toBase64(
|
||||
entry.noteEncrypted ?? encryptNote(entry.note),
|
||||
),
|
||||
}
|
||||
: {}),
|
||||
value: entry.value,
|
||||
unit: entry.unit,
|
||||
measuredAt: entry.measuredAt.toISOString(),
|
||||
note: entry.note,
|
||||
note: disasterRecovery ? null : readEntryNoteSoft(entry, decryptFailures),
|
||||
})),
|
||||
}));
|
||||
|
||||
@@ -571,6 +610,16 @@ function resolveProfileColumn(
|
||||
return plaintext == null ? null : encryptToBytes(plaintext);
|
||||
}
|
||||
|
||||
function resolveEntryNote(entry: {
|
||||
note?: string | null;
|
||||
noteEncrypted?: string | null;
|
||||
}): Uint8Array<ArrayBuffer> | null {
|
||||
if (entry.noteEncrypted !== undefined && entry.noteEncrypted !== null) {
|
||||
return decodeEncryptedField(entry.noteEncrypted, "customMetricEntry.note");
|
||||
}
|
||||
return encryptNote(entry.note);
|
||||
}
|
||||
|
||||
function resolveFactValue(
|
||||
entry: HealthProfileFactBackupEntry,
|
||||
): Uint8Array<ArrayBuffer> {
|
||||
@@ -848,7 +897,10 @@ export async function restoreProfileData(
|
||||
value: entry.value,
|
||||
unit: entry.unit,
|
||||
measuredAt: new Date(entry.measuredAt),
|
||||
note: entry.note,
|
||||
// v1.39.3 — ciphertext wins when a DR payload carried it; a
|
||||
// portable file's readable note is sealed under this instance's
|
||||
// key. The readable column is never written.
|
||||
noteEncrypted: resolveEntryNote(entry),
|
||||
deletedAt: entry.deletedAt ? new Date(entry.deletedAt) : null,
|
||||
...(entry.createdAt
|
||||
? { createdAt: new Date(entry.createdAt) }
|
||||
|
||||
@@ -0,0 +1,184 @@
|
||||
import { describe, it, expect, beforeEach, afterEach, vi } from "vitest";
|
||||
import { readNote } from "@/lib/crypto/note-cipher";
|
||||
import { encryptToBytes } from "@/lib/ai/coach/bytes-codec";
|
||||
|
||||
// ── in-memory store backing a minimal prisma mock ──────────────────────────
|
||||
interface ConversationRow {
|
||||
id: string;
|
||||
userId: string;
|
||||
title: string | null;
|
||||
titleEncrypted: Uint8Array | null;
|
||||
}
|
||||
interface EntryRow {
|
||||
id: string;
|
||||
userId: string;
|
||||
note: string | null;
|
||||
noteEncrypted: Uint8Array | null;
|
||||
}
|
||||
|
||||
const store = vi.hoisted(() => ({
|
||||
conversations: [] as ConversationRow[],
|
||||
entries: [] as EntryRow[],
|
||||
}));
|
||||
|
||||
vi.mock("@/lib/jobs/boss-instance", () => ({ getGlobalBoss: () => null }));
|
||||
vi.mock("@/lib/logging/context", () => ({ annotate: vi.fn() }));
|
||||
|
||||
vi.mock("@/lib/db", () => {
|
||||
function delegate<R extends { id: string; userId: string }>(
|
||||
rows: () => R[],
|
||||
readable: keyof R,
|
||||
) {
|
||||
return {
|
||||
// The handler pages on `{ userId, <readable>: { not: null } }`.
|
||||
findMany: async (args: { where: { userId: string }; take: number }) =>
|
||||
rows()
|
||||
.filter((r) => r.userId === args.where.userId && r[readable] !== null)
|
||||
.slice(0, args.take)
|
||||
.map((r) => ({ id: r.id })),
|
||||
findUnique: async (args: { where: { id: string } }) =>
|
||||
rows().find((x) => x.id === args.where.id) ?? null,
|
||||
update: async (args: { where: { id: string }; data: Partial<R> }) => {
|
||||
const r = rows().find((x) => x.id === args.where.id)!;
|
||||
Object.assign(r, args.data);
|
||||
return r;
|
||||
},
|
||||
};
|
||||
}
|
||||
const delegates = {
|
||||
coachConversation: delegate(() => store.conversations, "title"),
|
||||
customMetricEntry: delegate(() => store.entries, "note"),
|
||||
};
|
||||
return {
|
||||
prisma: {
|
||||
...delegates,
|
||||
$transaction: async (fn: (tx: typeof delegates) => unknown) =>
|
||||
fn(delegates),
|
||||
},
|
||||
};
|
||||
});
|
||||
|
||||
import { runFreeTextEncryptionBackfillForUser } from "@/lib/jobs/free-text-encryption-backfill";
|
||||
|
||||
const KEY = "a".repeat(64);
|
||||
|
||||
beforeEach(() => {
|
||||
vi.stubEnv("ENCRYPTION_KEYS", "");
|
||||
vi.stubEnv("ENCRYPTION_ACTIVE_KEY_ID", "");
|
||||
vi.stubEnv("ENCRYPTION_KEY", KEY);
|
||||
store.conversations = [
|
||||
{
|
||||
id: "c1",
|
||||
userId: "u1",
|
||||
title: "Why is my pressure up after the new tablets?",
|
||||
titleEncrypted: null,
|
||||
},
|
||||
// Already migrated: left alone.
|
||||
{
|
||||
id: "c2",
|
||||
userId: "u1",
|
||||
title: null,
|
||||
titleEncrypted: encryptToBytes("Sleep this week"),
|
||||
},
|
||||
{ id: "c3", userId: "u2", title: "other user", titleEncrypted: null },
|
||||
];
|
||||
store.entries = [
|
||||
{
|
||||
id: "e1",
|
||||
userId: "u1",
|
||||
note: "after the long ride",
|
||||
noteEncrypted: null,
|
||||
},
|
||||
{ id: "e2", userId: "u1", note: null, noteEncrypted: null }, // no note
|
||||
// An empty legacy note is still a readable value; it migrates to "none".
|
||||
{ id: "e3", userId: "u1", note: "", noteEncrypted: null },
|
||||
{ id: "e4", userId: "u2", note: "other user", noteEncrypted: null },
|
||||
];
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
vi.unstubAllEnvs();
|
||||
});
|
||||
|
||||
describe("runFreeTextEncryptionBackfillForUser", () => {
|
||||
it("seals titles and notes, nulls the readable column, and reads back the same text", async () => {
|
||||
const summary = await runFreeTextEncryptionBackfillForUser("u1");
|
||||
|
||||
expect(summary).toEqual({
|
||||
conversationTitlesMigrated: 1,
|
||||
metricNotesMigrated: 2,
|
||||
});
|
||||
|
||||
const c1 = store.conversations.find((r) => r.id === "c1")!;
|
||||
expect(c1.title).toBeNull();
|
||||
expect(readNote(c1.titleEncrypted, null)).toBe(
|
||||
"Why is my pressure up after the new tablets?",
|
||||
);
|
||||
|
||||
const e1 = store.entries.find((r) => r.id === "e1")!;
|
||||
expect(e1.note).toBeNull();
|
||||
expect(readNote(e1.noteEncrypted, null)).toBe("after the long ride");
|
||||
|
||||
const e3 = store.entries.find((r) => r.id === "e3")!;
|
||||
expect(e3.note).toBeNull();
|
||||
expect(e3.noteEncrypted).toBeNull();
|
||||
});
|
||||
|
||||
it("keeps an already sealed row and a row without a note as they were", async () => {
|
||||
const before = store.conversations.find(
|
||||
(r) => r.id === "c2",
|
||||
)!.titleEncrypted;
|
||||
await runFreeTextEncryptionBackfillForUser("u1");
|
||||
const c2 = store.conversations.find((r) => r.id === "c2")!;
|
||||
expect(c2.titleEncrypted).toBe(before);
|
||||
const e2 = store.entries.find((r) => r.id === "e2")!;
|
||||
expect(e2).toMatchObject({ note: null, noteEncrypted: null });
|
||||
});
|
||||
|
||||
it("seals the readable value when a row holds both, since only an older writer can have put it there", async () => {
|
||||
store.conversations[1].title = "Renamed by the previous release";
|
||||
await runFreeTextEncryptionBackfillForUser("u1");
|
||||
const c2 = store.conversations.find((r) => r.id === "c2")!;
|
||||
expect(c2.title).toBeNull();
|
||||
expect(readNote(c2.titleEncrypted, null)).toBe(
|
||||
"Renamed by the previous release",
|
||||
);
|
||||
});
|
||||
|
||||
it("does not touch another account's rows", async () => {
|
||||
await runFreeTextEncryptionBackfillForUser("u1");
|
||||
expect(store.conversations.find((r) => r.id === "c3")).toMatchObject({
|
||||
title: "other user",
|
||||
titleEncrypted: null,
|
||||
});
|
||||
expect(store.entries.find((r) => r.id === "e4")).toMatchObject({
|
||||
note: "other user",
|
||||
noteEncrypted: null,
|
||||
});
|
||||
});
|
||||
|
||||
it("is idempotent: a second run changes nothing", async () => {
|
||||
await runFreeTextEncryptionBackfillForUser("u1");
|
||||
const sealed = store.conversations.find(
|
||||
(r) => r.id === "c1",
|
||||
)!.titleEncrypted;
|
||||
const second = await runFreeTextEncryptionBackfillForUser("u1");
|
||||
expect(second).toEqual({
|
||||
conversationTitlesMigrated: 0,
|
||||
metricNotesMigrated: 0,
|
||||
});
|
||||
expect(store.conversations.find((r) => r.id === "c1")!.titleEncrypted).toBe(
|
||||
sealed,
|
||||
);
|
||||
});
|
||||
|
||||
it("is fail-closed: without a key every readable row stays intact", async () => {
|
||||
vi.stubEnv("ENCRYPTION_KEY", "");
|
||||
vi.stubEnv("ENCRYPTION_KEYS", "");
|
||||
await expect(runFreeTextEncryptionBackfillForUser("u1")).rejects.toThrow();
|
||||
expect(store.conversations.find((r) => r.id === "c1")).toMatchObject({
|
||||
title: "Why is my pressure up after the new tablets?",
|
||||
titleEncrypted: null,
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -107,6 +107,7 @@ describe("maintenance registrar queue policies", () => {
|
||||
["INTAKE_SLOT_DEDUP_QUEUE", "exclusive"],
|
||||
["NOTE_ENCRYPTION_BACKFILL_QUEUE", "exclusive"],
|
||||
["MED_NOTES_ENCRYPTION_BACKFILL_QUEUE", "exclusive"],
|
||||
["FREE_TEXT_ENCRYPTION_BACKFILL_QUEUE", "exclusive"],
|
||||
["DOCUMENT_THUMBNAIL_BACKFILL_QUEUE", "exclusive"],
|
||||
["CONTENT_INDEX_BACKFILL_QUEUE", "exclusive"],
|
||||
["ENCRYPTION_KEY_ROTATE_QUEUE", "exclusive"],
|
||||
|
||||
@@ -0,0 +1,245 @@
|
||||
/**
|
||||
* v1.39.3 — boot-time converging backfill that moves the last two readable
|
||||
* free-text columns to AES-256-GCM at rest:
|
||||
* `CoachConversation.title` -> `titleEncrypted` (the opening words of the
|
||||
* first Coach message, or a rename) and
|
||||
* `CustomMetricEntry.note` -> `noteEncrypted`.
|
||||
*
|
||||
* Modelled on `med-notes-encryption-backfill.ts`: a discovery query enqueues
|
||||
* one job per user still holding an un-migrated row, the per-user handler walks
|
||||
* that user's rows, and the pass is idempotent across reboots — once a row is
|
||||
* migrated it drops off the discovery + candidate sets.
|
||||
*
|
||||
* DATA-LOSS SAFETY:
|
||||
* - Per row, the encrypt-then-null happens in a SINGLE interactive
|
||||
* transaction guarded by a re-read inside the tx, so the readable column is
|
||||
* only nulled AFTER the ciphertext is written in the same atomic unit. A
|
||||
* row that had text is never left with neither.
|
||||
* - FAIL-CLOSED: the encryption helper throws on a missing / malformed key.
|
||||
* The throw aborts the transaction (the row is untouched) and propagates so
|
||||
* pg-boss retries.
|
||||
* - IDEMPOTENT: the guard (readable column IS NOT NULL) is re-checked inside
|
||||
* the tx, so a re-run, or two workers racing, migrates a row at most once;
|
||||
* a second pass migrates zero rows.
|
||||
* - The guard is the readable column alone, not "and no ciphertext yet".
|
||||
* From this release on nothing writes the readable column, so a row holding
|
||||
* both can only come from a writer of the previous release still running
|
||||
* after the upgrade (a rename, an edited note). Its readable value is then
|
||||
* the newer one, and it is the one sealed.
|
||||
*
|
||||
* The readable columns are NOT dropped in this release. That is a follow-up
|
||||
* release, once this backfill reports zero remaining rows on every instance;
|
||||
* both columns carry the marker in `schema.prisma`.
|
||||
*
|
||||
* Boot discovery is staggered past the startup storm (`startAfter`), like the
|
||||
* other boot backfills; the daily discovery tick passes no offset.
|
||||
*
|
||||
* The queue name MUST be registered in the maintenance registrar
|
||||
* (`src/lib/jobs/reminder/register-maintenance.ts`) so pg-boss provisions it at
|
||||
* boot; an unregistered queue silently never drains.
|
||||
*/
|
||||
import { prisma } from "@/lib/db";
|
||||
import { annotate } from "@/lib/logging/context";
|
||||
import { getGlobalBoss } from "@/lib/jobs/boss-instance";
|
||||
import { encryptToBytes } from "@/lib/ai/coach/bytes-codec";
|
||||
import { encryptNote } from "@/lib/crypto/note-cipher";
|
||||
|
||||
export const FREE_TEXT_ENCRYPTION_BACKFILL_QUEUE =
|
||||
"free-text-encryption-backfill";
|
||||
|
||||
/** Serial — one transaction per row, kept off the request pool. */
|
||||
export const FREE_TEXT_ENCRYPTION_BACKFILL_CONCURRENCY = 1;
|
||||
|
||||
/** How many candidate rows to pull per page within a user's pass. */
|
||||
const PAGE_SIZE = 200;
|
||||
|
||||
export interface FreeTextEncryptionBackfillPayload {
|
||||
/** Absent on the daily discovery tick; present for a per-user job. */
|
||||
userId?: string;
|
||||
enqueuedAt?: string;
|
||||
}
|
||||
|
||||
export interface FreeTextEncryptionBackfillSummary {
|
||||
conversationTitlesMigrated: number;
|
||||
metricNotesMigrated: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Migrate one conversation's readable title into `titleEncrypted` and null the
|
||||
* readable column, atomically and idempotently. Returns true if it changed the
|
||||
* row, false if a concurrent pass already did.
|
||||
*/
|
||||
async function migrateConversationTitle(id: string): Promise<boolean> {
|
||||
return prisma.$transaction(async (tx) => {
|
||||
const fresh = await tx.coachConversation.findUnique({
|
||||
where: { id },
|
||||
select: { title: true },
|
||||
});
|
||||
if (!fresh || fresh.title === null) return false;
|
||||
// FAIL-CLOSED: a key error throws here and rolls the tx back.
|
||||
await tx.coachConversation.update({
|
||||
where: { id },
|
||||
data: { titleEncrypted: encryptToBytes(fresh.title), title: null },
|
||||
});
|
||||
return true;
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Migrate one custom-metric reading's readable note into `noteEncrypted` and
|
||||
* null the readable column, atomically and idempotently.
|
||||
*/
|
||||
async function migrateMetricNote(id: string): Promise<boolean> {
|
||||
return prisma.$transaction(async (tx) => {
|
||||
const fresh = await tx.customMetricEntry.findUnique({
|
||||
where: { id },
|
||||
select: { note: true },
|
||||
});
|
||||
if (!fresh || fresh.note === null) return false;
|
||||
await tx.customMetricEntry.update({
|
||||
where: { id },
|
||||
// An empty note reads back as "no note" either way, so it stores none.
|
||||
data: { noteEncrypted: encryptNote(fresh.note), note: null },
|
||||
});
|
||||
return true;
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Per-user queue handler. Walks every un-migrated title and note of one
|
||||
* account and migrates each in its own transaction. Safe to re-run.
|
||||
*/
|
||||
export async function runFreeTextEncryptionBackfillForUser(
|
||||
userId: string,
|
||||
): Promise<FreeTextEncryptionBackfillSummary> {
|
||||
let conversationTitlesMigrated = 0;
|
||||
let metricNotesMigrated = 0;
|
||||
// Every row this pass has sealed. A row that comes back on a later page was
|
||||
// not cleared, so the pass would never converge; it stops loudly instead of
|
||||
// re-sealing the same rows forever.
|
||||
const sealed = new Set<string>();
|
||||
const once = (id: string) => {
|
||||
if (sealed.has(id)) {
|
||||
throw new Error(
|
||||
`free-text encryption backfill: row ${id} still readable after sealing`,
|
||||
);
|
||||
}
|
||||
sealed.add(id);
|
||||
};
|
||||
|
||||
// Each migration nulls the readable column, so a migrated row drops out of
|
||||
// the next `IS NOT NULL` page.
|
||||
for (;;) {
|
||||
const rows = await prisma.coachConversation.findMany({
|
||||
where: { userId, title: { not: null } },
|
||||
select: { id: true },
|
||||
take: PAGE_SIZE,
|
||||
});
|
||||
if (rows.length === 0) break;
|
||||
let migratedThisPage = 0;
|
||||
for (const { id } of rows) {
|
||||
once(id);
|
||||
if (await migrateConversationTitle(id)) {
|
||||
conversationTitlesMigrated += 1;
|
||||
migratedThisPage += 1;
|
||||
}
|
||||
}
|
||||
// A page where nothing moved (every row taken by a racing worker) ends
|
||||
// the loop, so it always terminates.
|
||||
if (migratedThisPage === 0) break;
|
||||
}
|
||||
|
||||
for (;;) {
|
||||
const rows = await prisma.customMetricEntry.findMany({
|
||||
where: { userId, note: { not: null } },
|
||||
select: { id: true },
|
||||
take: PAGE_SIZE,
|
||||
});
|
||||
if (rows.length === 0) break;
|
||||
let migratedThisPage = 0;
|
||||
for (const { id } of rows) {
|
||||
once(id);
|
||||
if (await migrateMetricNote(id)) {
|
||||
metricNotesMigrated += 1;
|
||||
migratedThisPage += 1;
|
||||
}
|
||||
}
|
||||
if (migratedThisPage === 0) break;
|
||||
}
|
||||
|
||||
annotate({
|
||||
action: {
|
||||
name: "free_text.encryption.backfill",
|
||||
details: {
|
||||
conversation_titles_migrated: conversationTitlesMigrated,
|
||||
metric_notes_migrated: metricNotesMigrated,
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
return { conversationTitlesMigrated, metricNotesMigrated };
|
||||
}
|
||||
|
||||
/**
|
||||
* Discovery. Finds every user still holding a readable title or note and
|
||||
* enqueues one backfill job per account. The 0357 partial indexes match both
|
||||
* predicates, so the scan is index-only and converges to empty. pg-boss
|
||||
* `singletonKey` coalesces duplicate sends. Best-effort: errors come back
|
||||
* through the result value so worker boot never fails on a miss.
|
||||
*/
|
||||
export async function enqueueBootTimeFreeTextEncryptionBackfill(
|
||||
// Boot-storm stagger in seconds; 0 (the cron tick) keeps immediate semantics.
|
||||
startAfterSeconds: number = 0,
|
||||
): Promise<{ enqueued: number; skipped: number; error: string | null }> {
|
||||
const boss = getGlobalBoss();
|
||||
if (!boss) {
|
||||
return { enqueued: 0, skipped: 0, error: null };
|
||||
}
|
||||
|
||||
try {
|
||||
const [titleUsers, noteUsers] = await Promise.all([
|
||||
prisma.$queryRaw<{ user_id: string }[]>`
|
||||
SELECT DISTINCT user_id FROM coach_conversations
|
||||
WHERE title IS NOT NULL`,
|
||||
prisma.$queryRaw<{ user_id: string }[]>`
|
||||
SELECT DISTINCT user_id FROM custom_metric_entries
|
||||
WHERE note IS NOT NULL`,
|
||||
]);
|
||||
|
||||
const userIds = new Set<string>();
|
||||
for (const { user_id } of titleUsers) userIds.add(user_id);
|
||||
for (const { user_id } of noteUsers) userIds.add(user_id);
|
||||
|
||||
let enqueued = 0;
|
||||
let skipped = 0;
|
||||
for (const userId of userIds) {
|
||||
const payload: FreeTextEncryptionBackfillPayload = {
|
||||
userId,
|
||||
enqueuedAt: new Date().toISOString(),
|
||||
};
|
||||
const jobId = await boss.send(
|
||||
FREE_TEXT_ENCRYPTION_BACKFILL_QUEUE,
|
||||
payload,
|
||||
{
|
||||
retryLimit: 5,
|
||||
retryDelay: 60,
|
||||
retryBackoff: true,
|
||||
singletonKey: `free-text-encryption-backfill|${userId}`,
|
||||
...(startAfterSeconds > 0 ? { startAfter: startAfterSeconds } : {}),
|
||||
},
|
||||
);
|
||||
if (jobId) {
|
||||
enqueued += 1;
|
||||
} else {
|
||||
skipped += 1;
|
||||
}
|
||||
}
|
||||
return { enqueued, skipped, error: null };
|
||||
} catch (err) {
|
||||
return {
|
||||
enqueued: 0,
|
||||
skipped: 0,
|
||||
error: err instanceof Error ? err.message : String(err),
|
||||
};
|
||||
}
|
||||
}
|
||||
@@ -57,6 +57,7 @@ export const JOB_FACT_ALLOWLIST: ReadonlySet<string> = new Set([
|
||||
"context_surfaced",
|
||||
"continuation",
|
||||
"continued",
|
||||
"conversation_titles_migrated",
|
||||
"created",
|
||||
"daily_rows_retired",
|
||||
"days_consolidated",
|
||||
@@ -129,6 +130,7 @@ export const JOB_FACT_ALLOWLIST: ReadonlySet<string> = new Set([
|
||||
"medications",
|
||||
"medications_evaluated",
|
||||
"metric_assessments_warmed",
|
||||
"metric_notes_migrated",
|
||||
"missed_doses_minted",
|
||||
"module_off",
|
||||
"mood_entries_migrated",
|
||||
|
||||
@@ -92,6 +92,13 @@ import {
|
||||
enqueueBootTimeMedNotesEncryptionBackfill,
|
||||
type MedNotesEncryptionBackfillPayload,
|
||||
} from "@/lib/jobs/med-notes-encryption-backfill";
|
||||
import {
|
||||
FREE_TEXT_ENCRYPTION_BACKFILL_QUEUE,
|
||||
FREE_TEXT_ENCRYPTION_BACKFILL_CONCURRENCY,
|
||||
runFreeTextEncryptionBackfillForUser,
|
||||
enqueueBootTimeFreeTextEncryptionBackfill,
|
||||
type FreeTextEncryptionBackfillPayload,
|
||||
} from "@/lib/jobs/free-text-encryption-backfill";
|
||||
import {
|
||||
CONTENT_INDEX_BACKFILL_QUEUE,
|
||||
CONTENT_INDEX_BACKFILL_CONCURRENCY,
|
||||
@@ -338,6 +345,13 @@ const MCP_TOKEN_CLEANUP_CRON = "0 4 * * *";
|
||||
// boss poll.
|
||||
|
||||
const MED_NOTES_ENCRYPTION_BACKFILL_CRON = "5 4 * * *";
|
||||
// v1.39.3 — converging backfill that moves the Coach conversation title and the
|
||||
// custom-metric reading note to AES-256-GCM at rest. Boot discovery (staggered
|
||||
// past the startup storm) enqueues one per-user job; a daily 04:07
|
||||
// Europe/Berlin discovery tick re-fans for rows written by a previous-release
|
||||
// process after the upgrade. Two minutes after the medication-note tick so the
|
||||
// two do not share a poll.
|
||||
const FREE_TEXT_ENCRYPTION_BACKFILL_CRON = "7 4 * * *";
|
||||
|
||||
const allQueues = [
|
||||
// v1.25 (W-ENV) — nightly environmental-context fetch. A daily discovery tick
|
||||
@@ -436,6 +450,10 @@ const allQueues = [
|
||||
// plaintext medication note; without this entry pg-boss never provisions the
|
||||
// queue and both the boot enqueue and the cron silently no-op.
|
||||
MED_NOTES_ENCRYPTION_BACKFILL_QUEUE,
|
||||
// v1.39.3 — Coach title + custom-metric note encryption backfill. Without
|
||||
// this entry pg-boss never provisions the queue and both the boot enqueue
|
||||
// and the cron silently no-op.
|
||||
FREE_TEXT_ENCRYPTION_BACKFILL_QUEUE,
|
||||
// Document vault — daily physical purge for tombstones past the 30-day
|
||||
// undo grace (returns the encrypted blob's TOAST space). Without this entry
|
||||
// the daily schedule silently no-ops and "deleted" documents hold backup
|
||||
@@ -593,6 +611,9 @@ const schedules: ScheduleEntry[] = [
|
||||
// fan out one per-user job per account still holding a plaintext medication
|
||||
// note.
|
||||
[MED_NOTES_ENCRYPTION_BACKFILL_QUEUE, MED_NOTES_ENCRYPTION_BACKFILL_CRON],
|
||||
// v1.39.3 — daily 04:07 Europe/Berlin discovery for the Coach title +
|
||||
// custom-metric note backfill. Empty payload = fan out per user.
|
||||
[FREE_TEXT_ENCRYPTION_BACKFILL_QUEUE, FREE_TEXT_ENCRYPTION_BACKFILL_CRON],
|
||||
// Document vault — daily 04:10 Europe/Berlin purge for tombstoned
|
||||
// documents past the 30-day undo grace.
|
||||
[DOCUMENT_PURGE_QUEUE, DOCUMENT_PURGE_CRON, cronIsTheRetry],
|
||||
@@ -651,6 +672,11 @@ const queuePolicies: QueuePolicyTable = {
|
||||
reason:
|
||||
"Per-user medication-note encryption backfill; discovery drops the user once no plaintext note remains.",
|
||||
},
|
||||
[FREE_TEXT_ENCRYPTION_BACKFILL_QUEUE]: {
|
||||
policy: "exclusive",
|
||||
reason:
|
||||
"Per-user Coach title + custom-metric note encryption backfill; discovery drops the user once no readable value remains.",
|
||||
},
|
||||
[DOCUMENT_THUMBNAIL_BACKFILL_QUEUE]: {
|
||||
policy: "exclusive",
|
||||
reason:
|
||||
@@ -1188,6 +1214,65 @@ export async function registerMaintenanceQueues(
|
||||
},
|
||||
);
|
||||
|
||||
// v1.39.3 — Coach title + custom-metric note encryption backfill worker.
|
||||
// Same shape as the medication-note worker above: an empty-userId payload is
|
||||
// the daily discovery tick, a userId payload migrates that account's rows.
|
||||
await createAndWork<Partial<FreeTextEncryptionBackfillPayload>>(
|
||||
boss,
|
||||
FREE_TEXT_ENCRYPTION_BACKFILL_QUEUE,
|
||||
{ localConcurrency: FREE_TEXT_ENCRYPTION_BACKFILL_CONCURRENCY },
|
||||
async (jobs) => {
|
||||
let users = 0;
|
||||
let discoveryEnqueued = 0;
|
||||
let discoverySkipped = 0;
|
||||
let discoveryFailed = 0;
|
||||
let titles = 0;
|
||||
let notes = 0;
|
||||
for (const job of jobs) {
|
||||
const { userId } = job.data;
|
||||
if (!userId) {
|
||||
const result = await enqueueBootTimeFreeTextEncryptionBackfill();
|
||||
discoveryEnqueued += result.enqueued;
|
||||
discoverySkipped += result.skipped;
|
||||
if (result.error) discoveryFailed++;
|
||||
workerLog(
|
||||
"info",
|
||||
`[free-text-encryption-backfill] daily discovery enqueued=${result.enqueued} skipped=${result.skipped}${result.error ? ` error=${result.error}` : ""}`,
|
||||
);
|
||||
continue;
|
||||
}
|
||||
try {
|
||||
const { conversationTitlesMigrated, metricNotesMigrated } =
|
||||
await runFreeTextEncryptionBackfillForUser(userId);
|
||||
users++;
|
||||
titles += conversationTitlesMigrated;
|
||||
notes += metricNotesMigrated;
|
||||
workerLog(
|
||||
"info",
|
||||
`[free-text-encryption-backfill] user=${userId} titles=${conversationTitlesMigrated} notes=${metricNotesMigrated}`,
|
||||
);
|
||||
} catch (err) {
|
||||
recordError();
|
||||
workerLog(
|
||||
"error",
|
||||
`[free-text-encryption-backfill] user=${userId} failed`,
|
||||
err,
|
||||
);
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
return jobDone({
|
||||
jobs: jobs.length,
|
||||
users,
|
||||
discovery_enqueued: discoveryEnqueued,
|
||||
discovery_skipped: discoverySkipped,
|
||||
discovery_failed: discoveryFailed,
|
||||
conversation_titles_migrated: titles,
|
||||
metric_notes_migrated: notes,
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
// Document vault P2 — on-demand content-search index backfill worker. The
|
||||
// trigger endpoint sends one per-user job; this handler indexes that user's
|
||||
// not-yet-indexed documents (consent + budget gated, bounded + resumable).
|
||||
@@ -1550,6 +1635,33 @@ export async function enqueueMaintenanceBootDiscovery(): Promise<void> {
|
||||
);
|
||||
}
|
||||
|
||||
// v1.39.3 — Coach title + custom-metric note encryption backfill. Staggered
|
||||
// one step past the note-encryption backfill so it starts after the boot
|
||||
// storm rather than on the first poll.
|
||||
try {
|
||||
const { enqueued, skipped, error } =
|
||||
await enqueueBootTimeFreeTextEncryptionBackfill(
|
||||
BOOT_BACKFILL_STAGGER_SECONDS * 6,
|
||||
);
|
||||
if (error) {
|
||||
workerLog(
|
||||
"error",
|
||||
`[free-text-encryption-backfill] boot discovery failed: ${error}`,
|
||||
);
|
||||
} else {
|
||||
workerLog(
|
||||
"info",
|
||||
`[free-text-encryption-backfill] boot discovery: enqueued=${enqueued} skipped=${skipped}`,
|
||||
);
|
||||
}
|
||||
} catch (err) {
|
||||
workerLog(
|
||||
"error",
|
||||
"[free-text-encryption-backfill] boot discovery threw an unexpected error",
|
||||
err,
|
||||
);
|
||||
}
|
||||
|
||||
// Document vault — preview-thumbnail backfill. Finds every account holding a
|
||||
// thumbnailable document (image or PDF) without a preview and enqueues one
|
||||
// per-user backfill pass. Idempotent across reboots: a rendered thumbnail
|
||||
|
||||
@@ -127,6 +127,7 @@ const subsystemSurface = {
|
||||
"encryption-key-rotate": { audience: "system" },
|
||||
"mcp-token-cleanup": { audience: "system" },
|
||||
"med-notes-encryption-backfill": { audience: "system" },
|
||||
"free-text-encryption-backfill": { audience: "system" },
|
||||
"document-tombstone-purge": { audience: "system" },
|
||||
// v1.37.19 (A7-4) — hourly heal of summary rows stuck on PENDING.
|
||||
"document-summary-reaper": { audience: "system" },
|
||||
|
||||
@@ -798,6 +798,8 @@ const customMetricEntryBackupSchema = z
|
||||
unit: z.string().min(1),
|
||||
measuredAt: isoDateTime,
|
||||
note: z.string().nullable().default(null),
|
||||
/// v1.39.3 — the note's ciphertext, DR payloads only.
|
||||
noteEncrypted: base64BytesSchema.nullable().optional(),
|
||||
createdAt: isoDateTime.optional(),
|
||||
/// v1.37.20 (A3-11) — the entry tombstone rides DR payloads so a restore
|
||||
/// brings the account back exactly as it stood, undo affordance included.
|
||||
@@ -1365,7 +1367,10 @@ const coachConversationDocumentBackupSchema = z
|
||||
const coachConversationBackupSchema = z
|
||||
.object({
|
||||
id: z.string().min(1),
|
||||
title: z.string(),
|
||||
// v1.39.3 — a portable file (and every file written before) carries the
|
||||
// readable title; a disaster-recovery file carries only the ciphertext.
|
||||
title: z.string().optional(),
|
||||
titleEncrypted: base64BytesSchema.optional(),
|
||||
// NOT optional with a default. The fence is permanent and a file that does
|
||||
// not state it is a file that cannot be trusted to re-fence the
|
||||
// conversation, so the absence has to be visible rather than defaulted to
|
||||
|
||||
@@ -112,6 +112,8 @@ const COACH_PLAN_CUE = "if the evening reading is over 140";
|
||||
const COACH_PLAN_ACTION = "then walk twenty minutes before dinner tomorrow";
|
||||
const COACH_REMINDER_NOTE = "ask how the evening walks are going";
|
||||
const COACH_SUMMARY = "earlier turns: weight trend and evening walks";
|
||||
const METRIC_LEGACY_NOTE = "left hand, after the long ride";
|
||||
const METRIC_SEALED_NOTE = "right hand, rested";
|
||||
const COACH_USER_TURN = "my readings look higher this week, is that real?";
|
||||
const COACH_ASSISTANT_TURN =
|
||||
"the last seven mornings average 4 mmHg above the fortnight before";
|
||||
@@ -723,13 +725,26 @@ async function seedEveryTwoEndedModel(prisma: PrismaClient): Promise<void> {
|
||||
userId: OWNER_ID,
|
||||
name: "Grip strength",
|
||||
unit: "kg",
|
||||
// v1.39.3 — one reading whose note is still readable (a row the
|
||||
// free-text backfill has not reached) and one sealed the current way.
|
||||
// Both must come back sealed, and neither readable.
|
||||
entries: {
|
||||
create: {
|
||||
userId: OWNER_ID,
|
||||
value: 44,
|
||||
unit: "kg",
|
||||
measuredAt: AT("2026-07-01T18:00:00.000Z"),
|
||||
},
|
||||
create: [
|
||||
{
|
||||
userId: OWNER_ID,
|
||||
value: 44,
|
||||
unit: "kg",
|
||||
measuredAt: AT("2026-07-01T18:00:00.000Z"),
|
||||
note: METRIC_LEGACY_NOTE,
|
||||
},
|
||||
{
|
||||
userId: OWNER_ID,
|
||||
value: 45,
|
||||
unit: "kg",
|
||||
measuredAt: AT("2026-07-02T18:00:00.000Z"),
|
||||
noteEncrypted: encryptToBytes(METRIC_SEALED_NOTE),
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
});
|
||||
@@ -923,7 +938,9 @@ async function seedEveryTwoEndedModel(prisma: PrismaClient): Promise<void> {
|
||||
await prisma.coachConversation.create({
|
||||
data: {
|
||||
userId: OWNER_ID,
|
||||
title: "About my June labs",
|
||||
// v1.39.3 — sealed the current way; the thread above keeps a readable
|
||||
// legacy title, the shape of a row the backfill has not reached.
|
||||
titleEncrypted: encryptToBytes("About my June labs"),
|
||||
documentScoped: true,
|
||||
messages: {
|
||||
create: [
|
||||
@@ -1524,6 +1541,20 @@ describe("every model the plan claims two-ended survives a real restore", () =>
|
||||
sourceId: "4711",
|
||||
aiReadDeferred: true,
|
||||
});
|
||||
const metricEntries = await prisma.customMetricEntry.findMany({
|
||||
where: { userId: OWNER_ID },
|
||||
orderBy: { measuredAt: "asc" },
|
||||
});
|
||||
expect(
|
||||
metricEntries.map((entry) => ({
|
||||
note: readNote(entry.noteEncrypted, entry.note),
|
||||
readable: entry.note,
|
||||
})),
|
||||
"both reading notes come back, sealed, with nothing readable",
|
||||
).toEqual([
|
||||
{ note: METRIC_LEGACY_NOTE, readable: null },
|
||||
{ note: METRIC_SEALED_NOTE, readable: null },
|
||||
]);
|
||||
const threads = await prisma.coachConversation.findMany({
|
||||
where: { userId: OWNER_ID },
|
||||
orderBy: { createdAt: "asc" },
|
||||
@@ -1534,7 +1565,9 @@ describe("every model the plan claims two-ended survives a real restore", () =>
|
||||
});
|
||||
expect(
|
||||
threads.map((thread) => ({
|
||||
title: thread.title,
|
||||
title: readNote(thread.titleEncrypted, thread.title),
|
||||
// A restore seals every title and never writes the readable column.
|
||||
readableTitle: thread.title,
|
||||
documentScoped: thread.documentScoped,
|
||||
summary: thread.summaryEncrypted
|
||||
? decryptFromBytes(thread.summaryEncrypted)
|
||||
@@ -1552,6 +1585,7 @@ describe("every model the plan claims two-ended survives a real restore", () =>
|
||||
).toEqual([
|
||||
{
|
||||
title: "How is my blood pressure trending?",
|
||||
readableTitle: null,
|
||||
documentScoped: false,
|
||||
summary: COACH_SUMMARY,
|
||||
summaryTurnCount: 4,
|
||||
@@ -1573,6 +1607,7 @@ describe("every model the plan claims two-ended survives a real restore", () =>
|
||||
},
|
||||
{
|
||||
title: "About my June labs",
|
||||
readableTitle: null,
|
||||
documentScoped: true,
|
||||
summary: null,
|
||||
summaryTurnCount: 0,
|
||||
|
||||
@@ -0,0 +1,181 @@
|
||||
/**
|
||||
* v1.39.3 — the Coach conversation title and the custom-metric reading note
|
||||
* move to AES-256-GCM at rest. Migration 0357 only adds the ciphertext
|
||||
* columns; the existing rows are sealed by the boot-time backfill. This runs
|
||||
* the real discovery and the real per-user pass against Postgres, with the
|
||||
* migrated schema:
|
||||
*
|
||||
* - discovery finds exactly the accounts still holding a readable value;
|
||||
* - the pass seals every such row in its own transaction, nulls the readable
|
||||
* column, and reads back the same text;
|
||||
* - a second pass and a second discovery find nothing (it converges);
|
||||
* - a row written the current way is left as it was.
|
||||
*/
|
||||
import { afterEach, beforeEach, describe, expect, it } from "vitest";
|
||||
|
||||
import { setGlobalBoss } from "@/lib/jobs/boss-instance";
|
||||
import { encryptToBytes } from "@/lib/ai/coach/bytes-codec";
|
||||
import { readNote } from "@/lib/crypto/note-cipher";
|
||||
import {
|
||||
enqueueBootTimeFreeTextEncryptionBackfill,
|
||||
runFreeTextEncryptionBackfillForUser,
|
||||
} from "@/lib/jobs/free-text-encryption-backfill";
|
||||
import { getPrismaClient, truncateAllTables } from "./setup";
|
||||
|
||||
function makeCapturingBoss() {
|
||||
const sent: { queue: string; userId: string; startAfter?: number }[] = [];
|
||||
const boss = {
|
||||
send: async (
|
||||
queue: string,
|
||||
payload: { userId?: string },
|
||||
opts: { startAfter?: number },
|
||||
) => {
|
||||
if (payload.userId) sent.push({ queue, userId: payload.userId, ...opts });
|
||||
return `job-${sent.length}`;
|
||||
},
|
||||
};
|
||||
return { boss, sent };
|
||||
}
|
||||
|
||||
async function seedUser(id: string) {
|
||||
await getPrismaClient().user.create({
|
||||
data: {
|
||||
id,
|
||||
username: id,
|
||||
email: `${id}@example.test`,
|
||||
timezone: "Europe/Berlin",
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
beforeEach(async () => {
|
||||
await truncateAllTables(getPrismaClient());
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
setGlobalBoss(null as never);
|
||||
});
|
||||
|
||||
describe("free-text encryption backfill (real Postgres)", () => {
|
||||
it("discovers, seals, and converges", async () => {
|
||||
const prisma = getPrismaClient();
|
||||
await Promise.all([seedUser("ft-a"), seedUser("ft-b"), seedUser("ft-c")]);
|
||||
|
||||
// ft-a: a legacy readable title and a sealed one.
|
||||
await prisma.coachConversation.create({
|
||||
data: { id: "conv-legacy", userId: "ft-a", title: "Legacy title" },
|
||||
});
|
||||
await prisma.coachConversation.create({
|
||||
data: {
|
||||
id: "conv-sealed",
|
||||
userId: "ft-a",
|
||||
titleEncrypted: encryptToBytes("Sealed title"),
|
||||
},
|
||||
});
|
||||
// ft-b: a legacy readable note on a custom-metric reading.
|
||||
const metric = await prisma.customMetric.create({
|
||||
data: { userId: "ft-b", name: "Grip", unit: "kg" },
|
||||
});
|
||||
await prisma.customMetricEntry.create({
|
||||
data: {
|
||||
id: "entry-legacy",
|
||||
userId: "ft-b",
|
||||
customMetricId: metric.id,
|
||||
value: 40,
|
||||
unit: "kg",
|
||||
measuredAt: new Date("2026-09-01T08:00:00.000Z"),
|
||||
note: "legacy note",
|
||||
},
|
||||
});
|
||||
await prisma.customMetricEntry.create({
|
||||
data: {
|
||||
id: "entry-none",
|
||||
userId: "ft-b",
|
||||
customMetricId: metric.id,
|
||||
value: 41,
|
||||
unit: "kg",
|
||||
measuredAt: new Date("2026-09-02T08:00:00.000Z"),
|
||||
},
|
||||
});
|
||||
// ft-c: nothing readable, only a sealed title.
|
||||
await prisma.coachConversation.create({
|
||||
data: { userId: "ft-c", titleEncrypted: encryptToBytes("Only sealed") },
|
||||
});
|
||||
|
||||
const first = makeCapturingBoss();
|
||||
setGlobalBoss(first.boss as never);
|
||||
const discovered = await enqueueBootTimeFreeTextEncryptionBackfill(180);
|
||||
expect(discovered.error).toBeNull();
|
||||
expect(first.sent.map((s) => s.userId).sort()).toEqual(["ft-a", "ft-b"]);
|
||||
// The boot call is deferred past the startup storm.
|
||||
expect(first.sent.every((s) => s.startAfter === 180)).toBe(true);
|
||||
|
||||
expect(await runFreeTextEncryptionBackfillForUser("ft-a")).toEqual({
|
||||
conversationTitlesMigrated: 1,
|
||||
metricNotesMigrated: 0,
|
||||
});
|
||||
expect(await runFreeTextEncryptionBackfillForUser("ft-b")).toEqual({
|
||||
conversationTitlesMigrated: 0,
|
||||
metricNotesMigrated: 1,
|
||||
});
|
||||
|
||||
const conversations = await prisma.coachConversation.findMany({
|
||||
where: { userId: "ft-a" },
|
||||
orderBy: { id: "asc" },
|
||||
});
|
||||
expect(
|
||||
conversations.map((c) => ({
|
||||
id: c.id,
|
||||
readable: c.title,
|
||||
text: readNote(c.titleEncrypted, c.title),
|
||||
})),
|
||||
).toEqual([
|
||||
{ id: "conv-legacy", readable: null, text: "Legacy title" },
|
||||
{ id: "conv-sealed", readable: null, text: "Sealed title" },
|
||||
]);
|
||||
const entries = await prisma.customMetricEntry.findMany({
|
||||
where: { userId: "ft-b" },
|
||||
orderBy: { id: "asc" },
|
||||
});
|
||||
expect(
|
||||
entries.map((e) => ({
|
||||
id: e.id,
|
||||
readable: e.note,
|
||||
text: readNote(e.noteEncrypted, e.note),
|
||||
})),
|
||||
).toEqual([
|
||||
{ id: "entry-legacy", readable: null, text: "legacy note" },
|
||||
{ id: "entry-none", readable: null, text: null },
|
||||
]);
|
||||
|
||||
// Converged: nothing left to discover, nothing left to seal.
|
||||
const second = makeCapturingBoss();
|
||||
setGlobalBoss(second.boss as never);
|
||||
expect(
|
||||
(await enqueueBootTimeFreeTextEncryptionBackfill()).error,
|
||||
).toBeNull();
|
||||
expect(second.sent).toEqual([]);
|
||||
expect(await runFreeTextEncryptionBackfillForUser("ft-a")).toEqual({
|
||||
conversationTitlesMigrated: 0,
|
||||
metricNotesMigrated: 0,
|
||||
});
|
||||
});
|
||||
|
||||
it("uses the partial indexes migration 0357 adds", async () => {
|
||||
const indexes = await getPrismaClient().$queryRaw<
|
||||
{ indexname: string; indexdef: string }[]
|
||||
>`
|
||||
SELECT indexname, indexdef FROM pg_indexes
|
||||
WHERE indexname IN (
|
||||
'coach_conversations_title_backfill_idx',
|
||||
'custom_metric_entries_note_backfill_idx'
|
||||
)
|
||||
ORDER BY indexname`;
|
||||
expect(indexes.map((i) => i.indexname)).toEqual([
|
||||
"coach_conversations_title_backfill_idx",
|
||||
"custom_metric_entries_note_backfill_idx",
|
||||
]);
|
||||
expect(indexes[0].indexdef).toContain("WHERE (title IS NOT NULL)");
|
||||
expect(indexes[1].indexdef).toContain("WHERE (note IS NOT NULL)");
|
||||
});
|
||||
});
|
||||
Reference in new issue
Block a user