diff --git a/CHANGELOG.md b/CHANGELOG.md index 154891c6c..364568d39 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,24 @@ ### Added +- The mood module can now say what a day would have looked like on your own + past patterns, and how far the day you recorded sits from that. It is a + second, separate reading: your own rating stays the leading one, it is + stored exactly as you gave it, and nothing overwrites it. The comparison + appears only once there are enough days to say anything honest, it always + carries the number of days it was built from and a range rather than a + single figure, and it names which of the things you recorded weighed most on + that day. It describes a connection between things you wrote down. It never + claims a cause. + + Below fifteen days there is nothing at all. Between fifteen and thirty the + page says how far along it is instead of guessing. Between thirty and sixty + the comparison is there and is labelled as provisional. Where the days are + there but nothing in them is steady enough to compare against, it says that + too, rather than producing a number to fill the space. The comparison is + rebuilt overnight from your entries, and if you delete enough of them it + goes away again instead of standing there with a fresh date on it. + - A mood entry can now carry the shape of the day around it. Four sections sit below the sliders — work, contacts, leisure, and one notable event — and every one of them is closed until you open it, so a quick check-in is still diff --git a/docs/api/openapi.yaml b/docs/api/openapi.yaml index 33fb729bd..b3f680fd2 100644 --- a/docs/api/openapi.yaml +++ b/docs/api/openapi.yaml @@ -8357,6 +8357,29 @@ paths: "403": *a6 "422": *a3 "429": *a4 + /api/mood/prognosis: + get: + tags: + - Mood + summary: The day's own rating and what the account's days imply (v1.38) + description: "The newest stored comparison for the account, with the person's own rating for that day beside it. Written + nightly by a job and never by a request: the value cannot be overwritten, which is what keeps the two readings + separate. Absence is explicit (`present: false` with a reason and a count) and is a normal answer, not an error + — the comparison refuses when the history does not support one. Whole-record read: the fit takes the day's + context AND the sleep, activity and recovery figures the owning modules hold, and it names them, so a + section-scoped grant is refused rather than served a filtered list. Gated: `module.disabled` 403 when the mood + module is off." + responses: + "200": + description: The day's two readings, or an explained absence. + content: + application/json: + schema: + $ref: "#/components/schemas/MoodPrognosisEnvelope" + "401": *a1 + "403": *a7 + "422": *a3 + "429": *a4 /api/mood-entries/bulk: post: tags: @@ -32815,6 +32838,184 @@ components: - generatedAt additionalProperties: false description: Per-analyte change between the two most-recent numeric lab panels. Neutral framing, never a diagnosis. + MoodPrognosisEnvelope: + type: object + properties: + data: + $ref: "#/components/schemas/MoodPrognosis" + error: + type: "null" + meta: + type: object + properties: + requestId: + type: string + additionalProperties: false + required: + - data + - error + additionalProperties: false + MoodPrognosis: + anyOf: + - type: object + properties: + present: + type: boolean + const: true + date: + type: string + description: The local day this is about, `YYYY-MM-DD`. + predicted: + type: number + description: The expected pleasantness (A1) for that day, 0-10, on the account's own past pattern. A counterfactual, + never a measurement — and never to be displayed without `n` and the band. + ciLow: + anyOf: + - type: number + - type: "null" + ciHigh: + anyOf: + - type: number + - type: "null" + description: The band, from the validation residuals. It widens when the model fits badly, so a wide band is the signal + to present the value cautiously. + n: + type: integer + minimum: -9007199254740991 + maximum: 9007199254740991 + description: "Complete days the fit used. Display it WHEREVER `predicted` is displayed: a value built from 30 days and + one built from 300 look identical otherwise." + modelVersion: + type: string + computedAt: + type: string + format: date-time + pattern: ^(?:(?:\d\d[2468][048]|\d\d[13579][26]|\d\d0[48]|[02468][048]00|[13579][26]00)-02-29|\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\d|30)|(?:02)-(?:0[1-9]|1\d|2[0-8])))T(?:(?:[01]\d|2[0-3]):[0-5]\d(?::[0-5]\d(?:\.\d+)?)?(?:Z|([+-](?:[01]\d|2[0-3]):[0-5]\d)))$ + ageDays: + anyOf: + - type: integer + minimum: -9007199254740991 + maximum: 9007199254740991 + - type: "null" + description: Whole days between that day and the reader's today. + current: + type: boolean + description: Whether it is fresh enough to back a present-tense sentence. False means the copy must name the day it is + about. + provisional: + type: boolean + description: "True below 60 days of history: label it as provisional." + stage: + type: string + enum: + - provisional + - regular + - seasonal + entries: + type: integer + minimum: -9007199254740991 + maximum: 9007199254740991 + description: Days of history behind the ladder. + seasonalUnlocked: + type: boolean + selfAssessment: + anyOf: + - type: integer + minimum: -9007199254740991 + maximum: 9007199254740991 + - type: "null" + description: "The person's own rating for that day, stored exactly as they gave it. It LEADS: it is the primary value on + any surface that shows both, and it is never merged with `predicted`." + deviation: + anyOf: + - type: string + enum: + - within + - above + - below + - type: "null" + description: Where the self-assessment sat against the BAND, not against `predicted`. `within` means the day went as the + account's own past days suggest. + deviationAmount: + anyOf: + - type: number + - type: "null" + description: How far outside the band, in scale points. `0` when inside. + contributions: + type: array + items: + $ref: "#/components/schemas/MoodPrognosisContribution" + required: + - present + - date + - predicted + - ciLow + - ciHigh + - n + - modelVersion + - computedAt + - ageDays + - current + - provisional + - stage + - entries + - seasonalUnlocked + - selfAssessment + - deviation + - deviationAmount + - contributions + additionalProperties: false + - type: object + properties: + present: + type: boolean + const: false + reason: + type: string + enum: + - no-output-yet + - learning-phase + - no-pattern + description: "`no-output-yet`: too few days for anything. `learning-phase`: enough to describe, not to compare. + `no-pattern`: enough days, nothing steady enough in them. None of the three is an error and none is a + zero." + entries: + type: integer + minimum: -9007199254740991 + maximum: 9007199254740991 + nextThreshold: + anyOf: + - type: integer + minimum: -9007199254740991 + maximum: 9007199254740991 + - type: "null" + description: Days at which the next step unlocks, when a count is missing. + required: + - present + - reason + - entries + - nextThreshold + additionalProperties: false + description: "Two separate readings of one day: what the person recorded, and what a comparison with their own past days + would have expected. They are never merged — the distance between them is the finding, and a single blended + figure destroys it. The comparison is a regularised regression over the account's own history: observational + only, never causal. Any copy built on it must state the value as a counterfactual (a value around X would have + been expected), must carry `n`, and must not say that anything caused anything." + MoodPrognosisContribution: + type: object + properties: + feature: + type: string + description: "Stable feature identifier: `dimension:a2`, `context:workLoad`, `context:workStatus=off` or `linked:steps`. + A client that does not recognise one skips it — an older row can name a feature a later model dropped." + contribution: + type: number + description: "Signed: how far this feature moved the expected value away from the account's own average day. Ordered by + absolute size, largest first." + required: + - feature + - contribution + additionalProperties: false BulkMoodResponseEnvelope: type: object properties: diff --git a/messages/de.json b/messages/de.json index 22e0b9015..fe298d368 100644 --- a/messages/de.json +++ b/messages/de.json @@ -3785,6 +3785,25 @@ "statement": "An Tagen mit „{label}“ lag die Stimmung im Durchschnitt {direction}: {withAvg} gegenüber {withoutAvg}.", "counts": "{withDays} Tage mit, {withoutDays} Tage ohne", "caveat": "Das zeigt einen Zusammenhang, keine Ursache." + }, + "prognosis": { + "title": "Deine Einschätzung und was deine bisherigen Tage nahelegen", + "selfLabel": "Deine Einschätzung", + "expectedLabel": "Auf Basis deiner Muster erwartet", + "band": "Bereich {low} bis {high}", + "statement": "Auf Basis deiner bisherigen Muster wäre für {date} ein Wert um {value} zu erwarten gewesen. Ermittelt aus {n} Tagen, die alles enthalten, was der Vergleich braucht.", + "provisional": "Noch vorläufig: das Muster stützt sich auf eine kurze Strecke von Tagen.", + "notToday": "Das beschreibt {date}, nicht heute.", + "deviationWithin": "Deine eigene Einschätzung liegt in diesem Bereich.", + "deviationAbove": "Deine eigene Einschätzung liegt {amount} Punkte über diesem Bereich.", + "deviationBelow": "Deine eigene Einschätzung liegt {amount} Punkte unter diesem Bereich.", + "weightedTitle": "Was an diesem Tag am stärksten gewichtet wurde", + "weightedUp": "{label} — zählte in Richtung eines höheren erwarteten Werts", + "weightedDown": "{label} — zählte in Richtung eines niedrigeren erwarteten Werts", + "caveat": "Das ist ein Vergleich mit deinen eigenen bisherigen Tagen, keine Aussage darüber, wie ein Tag zustande gekommen ist.", + "learning": "Lernphase: {entries} von {threshold} Tagen erfasst. Ein Vergleich erscheint, sobald genug davon zusammengekommen sind.", + "noPattern": "In den bisher erfassten {entries} Tagen ist nichts gleichmäßig genug, um einen Tag damit zu vergleichen. Wer den Kontext eines Tages mit erfasst, macht mehr davon vergleichbar.", + "featureValue": "{field}: {value}" } }, "derived": { diff --git a/messages/en.json b/messages/en.json index afacc42d2..60afd4fa5 100644 --- a/messages/en.json +++ b/messages/en.json @@ -3785,6 +3785,25 @@ "statement": "On days with “{label}” the mood averaged {direction}: {withAvg} against {withoutAvg}.", "counts": "{withDays} days with, {withoutDays} days without", "caveat": "That shows a connection, not a cause." + }, + "prognosis": { + "title": "Your rating, and what your own days suggest", + "selfLabel": "Your rating", + "expectedLabel": "Expected on your own pattern", + "band": "Range {low} to {high}", + "statement": "On your own patterns so far, a value around {value} would have been expected for {date}. Built from {n} days that carried everything the comparison needs.", + "provisional": "Still provisional: the pattern rests on a short stretch of days.", + "notToday": "This describes {date}, not today.", + "deviationWithin": "Your own rating sits inside that range.", + "deviationAbove": "Your own rating sits {amount} points above that range.", + "deviationBelow": "Your own rating sits {amount} points below that range.", + "weightedTitle": "What weighed most in this day's comparison", + "weightedUp": "{label} — counted towards a higher expected value", + "weightedDown": "{label} — counted towards a lower expected value", + "caveat": "This is a comparison with your own past days, not a statement about how a day came to go the way it did.", + "learning": "Learning phase: {entries} of {threshold} days recorded. A comparison appears once there are enough of them.", + "noPattern": "Nothing in the {entries} days recorded so far is steady enough to compare a day against. Recording a day's context makes more of it comparable.", + "featureValue": "{field}: {value}" } }, "derived": { diff --git a/messages/es.json b/messages/es.json index 7cec1ba45..ac386797c 100644 --- a/messages/es.json +++ b/messages/es.json @@ -3785,6 +3785,25 @@ "statement": "En los días con «{label}» el ánimo promedió {direction}: {withAvg} frente a {withoutAvg}.", "counts": "{withDays} días con, {withoutDays} días sin", "caveat": "Esto muestra una relación, no una causa." + }, + "prognosis": { + "title": "Tu valoración y lo que sugieren tus días anteriores", + "selfLabel": "Tu valoración", + "expectedLabel": "Esperado según tus propios patrones", + "band": "Rango de {low} a {high}", + "statement": "Según tus patrones hasta ahora, para {date} cabría esperar un valor en torno a {value}. Calculado a partir de {n} días que reúnen todo lo que la comparación necesita.", + "provisional": "Todavía provisional: el patrón se apoya en pocos días.", + "notToday": "Esto describe {date}, no hoy.", + "deviationWithin": "Tu propia valoración está dentro de ese rango.", + "deviationAbove": "Tu propia valoración está {amount} puntos por encima de ese rango.", + "deviationBelow": "Tu propia valoración está {amount} puntos por debajo de ese rango.", + "weightedTitle": "Lo que más pesó en la comparación de este día", + "weightedUp": "{label}: contó hacia un valor esperado más alto", + "weightedDown": "{label}: contó hacia un valor esperado más bajo", + "caveat": "Es una comparación con tus propios días anteriores, no una afirmación sobre cómo llegó a transcurrir un día.", + "learning": "Fase de aprendizaje: {entries} de {threshold} días registrados. La comparación aparece cuando haya suficientes.", + "noPattern": "En los {entries} días registrados hasta ahora no hay nada lo bastante constante para comparar un día. Registrar el contexto del día hace comparable una parte mayor.", + "featureValue": "{field}: {value}" } }, "derived": { diff --git a/messages/fr.json b/messages/fr.json index d91f2cf3d..9a51ce76f 100644 --- a/messages/fr.json +++ b/messages/fr.json @@ -3785,6 +3785,25 @@ "statement": "Les jours avec « {label} », l'humeur était en moyenne {direction} : {withAvg} contre {withoutAvg}.", "counts": "{withDays} jours avec, {withoutDays} jours sans", "caveat": "Cela montre un lien, pas une cause." + }, + "prognosis": { + "title": "Ton évaluation et ce que suggèrent tes journées passées", + "selfLabel": "Ton évaluation", + "expectedLabel": "Attendu selon tes propres habitudes", + "band": "Plage de {low} à {high}", + "statement": "D'après tes habitudes jusqu'ici, une valeur autour de {value} aurait été attendue pour {date}. Établi à partir de {n} journées réunissant tout ce dont la comparaison a besoin.", + "provisional": "Encore provisoire : le motif repose sur une courte série de journées.", + "notToday": "Cela décrit {date}, pas aujourd'hui.", + "deviationWithin": "Ta propre évaluation se situe dans cette plage.", + "deviationAbove": "Ta propre évaluation se situe {amount} points au-dessus de cette plage.", + "deviationBelow": "Ta propre évaluation se situe {amount} points en dessous de cette plage.", + "weightedTitle": "Ce qui a le plus pesé dans la comparaison de cette journée", + "weightedUp": "{label} — comptait vers une valeur attendue plus élevée", + "weightedDown": "{label} — comptait vers une valeur attendue plus basse", + "caveat": "C'est une comparaison avec tes propres journées passées, pas une affirmation sur la manière dont une journée s'est déroulée.", + "learning": "Phase d'apprentissage : {entries} journées sur {threshold} enregistrées. La comparaison apparaît une fois qu'il y en a assez.", + "noPattern": "Dans les {entries} journées enregistrées jusqu'ici, rien n'est assez régulier pour comparer une journée. Noter le contexte d'une journée en rend davantage comparable.", + "featureValue": "{field} : {value}" } }, "derived": { diff --git a/messages/it.json b/messages/it.json index 60bfd0bf4..9fc05689f 100644 --- a/messages/it.json +++ b/messages/it.json @@ -3785,6 +3785,25 @@ "statement": "Nei giorni con «{label}» l'umore è stato in media {direction}: {withAvg} contro {withoutAvg}.", "counts": "{withDays} giorni con, {withoutDays} giorni senza", "caveat": "Questo mostra una relazione, non una causa." + }, + "prognosis": { + "title": "La tua valutazione e ciò che suggeriscono le tue giornate passate", + "selfLabel": "La tua valutazione", + "expectedLabel": "Atteso in base alle tue abitudini", + "band": "Intervallo da {low} a {high}", + "statement": "In base alle tue abitudini finora, per {date} ci si sarebbe attesi un valore intorno a {value}. Ricavato da {n} giornate che contengono tutto ciò che serve al confronto.", + "provisional": "Ancora provvisorio: lo schema si appoggia su poche giornate.", + "notToday": "Questo descrive {date}, non oggi.", + "deviationWithin": "La tua valutazione rientra in questo intervallo.", + "deviationAbove": "La tua valutazione è {amount} punti sopra questo intervallo.", + "deviationBelow": "La tua valutazione è {amount} punti sotto questo intervallo.", + "weightedTitle": "Che cosa ha pesato di più nel confronto di questa giornata", + "weightedUp": "{label} — contava verso un valore atteso più alto", + "weightedDown": "{label} — contava verso un valore atteso più basso", + "caveat": "È un confronto con le tue giornate passate, non un'affermazione su come una giornata sia andata a finire così.", + "learning": "Fase di apprendimento: {entries} giornate su {threshold} registrate. Il confronto compare quando ce ne sono abbastanza.", + "noPattern": "Nelle {entries} giornate registrate finora non c'è nulla di abbastanza costante per confrontare una giornata. Registrare il contesto della giornata rende confrontabile una parte maggiore.", + "featureValue": "{field}: {value}" } }, "derived": { diff --git a/messages/pl.json b/messages/pl.json index cd1f4c71e..a0e9c163a 100644 --- a/messages/pl.json +++ b/messages/pl.json @@ -3785,6 +3785,25 @@ "statement": "W dniach z „{label}” nastrój był średnio {direction}: {withAvg} wobec {withoutAvg}.", "counts": "{withDays} dni z, {withoutDays} dni bez", "caveat": "To pokazuje związek, nie przyczynę." + }, + "prognosis": { + "title": "Twoja ocena i to, co sugerują twoje dotychczasowe dni", + "selfLabel": "Twoja ocena", + "expectedLabel": "Oczekiwane na podstawie twoich wzorców", + "band": "Zakres od {low} do {high}", + "statement": "Na podstawie twoich dotychczasowych wzorców dla {date} należałoby oczekiwać wartości około {value}. Wyliczone z {n} dni, które zawierają wszystko, czego potrzebuje porównanie.", + "provisional": "Jeszcze wstępne: wzorzec opiera się na krótkim odcinku dni.", + "notToday": "To opisuje {date}, nie dzisiaj.", + "deviationWithin": "Twoja własna ocena mieści się w tym zakresie.", + "deviationAbove": "Twoja własna ocena leży {amount} punktów powyżej tego zakresu.", + "deviationBelow": "Twoja własna ocena leży {amount} punktów poniżej tego zakresu.", + "weightedTitle": "Co najbardziej zaważyło w porównaniu tego dnia", + "weightedUp": "{label} — liczyło się w stronę wyższej oczekiwanej wartości", + "weightedDown": "{label} — liczyło się w stronę niższej oczekiwanej wartości", + "caveat": "To porównanie z twoimi wcześniejszymi dniami, nie stwierdzenie, jak doszło do takiego przebiegu dnia.", + "learning": "Faza uczenia: zapisano {entries} z {threshold} dni. Porównanie pojawi się, gdy będzie ich dość.", + "noPattern": "W zapisanych dotąd {entries} dniach nic nie jest na tyle regularne, by porównać z tym dzień. Zapisywanie kontekstu dnia sprawia, że więcej da się porównać.", + "featureValue": "{field}: {value}" } }, "derived": { diff --git a/prisma/migrations/0312_mood_prediction/migration.sql b/prisma/migrations/0312_mood_prediction/migration.sql new file mode 100644 index 000000000..941713ec1 --- /dev/null +++ b/prisma/migrations/0312_mood_prediction/migration.sql @@ -0,0 +1,58 @@ +-- v1.38 — the expected value of a day, beside the day's own rating. +-- +-- One row per account per local day, holding what the account's own past days +-- say a day like this one usually looks like. It is a second reading of a day, +-- never a correction of the first: the self-assessment on `mood_entries` is +-- stored exactly as the person set it and nothing here touches it. The +-- distance between the two is the whole point of having both. +-- +-- Its own table rather than four more columns on `mood_entries`, and the +-- reason is a permission rather than a shape: a value the user must not be +-- able to overwrite has no business sitting in the row their own edit form +-- writes. One writer, one table. +-- +-- Not carried in a disaster-recovery backup. Every value here is recomputable +-- from `mood_entries` + `mood_contexts` and the measurement tables, and the +-- nightly job rebuilds it. The verdict is written down in `DERIVED_MODELS` +-- (`src/lib/export/backup-plan.ts`) where the completeness guard reads it, +-- so a restore that comes back without these rows is doing what it was told. +-- +-- No CHECK constraints on the ranges, matching `mood_contexts`: the bounds are +-- server-side, and a later change to a scale must not strand rows written +-- under the old one. + +CREATE TABLE "mood_predictions" ( + "user_id" TEXT NOT NULL, + -- YYYY-MM-DD in the account's own zone, the same key `mood_entries.date` + -- carries, so the two readings of one day join without a second timezone + -- derivation. + "date" TEXT NOT NULL, + + -- The expected pleasantness (A1), 0-10, and the band around it. The band + -- comes from the rolling-origin validation residuals, so it widens when + -- the model is bad instead of staying confidently narrow. + "predicted" DOUBLE PRECISION NOT NULL, + "ci_low" DOUBLE PRECISION, + "ci_high" DOUBLE PRECISION, + -- Complete days the fit was built from. Displayed wherever the value is. + "n" INTEGER NOT NULL, + + -- Bumped when the feature set or the fit changes, so a re-fit is legible + -- in the stored rows rather than silently changing what they mean. + "model_version" TEXT NOT NULL, + -- JSON array of { feature, contribution }: the standardised coefficient + -- times that day's standardised value. Stored so the explanation line is + -- generated from what the model actually used. + "features" TEXT NOT NULL, + "computed_at" TIMESTAMPTZ(3) NOT NULL, + + CONSTRAINT "mood_predictions_pkey" PRIMARY KEY ("user_id", "date") +); + +-- The account cascade every other user-scoped table here carries. The primary +-- key already leads with `user_id`, so the per-account sweep the job and the +-- wipe both run needs no second index. +ALTER TABLE "mood_predictions" + ADD CONSTRAINT "mood_predictions_user_id_fkey" + FOREIGN KEY ("user_id") REFERENCES "users"("id") + ON DELETE CASCADE ON UPDATE CASCADE; diff --git a/prisma/schema.prisma b/prisma/schema.prisma index 3116419c1..fa8e9e929 100644 --- a/prisma/schema.prisma +++ b/prisma/schema.prisma @@ -774,6 +774,7 @@ model User { googleHealthOAuthStates GoogleHealthOAuthState[] moodEntries MoodEntry[] moodContexts MoodContext[] + moodPredictions MoodPrediction[] mentalHealthAssessments MentalHealthAssessment[] moodTagsCustom MoodTag[] @relation("UserMoodTags") moodTagCategoriesCustom MoodTagCategory[] @relation("UserMoodTagCategories") @@ -4265,6 +4266,68 @@ model MoodContext { @@map("mood_contexts") } +/// What the account's own past days say a day like this one usually looks +/// like — one row per local day, written by a job and never by a person. +/// +/// Four things about this table that a later reader would otherwise have to +/// infer, and would infer wrongly: +/// +/// 1. **Written by the `mood-prognosis-refresh` job only. Never by a route, +/// and the user cannot overwrite it.** The self-assessment on `MoodEntry` +/// and the value here are two separate readings of the same day and they +/// stay separate: the deviation between them is the entire finding, and a +/// surface that let somebody correct the prognosis into agreement would +/// destroy it. Two tables make that structural rather than a convention, +/// and `src/__tests__/mood-prediction-write-surface-guard.test.ts` holds +/// the writer set to one file. +/// 2. **`features` holds the evidence, not a hint.** It is the JSON list of +/// the standardised contributions the fit actually used for that day, so +/// the "what the model weighted" line is generated from what happened +/// rather than recomputed at render time against a model that may since +/// have been re-fitted. +/// 3. **Deliberately not in the disaster-recovery backup.** It is a derived +/// artefact, recomputable in full from `MoodEntry` + `MoodContext` and the +/// linked modules, and the nightly job rebuilds it after a restore. The +/// verdict is written down in `DERIVED_MODELS` +/// (`src/lib/export/backup-plan.ts`), where the completeness guard reads +/// it, so this is a decision rather than an omission. It IS in the wipe +/// plan: a derived row is still a statement about somebody's days. +/// 4. **`n` and the band are part of the value.** A consumer that renders +/// `predicted` without `ciLow` / `ciHigh` / `n` is a defect: a forecast +/// built from thirty days and one built from three hundred look identical +/// on screen and mean very different things. +model MoodPrediction { + userId String @map("user_id") + /// YYYY-MM-DD in the account's own zone — the same day key and the same + /// anchoring `MoodEntry.date` carries, so the two readings of a day join + /// without a second timezone derivation. + date String + + /// The expected pleasantness (A1) for that day, 0-10, on the account's own + /// past pattern. A counterfactual, never a measurement. + predicted Float + /// The band around it, from the rolling-origin validation residuals. Null + /// only when the validation produced no residuals at all, which the write + /// path treats as "no forecast" rather than as a bare point estimate. + ciLow Float? @map("ci_low") + ciHigh Float? @map("ci_high") + /// Complete days the fit was built from. Shown wherever `predicted` is. + n Int + + /// Bumped whenever the feature set or the fit changes, so a re-fit is + /// legible in the stored rows instead of silently changing their meaning. + modelVersion String @map("model_version") + /// JSON array of `{ feature, contribution }` — the standardised coefficient + /// times that day's standardised value, per feature. + features String + computedAt DateTime @map("computed_at") + + user User @relation(fields: [userId], references: [id], onDelete: Cascade) + + @@id([userId, date]) + @@map("mood_predictions") +} + // ─── Mood Tag Taxonomy (v1.8.5 W6-tax) ──────────────────────── // // Structured, icon-bearing tag catalog modelled on the standalone mood diff --git a/src/__tests__/copy-matches-behaviour-guard.test.ts b/src/__tests__/copy-matches-behaviour-guard.test.ts index 345f18467..da87e9644 100644 --- a/src/__tests__/copy-matches-behaviour-guard.test.ts +++ b/src/__tests__/copy-matches-behaviour-guard.test.ts @@ -1,8 +1,11 @@ /** * Copy-matches-behaviour guard. * - * Three classes of copy that drifted away from the code they describe, each - * pinned here so the pair fails loudly if either half moves again: + * Four classes of copy that must not drift away from the code they describe, + * each pinned here so the pair fails loudly if either half moves. The first + * three are repairs — copy that had already drifted. The fourth arrives with + * the feature it governs, because a wording rule that depends on review is a + * wording rule that survives until the fifth locale. * * 1. DELETE CONFIRMATIONS. A confirmation is the moment a user weighs the * consequence. Four dialogs promised permanence on surfaces that render @@ -224,3 +227,150 @@ describe("the glucose reference band control is translated everywhere", () => { expect(copy).toContain("never inferred"); }); }); + +/** + * 4. THE MOOD PROGNOSIS. The forecast is a comparison with somebody's own past + * days and it may never be phrased as a cause. "A value around 5.4 would + * have been expected" is what the arithmetic supports; "overtime lowered + * your mood" is not, and the difference is not a matter of taste — the model + * is a correlation over a dozen columns and a causal sentence would be a + * claim it cannot make about somebody's health. + * + * The second half is the count. A forecast built from thirty days and one + * built from three hundred render identically, so every statement that + * quotes a value has to quote what it rests on. Both halves are a test here + * rather than a review habit, because a review habit does not survive the + * fifth locale. + */ +describe("the mood prognosis never states a cause", () => { + /** + * Causal constructions, per locale. Multi-word where a single word would + * over-match — Italian `causa` is a noun in ordinary use and only `a causa + * di` is the construction being banned. + */ + const CAUSAL_PHRASES: Record = { + de: [ + "weil", + "verursacht", + "führt zu", + "sorgt für", + "wegen", + "verbessert deine", + "verschlechtert deine", + "senkt deine", + "hebt deine", + ], + en: [ + "because", + "causes", + "caused", + "leads to", + "due to", + "makes you", + "lowers your", + "raises your", + "improves your", + "worsens your", + ], + es: [ + "porque", + "provoca", + "debido a", + "a causa de", + "mejora tu", + "empeora tu", + ], + fr: [ + "parce que", + "provoque", + "en raison de", + "à cause de", + "améliore ton", + "dégrade ton", + ], + it: [ + "perché", + "provoca", + "a causa di", + "migliora il tuo", + "peggiora il tuo", + ], + pl: ["ponieważ", "powoduje", "z powodu", "wywołuje", "poprawia twój"], + }; + + /** Every leaf string under `insights.mood.prognosis`, flattened. */ + function prognosisStrings(locale: string): Array<[string, string]> { + const root = ( + bundle(locale).insights as Record> + ).mood.prognosis as Record; + expect( + root, + `${locale} carries no insights.mood.prognosis block`, + ).toBeTypeOf("object"); + return Object.entries(root).filter( + (entry): entry is [string, string] => typeof entry[1] === "string", + ); + } + + for (const locale of LOCALES) { + it(`${locale} carries the block, and every string in it`, () => { + const strings = prognosisStrings(locale); + // The positive control. Every assertion below passes on an empty list, + // so the count is the evidence and the empty offender set is only the + // verdict. + expect(strings.length).toBeGreaterThanOrEqual(17); + for (const [key, value] of strings) { + expect(value.length, `${locale}/${key} is empty`).toBeGreaterThan(0); + } + }); + + it(`${locale} names no cause`, () => { + const offenders = prognosisStrings(locale) + .filter(([, value]) => + CAUSAL_PHRASES[locale].some((phrase) => + value.toLowerCase().includes(phrase), + ), + ) + .map(([key]) => key); + expect( + offenders, + `${locale} prognosis copy states a cause; the forecast is a comparison with the account's own past days and cannot support one`, + ).toEqual([]); + }); + } + + /** + * Statements that quote a value must quote what it rests on. The list is + * explicit rather than derived, so adding a statement key forces a decision + * here instead of inheriting silence. + */ + const COUNTED_STATEMENTS: Record = { + statement: ["{value}", "{n}"], + learning: ["{entries}", "{threshold}"], + noPattern: ["{entries}"], + band: ["{low}", "{high}"], + }; + + for (const locale of LOCALES) { + it(`${locale} keeps the count beside the claim`, () => { + const strings = new Map(prognosisStrings(locale)); + for (const [key, placeholders] of Object.entries(COUNTED_STATEMENTS)) { + const copy = strings.get(key); + expect(copy, `${locale}/${key} is missing`).toBeTypeOf("string"); + for (const placeholder of placeholders) { + expect( + copy, + `${locale}/${key} quotes a value without ${placeholder}; a forecast without the days behind it cannot be weighed by the reader`, + ).toContain(placeholder); + } + } + }); + } + + it("the forecast is phrased as a counterfactual, not as a measurement (EN + DE)", () => { + const en = resolve(bundle("en"), "insights.mood.prognosis.statement"); + expect(en.toLowerCase()).toContain("would have been expected"); + const de = resolve(bundle("de"), "insights.mood.prognosis.statement"); + expect(de.toLowerCase()).toContain("zu erwarten gewesen"); + }); +}); diff --git a/src/__tests__/mood-prediction-write-surface-guard.test.ts b/src/__tests__/mood-prediction-write-surface-guard.test.ts new file mode 100644 index 000000000..af83e70d0 --- /dev/null +++ b/src/__tests__/mood-prediction-write-surface-guard.test.ts @@ -0,0 +1,158 @@ +/** + * `MoodPrediction` has exactly one writer, and it is not a request. + * + * The product rule underneath this guard: a day carries two readings — the + * one the person gave it and the one their own past days imply — and they are + * never merged. The self-assessment leads and is stored exactly as it was + * given; the prognosis is computed, carries a band and a case count, and the + * user cannot overwrite it. A route that wrote `moodPrediction` would make the + * second value editable, and an editable prognosis is not a prognosis: the + * deviation between the two, which is the only thing either number is for, + * would quietly become a number somebody could tune to zero. + * + * The second failure the single writer prevents is the one the cycle module + * already paid for: a forecast written as a side effect of a page view makes + * the output a function of browsing rather than of data + * (`src/lib/cycle/prediction-cache.ts`, `src/lib/jobs/cycle-prediction-refresh.ts`). + * + * **Every matcher here asserts a non-zero match count.** Legs (a) and (b) + * expect an EMPTY offender set, so "found nothing" and "nothing is wrong" look + * identical from the outside — which is precisely how the bearer-scope guard + * stayed green for months while matching nothing at all. Each leg therefore + * pins a separate positive: the number of files SCANNED, and the write calls + * the legitimate writer really does make. `describe("the matcher sees what it + * looks for")` proves the regex against synthetic sources, including the + * broken-across-lines form that defeated an earlier guard in this repository. + * + * **What this guard structurally cannot see, stated like the session-surface + * guard states its own limit.** The matcher keys on a literal `moodPrediction` + * delegate access. The "Delete All Data" wipe DOES delete this table — the + * model is in `WIPE_MODELS` (`src/lib/data-wipe/wipe-plan.ts`) — but it does so + * through a delegate resolved dynamically from the model name (`client[key] + * .deleteMany`), so no `moodPrediction.deleteMany(` literal exists for the + * matcher to catch, and that deletion is correctly NOT counted as a writer. + * The wipe path's own completeness is proven elsewhere, by + * `data-wipe-completeness.test.ts` enumerating the schema; this guard is only + * the one about request-time writers, and it cannot be the gate on a dynamic + * delegate. A future writer that reached the table through a computed property + * would slip this matcher for the same reason. + */ +import { existsSync, readFileSync, readdirSync } from "node:fs"; +import { join, sep } from "node:path"; + +import { describe, expect, it } from "vitest"; + +const SRC = join(process.cwd(), "src"); + +/** + * A write against the `moodPrediction` delegate, on any receiver. + * + * Whitespace-tolerant in every joint: Prettier breaks a long call across + * lines, and a matcher demanding `prisma.moodPrediction.upsert(` on one line + * is a matcher that stops matching the day the line gets long enough. The + * receiver is left open (`prisma`, `tx`, `db`, the worker client) because + * pinning one name would let a rename walk straight past. + */ +const WRITE_CALL = + /\bmoodPrediction\s*\.\s*(createMany|createManyAndReturn|create|upsert|updateMany|update|deleteMany|delete)\s*\(/g; + +/** The one file allowed to write, relative to `src/`. */ +const SOLE_WRITER = join("lib", "jobs", "mood-prognosis-refresh.ts"); + +function sourceFiles(dir: string): string[] { + const out: string[] = []; + for (const entry of readdirSync(dir, { withFileTypes: true })) { + const full = join(dir, entry.name); + if (entry.isDirectory()) { + if (entry.name === "generated" || entry.name === "node_modules") continue; + out.push(...sourceFiles(full)); + continue; + } + if (entry.name.endsWith(".ts") || entry.name.endsWith(".tsx")) { + out.push(full); + } + } + return out; +} + +function writeCallsIn(source: string): string[] { + return [...source.matchAll(WRITE_CALL)].map((m) => m[1]); +} + +const ALL_FILES = sourceFiles(SRC); +const PRODUCTION_FILES = ALL_FILES.filter( + (f) => !f.includes(`${sep}__tests__${sep}`) && !f.endsWith(".test.ts"), +); + +describe("MoodPrediction is written by one file", () => { + it("scans a plausible number of source files", () => { + // The positive control for legs (a) and (b) below, both of which pass on + // an empty result. If this collapses, they are asserting nothing. + expect(ALL_FILES.length).toBeGreaterThan(1000); + expect(PRODUCTION_FILES.length).toBeGreaterThan(800); + }); + + it("(a) no route, page or component writes it", () => { + const offenders = PRODUCTION_FILES.filter( + (f) => + f.startsWith(join(SRC, "app")) || f.startsWith(join(SRC, "components")), + ).filter((f) => writeCallsIn(readFileSync(f, "utf8")).length > 0); + + expect( + offenders.map((f) => f.slice(SRC.length + 1)), + "a request-time surface writes the prognosis — it is computed by the nightly job and the user cannot overwrite it", + ).toEqual([]); + }); + + it("(b) the writer set is exactly the refresh job", () => { + const writers = PRODUCTION_FILES.filter( + (f) => writeCallsIn(readFileSync(f, "utf8")).length > 0, + ).map((f) => f.slice(SRC.length + 1)); + + expect(writers.sort()).toEqual([SOLE_WRITER]); + }); + + it("(c) that writer really does write — the matcher is not matching nothing", () => { + expect( + existsSync(join(SRC, SOLE_WRITER)), + `${SOLE_WRITER} does not exist — the table has a guard and no writer`, + ).toBe(true); + const source = readFileSync(join(SRC, SOLE_WRITER), "utf8"); + const calls = writeCallsIn(source); + expect( + calls.length, + "the sole writer makes no write call the matcher can see; leg (b) is then satisfied by an empty set rather than by the truth", + ).toBeGreaterThan(0); + // It replaces rather than accumulates: a pass writes the current window + // and clears whatever fell out of it. + expect(calls).toContain("deleteMany"); + }); +}); + +describe("the matcher sees what it looks for", () => { + it("matches a call Prettier broke across lines", () => { + const wrapped = [ + "await prisma.moodPrediction", + " .upsert(", + " { where: { userId_date: { userId, date } } },", + " );", + ].join("\n"); + expect(writeCallsIn(wrapped)).toEqual(["upsert"]); + }); + + it("matches through an aliased client", () => { + expect(writeCallsIn("tx.moodPrediction.deleteMany({ where })")).toEqual([ + "deleteMany", + ]); + }); + + it("does not match a read", () => { + expect( + writeCallsIn("const rows = await prisma.moodPrediction.findMany({})"), + ).toEqual([]); + }); + + it("does not match a different model", () => { + expect(writeCallsIn("prisma.moodEntry.create({ data })")).toEqual([]); + }); +}); diff --git a/src/__tests__/sharing-surface-guard.test.ts b/src/__tests__/sharing-surface-guard.test.ts index 4af6c2557..38fe49fc5 100644 --- a/src/__tests__/sharing-surface-guard.test.ts +++ b/src/__tests__/sharing-surface-guard.test.ts @@ -891,6 +891,10 @@ const DELEGABLE_ROUTES: Record = { domain: "record", why: "What the sleep, activity, vitals and illness modules already hold for one of the record's days, read-only. It is reached from a mood surface and it is NOT a mind read: the payload is sleep minutes, steps, active energy, resting heart rate, heart-rate variability and an illness day-log with its symptom keys. Declared `mind` it let a mood-only delegate walk arbitrary dates and read two sections the owner had declined to share, with no entry needing to exist for the date. Admitted on the whole-record grant only, like the other cross-section reads; a scoped grant is refused rather than served a filtered day.", }, + "app/api/mood/prognosis/route.ts": { + domain: "record", + why: "The day's two readings — the record's own rating and what its past days imply — read-only. It is NOT a mind read: the fit takes the day's context AND the sleep, activity and recovery figures the owning modules hold, and the payload names the features that carried weight, so a reader sees that sleep or steps or resting heart rate mattered on a given day. Declared `mind` it would hand a mood-only delegate a derived view of sections the owner declined to share. Admitted on the whole-record grant only, like the linked-day read beside it; a scoped grant is refused rather than served a filtered contribution list, which would falsely read as \"the model weighted only the mood fields\".", + }, "app/api/mood-entries/[id]/route.ts": { domain: "mind", why: "One mood entry of the record, fetch-then-guard against the resolved user.", @@ -1860,9 +1864,12 @@ const ACTOR_ROUTES: Record = { * managed-record configuration adds one more: 196 → 197. The bounded * read-only profile summary makes the next explicit admission: 197 → 198. * The two initial Cycle reads follow as separate, scoped admissions: 198 → 200. - * The mood day's linked figures, read-only, add one: 200 → 201. + * The mood day's linked figures, read-only, add one: 200 → 201. The visit + * record and its address book bring twelve: 201 → 213. The mood prognosis, + * read-only and admitted on the whole record for the same reason as the linked + * figures, adds one: 213 → 214. */ -const FROZEN_ENTRY_COUNT = 213; +const FROZEN_ENTRY_COUNT = 214; /** * The two surfaces that authenticate a Bearer token outside `requireAuth` — diff --git a/src/app/api/mood/prognosis/route.ts b/src/app/api/mood/prognosis/route.ts new file mode 100644 index 000000000..8296d7ef3 --- /dev/null +++ b/src/app/api/mood/prognosis/route.ts @@ -0,0 +1,74 @@ +/** + * `GET /api/mood/prognosis` — the day's two readings, side by side. + * + * The person's own rating leads and is returned exactly as it was stored. The + * forecast rides beside it with its band, the number of days it was built + * from, and how old it is. The two are never merged: the distance between them + * is the whole finding, and a single blended number would destroy it. + * + * **This route declares `record`, not `mind`, and that is the security + * decision on it.** The payload is not only mood: the fit reads the day's + * level-B context AND the figures the sleep, activity and recovery modules + * hold, and the contributions it publishes name those features one by one — a + * reader can see that sleep or steps or resting heart rate carried weight on a + * given day, and roughly how much. Declared `mind` it would be a hole with a + * mood-shaped label on it: a delegate holding only the mind section would + * receive a derived view of two sections the owner had declined to share. The + * convention for a read that crosses sections is `record` + * (`/api/mood/linked-context`, `/api/dashboard/snapshot`, `/api/daily/digest` + * are the precedents) and a scoped grant never reaches one, so a scoped + * delegate loses the forecast entirely rather than receiving a filtered + * version of it. A filtered version would be worse: a contribution list with + * the linked features quietly removed would tell the reader that the model + * weighted only the mood fields, which is false. + * + * Nothing here writes. The rows come from the nightly job and the user cannot + * overwrite them — `src/__tests__/mood-prediction-write-surface-guard.test.ts` + * holds that, and this file is one of the ones it scans. + */ +import type { NextRequest } from "next/server"; + +import { apiHandler, requireRecordAuth } from "@/lib/api-handler"; +import { apiSuccess } from "@/lib/api-response"; +import { readMoodPrognosis } from "@/lib/analytics/mood-prognosis/read"; +import { annotate } from "@/lib/logging/context"; +import { requireModuleEnabled } from "@/lib/modules/gate"; + +export const dynamic = "force-dynamic"; + +// The parameter is declared although the handler body reads nothing off it, +// and the reason is the exported type rather than the runtime. `apiHandler` +// infers the type of `GET` from this callback, so a zero-argument handler +// would type `GET` as taking no arguments — and Next.js, and the integration +// tests, call it WITH the request. The wide event's method and path, and the +// read-vs-write check `requireRecordAuth` runs, both come from the request the +// wrapper receives from Next.js (not from this parameter), and both fall back +// to `GET` when no request is present — so an absent request keeps a read a +// read rather than escalating it. +export const GET = apiHandler(async (_request: NextRequest) => { + // `record`: see the file header. A scoped grant is refused here by + // construction rather than served a narrowed payload. + const { user } = await requireRecordAuth("read", "record"); + + const gate = await requireModuleEnabled(user.id, "mood"); + if (!gate.enabled) return gate.response; + + const reading = await readMoodPrognosis(user.id, new Date(), user.timezone); + + annotate({ + action: { name: "mood.prognosis.read" }, + meta: reading.present + ? { + present: true, + day: reading.date, + fit_rows: reading.n, + age_days: reading.ageDays, + current: reading.current, + provisional: reading.provisional, + deviation: reading.deviation, + } + : { present: false, reason: reading.reason, entries: reading.entries }, + }); + + return apiSuccess(reading); +}); diff --git a/src/app/insights/mood/page.tsx b/src/app/insights/mood/page.tsx index 6573fff92..50886b099 100644 --- a/src/app/insights/mood/page.tsx +++ b/src/app/insights/mood/page.tsx @@ -36,6 +36,21 @@ const MoodChart = dynamic( { ssr: false, loading: () => }, ); +/** + * The day's two readings. Deferred like every other block on this route that + * is not the first paint: it reads its own endpoint and renders below the + * calendar, so nothing about it belongs in the eager graph of a page whose + * first paint is a heatmap. This route sits close to its bundle ceiling and + * every addition to it goes through the same treatment. + */ +const MoodPrognosisCard = dynamic( + () => + import("@/components/insights/mood/mood-prognosis-card").then((mod) => ({ + default: mod.MoodPrognosisCard, + })), + { ssr: false, loading: () => null }, +); + interface ComprehensiveMoodData { moodSummary: { count: number } | null; } @@ -137,6 +152,12 @@ export default function InsightsStimmungPage() { + {/* The day's own rating beside what the account's past days imply. It + sits high because the self-assessment is the leading value on this + page, and directly under the line chart because that is where a + reader has just seen the rating it leads with. */} + + {/* The better-days Einschätzung sits directly under the Ziel card, ahead of the classification tiles and breakdowns. */} diff --git a/src/components/insights/mood/__tests__/mood-prognosis-card.test.tsx b/src/components/insights/mood/__tests__/mood-prognosis-card.test.tsx new file mode 100644 index 000000000..7423ac228 --- /dev/null +++ b/src/components/insights/mood/__tests__/mood-prognosis-card.test.tsx @@ -0,0 +1,170 @@ +/** + * What reaches the screen, not what the strings say. + * + * The copy guard reads the bundles; this reads the render. Between them they + * cover the two halves of the same rule — that the two readings of a day are + * shown side by side and never merged, that the forecast never appears without + * its band and its count, and that the deviation is measured against the band + * rather than against the midpoint. + */ +import { describe, expect, it } from "vitest"; +import { renderToStaticMarkup } from "react-dom/server"; + +import { I18nProvider } from "@/lib/i18n/context"; +import type { PrognosisReading } from "@/lib/analytics/mood-prognosis/read"; + +import { MoodPrognosisView } from "../mood-prognosis-card"; + +function render(reading: PrognosisReading): string { + return renderToStaticMarkup( + + + , + ); +} + +function present(over: Partial = {}): PrognosisReading { + return { + present: true, + date: "2026-08-07", + predicted: 5.4, + ciLow: 4.1, + ciHigh: 6.8, + n: 42, + modelVersion: "mood-ridge-1", + computedAt: "2026-08-08T02:35:00.000Z", + ageDays: 1, + current: true, + provisional: false, + stage: "regular", + entries: 64, + seasonalUnlocked: false, + selfAssessment: 6, + deviation: "within", + deviationAmount: 0, + contributions: [ + { feature: "dimension:a2", contribution: 0.9 }, + { feature: "linked:steps", contribution: -0.4 }, + ], + ...over, + } as PrognosisReading; +} + +describe("", () => { + it("shows both readings, and shows the person's own one first", () => { + const html = render(present()); + expect(html).toContain('data-slot="mood-prognosis-self"'); + expect(html).toContain('data-slot="mood-prognosis-expected"'); + // The self block precedes the expected block in the document, which is + // what "the self-assessment leads" means to a screen reader. + expect(html.indexOf('data-slot="mood-prognosis-self"')).toBeLessThan( + html.indexOf('data-slot="mood-prognosis-expected"'), + ); + // Both values are on screen as themselves. A merged single figure is the + // failure this whole feature is shaped to avoid. + expect(html).toContain("6.0"); + expect(html).toContain("5.4"); + }); + + it("never shows the value without its band and its count", () => { + const html = render(present()); + expect(html).toContain("4.1"); + expect(html).toContain("6.8"); + expect(html).toContain("42"); + }); + + it("phrases the forecast as a counterfactual", () => { + const html = render(present()); + expect(html).toContain("would have been expected"); + expect(html).not.toContain("your mood is"); + }); + + it("measures the deviation against the band, not the midpoint", () => { + // 6.0 is above the 5.4 midpoint and inside the 4.1-6.8 band, so it is not + // a finding. A midpoint comparison would have called it one. + expect(render(present())).toContain("inside that range"); + + const above = render( + present({ selfAssessment: 8, deviation: "above", deviationAmount: 1.2 }), + ); + expect(above).toContain("1.2 points above"); + }); + + it("labels a provisional forecast as provisional", () => { + expect(render(present({ provisional: true }))).toContain( + 'data-slot="mood-prognosis-provisional"', + ); + expect(render(present())).not.toContain( + 'data-slot="mood-prognosis-provisional"', + ); + }); + + it("says which day it is about when that day is not today", () => { + const stale = render(present({ current: false, ageDays: 4 })); + expect(stale).toContain('data-slot="mood-prognosis-age"'); + // Rendered through the account's own date-order preference rather than as + // the raw day key — under this render's `en` locale and the default AUTO + // preference that is month-first. The assertion is on the formatted form + // precisely because a raw `2026-08-07` on the screen would pass a looser + // one. + expect(stale).toContain("08/07/2026"); + expect(stale).not.toContain("2026-08-07"); + }); + + it("names what the model weighted, in the reader's own words", () => { + const html = render(present()); + // The dimension's own label, the one the slider carries. + expect(html).toContain("Stress"); + expect(html).toContain("Steps"); + expect(html).toContain("counted towards a higher expected value"); + expect(html).toContain("counted towards a lower expected value"); + }); + + it("skips a feature key this build no longer knows", () => { + const html = render( + present({ + contributions: [ + { feature: "retired:whatever", contribution: 2 }, + { feature: "dimension:a3", contribution: 1 }, + ], + }), + ); + expect(html).not.toContain("retired:whatever"); + expect(html).toContain('data-slot="mood-prognosis-explanation"'); + }); + + it("renders nothing at all below the first rung", () => { + expect( + render({ + present: false, + reason: "no-output-yet", + entries: 4, + nextThreshold: 15, + }), + ).toBe(""); + }); + + it("explains the learning phase with the count and the next step", () => { + const html = render({ + present: false, + reason: "learning-phase", + entries: 22, + nextThreshold: 30, + }); + expect(html).toContain("22"); + expect(html).toContain("30"); + // Absence, not an error and not a zero. + expect(html).not.toContain('data-slot="mood-prognosis-expected"'); + }); + + it("refuses honestly when there are days but no pattern", () => { + const html = render({ + present: false, + reason: "no-pattern", + entries: 140, + nextThreshold: null, + }); + expect(html).toContain("140"); + expect(html).not.toContain('data-slot="mood-prognosis-statement"'); + }); +}); diff --git a/src/components/insights/mood/mood-prognosis-card.tsx b/src/components/insights/mood/mood-prognosis-card.tsx new file mode 100644 index 000000000..4167d1941 --- /dev/null +++ b/src/components/insights/mood/mood-prognosis-card.tsx @@ -0,0 +1,248 @@ +"use client"; + +/** + * The day's two readings, side by side, with the person's own one leading. + * + * The copy rules this component exists to hold, stated once so nobody has to + * infer them from the strings: + * + * * **No sentence names a cause.** Not "overtime lowered your mood", not + * "poor sleep is why today was hard". What the data supports is "on your + * own past days, a value around X would have been expected", and that is + * what it says. The ban is a test, not a habit: + * `src/__tests__/copy-matches-behaviour-guard.test.ts` reads the six + * bundles and fails on a causal construction in any of them. + * * **The forecast is a counterfactual and the sentence says so.** Never + * "your mood is 5.4" — always "a value around 5.4 would have been + * expected". The difference is the whole distinction between the two + * readings. + * * **The count is on the same surface as the number**, not in a tooltip. A + * forecast built from thirty days and one built from three hundred look + * identical otherwise, and a reader who cannot see which one they are + * looking at cannot weigh it. + * * **The deviation is measured against the band, never against the + * midpoint.** Inside the band is "as expected"; outside it is the finding. + * + * The explanation is generated from the contributions the job stored for that + * day, so it names what the model actually weighted rather than a fresh + * calculation against a model that may since have been re-fitted. + */ +import { useQuery } from "@tanstack/react-query"; +import { Compass } from "lucide-react"; + +import { Card, CardContent, CardHeader } from "@/components/ui/card"; +import { TileHeader } from "@/components/insights/tile-header"; +import { Skeleton } from "@/components/ui/skeleton"; +import { useAuth } from "@/hooks/use-auth"; +import { apiGet } from "@/lib/api/api-fetch"; +import { formatDate } from "@/lib/date-format"; +import { useDateFormatPreference, useTranslations } from "@/lib/i18n/context"; +import { queryKeys } from "@/lib/query-keys"; +import type { PrognosisReading } from "@/lib/analytics/mood-prognosis/read"; + +import { prognosisFeatureLabel } from "./prognosis-feature-label"; + +/** How many weighted features the explanation names. */ +const EXPLAINED_FEATURES = 3; + +function formatValue(value: number, locale: string): string { + return new Intl.NumberFormat(locale, { + minimumFractionDigits: 1, + maximumFractionDigits: 1, + }).format(value); +} + +export function MoodPrognosisCard() { + const { isAuthenticated } = useAuth(); + + const { data, isLoading } = useQuery({ + queryKey: queryKeys.moodPrognosis(), + queryFn: async () => apiGet("/api/mood/prognosis"), + enabled: isAuthenticated, + staleTime: 60_000, + }); + + if (isLoading) { + return ; + } + if (!data) return null; + return ; +} + +/** + * The card itself, over a resolved reading. + * + * Split from the query so the copy rules above can be asserted against real + * renders rather than against the strings alone: "the two values are never + * merged" is a property of what reaches the screen, and a bundle check cannot + * see it. + */ +export function MoodPrognosisView({ reading }: { reading: PrognosisReading }) { + const { t, locale } = useTranslations(); + const dateFormat = useDateFormatPreference(); + const data = reading; + // The day key is a plain calendar date and is rendered as one, through the + // account's own date-order preference and pinned to UTC — the same treatment + // every other day key in the app gets, so this card cannot show a different + // day from the list beside it. + const day = data.present ? formatDate(data.date, dateFormat, locale) : ""; + + // Below the first rung there is nothing to say and the surface says nothing + // rather than showing an empty frame with an apology in it. + if (!data.present && data.reason === "no-output-yet") return null; + + return ( + + + + + + {!data.present ? ( +

+ {data.reason === "learning-phase" + ? t("insights.mood.prognosis.learning", { + entries: String(data.entries), + threshold: String(data.nextThreshold ?? 0), + }) + : t("insights.mood.prognosis.noPattern", { + entries: String(data.entries), + })} +

+ ) : ( + <> +
+ {/* The self-assessment leads: it is the larger number, it comes + first, and it is the only one of the two the person gave. */} +
+

+ {t("insights.mood.prognosis.selfLabel")} +

+

+ {data.selfAssessment === null + ? t("mood.dayContext.notRecorded") + : formatValue(data.selfAssessment, locale)} +

+
+
+

+ {t("insights.mood.prognosis.expectedLabel")} +

+

+ {formatValue(data.predicted, locale)} +

+ {/* The band and the count ride with the value, in the block a + screen reader reaches in the same breath. */} +

+ {data.ciLow !== null && data.ciHigh !== null + ? t("insights.mood.prognosis.band", { + low: formatValue(data.ciLow, locale), + high: formatValue(data.ciHigh, locale), + }) + : null} +

+
+
+ +

+ {t("insights.mood.prognosis.statement", { + value: formatValue(data.predicted, locale), + date: day, + n: String(data.n), + })} +

+ + {data.provisional ? ( +

+ {t("insights.mood.prognosis.provisional")} +

+ ) : null} + + {!data.current ? ( +

+ {t("insights.mood.prognosis.notToday", { date: day })} +

+ ) : null} + + {data.deviation !== null && data.selfAssessment !== null ? ( +

+ {data.deviation === "within" + ? t("insights.mood.prognosis.deviationWithin") + : data.deviation === "above" + ? t("insights.mood.prognosis.deviationAbove", { + amount: formatValue(data.deviationAmount ?? 0, locale), + }) + : t("insights.mood.prognosis.deviationBelow", { + amount: formatValue(data.deviationAmount ?? 0, locale), + })} +

+ ) : null} + + + +

+ {t("insights.mood.prognosis.caveat")} +

+ + )} + + + ); +} + +function PrognosisExplanation({ + contributions, +}: { + contributions: Array<{ feature: string; contribution: number }>; +}) { + const { t } = useTranslations(); + const named = contributions + .map((item) => ({ + ...item, + label: prognosisFeatureLabel(item.feature, t), + })) + .filter( + (item): item is typeof item & { label: string } => + item.label !== null && item.contribution !== 0, + ) + .slice(0, EXPLAINED_FEATURES); + + if (named.length === 0) return null; + + return ( +
+

+ {t("insights.mood.prognosis.weightedTitle")} +

+
    + {named.map((item) => ( +
  • + {item.contribution > 0 + ? t("insights.mood.prognosis.weightedUp", { label: item.label }) + : t("insights.mood.prognosis.weightedDown", { + label: item.label, + })} +
    • + ))} +
+
+ ); +} diff --git a/src/components/insights/mood/prognosis-feature-label.ts b/src/components/insights/mood/prognosis-feature-label.ts new file mode 100644 index 000000000..6c5d912a4 --- /dev/null +++ b/src/components/insights/mood/prognosis-feature-label.ts @@ -0,0 +1,71 @@ +/** + * A stored feature key, in the reader's own language. + * + * The keys in `MoodPrediction.features` are stable identifiers, not copy. This + * turns one into a label, and it does so out of the strings the capture + * surfaces already use: a person reads "Workload" here because that is what + * the slider they moved was called, not because a second wording was invented + * for the explanation. + * + * Every key it cannot place returns `null`, and that path is real rather than + * defensive: a row written by an older `modelVersion` can name a feature this + * build has dropped, and skipping it is better than rendering an identifier at + * somebody. + */ +import { parseFeatureKey } from "@/lib/analytics/mood-prognosis/feature-keys"; +import { contextValueLabelKey } from "@/lib/mood/context-vocabulary"; +import { getMoodDimension } from "@/lib/mood/dimensions"; + +type Translate = ( + key: string, + params?: Record, +) => string; + +/** + * The five linked figures, resolved through literal keys. + * + * A switch rather than a computed key so the call-site coverage guard can see + * every string this file can ask for. The keys are the capture sheet's own + * (`mood.dayContext.linked*`), which is the point: the label a figure carries + * in the explanation is the label it carries where it was read. + */ +function linkedLabel(metric: string, t: Translate): string | null { + switch (metric) { + case "sleepAsleep": + return t("mood.dayContext.linkedSleep"); + case "steps": + return t("mood.dayContext.linkedSteps"); + case "activeEnergy": + return t("mood.dayContext.linkedActiveEnergy"); + case "restingHeartRate": + return t("mood.dayContext.linkedRestingHeartRate"); + case "heartRateVariability": + return t("mood.dayContext.linkedHrv"); + default: + return null; + } +} + +export function prognosisFeatureLabel( + key: string, + t: Translate, +): string | null { + const parsed = parseFeatureKey(key); + if (parsed === null) return null; + + if (parsed.family === "dimension") { + const dimension = getMoodDimension(parsed.dimension); + return dimension ? t(dimension.labelKey) : null; + } + if (parsed.family === "linked") return linkedLabel(parsed.metric, t); + + const field = t(`mood.context.field.${parsed.field}`); + if (parsed.value === null) return field; + // "Working day: longer than usual" — the field and the value the person + // picked, joined by a pattern rather than by a hard-coded colon, because the + // punctuation between them is a locale's business. + return t("insights.mood.prognosis.featureValue", { + field, + value: t(contextValueLabelKey(parsed.field, parsed.value)), + }); +} diff --git a/src/lib/analytics/mood-prognosis/__tests__/features.test.ts b/src/lib/analytics/mood-prognosis/__tests__/features.test.ts new file mode 100644 index 000000000..19379217c --- /dev/null +++ b/src/lib/analytics/mood-prognosis/__tests__/features.test.ts @@ -0,0 +1,281 @@ +/** + * The matrix: what becomes a column, what never does, and what happens to a + * day that is missing something. + * + * The cases that matter are the absences. A day with no context row must not + * contribute a zero to a work-status column, because zero there means "worked + * a regular day and not overtime" and the truth is "nobody said". That + * distinction is invisible in the fitted numbers and would quietly teach the + * model that every unlogged day was a day off. + */ +import { describe, expect, it } from "vitest"; + +import { + MAX_MODEL_FEATURES, + MIN_FEATURE_COVERAGE, + buildFeatureMatrix, + extractDayFeatures, + screenFeatures, + standardiseDay, + type PrognosisContextInput, + type PrognosisDayInput, +} from "../features"; +import { MOOD_DIMENSIONS } from "@/lib/mood/dimensions"; + +const EMPTY_LINKED = { + sleepAsleep: null, + steps: null, + activeEnergy: null, + restingHeartRate: null, + heartRateVariability: null, +}; + +function context( + overrides: Partial = {}, +): PrognosisContextInput { + return { + workStatus: null, + workMinutes: null, + overtimeMinutes: null, + workLoad: null, + workSatisfaction: null, + contactCircles: [], + contactForm: null, + contactExtent: null, + contactQuality: null, + contactSupport: null, + leisureCategories: [], + leisureMinutes: null, + leisureJoy: null, + leisureRecovery: null, + eventType: null, + eventValence: null, + ...overrides, + }; +} + +function day( + index: number, + overrides: Partial = {}, +): PrognosisDayInput { + const date = new Date(Date.UTC(2026, 0, 1 + index)); + return { + day: date.toISOString().slice(0, 10), + a1: 5, + dimensions: {}, + context: null, + linked: { ...EMPTY_LINKED }, + ...overrides, + }; +} + +describe("extractDayFeatures", () => { + it("takes A2 through the dimension table's inverse flag, not inline", () => { + const a2 = MOOD_DIMENSIONS.find((d) => d.key === "a2"); + expect(a2?.inverse, "A2 is the inverse dimension this case is about").toBe( + true, + ); + const features = extractDayFeatures( + day(0, { dimensions: { a2: 8, a3: 8 } }), + ); + // Stress 8 is a heavy day, so on an up-is-better axis it reads as 2. + expect(features.get("dimension:a2")).toBe(2); + // A3 is not inverse and is carried literally. + expect(features.get("dimension:a3")).toBe(8); + }); + + it("never makes A1 an input", () => { + const features = extractDayFeatures( + day(0, { a1: 7, dimensions: { a1: 7, a2: 3 } }), + ); + expect(features.has("dimension:a1")).toBe(false); + }); + + it("a day with no context contributes no context column at all", () => { + const features = extractDayFeatures(day(0, { context: null })); + expect( + [...features.keys()].filter((k) => k.startsWith("context:")), + ).toEqual([]); + }); + + it("an unanswered field contributes no column; an answered one contributes zeros and a one", () => { + const answered = extractDayFeatures( + day(0, { context: context({ workStatus: "overtime" }) }), + ); + expect(answered.get("context:workStatus=overtime")).toBe(1); + expect(answered.get("context:workStatus=off")).toBe(0); + + const unanswered = extractDayFeatures( + day(0, { context: context({ workLoad: 4 }) }), + ); + expect(unanswered.has("context:workStatus=overtime")).toBe(false); + expect(unanswered.has("context:workStatus=off")).toBe(false); + expect(unanswered.get("context:workLoad")).toBe(4); + }); + + it("a recorded zero is a value, and absence is absence", () => { + const zero = extractDayFeatures( + day(0, { linked: { ...EMPTY_LINKED, steps: 0 } }), + ); + expect(zero.get("linked:steps")).toBe(0); + const absent = extractDayFeatures(day(0)); + expect(absent.has("linked:steps")).toBe(false); + }); + + it("an empty multi-select says nothing rather than saying no to everything", () => { + const features = extractDayFeatures( + day(0, { context: context({ contactCircles: [] }) }), + ); + expect(features.has("context:contactCircles=partner")).toBe(false); + }); +}); + +describe("screening", () => { + /** A2 drives A1 exactly; everything else is constant or noise. */ + function drivenDays(count: number): PrognosisDayInput[] { + return Array.from({ length: count }, (_, i) => { + const stress = i % 11; + return day(i, { + // Up-is-better A2 is `10 - stress`, and the target follows it. + a1: 10 - stress, + dimensions: { a2: stress }, + }); + }); + } + + it("keeps a feature that tracks the target", () => { + const screened = screenFeatures(drivenDays(40)); + const a2 = screened.find((s) => s.key === "dimension:a2"); + expect(a2?.selected, "a perfectly tracking feature was screened out").toBe( + true, + ); + expect(a2?.pairs).toBe(40); + }); + + it("refuses a feature recorded on too few of the days, before testing it", () => { + const days = drivenDays(40); + // Recorded on a quarter of the days — under the coverage floor whatever + // it correlates with. + const covered = Math.floor(40 * MIN_FEATURE_COVERAGE) - 5; + for (let i = 0; i < covered; i++) { + days[i] = { ...days[i], linked: { ...EMPTY_LINKED, steps: 1000 + i } }; + } + const screened = screenFeatures(days); + const steps = screened.find((s) => s.key === "linked:steps"); + expect(steps?.selected).toBe(false); + expect(steps?.reason).toBe("coverage"); + }); + + it("refuses a one-hot seen fewer times than the occurrence floor", () => { + const days = drivenDays(40).map((d, i) => ({ + ...d, + // Everybody worked; two days were overtime. The column is present on + // every day, so coverage passes and the occurrence floor is what + // refuses it. + context: context({ workStatus: i < 2 ? "overtime" : "regular" }), + })); + const screened = screenFeatures(days); + const overtime = screened.find( + (s) => s.key === "context:workStatus=overtime", + ); + expect(overtime?.selected).toBe(false); + expect(overtime?.reason).toBe("occurrences"); + }); + + it("refuses pure noise — the whole reason the FDR layer is here", () => { + // A deterministic pseudo-random series, so the case cannot flake. + let seed = 42; + const random = () => { + seed = (seed * 1103515245 + 12345) % 2147483648; + return seed / 2147483648; + }; + const days = Array.from({ length: 60 }, (_, i) => + day(i, { + a1: Math.round(random() * 10), + dimensions: { a2: Math.round(random() * 10) }, + linked: { ...EMPTY_LINKED, steps: Math.round(random() * 10000) }, + }), + ); + const selected = screenFeatures(days).filter((s) => s.selected); + expect(selected.map((s) => s.key)).toEqual([]); + }); + + it("caps the column count", () => { + expect(MAX_MODEL_FEATURES).toBe(12); + }); +}); + +describe("buildFeatureMatrix", () => { + function drivenDays(count: number): PrognosisDayInput[] { + return Array.from({ length: count }, (_, i) => + day(i, { a1: 10 - (i % 11), dimensions: { a2: i % 11 } }), + ); + } + + it("standardises per user, so the columns are that person's own units", () => { + const matrix = buildFeatureMatrix(drivenDays(40)); + expect(matrix).not.toBeNull(); + if (!matrix) return; + const column = matrix.rows.map((row) => row[0]); + const mean = column.reduce((s, v) => s + v, 0) / column.length; + const sd = Math.sqrt( + column.reduce((s, v) => s + (v - mean) * (v - mean), 0) / column.length, + ); + expect(mean).toBeCloseTo(0, 6); + expect(sd).toBeCloseTo(1, 6); + // The standardisation is kept so a later day can be scored on the same + // scale rather than re-standardised against itself. + expect(matrix.standardisation[0].key).toBe(matrix.features[0]); + expect(matrix.standardisation[0].sd).toBeGreaterThan(0); + }); + + it("orders rows oldest first, which is what makes the validation honest", () => { + const matrix = buildFeatureMatrix(drivenDays(40)); + expect(matrix).not.toBeNull(); + if (!matrix) return; + const sorted = [...matrix.days].sort(); + expect(matrix.days).toEqual(sorted); + }); + + it("drops a day that is missing a selected feature rather than filling it in", () => { + const days = drivenDays(40); + days[5] = { ...days[5], dimensions: {} }; + const matrix = buildFeatureMatrix(days); + expect(matrix).not.toBeNull(); + if (!matrix) return; + expect(matrix.days).not.toContain(days[5].day); + expect(matrix.rows).toHaveLength(39); + // Nothing was imputed in its place. + expect(matrix.targets).toHaveLength(39); + }); + + it("refuses when nothing survives the screening", () => { + const flat = Array.from({ length: 40 }, (_, i) => day(i, { a1: 5 })); + expect(buildFeatureMatrix(flat)).toBeNull(); + }); + + it("refuses a constant target instead of dividing by zero", () => { + const constant = Array.from({ length: 40 }, (_, i) => + day(i, { a1: 5, dimensions: { a2: i % 7 } }), + ); + expect(buildFeatureMatrix(constant)).toBeNull(); + }); +}); + +describe("standardiseDay", () => { + it("scores a day on the stored scale", () => { + const row = standardiseDay(day(0, { dimensions: { a2: 5 } }), [ + { key: "dimension:a2", mean: 5, sd: 2 }, + ]); + // Up-is-better A2 of a stress-5 day is 5, the mean, so zero. + expect(row).toEqual([0]); + }); + + it("refuses a day missing one of the fit's features", () => { + expect( + standardiseDay(day(0, { dimensions: {} }), [ + { key: "dimension:a2", mean: 5, sd: 2 }, + ]), + ).toBeNull(); + }); +}); diff --git a/src/lib/analytics/mood-prognosis/__tests__/forecast.test.ts b/src/lib/analytics/mood-prognosis/__tests__/forecast.test.ts new file mode 100644 index 000000000..78c7df9f6 --- /dev/null +++ b/src/lib/analytics/mood-prognosis/__tests__/forecast.test.ts @@ -0,0 +1,71 @@ +/** + * `computeForecast` — the two floors a fit has to clear, and the refusals. + * + * The first floor is the ladder, on rated days. The second is the one this + * file exists for: a complete-case matrix can shrink well below the rated-day + * count, because a day missing one of the fit's inputs is dropped rather than + * imputed. An account with thirty rated days but a feature recorded on only + * two thirds of them fits on twenty complete rows, and a forecast "built from + * twenty days" past a floor that was supposed to mean thirty is a floor that + * does not hold. The fit is refused below `MIN_ENTRIES_FOR_FORECAST` complete + * rows, not only below thirty rated days. + */ +import { describe, expect, it } from "vitest"; + +import { computeForecast } from "../forecast"; +import type { PrognosisDayInput } from "../features"; +import { MIN_ENTRIES_FOR_FORECAST } from "../thresholds"; + +const EMPTY_LINKED = { + sleepAsleep: null, + steps: null, + activeEnergy: null, + restingHeartRate: null, + heartRateVariability: null, +}; + +function dayKey(index: number): string { + return new Date(Date.UTC(2026, 0, 1 + index)).toISOString().slice(0, 10); +} + +/** + * `rated` days carrying a rating driven by A2, of which the most recent + * `withFeature` actually carry the A2 slider. The rest are older face-only + * days: rated, but with no input the fit can use, so they drop out of the + * complete-case matrix. The feature sits on the RECENT days on purpose, so the + * days the write window would score are scorable — otherwise the refusal is + * "no recent day to score" rather than the "too few complete rows" this file + * is about. + */ +function sparseDays(rated: number, withFeature: number): PrognosisDayInput[] { + return Array.from({ length: rated }, (_, i) => { + const stress = i % 11; + const hasFeature = i >= rated - withFeature; + return { + day: dayKey(i), + a1: Math.round(0.8 * (10 - stress) + 1), + dimensions: hasFeature ? { a2: stress } : {}, + context: null, + linked: { ...EMPTY_LINKED }, + }; + }); +} + +describe("computeForecast complete-row floor", () => { + it("refuses a fit that clears the rated-day floor but not the complete-row one", () => { + // 40 rated days, A2 on 25 of them: coverage 62.5% clears the candidate + // floor, so A2 is screened in, but the complete-case matrix is 25 rows — + // under the 30-row floor. + const result = computeForecast(sparseDays(40, 25)); + expect(result.status, JSON.stringify(result)).toBe("refused"); + }); + + it("writes once the complete rows themselves reach the floor", () => { + // Every rated day carries the feature, so complete rows equal rated days. + const result = computeForecast(sparseDays(40, 40)); + expect(result.status).toBe("written"); + if (result.status === "written") { + expect(result.n).toBeGreaterThanOrEqual(MIN_ENTRIES_FOR_FORECAST); + } + }); +}); diff --git a/src/lib/analytics/mood-prognosis/__tests__/gates.test.ts b/src/lib/analytics/mood-prognosis/__tests__/gates.test.ts new file mode 100644 index 000000000..7895459d0 --- /dev/null +++ b/src/lib/analytics/mood-prognosis/__tests__/gates.test.ts @@ -0,0 +1,205 @@ +/** + * The minimum-n ladder, at every boundary, from both sides. + * + * Each case names the threshold it is about, so a red here says which floor + * moved rather than "expected false to be true". That matters more than usual: + * the thing these gates prevent is invisible on screen, so the test output is + * the only place the failure is ever legible. + * + * Every one of these has been watched failing — the comparison flipped from + * `>=` to `>`, the red read, the file restored and the hash checked — and one + * check was deleted outright to prove the absence of a gate fails too. The log + * is in the phase's execution notes. + * + * The assertions name the REASON, not just the absence: "no forecast" and "no + * forecast because there are only eleven entries and fifteen is where anything + * starts" are different answers, and only the second one can be shown to + * somebody. + */ +import { readFileSync } from "node:fs"; +import { join } from "node:path"; + +import { describe, expect, it } from "vitest"; + +import { + MIN_ENTRIES_FOR_FORECAST, + MIN_ENTRIES_FOR_REGULAR_FORECAST, + MIN_ENTRIES_FOR_SEASONAL, + MIN_ENTRIES_LEARNING_PHASE, + MIN_TAG_OCCURRENCES, + forecastGate, + occurrencesSuffice, + prognosisStage, + seasonalComparisonsUnlocked, +} from "../thresholds"; +import { CONTEXT_MIN_PRESENT_DAYS } from "@/lib/insights/mood-context-crosstab"; + +describe("the ladder is ordered and its rungs are the documented numbers", () => { + it("holds the values the concept asks for", () => { + expect(MIN_ENTRIES_LEARNING_PHASE).toBe(15); + expect(MIN_ENTRIES_FOR_FORECAST).toBe(30); + expect(MIN_ENTRIES_FOR_REGULAR_FORECAST).toBe(60); + expect(MIN_ENTRIES_FOR_SEASONAL).toBe(90); + }); + + it("rises strictly, so no rung can be reached before the one below it", () => { + expect(MIN_ENTRIES_LEARNING_PHASE).toBeLessThan(MIN_ENTRIES_FOR_FORECAST); + expect(MIN_ENTRIES_FOR_FORECAST).toBeLessThan( + MIN_ENTRIES_FOR_REGULAR_FORECAST, + ); + expect(MIN_ENTRIES_FOR_REGULAR_FORECAST).toBeLessThan( + MIN_ENTRIES_FOR_SEASONAL, + ); + }); +}); + +describe("MIN_ENTRIES_LEARNING_PHASE — below it, nothing at all", () => { + it(`refuses at ${MIN_ENTRIES_LEARNING_PHASE - 1}, and says why`, () => { + const gate = forecastGate(MIN_ENTRIES_LEARNING_PHASE - 1); + expect(gate.present, "MIN_ENTRIES_LEARNING_PHASE let n-1 through").toBe( + false, + ); + expect(gate).toEqual({ + present: false, + reason: "no-output-yet", + stage: "insufficient", + entries: MIN_ENTRIES_LEARNING_PHASE - 1, + nextThreshold: MIN_ENTRIES_LEARNING_PHASE, + }); + }); + + it(`reaches the learning phase at exactly ${MIN_ENTRIES_LEARNING_PHASE}`, () => { + const gate = forecastGate(MIN_ENTRIES_LEARNING_PHASE); + expect( + gate, + "MIN_ENTRIES_LEARNING_PHASE did not admit its own value", + ).toEqual({ + present: false, + reason: "learning-phase", + stage: "learning", + entries: MIN_ENTRIES_LEARNING_PHASE, + nextThreshold: MIN_ENTRIES_FOR_FORECAST, + }); + }); + + it("an empty account is absence, never a zero", () => { + const gate = forecastGate(0); + expect(gate.present).toBe(false); + expect(gate).not.toHaveProperty("predicted"); + if (!gate.present) expect(gate.entries).toBe(0); + }); +}); + +describe("MIN_ENTRIES_FOR_FORECAST — below it, no forecast", () => { + it(`refuses at ${MIN_ENTRIES_FOR_FORECAST - 1}, naming the learning phase`, () => { + const gate = forecastGate(MIN_ENTRIES_FOR_FORECAST - 1); + expect(gate.present, "MIN_ENTRIES_FOR_FORECAST let n-1 through").toBe( + false, + ); + if (!gate.present) { + expect(gate.reason).toBe("learning-phase"); + expect(gate.nextThreshold).toBe(MIN_ENTRIES_FOR_FORECAST); + expect(gate.entries).toBe(MIN_ENTRIES_FOR_FORECAST - 1); + } + }); + + it(`forecasts at exactly ${MIN_ENTRIES_FOR_FORECAST}, labelled provisional`, () => { + const gate = forecastGate(MIN_ENTRIES_FOR_FORECAST); + expect(gate.present, "MIN_ENTRIES_FOR_FORECAST refused its own value").toBe( + true, + ); + if (gate.present) { + expect(gate.stage).toBe("provisional"); + expect(gate.provisional).toBe(true); + } + }); +}); + +describe("MIN_ENTRIES_FOR_REGULAR_FORECAST — below it, provisional", () => { + it(`is still provisional at ${MIN_ENTRIES_FOR_REGULAR_FORECAST - 1}`, () => { + const gate = forecastGate(MIN_ENTRIES_FOR_REGULAR_FORECAST - 1); + expect(gate.present).toBe(true); + if (gate.present) { + expect( + gate.provisional, + "MIN_ENTRIES_FOR_REGULAR_FORECAST dropped the provisional label at n-1", + ).toBe(true); + expect(gate.stage).toBe("provisional"); + } + }); + + it(`drops the label at exactly ${MIN_ENTRIES_FOR_REGULAR_FORECAST}`, () => { + const gate = forecastGate(MIN_ENTRIES_FOR_REGULAR_FORECAST); + expect(gate.present).toBe(true); + if (gate.present) { + expect( + gate.provisional, + "MIN_ENTRIES_FOR_REGULAR_FORECAST kept the provisional label at its own value", + ).toBe(false); + expect(gate.stage).toBe("regular"); + } + }); +}); + +describe("MIN_ENTRIES_FOR_SEASONAL — below it, no weekday or seasonal view", () => { + it(`refuses the comparison at ${MIN_ENTRIES_FOR_SEASONAL - 1}`, () => { + expect( + seasonalComparisonsUnlocked(MIN_ENTRIES_FOR_SEASONAL - 1), + "MIN_ENTRIES_FOR_SEASONAL unlocked the comparison at n-1", + ).toBe(false); + expect(prognosisStage(MIN_ENTRIES_FOR_SEASONAL - 1)).toBe("regular"); + }); + + it(`unlocks it at exactly ${MIN_ENTRIES_FOR_SEASONAL}`, () => { + expect( + seasonalComparisonsUnlocked(MIN_ENTRIES_FOR_SEASONAL), + "MIN_ENTRIES_FOR_SEASONAL refused its own value", + ).toBe(true); + expect(prognosisStage(MIN_ENTRIES_FOR_SEASONAL)).toBe("seasonal"); + }); +}); + +describe("MIN_TAG_OCCURRENCES — a value seen four times is not evidence", () => { + it(`refuses at ${MIN_TAG_OCCURRENCES - 1}`, () => { + expect( + occurrencesSuffice(MIN_TAG_OCCURRENCES - 1), + "MIN_TAG_OCCURRENCES admitted a value seen n-1 times", + ).toBe(false); + }); + + it(`admits at exactly ${MIN_TAG_OCCURRENCES}`, () => { + expect( + occurrencesSuffice(MIN_TAG_OCCURRENCES), + "MIN_TAG_OCCURRENCES refused its own value", + ).toBe(true); + }); + + it("is the crosstab's floor rather than a second opinion about it", () => { + expect(MIN_TAG_OCCURRENCES).toBe(CONTEXT_MIN_PRESENT_DAYS); + }); +}); + +describe("no second number is minted for an idea that already has one", () => { + const THRESHOLDS = readFileSync(join(__dirname, "../thresholds.ts"), "utf8"); + + it("the file really is the one under test", () => { + // The positive control: the two checks below pass on an empty read, so + // each needs proof the file was found and holds what it claims to. + expect(THRESHOLDS.length).toBeGreaterThan(1000); + expect(THRESHOLDS).toContain("MIN_ENTRIES_FOR_SEASONAL"); + }); + + it("declares no paired-observation floor of its own", () => { + // `MIN_PAIRED_N` lives in `correlations.ts` and the screening imports it + // from there. A constant here named for the same idea would be a second + // answer to one question, and the two would drift. + expect(THRESHOLDS).not.toMatch(/export\s+const\s+MIN_PAIRED/); + expect(THRESHOLDS).not.toMatch(/export\s+const\s+MIN_PAIRS/); + }); + + it("takes the occurrence floor from the crosstab rather than restating it", () => { + expect(THRESHOLDS).toMatch( + /MIN_TAG_OCCURRENCES\s*=\s*CONTEXT_MIN_PRESENT_DAYS/, + ); + }); +}); diff --git a/src/lib/analytics/mood-prognosis/__tests__/rolling-origin.test.ts b/src/lib/analytics/mood-prognosis/__tests__/rolling-origin.test.ts new file mode 100644 index 000000000..53f93277a --- /dev/null +++ b/src/lib/analytics/mood-prognosis/__tests__/rolling-origin.test.ts @@ -0,0 +1,269 @@ +/** + * The fit, and the one property that makes its band worth reading. + * + * The leakage case is the reason this file exists. A random k-fold over days + * produces a model that scores beautifully and predicts nothing, and there is + * nothing in the reported error that says so — which is why the assertion is + * on the FOLD INDICES and on the day keys themselves rather than on the error. + * An error-based check passes a leaking split whenever the leak happens to be + * small, which is exactly when nobody would look. + * + * Watched red: the split was replaced with a shuffle, and + * `describe("no training fold contains a day after its validation fold")` + * went red naming a future day inside a training fold. Reverted, hash checked. + */ +import { describe, expect, it } from "vitest"; + +import { buildFeatureMatrix, type PrognosisDayInput } from "../features"; +import { fitRidge, predictWithFit, solveLinearSystem } from "../ridge"; +import { + BAND_LOWER_QUANTILE, + BAND_UPPER_QUANTILE, + LAMBDA_GRID, + MIN_VALIDATION_ROWS, + MODEL_VERSION, + fitPrognosisModel, + quantile, + rollingOriginFolds, +} from "../validate"; + +const EMPTY_LINKED = { + sleepAsleep: null, + steps: null, + activeEnergy: null, + restingHeartRate: null, + heartRateVariability: null, +}; + +function dayKey(index: number): string { + return new Date(Date.UTC(2026, 0, 1 + index)).toISOString().slice(0, 10); +} + +/** A deterministic pseudo-random series — a flaky statistics test is useless. */ +function seeded(seed: number): () => number { + let state = seed; + return () => { + state = (state * 1103515245 + 12345) % 2147483648; + return state / 2147483648; + }; +} + +/** + * Days whose target is a clean linear function of one dimension. + * + * One driver rather than two, and that is a statement about the screening + * rather than about the fit: the screening is univariate, so a feature that + * only matters jointly with another is refused before the fit ever sees it. + * A two-driver fixture measures that property, not this one. + */ +function linearDays(count: number): PrognosisDayInput[] { + return Array.from({ length: count }, (_, i) => { + const stress = i % 11; + return { + day: dayKey(i), + // 0.8 × up-is-better stress, plus one — inside the 0-10 scale. + a1: 0.8 * (10 - stress) + 1, + dimensions: { a2: stress, a3: (i * 7) % 11 }, + context: null, + linked: { ...EMPTY_LINKED }, + }; + }); +} + +/** Days whose target is noise and whose inputs are unrelated noise. */ +function noisyDays(count: number, seed = 7): PrognosisDayInput[] { + const random = seeded(seed); + return Array.from({ length: count }, (_, i) => ({ + day: dayKey(i), + a1: Math.round(random() * 10), + dimensions: { + a2: Math.round(random() * 10), + a3: Math.round(random() * 10), + }, + context: null, + linked: { ...EMPTY_LINKED }, + })); +} + +describe("no training fold contains a day after its validation fold", () => { + const ROWS = 60; + const folds = rollingOriginFolds(ROWS); + + it("produces folds at all — the assertions below pass on an empty list", () => { + expect(folds.length).toBeGreaterThan(0); + expect(folds.length).toBeLessThanOrEqual(4); + // Every validation row is covered exactly once and the last fold reaches + // the end, so no day is silently dropped from the measurement. + expect(folds[folds.length - 1].validationEnd).toBe(ROWS); + }); + + it("trains on a prefix that ends exactly where the validation starts", () => { + for (const fold of folds) { + expect(fold.trainEnd).toBe(fold.validationStart); + expect(fold.trainEnd).toBeGreaterThan(0); + expect(fold.validationEnd).toBeGreaterThan(fold.validationStart); + } + }); + + it("holds on the day keys, not just on the indices", () => { + // The indices are only meaningful because the matrix is time-ordered, so + // the property is asserted against the dates the rows actually carry. + const matrix = buildFeatureMatrix(linearDays(ROWS)); + expect(matrix).not.toBeNull(); + if (!matrix) return; + + for (const fold of rollingOriginFolds(matrix.rows.length)) { + const trainingDays = matrix.days.slice(0, fold.trainEnd); + const validationDays = matrix.days.slice( + fold.validationStart, + fold.validationEnd, + ); + const latestTrainingDay = trainingDays[trainingDays.length - 1]; + const earliestValidationDay = validationDays[0]; + const leaked = trainingDays.filter((d) => d >= earliestValidationDay); + expect( + leaked, + `a training fold holds ${leaked.length} day(s) dated at or after the validation fold that starts on ${earliestValidationDay}`, + ).toEqual([]); + expect(latestTrainingDay < earliestValidationDay).toBe(true); + } + }); + + it("expands rather than slides — later folds train on more history", () => { + const sizes = folds.map((f) => f.trainEnd); + for (let i = 1; i < sizes.length; i++) { + expect(sizes[i]).toBeGreaterThan(sizes[i - 1]); + } + }); + + it("refuses to split a window too short to measure anything on", () => { + expect(rollingOriginFolds(MIN_VALIDATION_ROWS - 1)).toEqual([]); + }); +}); + +describe("the ridge solve", () => { + it("recovers the coefficients of a clean linear signal", () => { + // y = 2*x1 - 3*x2, standardised columns, tiny penalty. + const rows: number[][] = []; + const targets: number[] = []; + for (let i = 0; i < 40; i++) { + const x1 = ((i % 7) - 3) / 2; + const x2 = ((i % 5) - 2) / 1.5; + rows.push([x1, x2]); + targets.push(2 * x1 - 3 * x2 + 5); + } + const fit = fitRidge(rows, targets, 1e-6); + expect(fit).not.toBeNull(); + if (!fit) return; + expect(fit.coefficients[0]).toBeCloseTo(2, 2); + expect(fit.coefficients[1]).toBeCloseTo(-3, 2); + // The intercept is the mean of the training targets, and the columns are + // centred against their own means, so a row at the column means predicts + // the average day exactly. + expect(predictWithFit(fit, fit.columnMeans)).toBeCloseTo(fit.intercept, 6); + // And every training row is predicted back to its own target. + rows.forEach((row, i) => { + expect(predictWithFit(fit, row)).toBeCloseTo(targets[i], 2); + }); + }); + + it("does not divide by zero on a constant target", () => { + const rows = Array.from({ length: 20 }, (_, i) => [i / 10, (i % 3) / 2]); + const fit = fitRidge(rows, new Array(20).fill(4), 1); + expect(fit).not.toBeNull(); + if (!fit) return; + expect(fit.coefficients.every(Number.isFinite)).toBe(true); + expect(fit.intercept).toBe(4); + expect(predictWithFit(fit, [0, 0])).toBeCloseTo(4, 6); + }); + + it("refuses a singular system rather than answering zeros", () => { + // Two identical columns and no penalty: the system has no unique answer, + // and a vector of zeros would look exactly like "these features do not + // matter". + expect( + solveLinearSystem( + [ + [1, 1], + [1, 1], + ], + [1, 1], + ), + ).toBeNull(); + }); + + it("penalises: a larger lambda pulls the coefficients towards zero", () => { + const rows = Array.from({ length: 30 }, (_, i) => [((i % 7) - 3) / 2]); + const targets = rows.map((r) => 4 * r[0] + 5); + const light = fitRidge(rows, targets, 0.01); + const heavy = fitRidge(rows, targets, 100); + expect(light).not.toBeNull(); + expect(heavy).not.toBeNull(); + if (!light || !heavy) return; + expect(Math.abs(heavy.coefficients[0])).toBeLessThan( + Math.abs(light.coefficients[0]), + ); + }); +}); + +describe("the band reacts to how wrong the model is", () => { + it("is narrow on a clean signal and wide on noise", () => { + const clean = buildFeatureMatrix(linearDays(80)); + expect(clean).not.toBeNull(); + if (!clean) return; + const cleanModel = fitPrognosisModel(clean); + expect(cleanModel).not.toBeNull(); + if (!cleanModel) return; + const cleanWidth = cleanModel.bandHigh - cleanModel.bandLow; + + // Noise cannot go through the screening — it is refused before a fit is + // attempted, which is the stronger answer. So the band's reaction is + // measured on the matrix directly, with the screening bypassed by handing + // the fit a matrix built from the noisy targets and the clean columns. + const noisy = buildFeatureMatrix(linearDays(80)); + expect(noisy).not.toBeNull(); + if (!noisy) return; + const random = seeded(11); + const noisyMatrix = { + ...noisy, + targets: noisy.targets.map(() => Math.round(random() * 10)), + }; + const noisyModel = fitPrognosisModel(noisyMatrix); + expect(noisyModel).not.toBeNull(); + if (!noisyModel) return; + const noisyWidth = noisyModel.bandHigh - noisyModel.bandLow; + + expect( + noisyWidth, + "the band did not widen on noise — a confident wrong number is the failure this band exists to prevent", + ).toBeGreaterThan(cleanWidth * 2); + expect(cleanWidth).toBeLessThan(2); + }); + + it("refuses noise at the screening, before any fit is attempted", () => { + expect(buildFeatureMatrix(noisyDays(80))).toBeNull(); + }); + + it("takes the band from the validation residuals, at the stated quantiles", () => { + const matrix = buildFeatureMatrix(linearDays(80)); + expect(matrix).not.toBeNull(); + if (!matrix) return; + const model = fitPrognosisModel(matrix); + expect(model).not.toBeNull(); + if (!model) return; + expect(model.bandLow).toBeLessThanOrEqual(model.bandHigh); + expect(BAND_LOWER_QUANTILE).toBe(0.1); + expect(BAND_UPPER_QUANTILE).toBe(0.9); + expect(model.n).toBe(matrix.rows.length); + expect(model.modelVersion).toBe(MODEL_VERSION); + expect(LAMBDA_GRID).toContain(model.fit.lambda); + // Never zero: an unpenalised solve memorises the training days. + expect(Math.min(...LAMBDA_GRID)).toBeGreaterThan(0); + }); + + it("quantile interpolates rather than rounding to a sample", () => { + expect(quantile([0, 10], 0.5)).toBe(5); + expect(quantile([1, 2, 3, 4], 0)).toBe(1); + expect(quantile([1, 2, 3, 4], 1)).toBe(4); + }); +}); diff --git a/src/lib/analytics/mood-prognosis/feature-keys.ts b/src/lib/analytics/mood-prognosis/feature-keys.ts new file mode 100644 index 000000000..3f51c71b5 --- /dev/null +++ b/src/lib/analytics/mood-prognosis/feature-keys.ts @@ -0,0 +1,98 @@ +/** + * The names the model's inputs are known by, everywhere. + * + * One string per feature, stored in `MoodPrediction.features`, read back by + * the surface that names what the model weighted, and compared against by the + * tests. It is therefore a wire format in all but name: renaming a key + * silently orphans every stored row that used it, which is why the encoding is + * a function here rather than a template literal at four call sites. + * + * The shape is `:`, and the four families are the four places an + * input can come from: + * + * * `dimension:a2` — a level-A self-rating (A1 is the target) + * * `context:workLoad` — a numeric field of the day's context + * * `context:workStatus=off` — one value of a closed context vocabulary + * * `linked:steps` — a figure read from the module that owns it + * + * Nothing here resolves a label. The label is a locale lookup and this module + * is pure and bundle-safe, so the resolver lives beside the component that + * renders it and takes a parsed key from here. + */ +import type { MoodDimensionKey } from "@/lib/mood/dimensions"; + +/** The linked figures the model may read, and the day-figure each names. */ +export const LINKED_FEATURE_METRICS = [ + "sleepAsleep", + "steps", + "activeEnergy", + "restingHeartRate", + "heartRateVariability", +] as const; + +export type LinkedFeatureMetric = (typeof LINKED_FEATURE_METRICS)[number]; + +export type ParsedFeatureKey = + | { family: "dimension"; dimension: MoodDimensionKey } + | { family: "context"; field: string; value: null } + | { family: "context"; field: string; value: string } + | { family: "linked"; metric: LinkedFeatureMetric }; + +/** `dimension:a2` */ +export function dimensionFeatureKey(dimension: MoodDimensionKey): string { + return `dimension:${dimension}`; +} + +/** `context:workLoad` — a numeric field. */ +export function contextNumericFeatureKey(field: string): string { + return `context:${field}`; +} + +/** `context:workStatus=off` — one value of a closed vocabulary. */ +export function contextValueFeatureKey(field: string, value: string): string { + return `context:${field}=${value}`; +} + +/** `linked:steps` */ +export function linkedFeatureKey(metric: LinkedFeatureMetric): string { + return `linked:${metric}`; +} + +const DIMENSION_KEYS = new Set(["a1", "a2", "a3", "a4", "a5"]); +const LINKED_KEYS = new Set(LINKED_FEATURE_METRICS); + +/** + * Read a stored key back. + * + * `null` for anything this version does not recognise, and that case is real + * rather than defensive: a row written by an older `modelVersion` can name a + * feature the current build has dropped, and the surface has to skip it rather + * than render `undefined` at somebody. + */ +export function parseFeatureKey(key: string): ParsedFeatureKey | null { + const separator = key.indexOf(":"); + if (separator < 0) return null; + const family = key.slice(0, separator); + const rest = key.slice(separator + 1); + if (rest.length === 0) return null; + + if (family === "dimension") { + return DIMENSION_KEYS.has(rest) + ? { family: "dimension", dimension: rest as MoodDimensionKey } + : null; + } + if (family === "linked") { + return LINKED_KEYS.has(rest) + ? { family: "linked", metric: rest as LinkedFeatureMetric } + : null; + } + if (family === "context") { + const equals = rest.indexOf("="); + if (equals < 0) return { family: "context", field: rest, value: null }; + const field = rest.slice(0, equals); + const value = rest.slice(equals + 1); + if (field.length === 0 || value.length === 0) return null; + return { family: "context", field, value }; + } + return null; +} diff --git a/src/lib/analytics/mood-prognosis/features.ts b/src/lib/analytics/mood-prognosis/features.ts new file mode 100644 index 000000000..467b71236 --- /dev/null +++ b/src/lib/analytics/mood-prognosis/features.ts @@ -0,0 +1,489 @@ +/** + * One row per day, one column per thing the day was made of. + * + * The target is the day's own pleasantness (A1) and everything else is an + * input: the other four level-A dimensions, the numbers and the closed + * vocabularies of the day's context, and the figures the sleep, activity and + * recovery modules already hold for that day. A1 is the only target on + * purpose — five targets would be five ways to be wrong and five bands to + * explain. + * + * Four rules this file is built on, each of which has a way of being got + * wrong that looks fine on screen: + * + * 1. **Nothing is imputed. Ever.** A day with a missing input is not a day + * with an average input. The stated rule is complete cases: a day enters + * the fit only if it carries the target and every SELECTED feature, and a + * feature only becomes a candidate if it was recorded on at least + * {@link MIN_FEATURE_COVERAGE} of the days that carry a target. The + * coverage floor is what keeps the complete-case rule from quietly + * emptying the matrix: without it, one field somebody filled in twice + * would exclude every other day they ever logged. + * 2. **Standardised per user, not min-max.** A person whose stress never + * exceeds six must not have six read as "maximum stress". The mean and the + * spread come from their own fit window, so every feature is expressed in + * that person's own units — which is also what makes the stored + * contributions comparable to each other. + * 3. **Screened before it is fitted, with the multiple-comparison correction + * the rest of the product already runs.** A sweep over sixty context + * features at raw p < 0.05 finds three things that are not there. Pearson + * with the shared `MIN_PAIRED_N` floor, Benjamini-Hochberg at the shared + * `FDR_Q`, and the shared effect-size floor on the shrunk estimate — all + * imported, none re-tuned here. + * 4. **Orientation comes from the table that owns it.** A2 is inverse and is + * flipped through `MOOD_DIMENSIONS[i].inverse`, never inline, so the chart + * and the model read one source. Context ratings are NOT flipped: no table + * states their orientation, and inventing one here would be exactly the + * second opinion the dimension table exists to prevent. It costs nothing + * mathematically — a flip only changes the sign of a coefficient, and the + * stored contribution is signed anyway. + */ +import { + EFFECT_SIZE_FLOOR, + FDR_Q, + benjaminiHochberg, + shrinkEstimate, +} from "@/lib/insights/correlation-discovery"; +import { + MAX_P_VALUE, + MIN_PAIRED_N, + pearson, +} from "@/lib/insights/correlations"; +import { + CONTACT_CIRCLE_KEYS, + CONTACT_EXTENT_KEYS, + CONTACT_FORM_KEYS, + EVENT_TYPE_KEYS, + LEISURE_CATEGORY_KEYS, + WORK_STATUS_KEYS, +} from "@/lib/mood/context-vocabulary"; +import { MOOD_DIMENSIONS } from "@/lib/mood/dimensions"; +import type { MoodDimensionKey } from "@/lib/mood/dimensions"; + +import { + contextNumericFeatureKey, + contextValueFeatureKey, + dimensionFeatureKey, + linkedFeatureKey, + LINKED_FEATURE_METRICS, +} from "./feature-keys"; +import { occurrencesSuffice } from "./thresholds"; + +/** + * Most features the fit may carry. + * + * Twelve over thirty days is already generous — a ridge penalty is what makes + * that survivable at all — and the cap is what stops a well-logged account + * from being fitted with sixty columns and a hundred rows. + */ +export const MAX_MODEL_FEATURES = 12; + +/** + * Share of target-carrying days a feature must be recorded on to be a + * candidate at all. + * + * The other half of the complete-case rule. A field somebody filled in for a + * fortnight and then stopped is not a column of the matrix; it is a fortnight, + * and admitting it would silently discard every day outside that fortnight. + */ +export const MIN_FEATURE_COVERAGE = 0.6; + +/** The day's context as the model reads it. The note never comes here. */ +export interface PrognosisContextInput { + workStatus: string | null; + workMinutes: number | null; + overtimeMinutes: number | null; + workLoad: number | null; + workSatisfaction: number | null; + contactCircles: string[]; + contactForm: string | null; + contactExtent: string | null; + contactQuality: number | null; + contactSupport: number | null; + leisureCategories: string[]; + leisureMinutes: number | null; + leisureJoy: number | null; + leisureRecovery: number | null; + eventType: string | null; + eventValence: number | null; +} + +/** One day, as everything upstream of the fit has resolved it. */ +export interface PrognosisDayInput { + /** Local day key, `YYYY-MM-DD`. */ + day: string; + /** The target: the day's own pleasantness, 0-10. `null` days are not fitted. */ + a1: number | null; + /** The other four self-ratings, literal as the person set them. */ + dimensions: Partial>; + /** The day's stored context, or `null` when no section was opened. */ + context: PrognosisContextInput | null; + /** What the owning modules hold for the day. `null` is absence. */ + linked: { + sleepAsleep: number | null; + steps: number | null; + activeEnergy: number | null; + restingHeartRate: number | null; + heartRateVariability: number | null; + }; +} + +/** The numeric context fields, in capture order. */ +const CONTEXT_NUMERIC_FIELDS = [ + "workMinutes", + "overtimeMinutes", + "workLoad", + "workSatisfaction", + "contactQuality", + "contactSupport", + "leisureMinutes", + "leisureJoy", + "leisureRecovery", + "eventValence", +] as const satisfies readonly (keyof PrognosisContextInput)[]; + +/** The single-valued vocabularies, each expanded to one column per value. */ +const CONTEXT_SINGLE_VOCABULARIES = [ + { field: "workStatus", keys: WORK_STATUS_KEYS }, + { field: "contactForm", keys: CONTACT_FORM_KEYS }, + { field: "contactExtent", keys: CONTACT_EXTENT_KEYS }, + { field: "eventType", keys: EVENT_TYPE_KEYS }, +] as const; + +/** The multi-select vocabularies, same expansion. */ +const CONTEXT_MULTI_VOCABULARIES = [ + { field: "contactCircles", keys: CONTACT_CIRCLE_KEYS }, + { field: "leisureCategories", keys: LEISURE_CATEGORY_KEYS }, +] as const; + +function isNumber(value: unknown): value is number { + return typeof value === "number" && Number.isFinite(value); +} + +/** + * Every feature this day carries a value for. + * + * A feature the day says nothing about is ABSENT from the map — it is not + * zero. The difference is the whole of rule 1: a day with no context row has + * no work-status column at all, while a day whose work status is `off` has a + * zero in every other work-status column and a one in that one. + */ +export function extractDayFeatures( + day: PrognosisDayInput, +): Map { + const out = new Map(); + + for (const dimension of MOOD_DIMENSIONS) { + // A1 is the target, not an input. Fitting a value on itself would produce + // a perfect model of nothing. + if (dimension.key === "a1") continue; + const raw = day.dimensions[dimension.key]; + if (!isNumber(raw)) continue; + // The orientation flip goes through the table that declares it, so a + // dimension that becomes inverse later moves the chart and the model + // together. + out.set( + dimensionFeatureKey(dimension.key), + dimension.inverse ? dimension.max - raw : raw, + ); + } + + const context = day.context; + if (context) { + for (const field of CONTEXT_NUMERIC_FIELDS) { + const raw = context[field]; + if (isNumber(raw)) out.set(contextNumericFeatureKey(field), raw); + } + for (const { field, keys } of CONTEXT_SINGLE_VOCABULARIES) { + const value = context[field]; + // A field nobody answered contributes no columns. Zeroing them would + // tell the fit "not overtime" on a day nobody said anything about work. + if (typeof value !== "string" || value.length === 0) continue; + for (const key of keys) { + out.set(contextValueFeatureKey(field, key), value === key ? 1 : 0); + } + } + for (const { field, keys } of CONTEXT_MULTI_VOCABULARIES) { + const values = context[field]; + if (!Array.isArray(values) || values.length === 0) continue; + const chosen = new Set(values); + for (const key of keys) { + out.set(contextValueFeatureKey(field, key), chosen.has(key) ? 1 : 0); + } + } + } + + for (const metric of LINKED_FEATURE_METRICS) { + const raw = day.linked[metric]; + if (isNumber(raw)) out.set(linkedFeatureKey(metric), raw); + } + + return out; +} + +/** What the screening decided about one candidate, kept for the tests. */ +export interface ScreenedFeature { + key: string; + /** Days carrying both the feature and the target. */ + pairs: number; + r: number; + /** The shrunk estimate the effect floor is applied to. */ + shrunk: number; + pValue: number; + qValue: number; + selected: boolean; + /** Why it was refused, when it was. */ + reason?: + | "coverage" + | "too-few-pairs" + | "no-variance" + | "p-value" + | "fdr" + | "effect-size" + | "occurrences" + | "cap"; +} + +/** + * Which candidates earn a column, and why the others did not. + * + * Order of refusal matters and is the order below: coverage, then pairs, then + * the raw p, then the family-wide FDR, then the effect floor, then the cap. + * A feature that fails an earlier gate is not tested against a later one, so + * the family the FDR corrects over is the set that was actually tested. + */ +export function screenFeatures( + days: readonly PrognosisDayInput[], +): ScreenedFeature[] { + const targetDays = days.filter((d) => isNumber(d.a1)); + const perDay = new Map>(); + for (const day of targetDays) perDay.set(day.day, extractDayFeatures(day)); + + const candidates = new Set(); + for (const features of perDay.values()) { + for (const key of features.keys()) candidates.add(key); + } + + const coverageFloor = targetDays.length * MIN_FEATURE_COVERAGE; + const refused: ScreenedFeature[] = []; + const tested: Array<{ + key: string; + pairs: number; + r: number; + shrunk: number; + pValue: number; + }> = []; + + for (const key of [...candidates].sort()) { + const xs: number[] = []; + const ys: number[] = []; + let occurrences = 0; + for (const day of targetDays) { + const value = perDay.get(day.day)?.get(key); + if (value === undefined) continue; + xs.push(value); + ys.push(day.a1 as number); + if (value !== 0) occurrences += 1; + } + + if (xs.length < coverageFloor) { + refused.push(zero(key, xs.length, "coverage")); + continue; + } + // A one-hot column seen four times is not evidence, however well it + // correlates. This is the ladder's occurrence floor, not a second one. + if (!occurrencesSuffice(occurrences)) { + refused.push(zero(key, xs.length, "occurrences")); + continue; + } + + const result = pearson({ xs, ys, minPairs: MIN_PAIRED_N }); + if (result.status === "insufficient") { + refused.push( + zero( + key, + result.n, + result.reason === "no_variance" ? "no-variance" : "too-few-pairs", + ), + ); + continue; + } + if (result.pValue >= MAX_P_VALUE) { + refused.push({ + key, + pairs: result.n, + r: result.r, + shrunk: shrinkEstimate(result.r, result.n), + pValue: result.pValue, + qValue: 1, + selected: false, + reason: "p-value", + }); + continue; + } + tested.push({ + key, + pairs: result.n, + r: result.r, + shrunk: shrinkEstimate(result.r, result.n), + pValue: result.pValue, + }); + } + + const qValues = benjaminiHochberg(tested.map((t) => t.pValue)); + const survivors: ScreenedFeature[] = []; + tested.forEach((t, index) => { + const qValue = qValues[index] ?? 1; + if (qValue > FDR_Q) { + survivors.push({ ...t, qValue, selected: false, reason: "fdr" }); + return; + } + if (Math.abs(t.shrunk) < EFFECT_SIZE_FLOOR) { + survivors.push({ ...t, qValue, selected: false, reason: "effect-size" }); + return; + } + survivors.push({ ...t, qValue, selected: true }); + }); + + // The cap keeps the strongest, and a stable tie-break by key keeps two runs + // over the same data returning the same columns. + const admitted = survivors + .filter((s) => s.selected) + .sort( + (a, b) => + Math.abs(b.shrunk) - Math.abs(a.shrunk) || a.key.localeCompare(b.key), + ); + for (const feature of admitted.slice(MAX_MODEL_FEATURES)) { + feature.selected = false; + feature.reason = "cap"; + } + + return [...survivors, ...refused].sort((a, b) => a.key.localeCompare(b.key)); +} + +function zero( + key: string, + pairs: number, + reason: ScreenedFeature["reason"], +): ScreenedFeature { + return { + key, + pairs, + r: 0, + shrunk: 0, + pValue: 1, + qValue: 1, + selected: false, + reason, + }; +} + +/** Per-feature centre and spread, over this account's own fit window. */ +export interface FeatureStandardisation { + key: string; + mean: number; + sd: number; +} + +export interface FeatureMatrix { + /** The selected feature keys, in column order. */ + features: string[]; + /** Standardised rows, one per complete day, oldest first. */ + rows: number[][]; + /** The target for each row, unstandardised (0-10). */ + targets: number[]; + /** The day each row is, so the validation can stay time-ordered. */ + days: string[]; + /** What each column was standardised by, needed to score a new day. */ + standardisation: FeatureStandardisation[]; +} + +/** + * Build the matrix: screen, then keep the complete days, then standardise. + * + * Returns `null` when nothing survives the screening or when no day carries + * every surviving feature. That is a refusal, not an empty matrix — the caller + * writes no forecast rather than one built on nothing. + */ +export function buildFeatureMatrix( + days: readonly PrognosisDayInput[], +): FeatureMatrix | null { + const screened = screenFeatures(days); + const selected = screened.filter((s) => s.selected).map((s) => s.key); + if (selected.length === 0) return null; + + const ordered = [...days] + .filter((d) => isNumber(d.a1)) + .sort((a, b) => a.day.localeCompare(b.day)); + + const rawRows: number[][] = []; + const targets: number[] = []; + const dayKeys: string[] = []; + for (const day of ordered) { + const features = extractDayFeatures(day); + const row: number[] = []; + let complete = true; + for (const key of selected) { + const value = features.get(key); + if (value === undefined) { + complete = false; + break; + } + row.push(value); + } + // Complete cases only. A day missing one selected feature is a day the fit + // does not see; nothing is filled in for it. + if (!complete) continue; + rawRows.push(row); + targets.push(day.a1 as number); + dayKeys.push(day.day); + } + + if (rawRows.length === 0) return null; + + const standardisation = selected.map((key, column) => { + const values = rawRows.map((row) => row[column]); + const mean = values.reduce((sum, v) => sum + v, 0) / values.length; + const variance = + values.reduce((sum, v) => sum + (v - mean) * (v - mean), 0) / + values.length; + return { key, mean, sd: Math.sqrt(variance) }; + }); + + // A column with no spread inside the complete-case set carries no + // information and would divide by zero. It goes, and if that empties the + // set the whole fit is refused. + const keep = standardisation + .map((s, index) => ({ ...s, index })) + .filter((s) => s.sd > 0); + if (keep.length === 0) return null; + + return { + features: keep.map((k) => k.key), + rows: rawRows.map((row) => keep.map((k) => (row[k.index] - k.mean) / k.sd)), + targets, + days: dayKeys, + standardisation: keep.map(({ key, mean, sd }) => ({ key, mean, sd })), + }; +} + +/** + * Standardise one day's raw features against a stored fit, for scoring. + * + * `null` when the day does not carry every feature the fit uses — the same + * complete-case rule the matrix applies, asked of a single day. A day that + * cannot be scored gets no forecast, which is the honest answer. + */ +export function standardiseDay( + day: PrognosisDayInput, + standardisation: readonly FeatureStandardisation[], +): number[] | null { + const features = extractDayFeatures(day); + const row: number[] = []; + for (const { key, mean, sd } of standardisation) { + const value = features.get(key); + if (value === undefined || sd <= 0) return null; + row.push((value - mean) / sd); + } + return row; +} diff --git a/src/lib/analytics/mood-prognosis/forecast.ts b/src/lib/analytics/mood-prognosis/forecast.ts new file mode 100644 index 000000000..5a9ec91e4 --- /dev/null +++ b/src/lib/analytics/mood-prognosis/forecast.ts @@ -0,0 +1,160 @@ +/** + * One account's answer, computed and handed back as rows to write. + * + * Pure: it takes the days somebody logged and returns either the rows a pass + * should store or a refusal with the reason. It does not touch the database, + * and that is a boundary rather than a style — `MoodPrediction` has exactly + * one writer, the refresh job, and a guard holds it there + * (`src/__tests__/mood-prediction-write-surface-guard.test.ts`). Keeping the + * arithmetic here and the write there is what lets the whole decision path be + * tested without a container. + * + * **It refuses more often than it answers, and that is the design.** Too few + * days, no feature that clears the screening, no recent day carrying every + * feature the fit needs, a singular system — every one of them returns a + * refusal and the account is left with no forecast at all. The surface then + * says there is not enough to go on, which is true, instead of showing a + * number built out of three days and a coincidence. + */ +import { + buildFeatureMatrix, + standardiseDay, + type PrognosisDayInput, +} from "./features"; +import { PROGNOSIS_WRITE_WINDOW_DAYS } from "./load"; +import { featureContributions, predictWithFit } from "./ridge"; +import { MIN_ENTRIES_FOR_FORECAST, forecastGate } from "./thresholds"; +import { fitPrognosisModel } from "./validate"; + +/** The lowest and highest value the target scale can express. */ +export const TARGET_MIN = 0; +export const TARGET_MAX = 10; + +/** One stored contribution: what a feature moved the expected value by. */ +export interface FeatureContribution { + feature: string; + contribution: number; +} + +/** One day's forecast, in the shape the row is written in. */ +export interface ForecastRow { + date: string; + predicted: number; + ciLow: number; + ciHigh: number; + n: number; + modelVersion: string; + /** Ordered by absolute contribution, largest first. */ + contributions: FeatureContribution[]; +} + +/** Why an account gets no forecast. Every one of these is a normal outcome. */ +export type ForecastRefusal = + /** Below the ladder's forecast threshold. */ + | "too-few-days" + /** Nothing recorded often enough, or varied enough, to screen in. */ + | "no-pattern" + /** A pattern, but no recent day carrying every feature it needs. */ + | "no-scorable-day"; + +export type ForecastResult = + | { + status: "written"; + rows: ForecastRow[]; + /** Complete days the fit used. The `n` every surface must show. */ + n: number; + modelVersion: string; + /** How wide the band came out, for the job's event only. */ + bandWidth: number; + featureCount: number; + } + | { status: "refused"; reason: ForecastRefusal }; + +function clampToScale(value: number): number { + return Math.min(TARGET_MAX, Math.max(TARGET_MIN, value)); +} + +function round(value: number, digits: number): number { + const factor = 10 ** digits; + return Math.round(value * factor) / factor; +} + +/** + * Fit the account's window and score its most recent days. + * + * The ladder is read against the number of DAYS carrying a self-rating, which + * is the number a person can see themselves growing. The `n` stored on each + * row is the number of complete days the fit actually used, which can be + * smaller — a day that is missing one of the fit's inputs cannot be part of + * it. Both numbers are honest and they answer different questions; the one on + * the surface beside the value is `n`, because that is what the value rests on. + */ +export function computeForecast( + days: readonly PrognosisDayInput[], + writeWindowDays: number = PROGNOSIS_WRITE_WINDOW_DAYS, +): ForecastResult { + const rated = days.filter((d) => typeof d.a1 === "number"); + const gate = forecastGate(rated.length); + if (!gate.present) return { status: "refused", reason: "too-few-days" }; + + const matrix = buildFeatureMatrix(days); + const model = matrix === null ? null : fitPrognosisModel(matrix); + if (model === null) return { status: "refused", reason: "no-pattern" }; + + // The second floor, and the one the rated-day gate above cannot see. A day + // missing one of the selected features is dropped rather than imputed, so a + // sparse-context account can clear thirty rated days and still fit on a + // handful of complete rows. `n` shown beside the value and a wide band on a + // small fit keep it from being dishonest, but a forecast "built from twelve + // days" past a floor that was supposed to mean thirty is a floor that does + // not hold. The number of days the fit ACTUALLY used has to clear the same + // threshold, not only the number of days that carried a rating. + if (model.n < MIN_ENTRIES_FOR_FORECAST) { + return { status: "refused", reason: "no-pattern" }; + } + + const rows: ForecastRow[] = []; + for (const day of rated.slice(-writeWindowDays)) { + const standardised = standardiseDay(day, model.standardisation); + // A day missing one of the fit's features cannot be scored, and nothing is + // filled in for it. It simply has no forecast. + if (standardised === null) continue; + const raw = predictWithFit(model.fit, standardised); + if (raw === null) continue; + + rows.push({ + date: day.day, + predicted: round(clampToScale(raw), 2), + // The band travels with the prediction and is clamped to the same scale: + // a lower edge below zero would be a claim the scale cannot express. + ciLow: round(clampToScale(raw + model.bandLow), 2), + ciHigh: round(clampToScale(raw + model.bandHigh), 2), + n: model.n, + modelVersion: model.modelVersion, + contributions: featureContributions( + model.fit, + model.features, + standardised, + ) + .filter((c) => Number.isFinite(c.contribution)) + .sort((a, b) => Math.abs(b.contribution) - Math.abs(a.contribution)) + .map((c) => ({ + feature: c.feature, + contribution: round(c.contribution, 3), + })), + }); + } + + if (rows.length === 0) { + return { status: "refused", reason: "no-scorable-day" }; + } + + return { + status: "written", + rows, + n: model.n, + modelVersion: model.modelVersion, + bandWidth: round(model.bandHigh - model.bandLow, 2), + featureCount: model.features.length, + }; +} diff --git a/src/lib/analytics/mood-prognosis/load.ts b/src/lib/analytics/mood-prognosis/load.ts new file mode 100644 index 000000000..930326756 --- /dev/null +++ b/src/lib/analytics/mood-prognosis/load.ts @@ -0,0 +1,213 @@ +/** + * The database read behind the fit, and nothing else. + * + * Separate from `features.ts` on purpose. `features.ts` is pure, and it shares + * `feature-keys.ts` with the client component that labels a stored feature key + * (`components/insights/mood/prognosis-feature-label.ts` imports the parser + * from there) — so those two modules form a Prisma-free island the browser + * bundle can reach. A database import in `features.ts` would drag the client + * towards that bundle; keeping the read here is what holds the line. The split + * is the boundary, not a preference. + * + * **One row per day, taken whole from one entry.** A day with two entries + * contributes the later one — its target, its dimensions and its context + * together. Averaging the ratings and taking the context from whichever entry + * happened to carry one would build a row nobody logged, and the model would + * then be fitted on days that did not happen. + * + * The day key is the entry's own `date`, already resolved in the zone the + * entry was written in. Re-deriving it here from `moodLoggedAt` would be a + * second opinion about which day a row belongs to, which is the class of bug + * the per-row `tz` column was added to end. + */ +import { prisma } from "@/lib/db"; +import { decodeKeyList } from "@/lib/mood/context-vocabulary"; +import { DEFAULT_TIMEZONE } from "@/lib/mood/date-key"; +import { resolveLinkedDayFigures } from "@/lib/mood/linked-context"; + +import type { PrognosisDayInput } from "./features"; + +/** + * How far back the fit looks. + * + * A year: long enough to hold a seasonal swing, short enough that a life that + * changed two years ago is not still being fitted against today. Longer would + * also make the per-user standardisation describe somebody the person no + * longer is. + */ +export const PROGNOSIS_FIT_WINDOW_DAYS = 365; + +/** + * How many recent days the job writes a forecast for. + * + * Not the whole window: a forecast is worth storing for the days somebody + * might still be looking at. Older rows are recomputable and are deleted on + * every pass, which is also what keeps a stale forecast from outliving the + * data it was fitted on. + */ +export const PROGNOSIS_WRITE_WINDOW_DAYS = 7; + +/** + * Hard cap on the rows one fit reads. + * + * A year holds 365 days and the fold keeps one entry per day, so the cap only + * ever bites on an account that logs many times a day — and there it bites on + * the oldest days, which is the right end to lose. It is here because an + * unbounded `findMany` in a nightly job over an account with years of history + * is a table scan waiting for the account that has one. + */ +export const MAX_PROGNOSIS_ROWS = 5000; + +/** `YYYY-MM-DD`, this many days before the given instant, in UTC terms. */ +export function dayKeyBefore(now: Date, days: number): string { + return new Date(now.getTime() - days * 86_400_000).toISOString().slice(0, 10); +} + +/** + * Every day of the fit window, in day order, with everything the matrix needs. + * + * The linked figures come from the resolver that owns them, over the whole + * window in one query. They are read in the account's display zone: the entry + * rows carry their own zone for their own day key, but a step count has to be + * bucketed under some zone, and the account's own is the one every other + * surface uses. + */ +export async function loadPrognosisDays( + userId: string, + now: Date, + timezone: string | null, + windowDays: number = PROGNOSIS_FIT_WINDOW_DAYS, +): Promise { + const since = dayKeyBefore(now, windowDays); + const entries = await prisma.moodEntry.findMany({ + where: { userId, deletedAt: null, date: { gte: since } }, + // Descending, and the fold below keeps the FIRST row it sees per day, so + // "the last entry of a day wins" is the read order rather than a + // comparison. Descending also puts the cap on the right end: an account + // that logs many times a day loses its oldest days to the limit, never + // its newest ones, and the newest are the days a forecast is written for. + orderBy: [{ date: "desc" }, { moodLoggedAt: "desc" }], + take: MAX_PROGNOSIS_ROWS, + select: { + date: true, + moodA1: true, + stressA2: true, + energyA3: true, + connectionA4: true, + stabilityA5: true, + context: { + select: { + workStatus: true, + workMinutes: true, + overtimeMinutes: true, + workLoad: true, + workSatisfaction: true, + contactCircles: true, + contactForm: true, + contactExtent: true, + contactQuality: true, + contactSupport: true, + leisureCategories: true, + leisureMinutes: true, + leisureJoy: true, + leisureRecovery: true, + eventType: true, + eventValence: true, + // The note is deliberately not selected. It is encrypted free text + // about somebody's day and the model has no use for it; a fit does + // not read prose, and a query that decrypted it anyway would be + // reading a diary to compute an average. + }, + }, + }, + }); + + const byDay = new Map(); + for (const entry of entries) { + // The rows arrive newest first, so the first one seen for a day is that + // day's last entry. + if (byDay.has(entry.date)) continue; + byDay.set(entry.date, { + day: entry.date, + a1: entry.moodA1, + dimensions: { + a2: entry.stressA2, + a3: entry.energyA3, + a4: entry.connectionA4, + a5: entry.stabilityA5, + }, + context: entry.context + ? { + workStatus: entry.context.workStatus, + workMinutes: entry.context.workMinutes, + overtimeMinutes: entry.context.overtimeMinutes, + workLoad: entry.context.workLoad, + workSatisfaction: entry.context.workSatisfaction, + contactCircles: decodeKeyList(entry.context.contactCircles), + contactForm: entry.context.contactForm, + contactExtent: entry.context.contactExtent, + contactQuality: entry.context.contactQuality, + contactSupport: entry.context.contactSupport, + leisureCategories: decodeKeyList(entry.context.leisureCategories), + leisureMinutes: entry.context.leisureMinutes, + leisureJoy: entry.context.leisureJoy, + leisureRecovery: entry.context.leisureRecovery, + eventType: entry.context.eventType, + eventValence: entry.context.eventValence, + } + : null, + linked: { + sleepAsleep: null, + steps: null, + activeEnergy: null, + restingHeartRate: null, + heartRateVariability: null, + }, + }); + } + + const days = [...byDay.keys()].sort(); + if (days.length === 0) return []; + + const linked = await resolveLinkedDayFigures( + userId, + days, + timezone ?? DEFAULT_TIMEZONE, + ); + for (const day of days) { + const figures = linked.get(day); + if (figures) byDay.get(day)!.linked = figures; + } + + return days.map((day) => byDay.get(day)!); +} + +/** + * How many days of the window carry a self-rated pleasantness. + * + * The number the ladder is read against, and the number a surface quotes when + * it explains why there is nothing yet. Days, not entries: two entries on one + * Tuesday are one Tuesday, and counting them as two would let somebody reach + * a threshold by logging twice a day for a fortnight. + */ +export async function countQualifyingDays( + userId: string, + now: Date, + windowDays: number = PROGNOSIS_FIT_WINDOW_DAYS, +): Promise { + const since = dayKeyBefore(now, windowDays); + const days = await prisma.moodEntry.findMany({ + where: { + userId, + deletedAt: null, + moodA1: { not: null }, + date: { gte: since }, + }, + distinct: ["date"], + select: { date: true }, + // Bounded by the window either way — a year holds 366 distinct day keys — + // and the cap says so rather than trusting the window to be small. + take: 400, + }); + return days.length; +} diff --git a/src/lib/analytics/mood-prognosis/read.ts b/src/lib/analytics/mood-prognosis/read.ts new file mode 100644 index 000000000..480ea9fe5 --- /dev/null +++ b/src/lib/analytics/mood-prognosis/read.ts @@ -0,0 +1,231 @@ +/** + * What the surfaces read: the two values of a day, side by side, never merged. + * + * The self-assessment is the leading value and is returned exactly as it was + * stored. The prognosis is the second reading — computed, banded, counted, and + * not editable. The deviation between them is the finding, and it is measured + * against the BAND rather than against the midpoint: a rating inside the band + * is a day that went as the person's own past days suggest, and only a rating + * outside it is worth a sentence. + * + * One reader for every surface. The route, the Coach snapshot and the daily + * briefing all come through here, so a change to what "the day's forecast" + * means moves all three together instead of two of them. + * + * Absence is explicit and carries its reason and its count. `no-output-yet` + * and `learning-phase` come from the ladder; `no-pattern` is the honest answer + * when there are plenty of days and nothing in them the fit could hold on to. + * None of the three is an error and none of them is a zero. + */ +import { prisma } from "@/lib/db"; +import { + dayKeyAgeInDays, + isCurrentForTodayClaim, +} from "@/lib/insights/measurement-freshness"; +import { DEFAULT_TIMEZONE, moodDateKey } from "@/lib/mood/date-key"; + +import type { FeatureContribution } from "./forecast"; +import { countQualifyingDays } from "./load"; +import { + forecastGate, + seasonalComparisonsUnlocked, + type PrognosisStage, +} from "./thresholds"; + +/** How the day's own rating sat against the band. */ +export type PrognosisDeviation = "within" | "above" | "below"; + +export interface PrognosisPresent { + present: true; + /** The day the forecast is about, `YYYY-MM-DD`. */ + date: string; + /** The expected pleasantness, 0-10. Never shown without `n` and the band. */ + predicted: number; + ciLow: number | null; + ciHigh: number | null; + /** Complete days the fit used. Displayed wherever `predicted` is. */ + n: number; + modelVersion: string; + computedAt: string; + /** + * Whole days between the forecast's day and the reader's today. A forecast + * about the day before yesterday cannot back a sentence about today, and + * `current` is the same rule every other present-tense claim in the product + * uses — one constant, not a second one. + */ + ageDays: number | null; + current: boolean; + /** Below the regular threshold the forecast is labelled provisional. */ + provisional: boolean; + /** + * The rung the value is shown at. Only the three rungs that CARRY a value — + * a below-floor reading never reaches here, it becomes the absence shape — + * so the type matches the published enum exactly and an `insufficient` / + * `learning` stage cannot be constructed for a present reading. + */ + stage: Extract; + /** Days of history the ladder was read against. */ + entries: number; + /** Whether weekday and seasonal comparisons have unlocked. */ + seasonalUnlocked: boolean; + /** The person's own rating for that day. `null` when they gave none. */ + selfAssessment: number | null; + /** Where that rating sat against the band. `null` without a rating. */ + deviation: PrognosisDeviation | null; + /** How far outside the band it sat, in scale points. `0` when inside. */ + deviationAmount: number | null; + /** What the model weighted for that day, largest absolute first. */ + contributions: FeatureContribution[]; +} + +export interface PrognosisAbsent { + present: false; + /** + * `no-output-yet` — too few days for anything at all. + * `learning-phase` — enough to describe, not enough to forecast. + * `no-pattern` — enough days, nothing in them the fit could hold on to. + */ + reason: "no-output-yet" | "learning-phase" | "no-pattern"; + /** Days of history carrying a self-rating. */ + entries: number; + /** Days at which the next step unlocks, when a count is what is missing. */ + nextThreshold: number | null; +} + +export type PrognosisReading = PrognosisPresent | PrognosisAbsent; + +/** Parse the stored contributions. A malformed value reads as none. */ +export function parseContributions(stored: string): FeatureContribution[] { + try { + const parsed: unknown = JSON.parse(stored); + if (!Array.isArray(parsed)) return []; + return parsed.filter( + (item): item is FeatureContribution => + typeof item === "object" && + item !== null && + typeof (item as FeatureContribution).feature === "string" && + typeof (item as FeatureContribution).contribution === "number", + ); + } catch { + return []; + } +} + +/** Where a rating sits against a band. */ +export function deviationAgainstBand( + actual: number, + low: number | null, + high: number | null, +): { deviation: PrognosisDeviation; amount: number } | null { + if (low === null || high === null) return null; + if (actual > high) return { deviation: "above", amount: actual - high }; + if (actual < low) return { deviation: "below", amount: low - actual }; + return { deviation: "within", amount: 0 }; +} + +/** + * The newest forecast an account holds, with the day's own rating beside it. + * + * The job writes a trailing window and the newest row in it is normally + * yesterday's — the pass runs before anybody has logged today. That is why + * the age travels with the value instead of being assumed to be zero. + */ +export async function readMoodPrognosis( + userId: string, + now: Date = new Date(), + timezone?: string | null, +): Promise { + const row = await prisma.moodPrediction.findFirst({ + where: { userId }, + orderBy: { date: "desc" }, + select: { + date: true, + predicted: true, + ciLow: true, + ciHigh: true, + n: true, + modelVersion: true, + features: true, + computedAt: true, + }, + }); + + if (row === null) { + // No row is not automatically "too few days": an account can have a year + // of entries and still get no forecast, because nothing in them cleared + // the screening. The count decides which of the three answers is true. + const entries = await countQualifyingDays(userId, now); + const gate = forecastGate(entries); + if (!gate.present) { + return { + present: false, + reason: gate.reason, + entries, + nextThreshold: gate.nextThreshold, + }; + } + return { + present: false, + reason: "no-pattern", + entries, + nextThreshold: null, + }; + } + + const entries = await countQualifyingDays(userId, now); + + // A row on disk is not enough to answer with. The job writes nightly, so + // between an entry deletion and the next pass a row outlives its own + // eligibility. The read path re-reads the ladder and refuses below the + // floor, exactly as the job would — otherwise this route serves a value the + // account no longer has the data to support for up to a day, and it serves + // it under a stage ("insufficient" / "learning") that is not in the + // published enum, which fails a strict client decode of the whole payload. + const gate = forecastGate(entries); + if (!gate.present) { + return { + present: false, + reason: gate.reason, + entries, + nextThreshold: gate.nextThreshold, + }; + } + + const today = moodDateKey(now, timezone ?? DEFAULT_TIMEZONE); + const ageDays = dayKeyAgeInDays(row.date, today); + const entry = await prisma.moodEntry.findFirst({ + where: { userId, date: row.date, deletedAt: null, moodA1: { not: null } }, + orderBy: { moodLoggedAt: "desc" }, + select: { moodA1: true }, + }); + const selfAssessment = entry?.moodA1 ?? null; + const against = + selfAssessment === null + ? null + : deviationAgainstBand(selfAssessment, row.ciLow, row.ciHigh); + + return { + present: true, + date: row.date, + predicted: row.predicted, + ciLow: row.ciLow, + ciHigh: row.ciHigh, + n: row.n, + modelVersion: row.modelVersion, + computedAt: row.computedAt.toISOString(), + ageDays, + current: isCurrentForTodayClaim(ageDays), + // `gate` is the present variant here — the absence arm returned above — so + // the stage is one of provisional / regular / seasonal, never the + // out-of-enum "insufficient" or "learning" a stale row could otherwise + // carry onto the wire. + provisional: gate.provisional, + stage: gate.stage, + entries, + seasonalUnlocked: seasonalComparisonsUnlocked(entries), + selfAssessment, + deviation: against?.deviation ?? null, + deviationAmount: against?.amount ?? null, + contributions: parseContributions(row.features), + }; +} diff --git a/src/lib/analytics/mood-prognosis/ridge.ts b/src/lib/analytics/mood-prognosis/ridge.ts new file mode 100644 index 000000000..8f5335a4f --- /dev/null +++ b/src/lib/analytics/mood-prognosis/ridge.ts @@ -0,0 +1,173 @@ +/** + * Ridge regression, closed form, in about sixty lines of arithmetic. + * + * `(XᵀX + λI)β = Xᵀy`, solved by Gaussian elimination with partial pivoting. + * That is the whole model. It is here rather than in a dependency because a + * dozen standardised columns is a 12×12 system, and adding a package to solve + * one of those would be adding a supply chain to avoid writing a loop. + * + * Two properties the rest of the feature relies on: + * + * * **The intercept is not penalised, because there is no intercept in the + * system.** The target is centred before the solve and the mean is added + * back afterwards, which is the standard treatment: penalising the + * intercept would shrink every prediction towards zero rather than + * towards the person's own average day, and towards zero on a 0-10 + * pleasantness scale means "towards the worst day imaginable". + * * **λ is never zero.** With more columns than the ridge penalty can + * afford, an unpenalised solve is a memoriser: it fits the training days + * perfectly and says nothing about any other day. The grid the validation + * searches starts above zero for that reason. + * + * This is arithmetic, not an AI surface. No provider, no consent gate, no + * egress — see the note at the top of the refresh job. + */ + +/** A fitted model: one coefficient per standardised column, plus the mean. */ +export interface RidgeFit { + /** One per column of the matrix it was fitted on, in the same order. */ + coefficients: number[]; + /** The mean of the training targets, added back on every prediction. */ + intercept: number; + /** + * The mean of each training column, subtracted before every prediction. + * + * Usually zero to floating point, because the matrix arrives z-scored. It is + * carried anyway rather than assumed: centring the target while leaving the + * columns uncentred biases every coefficient, and the bias is small enough + * to look like rounding. Somebody handing this function a raw matrix should + * get the right answer, not a plausible one. + */ + columnMeans: number[]; + /** The penalty this fit used. */ + lambda: number; +} + +/** + * Solve `A x = b` by Gaussian elimination with partial pivoting. + * + * `null` when the system is singular to working precision. The caller treats + * that as "no fit" rather than as a fit of zeros: a singular system means the + * columns are telling the same story twice, and a zero vector would silently + * become a model that predicts the average and calls it a pattern. + */ +export function solveLinearSystem( + matrix: readonly number[][], + vector: readonly number[], +): number[] | null { + const n = vector.length; + if (matrix.length !== n) return null; + // Copy: the caller's XᵀX is reused across the λ grid and elimination is + // destructive. + const a = matrix.map((row, i) => [...row, vector[i]]); + + for (let column = 0; column < n; column++) { + let pivot = column; + for (let row = column + 1; row < n; row++) { + if (Math.abs(a[row][column]) > Math.abs(a[pivot][column])) pivot = row; + } + if (Math.abs(a[pivot][column]) < 1e-10) return null; + if (pivot !== column) { + const swap = a[pivot]; + a[pivot] = a[column]; + a[column] = swap; + } + const pivotValue = a[column][column]; + for (let row = column + 1; row < n; row++) { + const factor = a[row][column] / pivotValue; + if (factor === 0) continue; + for (let k = column; k <= n; k++) a[row][k] -= factor * a[column][k]; + } + } + + const solution = new Array(n).fill(0); + for (let row = n - 1; row >= 0; row--) { + let sum = a[row][n]; + for (let k = row + 1; k < n; k++) sum -= a[row][k] * solution[k]; + solution[row] = sum / a[row][row]; + } + return solution.every((v) => Number.isFinite(v)) ? solution : null; +} + +/** + * Fit `rows` against `targets` at one penalty. + * + * `rows` are already standardised per user (see `features.ts`), so the columns + * are comparable and one λ is a defensible penalty across all of them — + * which is not true of raw minutes beside a 0-10 rating. + */ +export function fitRidge( + rows: readonly (readonly number[])[], + targets: readonly number[], + lambda: number, +): RidgeFit | null { + const n = rows.length; + if (n === 0 || targets.length !== n) return null; + const columns = rows[0].length; + if (columns === 0) return null; + if (rows.some((row) => row.length !== columns)) return null; + + const intercept = targets.reduce((sum, v) => sum + v, 0) / n; + const centredTargets = targets.map((v) => v - intercept); + const columnMeans = Array.from({ length: columns }, (_, j) => { + let sum = 0; + for (let i = 0; i < n; i++) sum += rows[i][j]; + return sum / n; + }); + const centredRows = rows.map((row) => row.map((v, j) => v - columnMeans[j])); + + const xtx: number[][] = Array.from({ length: columns }, () => + new Array(columns).fill(0), + ); + const xty = new Array(columns).fill(0); + for (let i = 0; i < n; i++) { + const row = centredRows[i]; + for (let j = 0; j < columns; j++) { + xty[j] += row[j] * centredTargets[i]; + for (let k = j; k < columns; k++) { + const product = row[j] * row[k]; + xtx[j][k] += product; + if (k !== j) xtx[k][j] += product; + } + } + } + for (let j = 0; j < columns; j++) xtx[j][j] += lambda; + + const coefficients = solveLinearSystem(xtx, xty); + if (coefficients === null) return null; + return { coefficients, intercept, columnMeans, lambda }; +} + +/** Score one standardised row. */ +export function predictWithFit( + fit: RidgeFit, + row: readonly number[], +): number | null { + if (row.length !== fit.coefficients.length) return null; + let sum = fit.intercept; + for (let i = 0; i < row.length; i++) { + sum += fit.coefficients[i] * (row[i] - fit.columnMeans[i]); + } + return Number.isFinite(sum) ? sum : null; +} + +/** + * The per-feature contributions of one scored day. + * + * `coefficient × the day's standardised value`, which is what the stored + * `features` column holds and what the explanation line is generated from. + * Signed, so a feature that pulled the expected value down reads as having + * pulled it down, and the parts sum to the distance between the prediction and + * the person's own average day. + */ +export function featureContributions( + fit: RidgeFit, + featureKeys: readonly string[], + row: readonly number[], +): Array<{ feature: string; contribution: number }> { + return featureKeys.map((feature, index) => ({ + feature, + contribution: + fit.coefficients[index] * (row[index] - fit.columnMeans[index]), + })); +} diff --git a/src/lib/analytics/mood-prognosis/thresholds.ts b/src/lib/analytics/mood-prognosis/thresholds.ts new file mode 100644 index 000000000..88aab2821 --- /dev/null +++ b/src/lib/analytics/mood-prognosis/thresholds.ts @@ -0,0 +1,181 @@ +/** + * How much history a prognosis needs before it is allowed to say anything. + * + * This is the most dangerous file in the feature, and it is dangerous for one + * reason: **the failure mode is silent.** A forecast built from eight days + * renders exactly like a forecast built from eighty. Nobody looking at the + * screen can tell the difference, and the person reading it is being told + * something about their own health record. So every floor is a named constant + * with a boundary test, and every one of those tests has been watched failing + * — a gate nobody has seen go red is a decoration, not a gate. + * + * The ladder has five rungs, and each rung says what the surface may claim: + * + * | entries | stage | what may be shown | + * |---------|---------------|------------------------------------------------| + * | 0-14 | `insufficient`| nothing beyond the self-assessment trend | + * | 15-29 | `learning` | descriptive hints, labelled as a learning phase | + * | 30-59 | `provisional` | a forecast, labelled provisional | + * | 60-89 | `regular` | a forecast without the provisional label | + * | 90+ | `seasonal` | plus weekday and seasonal comparisons | + * + * Absence is `{ present: false }` with a reason and the count, never `0` and + * never an empty forecast object with a null inside it. That shape is a + * contract elsewhere in the product already — the MCP surface means exactly + * "honestly not recorded" by it — and a surface that answered zero here would + * be claiming a bad day where the truth is that nothing can be said yet. + * + * **Two numbers are deliberately NOT minted here.** The paired-observation + * floor for a correlation is `MIN_PAIRED_N` in + * `src/lib/insights/correlations.ts`, and the screening in `features.ts` + * imports it from there; a second constant for the same idea is how two + * surfaces start disagreeing about what counts as enough. The occurrence floor + * below is the crosstab's own floor by assignment rather than by coincidence, + * for the same reason — Phase 2's context comparison says at its declaration + * that this file would import it, and this file does. + */ +import { CONTEXT_MIN_PRESENT_DAYS } from "@/lib/insights/mood-context-crosstab"; + +/** + * Below this, no output at all — not even a descriptive hint. + * + * Fifteen days of a person's life is not a pattern, and a surface that + * described one would be describing noise back to somebody who would + * reasonably believe it. + */ +export const MIN_ENTRIES_LEARNING_PHASE = 15; + +/** + * Below this, descriptive hints only, labelled as a learning phase. At or + * above it, a fitted forecast is allowed. + */ +export const MIN_ENTRIES_FOR_FORECAST = 30; + +/** Below this, a forecast is shown but labelled provisional. */ +export const MIN_ENTRIES_FOR_REGULAR_FORECAST = 60; + +/** + * Below this, a weekday or seasonal comparison built on the forecast may not + * be shown. + * + * Two things it does NOT do, both worth stating because the omission is a + * decision rather than an oversight: + * + * * It does not retro-gate the weekday board the mood insights page has + * carried for releases. That board is built from the entry aggregates, + * not from this fit, and it has its own floors; hiding it from accounts + * under ninety days would take a surface away from people who have been + * reading it, to enforce a rule about a different calculation. + * * It does not build a second weekday board. One exists, and a second + * answer to "how do your Mondays go" is exactly the drift this milestone + * spent its effort avoiding. + * + * What it does is gate the `seasonalUnlocked` flag the forecast publishes, so + * any surface that wants to hang a weekday claim off THIS comparison — the + * clients included — asks one place whether the history supports one. + */ +export const MIN_ENTRIES_FOR_SEASONAL = 90; + +/** + * How often a tag or a context value must have been recorded before it may + * appear in an explanation at all. + * + * The crosstab's floor, taken rather than re-declared: it is the same + * question ("has this been seen often enough to be worth a sentence") asked + * by a different surface, and two answers to it would drift. + */ +export const MIN_TAG_OCCURRENCES = CONTEXT_MIN_PRESENT_DAYS; + +// A per-weekday observation floor was drafted here for a weekday-comparison +// board built on the forecast. That board was not built — the forecast +// publishes `seasonalUnlocked` and leaves the weekday view to the mood +// insights page, which has its own floors — so the constant had no consumer +// and was removed rather than left as a number nothing reads. If a weekday +// comparison on THIS fit is ever built, reintroduce the floor with it. + +/** Where an account sits on the ladder. */ +export type PrognosisStage = + "insufficient" | "learning" | "provisional" | "regular" | "seasonal"; + +/** + * Why there is no forecast. The reason travels with the count and with the + * next rung, so a surface can say "eleven of fifteen" instead of an error. + */ +export type PrognosisAbsenceReason = "no-output-yet" | "learning-phase"; + +export interface PrognosisAbsent { + present: false; + reason: PrognosisAbsenceReason; + stage: Extract; + /** Qualifying entries the account has. */ + entries: number; + /** Entries at which the next thing becomes available. */ + nextThreshold: number; +} + +export interface PrognosisAvailable { + present: true; + stage: Extract; + entries: number; + /** The forecast carries a provisional label below the regular threshold. */ + provisional: boolean; +} + +export type PrognosisGate = PrognosisAbsent | PrognosisAvailable; + +/** + * Which rung this many qualifying entries reaches. + * + * Every comparison is `>=`, so the constant names the first value that + * qualifies rather than the last that does not. The boundary tests pin both + * sides of each one. + */ +export function prognosisStage(entries: number): PrognosisStage { + if (entries >= MIN_ENTRIES_FOR_SEASONAL) return "seasonal"; + if (entries >= MIN_ENTRIES_FOR_REGULAR_FORECAST) return "regular"; + if (entries >= MIN_ENTRIES_FOR_FORECAST) return "provisional"; + if (entries >= MIN_ENTRIES_LEARNING_PHASE) return "learning"; + return "insufficient"; +} + +/** + * Whether a forecast may be shown for an account with this many qualifying + * entries, and if not, why not and what would change it. + */ +export function forecastGate(entries: number): PrognosisGate { + const stage = prognosisStage(entries); + if (stage === "insufficient") { + return { + present: false, + reason: "no-output-yet", + stage, + entries, + nextThreshold: MIN_ENTRIES_LEARNING_PHASE, + }; + } + if (stage === "learning") { + return { + present: false, + reason: "learning-phase", + stage, + entries, + nextThreshold: MIN_ENTRIES_FOR_FORECAST, + }; + } + return { + present: true, + stage, + entries, + provisional: stage === "provisional", + }; +} + +/** Whether weekday and seasonal comparisons have unlocked for this account. */ +export function seasonalComparisonsUnlocked(entries: number): boolean { + return entries >= MIN_ENTRIES_FOR_SEASONAL; +} + +/** Whether a tag or context value has been recorded often enough to name. */ +export function occurrencesSuffice(occurrences: number): boolean { + return occurrences >= MIN_TAG_OCCURRENCES; +} diff --git a/src/lib/analytics/mood-prognosis/validate.ts b/src/lib/analytics/mood-prognosis/validate.ts new file mode 100644 index 000000000..fb86a9752 --- /dev/null +++ b/src/lib/analytics/mood-prognosis/validate.ts @@ -0,0 +1,230 @@ +/** + * Choosing the penalty, and measuring how wrong the model is, without letting + * it see the future. + * + * **Rolling origin, expanding window. Never a random k-fold.** Days are not + * exchangeable: a Tuesday sits between a Monday and a Wednesday that resemble + * it, and a shuffled split puts Wednesday in the training set and asks the + * model about Tuesday. The result is a model that looks excellent and is + * useless, and nothing about the number it reports says so. So the folds are + * ranges of a time-ordered array, the training range always ends where the + * validation range starts, and `rolling-origin.test.ts` asserts that on the + * fold indices and on the day keys themselves rather than on the resulting + * error — an error-based assertion would pass a leaking split whenever the + * leak happened to be small. + * + * **The band comes from the validation residuals**, not from a textbook + * standard error on the training fit. A training-fit interval on a model with + * twelve columns and thirty rows is narrow by construction and says nothing; + * the spread of the errors the model actually made on days it had not seen is + * the honest width. It widens when the model is bad, which is the property the + * surface depends on to report low confidence instead of a confident wrong + * number. + * + * **What is out-of-sample here, and what is not — stated exactly, because the + * boundary is easy to overclaim.** The RIDGE FIT is honest: within every fold + * the coefficients are solved on the training prefix alone and scored on + * validation rows the fit never saw, and no training fold holds a row dated + * after its validation fold. That is the claim `rolling-origin.test.ts` + * proves, and it is the one that matters for "does λ generalise". + * + * Two steps sit OUTSIDE that guarantee, on purpose: + * + * * **Feature selection.** The screening in `features.ts` runs once over the + * whole window before the split, so the feature set a fold trains on was + * chosen with that fold's validation rows in view. This is the classic + * "screen, then cross-validate" optimism. + * * **Standardisation.** The per-user mean and spread are taken over the + * whole window too, so a validation row is z-scored against statistics its + * own value helped compute. + * + * Both make the reported error slightly better than a fully-nested procedure + * would — measured here at roughly one part in sixty on the mean absolute + * error and a couple of points of coverage, i.e. small enough that the band is + * not dishonest but not zero. Nesting the screening and the standardisation + * inside each fold would remove even that, at the cost of a per-fold feature + * set and a materially larger amount of machinery; the trade was taken the + * cheap way deliberately, and this comment is the price of taking it — a + * reader must not read "no fold sees ahead" as covering the SELECTION, only + * the FIT. If the feature set ever grows past a handful of strong columns, or + * the optimism is ever measured larger, nest it. + */ +import { fitRidge, predictWithFit, type RidgeFit } from "./ridge"; +import type { FeatureMatrix } from "./features"; + +/** + * The stored model identity. Bump it whenever the feature set, the screening + * or the fit changes, so a re-fit is legible in the rows rather than silently + * changing what they mean. Rows carrying an older version stay readable; the + * surface skips a feature key the current build no longer knows. + */ +export const MODEL_VERSION = "mood-ridge-1"; + +/** + * The penalties the validation searches. + * + * Never zero: with a dozen standardised columns over a few dozen days, an + * unpenalised solve memorises the training days and describes no other day. + * The grid is coarse on purpose — the difference between λ=8 and λ=10 is far + * below the noise in thirty days of self-ratings, and a fine grid would only + * be a slower way to overfit the validation set. + */ +export const LAMBDA_GRID = [0.5, 1, 2, 5, 10, 25, 50] as const; + +/** How many validation blocks the window is cut into. */ +export const VALIDATION_FOLDS = 4; + +/** + * The band is the central 80% of the validation residuals: the 10th and the + * 90th percentile. A range wide enough to be honest and narrow enough to be + * worth reading — and empirical, so an asymmetric error stays asymmetric + * instead of being averaged into a symmetric claim. + */ +export const BAND_LOWER_QUANTILE = 0.1; +export const BAND_UPPER_QUANTILE = 0.9; + +/** Fewest rows a fold split is attempted on at all. */ +export const MIN_VALIDATION_ROWS = 10; + +/** One expanding-window split. Every index is into the time-ordered matrix. */ +export interface RollingOriginFold { + /** Training rows are `[0, trainEnd)`. */ + trainEnd: number; + /** Validation rows are `[validationStart, validationEnd)`. */ + validationStart: number; + validationEnd: number; +} + +/** + * Cut a time-ordered window into expanding-window folds. + * + * `trainEnd === validationStart` in every fold, which is the whole invariant: + * the training range is a prefix and the validation range starts exactly where + * it ends. No gap (a gap would waste days), no overlap (an overlap is the + * leak). + */ +export function rollingOriginFolds( + rowCount: number, + folds: number = VALIDATION_FOLDS, +): RollingOriginFold[] { + if (rowCount < MIN_VALIDATION_ROWS || folds < 1) return []; + const block = Math.floor(rowCount / (folds + 1)); + if (block < 1) return []; + + const out: RollingOriginFold[] = []; + for (let index = 1; index <= folds; index++) { + const validationStart = block * index; + const validationEnd = + index === folds ? rowCount : Math.min(rowCount, block * (index + 1)); + if (validationEnd <= validationStart) continue; + out.push({ trainEnd: validationStart, validationStart, validationEnd }); + } + return out; +} + +/** Linear-interpolated quantile of an unsorted sample. */ +export function quantile(values: readonly number[], q: number): number { + if (values.length === 0) return 0; + const sorted = [...values].sort((a, b) => a - b); + if (sorted.length === 1) return sorted[0]; + const position = (sorted.length - 1) * Math.min(1, Math.max(0, q)); + const lower = Math.floor(position); + const upper = Math.ceil(position); + if (lower === upper) return sorted[lower]; + return sorted[lower] + (sorted[upper] - sorted[lower]) * (position - lower); +} + +/** What one penalty scored across every fold. */ +export interface LambdaScore { + lambda: number; + /** Residuals (`actual − predicted`) from validation rows only. */ + residuals: number[]; + meanAbsoluteError: number; +} + +/** + * Score one penalty over the folds. Training folds never contain a row dated + * after their validation fold, by construction of {@link rollingOriginFolds}. + */ +export function scoreLambda( + matrix: FeatureMatrix, + lambda: number, + folds: readonly RollingOriginFold[], +): LambdaScore | null { + const residuals: number[] = []; + for (const fold of folds) { + const fit = fitRidge( + matrix.rows.slice(0, fold.trainEnd), + matrix.targets.slice(0, fold.trainEnd), + lambda, + ); + if (fit === null) continue; + for (let i = fold.validationStart; i < fold.validationEnd; i++) { + const predicted = predictWithFit(fit, matrix.rows[i]); + if (predicted === null) continue; + residuals.push(matrix.targets[i] - predicted); + } + } + if (residuals.length === 0) return null; + return { + lambda, + residuals, + meanAbsoluteError: + residuals.reduce((sum, r) => sum + Math.abs(r), 0) / residuals.length, + }; +} + +/** A model, ready to score a day, with the band its own errors earned it. */ +export interface PrognosisModel { + fit: RidgeFit; + features: string[]; + standardisation: FeatureMatrix["standardisation"]; + /** Rows the final fit used. This is the `n` every surface must show. */ + n: number; + /** Offsets from a prediction to the edges of the band, in target units. */ + bandLow: number; + bandHigh: number; + /** Mean absolute error over the validation folds, for the record. */ + meanAbsoluteError: number; + modelVersion: string; +} + +/** + * Validate, choose λ, refit on the whole window, and take the band from the + * validation residuals. + * + * `null` is a refusal and is a normal outcome: too few complete days to split, + * a singular system, or a fold set that produced no residual at all. The + * caller writes no forecast rather than a forecast nobody measured. + */ +export function fitPrognosisModel( + matrix: FeatureMatrix, + lambdaGrid: readonly number[] = LAMBDA_GRID, +): PrognosisModel | null { + const folds = rollingOriginFolds(matrix.rows.length); + if (folds.length === 0) return null; + + let best: LambdaScore | null = null; + for (const lambda of lambdaGrid) { + const score = scoreLambda(matrix, lambda, folds); + if (score === null) continue; + if (best === null || score.meanAbsoluteError < best.meanAbsoluteError) { + best = score; + } + } + if (best === null) return null; + + const fit = fitRidge(matrix.rows, matrix.targets, best.lambda); + if (fit === null) return null; + + return { + fit, + features: matrix.features, + standardisation: matrix.standardisation, + n: matrix.rows.length, + bandLow: quantile(best.residuals, BAND_LOWER_QUANTILE), + bandHigh: quantile(best.residuals, BAND_UPPER_QUANTILE), + meanAbsoluteError: best.meanAbsoluteError, + modelVersion: MODEL_VERSION, + }; +} diff --git a/src/lib/data-wipe/wipe-plan.ts b/src/lib/data-wipe/wipe-plan.ts index 98092efe2..c7f53c196 100644 --- a/src/lib/data-wipe/wipe-plan.ts +++ b/src/lib/data-wipe/wipe-plan.ts @@ -75,6 +75,12 @@ export const WIPE_MODELS = [ // but a parent deleted first leaves the child's own count reporting zero for // rows that existed, and the confirmation screen quotes those counts. "MoodContext", + // Derived, and wiped anyway. The forecast rows are recomputable from the + // entries above — which is why they stay out of the backup — but they are + // still a day-by-day statement about somebody's mood, and leaving them + // behind after a screen that promised to delete all health data would leave + // the shape of the record standing where the record used to be. + "MoodPrediction", "MoodEntry", "MoodEntryRollup", "MoodReminderDispatch", diff --git a/src/lib/export/backup-plan.ts b/src/lib/export/backup-plan.ts index ac3e0cdc5..031dc3852 100644 --- a/src/lib/export/backup-plan.ts +++ b/src/lib/export/backup-plan.ts @@ -367,6 +367,8 @@ export const DERIVED_MODELS: Readonly> = { "Recomputed from `MedicationIntakeEvent` against the schedule by the compliance analytics tier.", MoodEntryRollup: "A per-day reconstruction of `MoodEntry`, rebuilt by the same rollup tier that rebuilds the measurement one.", + MoodPrediction: + "What the account's own past days imply about a day, fitted from `MoodEntry` + `MoodContext` and the linked modules. The nightly `mood-prognosis-refresh` job rebuilds it after a restore, and rebuilding it is better than carrying it: a restored row would state a forecast made by a model version that may no longer exist, against data the restore has just moved.", CyclePrediction: "A forecast derived from `MenstrualCycle` history; regenerated on the next cycle read.", StrainTrimpCache: diff --git a/src/lib/insights/features.ts b/src/lib/insights/features.ts index 129fef2dc..b0598de15 100644 --- a/src/lib/insights/features.ts +++ b/src/lib/insights/features.ts @@ -17,6 +17,7 @@ import { userDayKey } from "@/lib/tz/format"; import { isCurrentForTodayClaim } from "@/lib/insights/measurement-freshness"; import { computeMoodDimensionSeries } from "@/lib/insights/mood-dimension-series"; import { computeContextMoodComparison } from "@/lib/insights/mood-context-crosstab"; +import { readMoodPrognosis } from "@/lib/analytics/mood-prognosis/read"; import { MOOD_DIMENSIONS } from "@/lib/mood/dimensions"; import { getServerTranslator } from "@/lib/i18n/server-translator"; import { @@ -321,6 +322,113 @@ async function buildMoodDimensionBlock(userId: string): Promise<{ }; } +/** + * The day's two readings for the model, with the age of the second one. + * + * Absence is a present field rather than a missing one: `{ present: false }` + * with the reason and the count, so the model says "not enough entries yet" + * instead of inventing a number. That shape is a contract elsewhere in the + * product already and it is the reason this block is built even when there is + * nothing to report. + * + * The wording rule travels with the values. A model handed a number and no + * instruction writes the sentence its training suggests, and the sentence its + * training suggests about a forecast is a causal one. + */ +interface MoodPrognosisFeature { + present: boolean; + /** Why there is nothing, when there is nothing. */ + reason?: string; + /** Days of the account's own history the ladder was read against. */ + entries: number; + /** The day the forecast is about. */ + date?: string; + /** The expected value, 0-10. Never to be stated without `n` and the band. */ + predicted?: number; + ciLow?: number | null; + ciHigh?: number | null; + /** Complete days the fit used. State it in any sentence that uses the value. */ + n?: number; + modelVersion?: string; + /** The person's own rating for that day, unchanged. */ + selfAssessment?: number | null; + /** Where that rating sat against the band: within, above or below. */ + deviation?: string | null; + /** Whole days between the forecast's day and today. */ + daysAgo?: number | null; + /** Whether it is fresh enough to back a present-tense sentence. */ + current?: boolean; + /** Present only when it is not, and says in words what the flag says. */ + staleNotice?: string; + /** The wording rule, stated rather than implied. */ + note: string; +} + +/** + * The forecast block, or an honest absence. + * + * One reader for every surface: this calls the same `readMoodPrognosis` the + * route calls, so the Coach and the page cannot disagree about what today's + * comparison is. The freshness rule is the shared one — `isCurrentForTodayClaim` + * against the same `TODAY_CLAIM_MAX_AGE_DAYS` — because a second constant for + * "how old may a value be before today stops being true about it" is how two + * surfaces start narrating the same reading differently. + */ +async function buildMoodPrognosisBlock(userId: string): Promise<{ + prognosis?: MoodPrognosisFeature; +}> { + const reading = await readMoodPrognosis( + userId, + new Date(), + await resolveUserTimezone(userId), + ); + + if (!reading.present) { + return { + prognosis: { + present: false, + reason: reading.reason, + entries: reading.entries, + note: + "There is no expected-value comparison for this account yet. Say so " + + "plainly if it comes up; do not estimate one, and do not describe " + + "the absence as a bad sign.", + }, + }; + } + + return { + prognosis: { + present: true, + entries: reading.entries, + date: reading.date, + predicted: reading.predicted, + ciLow: reading.ciLow, + ciHigh: reading.ciHigh, + n: reading.n, + modelVersion: reading.modelVersion, + selfAssessment: reading.selfAssessment, + deviation: reading.deviation, + daysAgo: reading.ageDays, + current: reading.current, + ...(reading.current + ? {} + : { + staleNotice: `This comparison is about ${reading.date}. State it as that day, with the date; do not describe it as today.`, + }), + note: + "Two separate readings of one day: `selfAssessment` is what the person " + + "recorded and leads; `predicted` is what a comparison with their own " + + "past days would have expected, built from `n` days. Never merge them, " + + "never present `predicted` as the person's mood, and never state it " + + "without `n` and the band. Phrase it as a counterfactual — a value " + + "around X would have been expected — and never say that anything " + + "caused anything: the comparison is an association over the account's " + + "own days and supports no cause.", + }, + }; +} + export interface AggregatedFeatures { weight?: { latest: number; @@ -467,6 +575,16 @@ export interface AggregatedFeatures { withoutAvg: number; }>; }; + /** + * v1.38 — the day's two readings and the distance between them. + * + * Always present once the mood module is on, because an honest absence is + * information: `present: false` with a reason and a count keeps the model + * from filling the gap with an estimate. The block carries its own age, so + * a comparison about the day before yesterday cannot back a sentence about + * today. + */ + prognosis?: MoodPrognosisFeature; }; sleep?: { avg7: number | null; @@ -1363,6 +1481,7 @@ export async function extractFeatures( totalEntries: moodTotalEntries, ...(await buildMoodDimensionBlock(userId)), ...(await buildMoodContextBlock(userId)), + ...(await buildMoodPrognosisBlock(userId)), coverage: { count: moodTotalEntries, spanDays, diff --git a/src/lib/jobs/__tests__/mood-prognosis-refresh.test.ts b/src/lib/jobs/__tests__/mood-prognosis-refresh.test.ts new file mode 100644 index 000000000..b3c77861d --- /dev/null +++ b/src/lib/jobs/__tests__/mood-prognosis-refresh.test.ts @@ -0,0 +1,152 @@ +/** + * The nightly mood-forecast rebuild. + * + * Three properties, each of which fails silently if it breaks: + * + * 1. **The sweep visits accounts that hold rows, not only accounts that + * qualify.** An account that deleted its entries drops out of the + * eligibility half of the cohort, and if that were the whole cohort it + * would keep last month's forecast forever, under whatever date the last + * pass stamped on it. Nothing on the surface would say so. + * 2. **A refusal clears.** Below the threshold, module off, no pattern — all + * three leave the account with no rows rather than with stale ones. + * 3. **One account's bad data costs one account.** The pass counts the + * failure and carries on; only an all-accounts failure fails the job. + * + * The cohort query and the module gate are mocked, which is what makes the + * decision path testable without a container. The arithmetic underneath has + * its own tests, and the whole thing is exercised end to end against real + * Postgres in `tests/integration/mood-prognosis-ladder.test.ts`. + */ +import { beforeEach, describe, expect, it, vi } from "vitest"; + +vi.mock("@/lib/jobs/reminder/shared", () => ({ + getWorkerPrisma: () => ({ + moodEntry: { groupBy: groupByMock }, + moodPrediction: { + findMany: predictionFindManyMock, + deleteMany: deleteManyMock, + }, + }), +})); + +vi.mock("@/lib/modules/gate", () => ({ + isModuleEnabled: vi.fn(async () => true), +})); + +vi.mock("@/lib/logging/background", () => ({ + withBackgroundEvent: async ( + _name: string, + fn: (evt: { addMeta: () => void }) => Promise, + ) => fn({ addMeta: () => {} }), +})); + +const groupByMock = vi.fn(); +const predictionFindManyMock = vi.fn(); +const deleteManyMock = vi.fn(); + +import { + MOOD_PROGNOSIS_REFRESH_CRON, + MOOD_PROGNOSIS_REFRESH_QUEUE, + handleMoodPrognosisRefresh, + runMoodPrognosisRefresh, +} from "@/lib/jobs/mood-prognosis-refresh"; +import { MIN_ENTRIES_FOR_FORECAST } from "@/lib/analytics/mood-prognosis/thresholds"; + +const NOW = new Date("2026-08-08T02:35:00.000Z"); + +beforeEach(() => { + vi.clearAllMocks(); + groupByMock.mockResolvedValue([ + { userId: "user-1", _count: { _all: MIN_ENTRIES_FOR_FORECAST } }, + { userId: "user-2", _count: { _all: 120 } }, + ]); + predictionFindManyMock.mockResolvedValue([]); + deleteManyMock.mockResolvedValue({ count: 0 }); +}); + +describe("mood-prognosis-refresh", () => { + it("runs at 04:35, in the slot the neighbouring jobs leave free", () => { + expect(MOOD_PROGNOSIS_REFRESH_CRON).toBe("35 4 * * *"); + expect(MOOD_PROGNOSIS_REFRESH_QUEUE).toBe("mood-prognosis-refresh"); + }); + + it("refreshes every eligible account", async () => { + const refresh = vi.fn(async () => ({ rows: 7, refusal: null })); + const summary = await runMoodPrognosisRefresh(NOW, { refresh }); + + expect(summary.scanned).toBe(2); + // The count is the point: a cohort query that matched nothing would + // satisfy "no bad rows" exactly as well as a working sweep. + expect(refresh).toHaveBeenCalledTimes(2); + expect(refresh).toHaveBeenCalledWith("user-1", NOW); + expect(summary.refreshed).toBe(2); + expect(summary.rowsWritten).toBe(14); + }); + + it("leaves an account below the entry cut out of the cohort", async () => { + groupByMock.mockResolvedValue([ + { userId: "user-1", _count: { _all: MIN_ENTRIES_FOR_FORECAST - 1 } }, + ]); + const refresh = vi.fn(async () => ({ rows: 1, refusal: null })); + const summary = await runMoodPrognosisRefresh(NOW, { refresh }); + + expect(summary.scanned).toBe(0); + expect(refresh).not.toHaveBeenCalled(); + }); + + it("still visits an account that holds rows but no longer qualifies", async () => { + groupByMock.mockResolvedValue([]); + predictionFindManyMock.mockResolvedValue([{ userId: "user-stale" }]); + const refresh = vi.fn(async () => ({ + rows: 0, + refusal: "too-few-days" as const, + })); + + const summary = await runMoodPrognosisRefresh(NOW, { refresh }); + + // Without this arm the account would keep its forecast forever: it is not + // in the eligible set any more, so nothing would ever come back for it. + expect(summary.scanned).toBe(1); + expect(refresh).toHaveBeenCalledWith("user-stale", NOW); + expect(summary.refused).toBe(1); + expect(summary.refreshed).toBe(0); + }); + + it("computes nothing and stores nothing while the module is off, and clears what it stored before", async () => { + const refresh = vi.fn(async () => ({ rows: 7, refusal: null })); + const summary = await runMoodPrognosisRefresh(NOW, { + isAvailable: async () => false, + refresh, + }); + + expect(refresh).not.toHaveBeenCalled(); + expect(summary.moduleOff).toBe(2); + expect(summary.rowsWritten).toBe(0); + expect(deleteManyMock).toHaveBeenCalledWith({ + where: { userId: "user-1" }, + }); + expect(deleteManyMock).toHaveBeenCalledTimes(2); + }); + + it("counts an account whose pass throws and carries on", async () => { + const refresh = vi + .fn() + .mockRejectedValueOnce(new Error("bad rows")) + .mockResolvedValueOnce({ rows: 3, refusal: null }); + + const summary = await runMoodPrognosisRefresh(NOW, { refresh }); + + expect(summary.failed).toBe(1); + expect(summary.refreshed).toBe(1); + expect(summary.rowsWritten).toBe(3); + }); + + it("fails the job only when every account failed", async () => { + const outcome = await handleMoodPrognosisRefresh([]); + // The default refresh path reaches the real loader, which the mocked + // client cannot serve, so every account throws — which is exactly the + // all-accounts case. + expect(outcome.ok).toBe(false); + }); +}); diff --git a/src/lib/jobs/job-outcome.ts b/src/lib/jobs/job-outcome.ts index 2c84daf4b..2c255d692 100644 --- a/src/lib/jobs/job-outcome.ts +++ b/src/lib/jobs/job-outcome.ts @@ -121,6 +121,7 @@ export const JOB_FACT_ALLOWLIST: ReadonlySet = new Set([ "medications_evaluated", "metric_assessments_warmed", "missed_doses_minted", + "module_off", "mood_entries_migrated", "mood_pruned", "notifications_dispatched", @@ -147,8 +148,10 @@ export const JOB_FACT_ALLOWLIST: ReadonlySet = new Set([ "rearmed", "records_read", "refreshed", + "refused", "reminders_due", "removed", + "rows_written", "restore_drill_age_days", "restore_drill_ciphertext_bytes", "restore_drill_intake_events", diff --git a/src/lib/jobs/mood-prognosis-refresh.ts b/src/lib/jobs/mood-prognosis-refresh.ts new file mode 100644 index 000000000..30ab95fba --- /dev/null +++ b/src/lib/jobs/mood-prognosis-refresh.ts @@ -0,0 +1,241 @@ +/** + * Nightly rebuild of the mood forecast rows. + * + * **This is not an AI surface.** A ridge regression over a dozen standardised + * columns is arithmetic: no provider, no key, no egress, no consent gate. It + * is stated here because the shape of the feature invites the assumption in + * both directions — nobody should wire a consent check that does not apply, + * and nobody should assume one exists where none is needed. If a later change + * narrates this forecast through a language model, THAT call site inherits + * `ai-consent-enforcement-guard.test.ts` and has to declare its surface. + * + * **This file is the only writer of `MoodPrediction`**, held there by + * `src/__tests__/mood-prediction-write-surface-guard.test.ts`. The reason is + * the product rule: a day carries the person's own rating and the value their + * past days imply, the two are never merged, and the second one is not + * editable. A route that wrote this table would make it editable. The cycle + * module learned the other half of the same lesson the expensive way — a + * forecast written as a side effect of opening a page is a function of + * browsing rather than of data. + * + * **A pass replaces; it never accumulates.** Every account it touches ends the + * pass holding exactly the rows this run wrote, or none. An account that has + * fallen below the threshold, switched the module off, or stopped showing a + * pattern is left with nothing rather than with last week's answer under a + * fresh-looking date. That is the stale rule, chosen over marking rows + * superseded because a derived row is cheap to rebuild and a superseded row is + * one more state every reader has to handle correctly. + * + * Runs at 04:35 Europe/Berlin. The 03:xx block is retention; 04:10 is the + * document purge, 04:20 the cycle forecast, 04:25 the achievement sweep, 04:30 + * the insight pre-generation, and 04:45/50/55 the recovery, stress and strain + * scores. 04:35 is free, and it sits after the inputs this job reads have + * settled. (04:40 is not free: the geo backfill runs at `40 * * * *`.) + */ +import type { Job } from "pg-boss"; + +import { + computeForecast, + type ForecastRefusal, +} from "@/lib/analytics/mood-prognosis/forecast"; +import { + PROGNOSIS_FIT_WINDOW_DAYS, + dayKeyBefore, + loadPrognosisDays, +} from "@/lib/analytics/mood-prognosis/load"; +import { MIN_ENTRIES_FOR_FORECAST } from "@/lib/analytics/mood-prognosis/thresholds"; +import { jobDone, jobFailed, type JobOutcome } from "@/lib/jobs/job-outcome"; +import { withBackgroundEvent } from "@/lib/logging/background"; +import { isModuleEnabled } from "@/lib/modules/gate"; + +import { getWorkerPrisma } from "./reminder/shared"; + +export const MOOD_PROGNOSIS_REFRESH_QUEUE = "mood-prognosis-refresh"; + +export const MOOD_PROGNOSIS_REFRESH_CRON = "35 4 * * *"; + +export interface MoodPrognosisRefreshPayload { + triggeredAt: string; +} + +export interface MoodPrognosisRefreshSummary { + /** Accounts the cohort query returned, plus any holding stale rows. */ + scanned: number; + /** Accounts whose forecast rows now reflect their data. */ + refreshed: number; + /** Rows written across every account. */ + rowsWritten: number; + /** Accounts the fit refused, whose rows were cleared. */ + refused: number; + /** Accounts whose module is off, whose rows were cleared. */ + moduleOff: number; + /** Accounts whose pass threw. */ + failed: number; +} + +/** + * Rebuild one account's rows, or clear them. + * + * The write and the delete are the two calls the guard allows in this file, + * and they are here rather than beside the arithmetic so the single-writer + * claim is a property of one module rather than a convention across two. + */ +async function refreshAccount( + userId: string, + now: Date, +): Promise<{ rows: number; refusal: ForecastRefusal | null }> { + const prisma = getWorkerPrisma(); + const user = await prisma.user.findUnique({ + where: { id: userId }, + select: { timezone: true }, + }); + const days = await loadPrognosisDays(userId, now, user?.timezone ?? null); + const result = computeForecast(days); + + if (result.status === "refused") { + await prisma.moodPrediction.deleteMany({ where: { userId } }); + return { rows: 0, refusal: result.reason }; + } + + const written: string[] = []; + for (const row of result.rows) { + const data = { + predicted: row.predicted, + ciLow: row.ciLow, + ciHigh: row.ciHigh, + n: row.n, + modelVersion: row.modelVersion, + features: JSON.stringify(row.contributions), + computedAt: now, + }; + await prisma.moodPrediction.upsert({ + where: { userId_date: { userId, date: row.date } }, + create: { userId, date: row.date, ...data }, + update: data, + }); + written.push(row.date); + } + // Everything outside this pass goes, so no account holds a row from an older + // fit beside a row from this one. + await prisma.moodPrediction.deleteMany({ + where: { userId, date: { notIn: written } }, + }); + return { rows: written.length, refusal: null }; +} + +/** + * Refresh every eligible account. Exported for the test; the handler below is + * the queue binding. + * + * The cohort is the union of two sets, and the second one is the whole point: + * accounts with enough recent rated days to be worth fitting, AND accounts + * that already hold forecast rows. Without the second, an account that deleted + * its entries would drop out of the cohort and keep its old forecast forever — + * the sweep would never visit it again to clear it. + */ +export async function runMoodPrognosisRefresh( + now: Date = new Date(), + options: { + isAvailable?: (userId: string) => Promise; + refresh?: typeof refreshAccount; + } = {}, +): Promise { + const isAvailable = + options.isAvailable ?? + ((userId: string) => isModuleEnabled(userId, "mood")); + const refresh = options.refresh ?? refreshAccount; + const prisma = getWorkerPrisma(); + + const since = dayKeyBefore(now, PROGNOSIS_FIT_WINDOW_DAYS); + const [eligible, holders] = await Promise.all([ + prisma.moodEntry.groupBy({ + by: ["userId"], + where: { + deletedAt: null, + moodA1: { not: null }, + date: { gte: since }, + }, + _count: { _all: true }, + }), + prisma.moodPrediction.findMany({ + distinct: ["userId"], + select: { userId: true }, + }), + ]); + + const userIds = new Set(); + for (const group of eligible) { + // The cheap cut, so an account with a handful of entries never pays for a + // window read. The real gate is the ladder inside the fit, which counts + // DAYS rather than rows. + if (group._count._all >= MIN_ENTRIES_FOR_FORECAST) + userIds.add(group.userId); + } + for (const holder of holders) userIds.add(holder.userId); + + const summary: MoodPrognosisRefreshSummary = { + scanned: userIds.size, + refreshed: 0, + rowsWritten: 0, + refused: 0, + moduleOff: 0, + failed: 0, + }; + + for (const userId of userIds) { + try { + if (!(await isAvailable(userId))) { + // An operator- or user-disabled module computes nothing and stores + // nothing — and gives back anything it stored before. + await prisma.moodPrediction.deleteMany({ where: { userId } }); + summary.moduleOff += 1; + continue; + } + const outcome = await refresh(userId, now); + if (outcome.refusal !== null) { + summary.refused += 1; + continue; + } + summary.refreshed += 1; + summary.rowsWritten += outcome.rows; + } catch { + // One account's bad data must not cost every other account its forecast. + // The count is what makes a systemic failure visible. + summary.failed += 1; + } + } + + return summary; +} + +export async function handleMoodPrognosisRefresh( + jobs: Job[], +): Promise { + void jobs; + return withBackgroundEvent("job.mood-prognosis-refresh", async (evt) => { + const summary = await runMoodPrognosisRefresh(); + evt.addMeta("mood_prognosis_scanned", summary.scanned); + evt.addMeta("mood_prognosis_refreshed", summary.refreshed); + evt.addMeta("mood_prognosis_rows", summary.rowsWritten); + evt.addMeta("mood_prognosis_refused", summary.refused); + evt.addMeta("mood_prognosis_module_off", summary.moduleOff); + evt.addMeta("mood_prognosis_failed", summary.failed); + + // Every account failing is a systemic fault (a dead pool, a schema drift), + // not one account's bad data. Anything short of that stays a success with + // the count on the event. + if (summary.failed > 0 && summary.failed === summary.scanned) { + return jobFailed( + `mood-prognosis-refresh failed for all ${summary.failed} accounts`, + ); + } + return jobDone({ + users_scanned: summary.scanned, + refreshed: summary.refreshed, + rows_written: summary.rowsWritten, + refused: summary.refused, + module_off: summary.moduleOff, + users_failed: summary.failed, + }); + }); +} diff --git a/src/lib/jobs/reminder/register-maintenance.ts b/src/lib/jobs/reminder/register-maintenance.ts index 332dd8c1d..6eee62d3b 100644 --- a/src/lib/jobs/reminder/register-maintenance.ts +++ b/src/lib/jobs/reminder/register-maintenance.ts @@ -195,6 +195,12 @@ import { handleCyclePredictionRefresh, type CyclePredictionRefreshPayload, } from "@/lib/jobs/cycle-prediction-refresh"; +import { + MOOD_PROGNOSIS_REFRESH_QUEUE, + MOOD_PROGNOSIS_REFRESH_CRON, + handleMoodPrognosisRefresh, + type MoodPrognosisRefreshPayload, +} from "@/lib/jobs/mood-prognosis-refresh"; import { ACHIEVEMENT_UNLOCK_SWEEP_QUEUE, ACHIEVEMENT_UNLOCK_SWEEP_CRON, @@ -365,6 +371,10 @@ const allQueues = [ // scans. Without this entry the schedule silently no-ops and the forecast // is only as fresh as the last time somebody opened the calendar. CYCLE_PREDICTION_REFRESH_QUEUE, + // v1.38 — nightly rebuild of the mood forecast rows. Without this entry the + // schedule silently no-ops: nothing else writes them, so the surface would + // answer "not enough data yet" forever on an account that has plenty. + MOOD_PROGNOSIS_REFRESH_QUEUE, // v1.35.3 — nightly pin for newly unlocked achievements. Without this entry // the schedule silently no-ops and no unlock date is ever made durable. ACHIEVEMENT_UNLOCK_SWEEP_QUEUE, @@ -521,6 +531,10 @@ const schedules: ScheduleEntry[] = [ CYCLE_PREDICTION_REFRESH_CRON, cronIsTheRetry, ], + // v1.38 — daily 04:35 Europe/Berlin rebuild of the mood forecast rows. + // Tomorrow's tick is the retry: the fit moves on a day scale and a night + // late writes the same rows from the same days. + [MOOD_PROGNOSIS_REFRESH_QUEUE, MOOD_PROGNOSIS_REFRESH_CRON, cronIsTheRetry], // v1.35.3 — daily 04:25 Europe/Berlin pin for newly unlocked achievements. // Tomorrow's tick is the retry: the date written is the computed completion // date, so a night late writes the identical row. @@ -743,6 +757,14 @@ export async function registerMaintenanceQueues( { localConcurrency: 1 }, handleCyclePredictionRefresh, ); + // v1.38 — nightly mood-forecast rebuild. Single-flight: two ticks would fit + // the same windows twice and the second pass would write the identical rows. + await createAndWork( + boss, + MOOD_PROGNOSIS_REFRESH_QUEUE, + { localConcurrency: 1 }, + handleMoodPrognosisRefresh, + ); // v1.35.3 — nightly achievement-unlock pin. Single-flight: the underlying // createMany is idempotent on the (userId, achievementId) unique, so a // second concurrent tick would only re-walk the same accounts. diff --git a/src/lib/jobs/subsystem-surface.ts b/src/lib/jobs/subsystem-surface.ts index 04550e269..89dd1f05f 100644 --- a/src/lib/jobs/subsystem-surface.ts +++ b/src/lib/jobs/subsystem-surface.ts @@ -117,6 +117,7 @@ const subsystemSurface = { "push-attempt-cleanup": { audience: "system" }, "arrival-reaction-cleanup": { audience: "system" }, "cycle-prediction-refresh": { audience: "system" }, + "mood-prognosis-refresh": { audience: "system" }, "achievement-unlock-sweep": { audience: "system" }, "measurement-tombstone-cleanup": { audience: "system" }, "coach-message-cleanup": { audience: "system" }, diff --git a/src/lib/mood/linked-context.ts b/src/lib/mood/linked-context.ts index f7c858c41..318dcc6cd 100644 --- a/src/lib/mood/linked-context.ts +++ b/src/lib/mood/linked-context.ts @@ -99,6 +99,20 @@ export interface LinkedBody { episodeId: string | null; } +/** + * The five linked figures of one day as plain numbers, for callers that need + * a window of days rather than one day's blocks — the prognosis fit is the + * only one so far. `null` is absence, and a module switched off makes its own + * figures absent rather than zero. + */ +export interface LinkedDayFigures { + sleepAsleep: number | null; + steps: number | null; + activeEnergy: number | null; + restingHeartRate: number | null; + heartRateVariability: number | null; +} + export interface LinkedDayContext { /** The entry's local day, resolved through the entry's own `tz`. */ day: string; @@ -396,6 +410,17 @@ export async function resolveLinkedDayContext( } : MODULE_OFF; + const bodyBlock = await resolveBody(userId, day, enabled); + + return { day, sleep, activity, vitals, body: bodyBlock }; +} + +/** The illness day-log block, split out so the day loop above reads straight. */ +async function resolveBody( + userId: string, + day: string, + enabled: ReadonlyMap, +): Promise> { let body: LinkedBlock = MODULE_OFF; if (enabled.get("illness") ?? true) { // Read only. The illness module owns symptom capture and its severity @@ -419,6 +444,152 @@ export async function resolveLinkedDayContext( episodeId: dayLog?.episodeId ?? null, }; } + return body; +} - return { day, sleep, activity, vitals, body }; +/** + * The five linked figures of one day, as plain numbers, for a whole window. + * + * The same figures the block above answers with, resolved for many days in + * one query instead of one query per day. It exists because the prognosis fits + * over a year of days and the per-day resolver would have run several hundred + * round trips to build one matrix. + * + * It is the SAME engine, deliberately: the module gates come from + * `moduleForMeasurementType` through `typeAvailable`, the cross-source de-dup + * runs through `canonicalRowsOfDay`, the night reconstruction is the one in + * `sleep-night.ts`, and the day totals are the same `sumOfDay` / `latestOfDay` + * the single-day path uses. A second implementation of "what did this day look + * like" would be a second answer, and the mood sheet and the model would start + * disagreeing about the same Tuesday. + * + * A switched-off module answers `null` for its metrics, not zero and not a + * value the owner declined to surface: the model must not learn from a module + * whose figures the account asked not to see. + */ +export async function resolveLinkedDayFigures( + userId: string, + days: readonly string[], + tz: string, +): Promise> { + const out = new Map(); + if (days.length === 0) return out; + + const sorted = [...days].sort(); + const { from } = localDayWindow(sorted[0]); + const { to } = localDayWindow(sorted[sorted.length - 1]); + + const gateStates = await Promise.all( + LINKED_MODULE_KEYS.map( + async (key) => [key, await isModuleEnabled(userId, key)] as const, + ), + ); + const enabled = new Map(gateStates); + const priority = await loadUserSourcePriority(userId); + + const measurements: LinkedRow[] = await prisma.measurement.findMany({ + where: { + userId, + deletedAt: null, + type: { in: [...LINKED_MEASUREMENT_TYPES] }, + measuredAt: { gte: from, lte: to }, + }, + select: { + type: true, + value: true, + measuredAt: true, + sleepStage: true, + source: true, + deviceType: true, + }, + // The canonical-source picker keeps insertion order inside a bucket, so a + // stable read order is what makes its device-type tie-break reproducible + // rather than whatever the planner returned this time. + orderBy: [{ measuredAt: "asc" }, { id: "asc" }], + }); + + // One pass to bucket by local day, rather than filtering the whole window + // once per day: over a year of history that difference is the whole cost of + // the fit's data preparation. + const byDay = new Map(); + for (const row of measurements) { + const key = moodDateKey(row.measuredAt, tz); + const bucket = byDay.get(key); + if (bucket) bucket.push(row); + else byDay.set(key, [row]); + } + + const sleepAvailable = typeAvailable("SLEEP_DURATION", enabled); + const activityAvailable = typeAvailable("ACTIVITY_STEPS", enabled); + const vitalsAvailable = typeAvailable("RESTING_HEART_RATE", enabled); + + const nightByDay = new Map(); + const inBedByDay = new Map(); + if (sleepAvailable) { + const stageRows = measurements.filter((r) => r.type === "SLEEP_DURATION"); + if (stageRows.length > 0) { + for (const night of reconstructSleepNights( + stageRows as unknown as SleepStageRow[], + tz, + priority, + )) { + nightByDay.set(night.night, night.asleepMinutes ?? null); + inBedByDay.set(night.night, night.inBedMinutes ?? null); + } + } + } + + for (const day of days) { + const dayRows = byDay.get(day) ?? []; + out.set(day, { + sleepAsleep: sleepAvailable ? (nightByDay.get(day) ?? null) : null, + steps: activityAvailable + ? valueOf( + sumOfDay(dayRows, "ACTIVITY_STEPS", "steps", day, tz, priority), + ) + : null, + activeEnergy: activityAvailable + ? valueOf( + sumOfDay( + dayRows, + "ACTIVE_ENERGY_BURNED", + "kcal", + day, + tz, + priority, + ), + ) + : null, + restingHeartRate: vitalsAvailable + ? valueOf( + latestOfDay( + dayRows, + "RESTING_HEART_RATE", + "bpm", + day, + tz, + priority, + ), + ) + : null, + heartRateVariability: vitalsAvailable + ? valueOf( + latestOfDay( + dayRows, + "HEART_RATE_VARIABILITY", + "ms", + day, + tz, + priority, + ), + ) + : null, + }); + } + return out; +} + +/** A figure as a plain number, with absence staying absence. */ +function valueOf(figure: LinkedFigure): number | null { + return figure.present ? figure.value : null; } diff --git a/src/lib/openapi/routes/mood.ts b/src/lib/openapi/routes/mood.ts index 1ab95c051..7589199e4 100644 --- a/src/lib/openapi/routes/mood.ts +++ b/src/lib/openapi/routes/mood.ts @@ -35,7 +35,9 @@ import { import { dataEnvelope, errorEnvelope, + recordRefusal, stdResponses, + MODULE_DISABLED_DESCRIPTION, baseUpdatedAtField, updatedAtTokenField, conflictResponse409, @@ -386,7 +388,137 @@ const bulkMoodResponse = z }) .meta({ id: "BulkMoodResponse" }); +// ── The day's two readings ─────────────────────────────────────────── +// +// Published as the RESOLVED value. The server computes the expected value, the +// band, the count, the deviation and the age; the client renders them. Nothing +// here is a hint the client is meant to re-derive — two clients deriving the +// same figure is how they start disagreeing about it, and this figure is about +// somebody's health record. + +const moodPrognosisContribution = z + .object({ + feature: z + .string() + .describe( + "Stable feature identifier: `dimension:a2`, `context:workLoad`, `context:workStatus=off` or `linked:steps`. A client that does not recognise one skips it — an older row can name a feature a later model dropped.", + ), + contribution: z + .number() + .describe( + "Signed: how far this feature moved the expected value away from the account's own average day. Ordered by absolute size, largest first.", + ), + }) + .meta({ id: "MoodPrognosisContribution" }); + +const moodPrognosisPresent = z.object({ + present: z.literal(true), + date: z.string().describe("The local day this is about, `YYYY-MM-DD`."), + predicted: z + .number() + .describe( + "The expected pleasantness (A1) for that day, 0-10, on the account's own past pattern. A counterfactual, never a measurement — and never to be displayed without `n` and the band.", + ), + ciLow: z.number().nullable(), + ciHigh: z + .number() + .nullable() + .describe( + "The band, from the validation residuals. It widens when the model fits badly, so a wide band is the signal to present the value cautiously.", + ), + n: z + .number() + .int() + .describe( + "Complete days the fit used. Display it WHEREVER `predicted` is displayed: a value built from 30 days and one built from 300 look identical otherwise.", + ), + modelVersion: z.string(), + computedAt: z.iso.datetime({ offset: true }), + ageDays: z + .number() + .int() + .nullable() + .describe("Whole days between that day and the reader's today."), + current: z + .boolean() + .describe( + "Whether it is fresh enough to back a present-tense sentence. False means the copy must name the day it is about.", + ), + provisional: z + .boolean() + .describe("True below 60 days of history: label it as provisional."), + stage: z.enum(["provisional", "regular", "seasonal"]), + entries: z.number().int().describe("Days of history behind the ladder."), + seasonalUnlocked: z.boolean(), + selfAssessment: z + .number() + .int() + .nullable() + .describe( + "The person's own rating for that day, stored exactly as they gave it. It LEADS: it is the primary value on any surface that shows both, and it is never merged with `predicted`.", + ), + deviation: z + .enum(["within", "above", "below"]) + .nullable() + .describe( + "Where the self-assessment sat against the BAND, not against `predicted`. `within` means the day went as the account's own past days suggest.", + ), + deviationAmount: z + .number() + .nullable() + .describe("How far outside the band, in scale points. `0` when inside."), + contributions: z.array(moodPrognosisContribution), +}); + +const moodPrognosisAbsent = z.object({ + present: z.literal(false), + reason: z + .enum(["no-output-yet", "learning-phase", "no-pattern"]) + .describe( + "`no-output-yet`: too few days for anything. `learning-phase`: enough to describe, not to compare. `no-pattern`: enough days, nothing steady enough in them. None of the three is an error and none is a zero.", + ), + entries: z.number().int(), + nextThreshold: z + .number() + .int() + .nullable() + .describe("Days at which the next step unlocks, when a count is missing."), +}); + +const moodPrognosisResponse = z + .union([moodPrognosisPresent, moodPrognosisAbsent]) + .meta({ + id: "MoodPrognosis", + description: + "Two separate readings of one day: what the person recorded, and what a comparison with their own past days would have expected. They are never merged — the distance between them is the finding, and a single blended figure destroys it. The comparison is a regularised regression over the account's own history: observational only, never causal. Any copy built on it must state the value as a counterfactual (a value around X would have been expected), must carry `n`, and must not say that anything caused anything.", + }); + export const moodPaths: NonNullable = { + "/api/mood/prognosis": { + get: { + tags: ["Mood"], + summary: "The day's own rating and what the account's days imply (v1.38)", + description: + "The newest stored comparison for the account, with the person's own rating for that day beside it. Written nightly by a job and never by a request: the value cannot be overwritten, which is what keeps the two readings separate. Absence is explicit (`present: false` with a reason and a count) and is a normal answer, not an error — the comparison refuses when the history does not support one. Whole-record read: the fit takes the day's context AND the sleep, activity and recovery figures the owning modules hold, and it names them, so a section-scoped grant is refused rather than served a filtered list. Gated: `module.disabled` 403 when the mood module is off.", + responses: { + "200": { + description: "The day's two readings, or an explained absence.", + content: { + "application/json": { + schema: dataEnvelope( + moodPrognosisResponse, + "MoodPrognosisEnvelope", + ), + }, + }, + }, + // Two reasons, one status: the module gate and the sharing fence both + // answer 403, and OpenAPI allows one response per status. + ...recordRefusal(MODULE_DISABLED_DESCRIPTION), + ...stdResponses, + }, + }, + }, "/api/mood-entries/bulk": { post: { tags: ["Mood"], diff --git a/src/lib/query-keys/mood.ts b/src/lib/query-keys/mood.ts index bde824988..99ac6989e 100644 --- a/src/lib/query-keys/mood.ts +++ b/src/lib/query-keys/mood.ts @@ -58,6 +58,18 @@ export const moodKeys = { moodLinkedContext: (date: string, tz: string) => ["mood-linked-context", date, tz] as const, + /** + * v1.38 — the day's two readings: the person's own rating and what their + * past days imply, with the band and the case count. + * + * Under the `["mood-entries"]` prefix so `moodDependentKeys` reaches it: the + * self-assessment half of the payload comes straight off a mood row, and an + * edit to today's entry changes the deviation the card shows. The forecast + * half does not move until the nightly job runs again, which is why the read + * is cheap enough to refetch on a mood write. + */ + moodPrognosis: () => ["mood-entries", "prognosis"] as const, + moodAnalytics: () => ["mood-analytics"] as const, /** * v1.8.5 — pre-computed mood-insights aggregates (heatmap, distribution, diff --git a/tests/integration/mood-dimension-features.test.ts b/tests/integration/mood-dimension-features.test.ts index b722528ee..2dac517fe 100644 --- a/tests/integration/mood-dimension-features.test.ts +++ b/tests/integration/mood-dimension-features.test.ts @@ -175,6 +175,86 @@ describe("level-A blocks reach the feature set with their own ages", () => { expect(features.mood?.dimensionsAsOfNote).toBeUndefined(); }); + it("carries the day's forecast with its own age, and its own instruction", async () => { + await seedUser(); + // Enough rated days that the account is above the forecast floor: the read + // path re-checks the ladder and refuses below it, stale row or not, so a + // one-entry account would (correctly) get an absence here rather than the + // present forecast this case is about. One statement rather than a loop of + // awaits, to keep the write short. + await getPrismaClient().moodEntry.createMany({ + data: Array.from({ length: 35 }, (_, d) => ({ + userId: USER, + date: shiftDateKey(todayKey(), -d), + tz: TZ, + mood: "GUT", + score: 4, + moodA1: 7, + stressA2: 3, + source: "MANUAL" as const, + moodLoggedAt: new Date(Date.now() - d * DAY_MS), + })), + }); + // A forecast about the day before yesterday. Written straight to the + // table: the job's own path is proven in `mood-prognosis-ladder.test.ts`, + // and what this case is about is what the assistant is handed. + await getPrismaClient().moodPrediction.create({ + data: { + userId: USER, + date: shiftDateKey(todayKey(), -3), + predicted: 5.4, + ciLow: 4.1, + ciHigh: 6.8, + n: 42, + modelVersion: "mood-ridge-1", + features: JSON.stringify([ + { feature: "dimension:a2", contribution: 0.9 }, + ]), + computedAt: new Date(Date.now() - 3 * DAY_MS), + }, + }); + + const features = await extractFeatures(USER, false); + const prognosis = features.mood?.prognosis; + expect( + prognosis, + "features.mood carries no prognosis block; the assistant sees neither the value nor the fact that there is none", + ).toBeDefined(); + expect(prognosis!.present).toBe(true); + expect(prognosis!.predicted).toBe(5.4); + expect(prognosis!.n).toBe(42); + expect(prognosis!.ciLow).toBe(4.1); + expect(prognosis!.ciHigh).toBe(6.8); + // Three days old: the same freshness rule the dimensions use, from the + // same module, so nothing may state it as today. + expect(prognosis!.daysAgo).toBe(3); + expect(prognosis!.current).toBe(false); + expect( + prognosis!.staleNotice, + "a stale forecast ships without the instruction that makes it safe to read", + ).toBeTruthy(); + expect(prognosis!.staleNotice).toContain(shiftDateKey(todayKey(), -3)); + expect(prognosis!.staleNotice!.toLowerCase()).toContain("do not describe"); + // And the wording rule rides with the values rather than being inferred. + expect(prognosis!.note.toLowerCase()).toContain("counterfactual"); + expect(prognosis!.note.toLowerCase()).toContain("never merge"); + }); + + it("states the absence of a forecast rather than omitting it", async () => { + await seedUser(); + await seedEntry(0, { mood: "GUT", score: 4, a1: 7 }); + + const features = await extractFeatures(USER, false); + const prognosis = features.mood?.prognosis; + expect(prognosis).toBeDefined(); + // Absence is a field, not a missing key: a model told "there is none yet" + // says so, and a model told nothing estimates one. + expect(prognosis!.present).toBe(false); + expect(prognosis!.reason).toBe("no-output-yet"); + expect(prognosis!.entries).toBe(1); + expect(prognosis!.predicted).toBeUndefined(); + }); + it("leaves the legacy scale line untouched", async () => { await seedUser(); await seedEntry(0, { mood: "GUT", score: 4, a1: 7 }); diff --git a/tests/integration/mood-prognosis-ladder.test.ts b/tests/integration/mood-prognosis-ladder.test.ts new file mode 100644 index 000000000..888e69581 --- /dev/null +++ b/tests/integration/mood-prognosis-ladder.test.ts @@ -0,0 +1,432 @@ +/** + * The ladder, the job and the route, against real Postgres and the real rows. + * + * The unit tests prove the arithmetic and the gates in isolation. This proves + * the assembly: that an account one day short of the threshold really gets + * nothing written, that the day it crosses really gets a row, that switching + * the module off really takes the rows away again, and that deleting back + * below the threshold does not leave a forecast standing with a fresh date on + * it. Every one of those is a claim about what is IN THE TABLE, so every + * assertion reads the table rather than a return value. + * + * It is also the automated half of the ladder walk the phase plan asks a human + * to do. What is left for a browser is what a browser is for: that the band is + * legible at 390 px and that the count sits beside the number rather than in a + * tooltip. + */ +import { NextRequest } from "next/server"; +import { beforeEach, describe, expect, it, vi } from "vitest"; + +import { cookieJar } from "./mock-next-headers"; +import { getPrismaClient, switchSessionTo, truncateAllTables } from "./setup"; + +const USER = "user-mood-prognosis"; +const TZ = "Europe/Berlin"; +const DAY_MS = 24 * 60 * 60 * 1000; + +vi.mock("next/headers", async () => { + const { cookieJar, headerJar } = await import("./mock-next-headers"); + return { + headers: vi.fn(async () => ({ + get: (name: string) => headerJar.get(name.toLowerCase()) ?? null, + })), + cookies: vi.fn(async () => ({ + get: (name: string) => { + const value = cookieJar.get(name); + return value ? { name, value } : undefined; + }, + set: (name: string, value: string) => cookieJar.set(name, value), + delete: (name: string) => cookieJar.delete(name), + })), + }; +}); + +vi.mock("@/lib/db-compat", () => ({ + ensureDbCompatibility: vi.fn().mockResolvedValue(undefined), +})); + +/** Fixed clock: a ladder test that slides with the calendar is a date bomb. */ +const NOW = new Date("2026-08-08T09:00:00.000Z"); + +function dayKey(daysAgo: number): string { + return new Date(NOW.getTime() - daysAgo * DAY_MS).toISOString().slice(0, 10); +} + +async function seedUser(): Promise { + await getPrismaClient().user.create({ + data: { + id: USER, + username: "mood-prognosis", + email: "mood-prognosis@example.test", + timezone: TZ, + }, + }); +} + +/** + * `count` consecutive days, each carrying a rating and a stress value the + * rating follows. A clean relationship on purpose: the point of this file is + * the plumbing, and a fixture the screening refuses would test the refusal + * path twice and the write path never. + */ +async function seedDays(count: number): Promise { + for (let i = 0; i < count; i++) { + const stress = i % 11; + await getPrismaClient().moodEntry.create({ + data: { + userId: USER, + date: dayKey(i), + tz: TZ, + mood: "OKAY", + score: 3, + moodA1: Math.round(0.8 * (10 - stress) + 1), + stressA2: stress, + source: "MANUAL", + moodLoggedAt: new Date(NOW.getTime() - i * DAY_MS), + }, + }); + } +} + +async function runJob() { + const { runMoodPrognosisRefresh } = + await import("@/lib/jobs/mood-prognosis-refresh"); + return runMoodPrognosisRefresh(NOW); +} + +function storedRows() { + return getPrismaClient().moodPrediction.findMany({ + where: { userId: USER }, + orderBy: { date: "asc" }, + }); +} + +beforeEach(async () => { + await truncateAllTables(getPrismaClient()); + cookieJar.clear(); +}); + +describe("the minimum-n ladder, end to end", () => { + it("writes nothing at one day short of the threshold, and says why", async () => { + await seedUser(); + await seedDays(29); + + const summary = await runJob(); + expect(summary.refreshed).toBe(0); + // The account is not in the cohort at all: the job's cheap cut is the same + // threshold, so an account a day short is never even read. The next case + // is the other half of the evidence — one more day and `scanned` is 1, so + // this zero is the cut landing in the right place rather than a sweep that + // matched nothing. + expect(summary.scanned).toBe(0); + expect(await storedRows()).toEqual([]); + + const { readMoodPrognosis } = + await import("@/lib/analytics/mood-prognosis/read"); + const reading = await readMoodPrognosis(USER, NOW, TZ); + expect(reading.present).toBe(false); + if (!reading.present) { + expect(reading.reason).toBe("learning-phase"); + expect(reading.entries).toBe(29); + expect(reading.nextThreshold).toBe(30); + } + }); + + it("writes a banded, counted, provisional forecast on the day it crosses", async () => { + await seedUser(); + await seedDays(30); + + const summary = await runJob(); + expect(summary.scanned).toBe(1); + expect(summary.refreshed).toBe(1); + + const rows = await storedRows(); + expect(rows.length).toBeGreaterThan(0); + // The trailing window, not the whole year of days. + expect(rows.length).toBeLessThanOrEqual(7); + + for (const row of rows) { + expect(row.predicted).toBeGreaterThanOrEqual(0); + expect(row.predicted).toBeLessThanOrEqual(10); + // The band and the count are part of the value, not decoration. + expect(row.ciLow).not.toBeNull(); + expect(row.ciHigh).not.toBeNull(); + expect(row.ciLow!).toBeLessThanOrEqual(row.predicted); + expect(row.ciHigh!).toBeGreaterThanOrEqual(row.predicted); + expect(row.n).toBe(30); + expect(row.modelVersion).toBe("mood-ridge-1"); + // The stored evidence the explanation is generated from. + const contributions = JSON.parse(row.features) as Array<{ + feature: string; + contribution: number; + }>; + expect(contributions.length).toBeGreaterThan(0); + expect(contributions[0].feature).toBe("dimension:a2"); + expect(typeof contributions[0].contribution).toBe("number"); + } + + const { readMoodPrognosis } = + await import("@/lib/analytics/mood-prognosis/read"); + const reading = await readMoodPrognosis(USER, NOW, TZ); + expect(reading.present).toBe(true); + if (reading.present) { + // Thirty days is a forecast, and a forecast at thirty days is labelled. + expect(reading.provisional).toBe(true); + expect(reading.stage).toBe("provisional"); + expect(reading.n).toBe(30); + // The self-assessment is returned unchanged beside it, never merged. + const entry = await getPrismaClient().moodEntry.findFirstOrThrow({ + where: { userId: USER, date: reading.date }, + }); + expect(reading.selfAssessment).toBe(entry.moodA1); + expect(reading.deviation).not.toBeNull(); + } + }); + + it("drops the provisional label once the account has enough days", async () => { + await seedUser(); + await seedDays(70); + await runJob(); + + const { readMoodPrognosis } = + await import("@/lib/analytics/mood-prognosis/read"); + const reading = await readMoodPrognosis(USER, NOW, TZ); + expect(reading.present).toBe(true); + if (reading.present) { + expect(reading.provisional).toBe(false); + expect(reading.stage).toBe("regular"); + expect(reading.seasonalUnlocked).toBe(false); + } + }); + + it("computes nothing and stores nothing while the mood module is off", async () => { + await seedUser(); + await seedDays(40); + await runJob(); + expect((await storedRows()).length).toBeGreaterThan(0); + + await getPrismaClient().user.update({ + where: { id: USER }, + data: { modulePreferencesJson: { mood: false } }, + }); + + const summary = await runJob(); + expect(summary.moduleOff).toBe(1); + expect(summary.refreshed).toBe(0); + // And the rows it wrote before are gone: a forecast that outlives the + // switch that turned its module off is a claim nobody asked for. + expect(await storedRows()).toEqual([]); + }); + + it("clears a stale forecast when the account falls back below the threshold", async () => { + await seedUser(); + await seedDays(40); + await runJob(); + const before = await storedRows(); + expect(before.length).toBeGreaterThan(0); + + // Delete back under the threshold. The account is no longer in the + // eligibility half of the cohort at all, so the pass only reaches it + // through the rows it already holds. + await getPrismaClient().moodEntry.deleteMany({ + where: { userId: USER, date: { gte: dayKey(28) } }, + }); + + const summary = await runJob(); + expect(summary.scanned).toBe(1); + expect(summary.refused).toBe(1); + expect( + await storedRows(), + "the forecast outlived its own eligibility — a stale claim with a fresh date on it", + ).toEqual([]); + }); + + it("replaces rather than accumulates across passes", async () => { + await seedUser(); + await seedDays(40); + await runJob(); + const first = await storedRows(); + + // A row from a pass that will not be repeated: an older day, written by + // hand the way a previous window would have left it. + await getPrismaClient().moodPrediction.create({ + data: { + userId: USER, + date: dayKey(200), + predicted: 5, + ciLow: 4, + ciHigh: 6, + n: 99, + modelVersion: "mood-ridge-0", + features: "[]", + computedAt: new Date(NOW.getTime() - 200 * DAY_MS), + }, + }); + + await runJob(); + const after = await storedRows(); + expect(after.map((r) => r.date)).toEqual(first.map((r) => r.date)); + expect(after.some((r) => r.modelVersion === "mood-ridge-0")).toBe(false); + }); +}); + +describe("GET /api/mood/prognosis", () => { + /** + * A signed-in browser session. + * + * The expiry is off the REAL clock, not the fixture's: the seeded days are + * dated against `NOW` so the ladder cannot slide with the calendar, but a + * session stamped from a fixed instant expires the moment the wall clock + * passes it, and every route then answers 401 for a reason that has nothing + * to do with what the case is about. + * + * `actingOn` is passed only for a delegate. Switching an owner's session + * into their own record would make the request a delegated one against a + * grant that does not exist, and the refusal would look like the scope + * refusal this file is trying to prove. + */ + async function signIn(userId: string, actingOn?: string) { + const session = await getPrismaClient().session.create({ + data: { + userId, + expiresAt: new Date(Date.now() + 60 * 60 * 1000), + }, + }); + cookieJar.set("healthlog_session", session.id); + if (actingOn) await switchSessionTo(session.id, actingOn); + return session; + } + + it("answers the owner with both readings", async () => { + await seedUser(); + await seedDays(40); + await runJob(); + await signIn(USER); + + const { GET } = await import("@/app/api/mood/prognosis/route"); + const res = await GET( + new NextRequest("http://localhost/api/mood/prognosis"), + ); + expect(res.status).toBe(200); + const body = (await res.json()) as { + data: { present: boolean; n: number; predicted: number; ciLow: number }; + }; + expect(body.data.present).toBe(true); + expect(body.data.n).toBe(40); + expect(typeof body.data.predicted).toBe("number"); + expect(typeof body.data.ciLow).toBe("number"); + }); + + it("refuses below the floor even while a stale row survives, and never emits an out-of-enum stage", async () => { + // The gap the read path has to close on its own. The job runs nightly, so + // between an entry deletion and the next 04:35 pass a MoodPrediction row + // outlives its own eligibility. If the read path trusted the row it would + // answer present:true with a value the account no longer has the data to + // support — and with stage "insufficient", which is not in the published + // enum, so a strict client decode fails the whole payload. The read path + // must agree with the job's own stale rule: below the floor is absence. + await seedUser(); + await seedDays(40); + await runJob(); + expect((await storedRows()).length).toBeGreaterThan(0); + + // Delete back to 20 rated days — under the 30-day forecast floor, but + // still inside the learning band, so the refusal reason is a real rung + // rather than the empty-account one. Deliberately DO NOT run the job: the + // point is what the route answers in the window before the sweep clears + // the row. The existing "clears a stale forecast" case runs the job before + // reading, which is exactly why it never saw this. + await getPrismaClient().moodEntry.deleteMany({ + where: { userId: USER, date: { gte: dayKey(20) } }, + }); + expect( + (await storedRows()).length, + "the row must still be on disk — otherwise this proves nothing about the read path", + ).toBeGreaterThan(0); + + await signIn(USER); + const { GET } = await import("@/app/api/mood/prognosis/route"); + const res = await GET( + new NextRequest("http://localhost/api/mood/prognosis"), + ); + expect(res.status).toBe(200); + const body = (await res.json()) as { + data: { + present: boolean; + reason?: string; + stage?: string; + predicted?: number; + }; + }; + expect( + body.data.present, + "the read path served a forecast to an account below every floor", + ).toBe(false); + expect(body.data).not.toHaveProperty("predicted"); + // The out-of-enum stage a strict Codable decode chokes on must never leave + // the route at all. + expect(body.data.stage).toBeUndefined(); + expect(body.data.reason).toBe("learning-phase"); + }); + + it("refuses a delegate whose grant does not open the whole record", async () => { + await seedUser(); + await seedDays(40); + await runJob(); + const rows = await storedRows(); + expect(rows.length).toBeGreaterThan(0); + + const delegate = await getPrismaClient().user.create({ + data: { + id: "user-mood-prognosis-delegate", + username: "mood-prognosis-delegate", + email: "mood-prognosis-delegate@example.test", + timezone: TZ, + }, + }); + await getPrismaClient().accountGrant.create({ + data: { + grantorId: USER, + granteeId: delegate.id, + access: "READ", + acceptedAt: new Date(), + // Only the mind section. The forecast reads across sections and names + // what it read, so it is not a mind payload. + scopeJson: ["mind"] as never, + }, + }); + await signIn(delegate.id, USER); + + const { GET } = await import("@/app/api/mood/prognosis/route"); + const res = await GET( + new NextRequest("http://localhost/api/mood/prognosis"), + ); + expect(res.status).toBe(403); + + // Asserted on the bytes, not on a parsed field: the point is that not one + // figure of the owner's reached the response. + const raw = await res.text(); + expect(raw).not.toContain("predicted"); + expect(raw).not.toContain("ciLow"); + expect(raw).not.toContain("dimension:a2"); + expect(raw).not.toContain(String(rows[0].n)); + expect(raw).not.toContain(rows[0].date); + }); + + it("blanks the surface when the mood module is off", async () => { + await seedUser(); + await seedDays(40); + await runJob(); + await getPrismaClient().user.update({ + where: { id: USER }, + data: { modulePreferencesJson: { mood: false } }, + }); + await signIn(USER); + + const { GET } = await import("@/app/api/mood/prognosis/route"); + const res = await GET( + new NextRequest("http://localhost/api/mood/prognosis"), + ); + expect(res.status).toBe(403); + }); +});