Files
HealthLog/src/lib/analytics/sleep-night.ts
T
Marc-André Bombeck 2a7195601b fix(sleep): stop double-counting bare ASLEEP against granular stages
Apple Health (and some WHOOP shapes) write BOTH an unspecified ASLEEP
aggregate row AND the granular CORE/DEEP/REM breakdown for the same
sleep period. The per-night reconstruction summed both, so a real ~8 h
night surfaced as ~16 h across every sleep surface — the chart showed
~16-18 h points and the target-range 30-day average read ~16.4 h/day.

Prefer the granular partition: when any CORE/DEEP/REM stage is present
for a night/source, count only those; fall back to the bare ASLEEP (or
a stage-less legacy/manual) row only when no granular stage exists.
IN_BED and AWAKE stay excluded; dual-source nights still collapse to one
canonical source via the user's sleep priority ladder. The same rule
applies inside the per-source asleep tiebreak so a source writing both
shapes is not over-weighted.

Align the surfaces that bypassed the shared helper:

- insights/targets: SLEEP_DURATION was averaged as raw per-stage rows
  (minutes) and fed into a target band expressed in hours, producing the
  bogus 16.4 h/day. Reconstruct per-night time-asleep, convert to hours,
  and classify + roll up consistency on the nightly hour totals.
- coach snapshot: route the sleep block through reconstructSleepNights
  so the prompt narrates the same nightly numbers (was summing every
  stage row, including IN_BED + AWAKE).
- sleep card chart: read /api/measurements/series?kind=sleep (night
  reconstruction, hours) instead of the generic measurement
  daily-average path, and label the axis in hours instead of minutes.

Add regression fixtures: a night with both a bare ASLEEP row and the
granular breakdown asserts the granular total (no double-count); a
dual-source night with both shapes per source; and a multi-night 30-day
average that lands at ~8 h, not ~16 h.
2026-06-04 18:22:55 +02:00

359 lines
15 KiB
TypeScript
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
/**
* v1.11.4 — per-night sleep aggregation (iOS #1, HIGH).
*
* `SLEEP_DURATION` is stored in MINUTES, one row per sleep STAGE per
* night (IN_BED / AWAKE / ASLEEP / REM / CORE / DEEP) since v1.4.23. The
* value is NOT cumulative — a single stage row carries only that stage's
* minutes. Every "last night's sleep" surface therefore has to SUM the
* stage rows belonging to one night before it shows a headline number;
* reading the single most-recent row gives one stage, not the night.
*
* This helper is the one shared place that turns the raw per-stage rows
* into per-night totals. The headline "time asleep" is light(CORE) +
* deep + REM; IN_BED and AWAKE are excluded from the asleep total,
* matching the AASM convention.
*
* Granular-over-bare (v1.11.5)
* ----------------------------
* Apple Health (and some WHOOP shapes) writes BOTH an unspecified `ASLEEP`
* AGGREGATE row AND the granular `CORE` / `DEEP` / `REM` breakdown for the
* SAME sleep period. The granular rows partition the same time the aggregate
* covers, so summing bare-ASLEEP together with the granular rows ~doubles the
* night. We therefore prefer the granular partition: when ANY granular stage
* (CORE / DEEP / REM) is present for a night/source we count ONLY the granular
* stages; the bare `ASLEEP` aggregate (or a stage-less legacy / manual row) is
* the night's asleep total only when no granular stage exists.
*
* Night grouping
* --------------
* A "night" is one SLEEP SESSION, not a calendar day. `measuredAt` is the
* stage segment's END instant (`apple-health-mapping.ts` sets
* `takenAt = endDate`), and a device writes one row per stage segment, so a
* normal overnight sleep (asleep before local midnight) spreads its segments
* across BOTH sides of midnight. Keying each segment by its own calendar day
* would split one physical night into two partial nights and undercount the
* "last night" headline.
*
* Instead we cluster the segments into sessions by time gap: sort by
* `measuredAt`, and start a new session whenever the gap from the previous
* segment exceeds `SESSION_GAP_MS` (3 h). Stage segments inside one sleep
* session are contiguous (minutes to ~2 h apart), so a > 3 h gap reliably
* marks the boundary between a daytime nap and the overnight session, or
* between two genuine sessions in one day — they stay separable. The gap is
* measured on the absolute UTC instant, so it is immune to DST shifts.
*
* Each session is keyed by the LOCAL WAKE DAY — `userDayKey` of the session's
* LAST segment (its end instant) in the user's timezone. This matches the
* Apple Health convention (a sleep session is attributed to the date you wake
* up) and keeps the keying tz/DST-correct because it runs on the real instant.
*
* Source de-dup
* -------------
* A user paired to more than one sleep source (e.g. WHOOP + Apple Health)
* gets per-stage rows for the SAME night from each source. Summing them all
* would ~double the night total. Before summing a session we collapse it to
* ONE source using the user's `sleep` source-priority ladder
* (`DEFAULT_SOURCE_PRIORITY.sleep` = WHOOP > APPLE_HEALTH > WITHINGS): pick the
* highest-ranked source present in the session, falling back to the source
* with the most asleep minutes (then source name) when none of the session's
* sources sits on the ladder. Only that source's segments are summed — sources
* are never blended within a night. This mirrors the per-(type, day) collapse
* the rest of the v1.11.x numeric surfaces apply.
*
* Unit
* ----
* Totals are returned in MINUTES — the canonical `SLEEP_DURATION` unit
* (`getUnitForType` → "minutes") — so the existing web consumers that
* already treat the summary as minutes (e.g. the insights overview's
* `formatHoursMinutes`) keep working unchanged; they now average per-
* night totals instead of per-stage rows, which is strictly more
* correct. Callers that want hours convert at the edge (`/ 60`).
*/
import type { MeasurementSource, SleepStage } from "@/generated/prisma/client";
import { userDayKey } from "@/lib/tz/resolver";
import { summarize, type DataSummary } from "@/lib/analytics/trends";
import {
getSourceLadder,
parseSourcePriority,
} from "@/lib/validations/source-priority";
/**
* The GRANULAR asleep stages (iOS 16+ `asleepCore` / `asleepDeep` /
* `asleepREM`). A bare `ASLEEP` row is the legacy iOS 15- `asleepUnspecified`
* AGGREGATE — many sources (Apple Health, some WHOOP shapes) write BOTH the
* unspecified aggregate AND the granular breakdown for the SAME period, so
* summing bare-ASLEEP together with the granular rows ~doubles the night. The
* granular rows partition the same time the aggregate covers, so when any
* granular stage is present for a night/source we count ONLY the granular
* stages and drop the bare aggregate; the bare row is the asleep total only
* when no granular stage exists.
*/
const GRANULAR_ASLEEP_STAGES: ReadonlySet<SleepStage> = new Set<SleepStage>([
"REM",
"CORE",
"DEEP",
]);
/**
* Sum a set of stage rows into time-asleep minutes WITHOUT double-counting a
* bare `ASLEEP` aggregate against the granular CORE/DEEP/REM rows it overlaps.
* IN_BED + AWAKE are never asleep; a stage-less (`null`) row is the night's
* total only when no granular stage is present.
*/
function asleepMinutesOf(rows: readonly SleepStageRow[]): number {
let granular = 0;
let fallback = 0;
let sawGranular = false;
for (const r of rows) {
const minutes = Number.isFinite(r.value) ? r.value : 0;
const stage = r.sleepStage;
if (stage != null && GRANULAR_ASLEEP_STAGES.has(stage)) {
granular += minutes;
sawGranular = true;
} else if (stage === "ASLEEP" || stage == null) {
// Bare unspecified aggregate (`ASLEEP`) or a stage-less legacy/manual
// row — only used when no granular stage carries the night.
fallback += minutes;
}
// IN_BED / AWAKE never count toward asleep.
}
return sawGranular ? granular : fallback;
}
/**
* Gap (ms) between consecutive stage segments that starts a new sleep
* session. Stage segments within one night are contiguous (minutes to a
* couple of hours apart); a > 3 h gap separates a nap from the overnight
* block, or two genuine sessions in one day.
*/
const SESSION_GAP_MS = 3 * 60 * 60 * 1000;
export interface SleepStageRow {
value: number;
measuredAt: Date;
sleepStage: SleepStage | null;
/**
* Ingest source of the stage row. Used to collapse a multi-source night
* (e.g. WHOOP + Apple Health) to one canonical source before summing.
* Optional so legacy callers and fixtures that don't carry a source still
* type-check (a missing source is treated as a single anonymous source —
* the de-dup is a no-op for single-source nights).
*/
source?: MeasurementSource | null;
}
/** One reconstructed night: the asleep total + the per-stage breakdown. */
export interface SleepNight {
/** Wake-day key (YYYY-MM-DD) in the user's timezone. */
night: string;
/** Latest stage instant in the night — the night's representative timestamp. */
measuredAt: Date;
/** Time asleep in minutes = CORE + DEEP + REM, or bare ASLEEP when granular stages are absent. */
asleepMinutes: number;
/** In-bed minutes when an IN_BED row exists, else null. */
inBedMinutes: number | null;
/** Awake-in-bed minutes when an AWAKE row exists, else null. */
awakeMinutes: number | null;
/** Per-stage minutes for the night (only stages the device reported). */
stages: Partial<Record<SleepStage, number>>;
}
/** Sentinel for rows that carry no source — collapses to one bucket. */
const NO_SOURCE = "__none__";
/**
* Pick the one canonical source for a session. The user's `sleep`
* source-priority ladder wins (lowest ladder index = highest priority).
* Sources absent from the ladder fall back to "most asleep minutes", then
* a stable source-name tiebreak. Single-source (or source-less) sessions
* resolve to that one bucket with no work.
*/
function pickSessionSource(
rows: SleepStageRow[],
sleepLadder: readonly MeasurementSource[],
): string {
// Group rows by source, then sum each source's asleep minutes with the
// granular-over-bare rule so a source that writes BOTH a bare ASLEEP
// aggregate and the granular breakdown is not over-weighted in the tiebreak.
const rowsBySource = new Map<string, SleepStageRow[]>();
for (const r of rows) {
const src = r.source ?? NO_SOURCE;
const list = rowsBySource.get(src) ?? [];
list.push(r);
rowsBySource.set(src, list);
}
const asleepBySource = new Map<string, number>();
for (const [src, srcRows] of rowsBySource) {
const minutes = asleepMinutesOf(srcRows);
if (minutes > 0) asleepBySource.set(src, minutes);
}
// No asleep minutes anywhere (IN_BED / AWAKE only) — fall back to the set
// of all present sources so the session still resolves to a single bucket.
const sources =
asleepBySource.size > 0
? [...asleepBySource.keys()]
: [...new Set(rows.map((r) => r.source ?? NO_SOURCE))];
if (sources.length <= 1) return sources[0] ?? NO_SOURCE;
const rankOf = (src: string): number => {
const i = sleepLadder.indexOf(src as MeasurementSource);
return i === -1 ? Number.MAX_SAFE_INTEGER : i;
};
return sources.sort((a, b) => {
const ra = rankOf(a);
const rb = rankOf(b);
if (ra !== rb) return ra - rb;
// Neither (or both) on the ladder — most asleep minutes wins, then name.
const ma = asleepBySource.get(a) ?? 0;
const mb = asleepBySource.get(b) ?? 0;
if (ma !== mb) return mb - ma;
return a < b ? -1 : 1;
})[0];
}
/**
* Group raw per-stage rows into per-night totals. Stages are clustered into
* sleep SESSIONS by time gap (a > `SESSION_GAP_MS` gap starts a new session),
* each session keyed by the LOCAL WAKE DAY of its last segment, then collapsed
* to ONE source via the user's `sleep` priority ladder before summing.
*
* Pure — the caller does the bounded DB read. `priorityJson` is the user's
* persisted `sourcePriorityJson` (or null for the defaults). Nights are
* returned sorted ascending by key so the last element is the most recent
* night.
*/
export function reconstructSleepNights(
rows: SleepStageRow[],
tz: string,
priorityJson: unknown = null,
): SleepNight[] {
if (rows.length === 0) return [];
const sleepLadder = getSourceLadder(parseSourcePriority(priorityJson), "sleep");
// Cluster into sessions by the gap between a segment's START and the latest
// END seen so far in the current session. `measuredAt` is the segment END;
// the segment START is `end − value minutes`. A new session begins only when
// the next segment STARTS more than SESSION_GAP_MS after the running end —
// so a single long stage block (a 4 h CORE) does NOT split a night, and two
// sources' interleaved/overlapping rows for the same night stay together
// (their gap is ≤ 0). Gaps are absolute-time, so the clustering is DST-immune.
// Sort by START so contiguous segments compare end-to-start in order.
const startOf = (r: SleepStageRow): number =>
r.measuredAt.getTime() -
(Number.isFinite(r.value) ? r.value : 0) * 60_000;
const sorted = [...rows].sort((a, b) => startOf(a) - startOf(b));
const sessions: SleepStageRow[][] = [];
let current: SleepStageRow[] = [];
let sessionEnd = Number.NEGATIVE_INFINITY;
for (const r of sorted) {
const start = startOf(r);
const end = r.measuredAt.getTime();
if (current.length > 0 && start - sessionEnd > SESSION_GAP_MS) {
sessions.push(current);
current = [];
sessionEnd = Number.NEGATIVE_INFINITY;
}
current.push(r);
if (end > sessionEnd) sessionEnd = end;
}
if (current.length > 0) sessions.push(current);
// Two sessions can land on the same wake day (e.g. a genuine second sleep);
// merge their rows under one key after the source-collapse so the night key
// stays unique. Collapse each session to its canonical source first.
const byNight = new Map<string, SleepStageRow[]>();
for (const session of sessions) {
const canonical = pickSessionSource(session, sleepLadder);
const kept = session.filter((r) => (r.source ?? NO_SOURCE) === canonical);
const pool = kept.length > 0 ? kept : session;
// The wake day is the LATEST segment END in the session (Apple Health
// attributes a session to the morning you wake up).
const wakeInstant = pool.reduce(
(max, r) => (r.measuredAt.getTime() > max.getTime() ? r.measuredAt : max),
pool[0].measuredAt,
);
const key = userDayKey(wakeInstant, tz);
const list = byNight.get(key) ?? [];
list.push(...kept);
byNight.set(key, list);
}
const nights: SleepNight[] = [];
for (const [night, nightRows] of byNight) {
let inBed = 0;
let awake = 0;
let sawInBed = false;
let sawAwake = false;
let latest = nightRows[0].measuredAt;
const stages: Partial<Record<SleepStage, number>> = {};
for (const r of nightRows) {
const minutes = Number.isFinite(r.value) ? r.value : 0;
if (r.measuredAt.getTime() > latest.getTime()) latest = r.measuredAt;
const stage = r.sleepStage;
if (stage) {
stages[stage] = (stages[stage] ?? 0) + minutes;
}
if (stage === "IN_BED") {
inBed += minutes;
sawInBed = true;
} else if (stage === "AWAKE") {
awake += minutes;
sawAwake = true;
}
// Asleep minutes are summed with the granular-over-bare rule below so a
// bare ASLEEP aggregate is never double-counted against the granular
// CORE/DEEP/REM rows it overlaps.
}
// v1.11.5 — single source of truth for the asleep total: prefer the
// granular CORE/DEEP/REM partition; fall back to the bare ASLEEP /
// stage-less row only when no granular stage exists for the night.
const asleep = asleepMinutesOf(nightRows);
nights.push({
night,
measuredAt: latest,
asleepMinutes: asleep,
inBedMinutes: sawInBed ? inBed : null,
awakeMinutes: sawAwake ? awake : null,
stages,
});
}
return nights.sort((a, b) => (a.night < b.night ? -1 : 1));
}
export interface SleepNightSummary {
/**
* A `DataSummary` whose every value is a per-night TIME-ASLEEP total
* (minutes). Drop-in replacement for the per-stage `summaries.
* SLEEP_DURATION` the slim slice produced. Null when no night has any
* asleep minutes in the supplied rows.
*/
summary: DataSummary;
/** The most recent night (for the dashboard/series headline), or null. */
latestNight: SleepNight | null;
}
/**
* Reconstruct nights from raw rows and summarise the per-night asleep
* totals. Nights with zero asleep minutes are dropped (an IN_BED-only or
* AWAKE-only fragment is not a scorable night). The returned `summary`
* uses one DataPoint per night so `summarize()`'s windowed avg7 / avg30
* become "average nightly sleep over the trailing N days".
*/
export function summarizeSleepNights(
rows: SleepStageRow[],
tz: string,
priorityJson: unknown = null,
): SleepNightSummary {
const nights = reconstructSleepNights(rows, tz, priorityJson).filter(
(n) => n.asleepMinutes > 0,
);
const dataPoints = nights.map((n) => ({
date: n.measuredAt,
value: n.asleepMinutes,
}));
return {
summary: summarize(dataPoints),
latestNight: nights.length > 0 ? nights[nights.length - 1] : null,
};
}