mirror of
https://github.com/MBombeck/HealthLog.git
synced 2026-11-07 04:12:04 +00:00
chore(release): v1.28.18 — Google Health sleep reads its true total
Google re-scores a night after the fact, and the sync re-reads recent nights to pick that up. The per-segment sleep externalId keyed on a volatile anchor (the session end instant plus a positional index), so a re-read minted fresh ids and landed as a SECOND parallel copy of the night instead of overwriting the first — the night-total then summed both (a real 7h35 night could read as 10h+). Each segment now keys on a stable identity: the session's own resource id (as the workout mapper already does) plus the segment's own start, dropping the shifting end anchor and the positional index. A mere re-classification of a block now updates in place. And before writing a re-read night, the sync soft-deletes any superseded rows left in that night's window (bounded to the session's own span, never a fresh row), so the night collapses to one canonical set. Nights already stored with duplicates heal as they are re-read within the sync overlap; scripts/repair-google-health-sleep.ts re-syncs the full history at once for older nights (dry-run by default, --run to execute).
This commit is contained in:
1 parent
3f4a845aae
commit
ef1b5f5de9
10 files changed
+436
-41
No files matched your search
+17
-1
@@ -2,7 +2,23 @@
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
## [1.28.17] — 2026-07-10 — Dashboard edits stick; share scope honoured
|
||||
## [1.28.18] — 2026-07-10 — Google Health sleep reads its true total
|
||||
|
||||
Sleep imported from Google Health could read far too long — a night of about
|
||||
seven and a half hours showing as ten. Google re-scores a night after the fact,
|
||||
and the sync re-reads recent nights to pick that up. The re-read was landing as a
|
||||
second, parallel copy of the night rather than replacing the first, so the
|
||||
night-total added both together.
|
||||
|
||||
Each sleep segment now keys on a stable identity (the session's own id plus the
|
||||
segment's start) instead of a value that shifted every time Google refined the
|
||||
night, so a re-read overwrites in place. And before writing a re-read night, the
|
||||
sync clears any superseded rows left in that night's window — so a re-scored
|
||||
night reads its true total, and a mere re-classification of a block (light to
|
||||
deep) updates in place instead of duplicating.
|
||||
|
||||
Nights already stored with duplicates heal as they are re-read; operators can
|
||||
repair the full history at once with a one-shot re-sync.
|
||||
|
||||
Fixes two bugs and hardens two edges.
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
openapi: 3.1.0
|
||||
info:
|
||||
title: HealthLog API
|
||||
version: 1.28.17
|
||||
version: 1.28.18
|
||||
description: >-
|
||||
Self-hosted personal-health-tracking PWA — public API surface for the iOS native client and external ingest.
|
||||
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "healthlog",
|
||||
"version": "1.28.17",
|
||||
"version": "1.28.18",
|
||||
"description": "Self-hosted personal-health-tracking PWA with Withings integration, AI insights, and doctor-report PDF export.",
|
||||
"license": "PolyForm-Noncommercial-1.0.0",
|
||||
"homepage": "https://healthlog.dev",
|
||||
|
||||
+1
-1
@@ -36,7 +36,7 @@ try {
|
||||
// v1.4.38.4 → v1.4.42. Do not hand-edit; bump `package.json` and rebuild.
|
||||
const CACHE_VERSION =
|
||||
(typeof self !== "undefined" && self.__APP_VERSION__) ||
|
||||
/* @sw-version-fallback */ "v1.28.17";
|
||||
/* @sw-version-fallback */ "v1.28.18";
|
||||
const STATIC_CACHE = `healthlog-static-${CACHE_VERSION}`;
|
||||
const PAGE_CACHE = `healthlog-pages-${CACHE_VERSION}`;
|
||||
// v1.18.6 — read-only data cache for a curated allowlist of safe GET `/api/*`
|
||||
|
||||
@@ -0,0 +1,90 @@
|
||||
/**
|
||||
* scripts/repair-google-health-sleep.ts — operator repair for the historic
|
||||
* Google Health sleep OVER-COUNT.
|
||||
*
|
||||
* THE DEFECT (fixed in code from v1.28.18): the per-segment sleep externalId was
|
||||
* keyed on a VOLATILE anchor (the session's `interval.endTime` plus a positional
|
||||
* segment index). When Google re-scored a night after the fact — which the sync
|
||||
* deliberately re-fetches on a 24 h overlap — the shifted anchor/index minted
|
||||
* FRESH externalIds, so the upsert created a SECOND parallel set of stage rows
|
||||
* for the same night instead of overwriting. The night-total then summed both
|
||||
* copies (a real 7h35 night could read as 10h+).
|
||||
*
|
||||
* THE CODE FIX anchors the externalId on Google's stable resource `name` plus
|
||||
* the segment's own start, and — via `replaceStaleGoogleHealthSleep` —
|
||||
* soft-deletes any stale row left in a re-fetched night's window before the
|
||||
* fresh set upserts. That makes every FUTURE sync idempotent and self-healing
|
||||
* for any night inside the incremental overlap.
|
||||
*
|
||||
* THIS SCRIPT heals HISTORY: nights older than the incremental window keep their
|
||||
* duplicate rows until re-fetched. It runs a FULL Google Health re-sync per
|
||||
* connected user, which re-imports every sleep session and lets the
|
||||
* replace-by-window cleanup collapse each night to a single canonical set (and
|
||||
* re-folds the rollup tier). It reuses the production sync path verbatim — no
|
||||
* bespoke SQL — so it can only do what an ordinary re-sync does.
|
||||
*
|
||||
* SAFETY: dry-run by default (lists the connections that WOULD be re-synced and
|
||||
* exits). Pass `--run` to execute. Runs users sequentially to avoid a burst
|
||||
* against Google's API. A per-user failure is reported and skipped, never fatal.
|
||||
*
|
||||
* RUN (never `pnpm tsx` — the standalone image strips tsx):
|
||||
* pnpm dlx tsx scripts/repair-google-health-sleep.ts # dry run
|
||||
* pnpm dlx tsx scripts/repair-google-health-sleep.ts --run # execute
|
||||
*/
|
||||
import { prisma } from "@/lib/db";
|
||||
import { syncUserGoogleHealth } from "@/lib/google-health/sync";
|
||||
|
||||
async function main(): Promise<void> {
|
||||
const execute = process.argv.includes("--run");
|
||||
|
||||
const connections = await prisma.googleHealthConnection.findMany({
|
||||
select: { userId: true },
|
||||
});
|
||||
|
||||
if (connections.length === 0) {
|
||||
console.log("No Google Health connections found — nothing to repair.");
|
||||
return;
|
||||
}
|
||||
|
||||
console.log(
|
||||
`${connections.length} Google Health connection(s) found.` +
|
||||
(execute
|
||||
? " Running a full re-sync per user (sequential)…"
|
||||
: " Dry run — pass --run to execute. Would re-sync:"),
|
||||
);
|
||||
|
||||
if (!execute) {
|
||||
for (const { userId } of connections) console.log(` • ${userId}`);
|
||||
console.log(
|
||||
"\nA full re-sync re-imports all Google Health data for each user and " +
|
||||
"collapses each re-scored night to one canonical set. Re-run with --run.",
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
let ok = 0;
|
||||
let failed = 0;
|
||||
for (const { userId } of connections) {
|
||||
try {
|
||||
const imported = await syncUserGoogleHealth(userId, { fullSync: true });
|
||||
ok += 1;
|
||||
console.log(` ✓ ${userId} — re-synced (${imported} rows imported)`);
|
||||
} catch (err) {
|
||||
failed += 1;
|
||||
console.error(
|
||||
` ✗ ${userId} — re-sync failed: ${
|
||||
err instanceof Error ? err.message : String(err)
|
||||
}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
console.log(`\nDone. ${ok} re-synced, ${failed} failed.`);
|
||||
}
|
||||
|
||||
main()
|
||||
.catch((err) => {
|
||||
console.error(err);
|
||||
process.exitCode = 1;
|
||||
})
|
||||
.finally(() => prisma.$disconnect());
|
||||
@@ -405,13 +405,84 @@ describe("mapSleepSession — per-stage segments", () => {
|
||||
expect(deep.value).toBe(45);
|
||||
expect(deep.sleepStage).toBe("DEEP");
|
||||
expect(deep.measuredAt.toISOString()).toBe("2026-06-02T02:45:00.000Z");
|
||||
// Per-stage rows key off the session END anchor + an indexed stage tag.
|
||||
expect(deep.fieldTag).toBe("2026-06-02T06:30:00.000Z:sleep_deep:0");
|
||||
// Segments key off the session anchor (here the fallback session-END, since
|
||||
// this fixture carries no resource `name`) plus the segment's own START —
|
||||
// NOT a positional index and NOT the stage label.
|
||||
expect(deep.fieldTag).toBe(
|
||||
"2026-06-02T06:30:00.000Z:sleep:2026-06-02T02:00:00.000Z",
|
||||
);
|
||||
|
||||
const rem = rows[1]!;
|
||||
expect(rem.value).toBe(30);
|
||||
expect(rem.sleepStage).toBe("REM");
|
||||
expect(rem.fieldTag).toBe("2026-06-02T06:30:00.000Z:sleep_rem:1");
|
||||
expect(rem.fieldTag).toBe(
|
||||
"2026-06-02T06:30:00.000Z:sleep:2026-06-02T03:00:00.000Z",
|
||||
);
|
||||
});
|
||||
|
||||
it("keeps stable per-segment externalIds across a re-score (anchor = resource name)", () => {
|
||||
// Two fetches of the SAME night. Google carries a stable resource `name`
|
||||
// and, on the re-score, refines the session END and RE-CLASSIFIES the first
|
||||
// block (LIGHT→DEEP) while keeping the segment START instants. The stable
|
||||
// anchor + start-keyed fieldTag must yield the SAME externalIds, so the
|
||||
// upsert overwrites in place rather than minting parallel duplicates.
|
||||
const name = "users/me/dataTypes/sleep/dataPoints/night-42";
|
||||
const first = mapSleepSession({
|
||||
name,
|
||||
sleep: {
|
||||
interval: {
|
||||
startTime: "2026-06-01T22:30:00.000Z",
|
||||
endTime: "2026-06-02T06:30:00.000Z",
|
||||
},
|
||||
stages: [
|
||||
{
|
||||
type: "LIGHT",
|
||||
startTime: "2026-06-02T02:00:00.000Z",
|
||||
endTime: "2026-06-02T02:45:00.000Z",
|
||||
},
|
||||
{
|
||||
type: "REM",
|
||||
startTime: "2026-06-02T03:00:00.000Z",
|
||||
endTime: "2026-06-02T03:30:00.000Z",
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
const rescored = mapSleepSession({
|
||||
name,
|
||||
sleep: {
|
||||
interval: {
|
||||
startTime: "2026-06-01T22:30:00.000Z",
|
||||
// Google refined the wake instant — the OLD anchor would have shifted.
|
||||
endTime: "2026-06-02T06:42:00.000Z",
|
||||
},
|
||||
stages: [
|
||||
{
|
||||
// Re-classified block, SAME start — must map to the same externalId.
|
||||
type: "DEEP",
|
||||
startTime: "2026-06-02T02:00:00.000Z",
|
||||
endTime: "2026-06-02T02:45:00.000Z",
|
||||
},
|
||||
{
|
||||
type: "REM",
|
||||
startTime: "2026-06-02T03:00:00.000Z",
|
||||
endTime: "2026-06-02T03:30:00.000Z",
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
|
||||
expect(first.map((r) => r.fieldTag)).toEqual([
|
||||
`${name}:sleep:2026-06-02T02:00:00.000Z`,
|
||||
`${name}:sleep:2026-06-02T03:00:00.000Z`,
|
||||
]);
|
||||
// Identical externalIds across the re-score → overwrite, not duplicate.
|
||||
expect(rescored.map((r) => r.fieldTag)).toEqual(
|
||||
first.map((r) => r.fieldTag),
|
||||
);
|
||||
// The re-classification is reflected in the stage axis (in-place update).
|
||||
expect(first[0]!.sleepStage).toBe("CORE"); // LIGHT → shallow-NREM band
|
||||
expect(rescored[0]!.sleepStage).toBe("DEEP");
|
||||
});
|
||||
|
||||
it("maps the documented SLEEP_STAGE_TYPE enum onto the shared bands", () => {
|
||||
|
||||
@@ -0,0 +1,99 @@
|
||||
/**
|
||||
* Pins the safety contract of `replaceStaleGoogleHealthSleep` — the
|
||||
* replace-by-window cleanup that keeps a re-scored Google night from
|
||||
* double-counting. The query MUST be tightly bounded: only LIVE
|
||||
* `GOOGLE_HEALTH` `SLEEP_DURATION` rows, only inside the session window, and
|
||||
* NEVER a row in the fresh keep-set. A session with no window is skipped
|
||||
* entirely (nothing to clean, and an unbounded delete would be data loss).
|
||||
*/
|
||||
import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||
|
||||
const { updateManyMock } = vi.hoisted(() => ({
|
||||
updateManyMock: vi.fn(async (_args: unknown) => ({ count: 0 })),
|
||||
}));
|
||||
|
||||
vi.mock("@/lib/db", () => ({
|
||||
prisma: { measurement: { updateMany: updateManyMock } },
|
||||
}));
|
||||
vi.mock("@/lib/crypto", () => ({ encrypt: vi.fn(), decrypt: vi.fn() }));
|
||||
vi.mock("@/lib/integrations/status", () => ({
|
||||
isReauthRequired: vi.fn(async () => false),
|
||||
recordSyncFailure: vi.fn(async () => {}),
|
||||
recordSyncSuccess: vi.fn(async () => {}),
|
||||
}));
|
||||
vi.mock("@/lib/rollups/measurement-rollups", () => ({
|
||||
collapseToTypeDayKeys: vi.fn(() => []),
|
||||
recomputeBucketsForMeasurement: vi.fn(async () => {}),
|
||||
recomputeUserRollups: vi.fn(async () => {}),
|
||||
}));
|
||||
vi.mock("@/lib/insights/comprehensive-generate", () => ({
|
||||
invalidateStatusInsightsForTypes: vi.fn(async () => {}),
|
||||
}));
|
||||
vi.mock("../credentials", () => ({
|
||||
getUserGoogleHealthCredentials: vi.fn(async () => null),
|
||||
}));
|
||||
vi.mock("../client", () => ({ refreshAccessToken: vi.fn() }));
|
||||
|
||||
import { replaceStaleGoogleHealthSleep } from "../sync";
|
||||
|
||||
beforeEach(() => {
|
||||
updateManyMock.mockClear();
|
||||
updateManyMock.mockResolvedValue({ count: 1 });
|
||||
});
|
||||
|
||||
describe("replaceStaleGoogleHealthSleep — bounded replace-by-window", () => {
|
||||
it("soft-deletes only live GOOGLE_HEALTH sleep rows in the window, excluding the fresh set", async () => {
|
||||
const windowStart = new Date("2026-06-01T22:30:00.000Z");
|
||||
const windowEnd = new Date("2026-06-02T06:30:00.000Z");
|
||||
const keepIds = ["anchor:sleep:a", "anchor:sleep:b"];
|
||||
|
||||
const removed = await replaceStaleGoogleHealthSleep("user-1", [
|
||||
{ windowStart, windowEnd, keepIds },
|
||||
]);
|
||||
|
||||
expect(removed).toBe(1);
|
||||
expect(updateManyMock).toHaveBeenCalledTimes(1);
|
||||
const arg = updateManyMock.mock.calls[0]![0] as {
|
||||
where: Record<string, unknown>;
|
||||
data: Record<string, unknown>;
|
||||
};
|
||||
expect(arg.where).toEqual({
|
||||
userId: "user-1",
|
||||
source: "GOOGLE_HEALTH",
|
||||
type: "SLEEP_DURATION",
|
||||
deletedAt: null,
|
||||
measuredAt: { gte: windowStart, lte: windowEnd },
|
||||
externalId: { notIn: keepIds },
|
||||
});
|
||||
// A soft delete — the row is tombstoned, never hard-removed.
|
||||
expect(arg.data).toHaveProperty("deletedAt");
|
||||
expect(arg.data.deletedAt).toBeInstanceOf(Date);
|
||||
});
|
||||
|
||||
it("skips a session with a null window (never issues an unbounded delete)", async () => {
|
||||
await replaceStaleGoogleHealthSleep("user-1", [
|
||||
{ windowStart: null, windowEnd: null, keepIds: ["x"] },
|
||||
{
|
||||
windowStart: new Date("2026-06-01T22:00:00.000Z"),
|
||||
windowEnd: new Date("2026-06-02T06:00:00.000Z"),
|
||||
keepIds: [],
|
||||
},
|
||||
]);
|
||||
// Both sessions are unsafe to clean (no window, or no fresh ids to keep) —
|
||||
// neither may issue a delete.
|
||||
expect(updateManyMock).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("never fails the sync when the cleanup query throws", async () => {
|
||||
updateManyMock.mockRejectedValueOnce(new Error("db down"));
|
||||
await expect(
|
||||
replaceStaleGoogleHealthSleep("user-1", [
|
||||
{
|
||||
windowStart: new Date("2026-06-01T22:00:00.000Z"),
|
||||
windowEnd: new Date("2026-06-02T06:00:00.000Z"),
|
||||
keepIds: ["anchor:sleep:a"],
|
||||
},
|
||||
]),
|
||||
).resolves.toBe(0);
|
||||
});
|
||||
});
|
||||
@@ -1806,13 +1806,23 @@ function readSleepSegments(
|
||||
}
|
||||
|
||||
/**
|
||||
* The stable session anchor for a sleep `DataPoint`'s externalId. The session
|
||||
* end (or start) ISO instant is unique per night, so per-stage rows key as
|
||||
* `<session-anchor>:sleep_<stage>` — a re-scored night overwrites in place.
|
||||
* Paths are camelCase (`sleep.interval.endTime`) — the response payload never
|
||||
* uses snake_case.
|
||||
* The stable session anchor for a sleep `DataPoint`'s externalId.
|
||||
*
|
||||
* Prefers the Google resource id (`name`) — it is INVARIANT across Google's
|
||||
* after-the-fact re-scoring of a night, so the per-segment rows keep the SAME
|
||||
* externalId on a re-fetch and overwrite in place instead of minting parallel
|
||||
* duplicates. This mirrors `mapWorkout`, which anchors on `name` for exactly
|
||||
* this reason.
|
||||
*
|
||||
* The session interval end/start is only a FALLBACK (for a payload without a
|
||||
* resource name): it SHIFTS whenever Google refines the wake/onset instant on a
|
||||
* re-score, which is precisely what used to duplicate a night's rows. Paths are
|
||||
* camelCase (`sleep.interval.endTime`) — the response payload never uses
|
||||
* snake_case.
|
||||
*/
|
||||
function sleepSessionAnchor(point: GoogleHealthDataPoint, tz?: string): string {
|
||||
const name = readPath(point, "name");
|
||||
if (typeof name === "string" && name !== "") return name;
|
||||
const end = readPath(point, "sleep.interval.endTime");
|
||||
if (typeof end === "string") {
|
||||
const d = parseLocalInstant(end, tz);
|
||||
@@ -1827,51 +1837,86 @@ function sleepSessionAnchor(point: GoogleHealthDataPoint, tz?: string): string {
|
||||
}
|
||||
|
||||
/**
|
||||
* Map one Google sleep session into per-SEGMENT `SLEEP_DURATION` rows. The
|
||||
* Google sleep payload carries a real per-stage segment series (each with its
|
||||
* own start/end), so one row is emitted PER SEGMENT — `measuredAt = that
|
||||
* segment's END` — rather than collapsing a stage's segments onto a single
|
||||
* lastEnd instant. The timeline is MEASURED (real onsets), so these rows are NOT
|
||||
* flagged reconstructed — unlike WHOOP, which has no onsets.
|
||||
* One mapped Google sleep session: the stable anchor, the session's covered
|
||||
* time window (earliest segment start → latest segment end, both UTC), and the
|
||||
* per-segment rows. The window drives the sync's replace-by-window cleanup of
|
||||
* stale rows a re-score orphaned; `rows` is what gets upserted.
|
||||
*/
|
||||
export interface GoogleHealthSleepSession {
|
||||
/** Stable session anchor — also the `<anchor>:sleep:` externalId prefix. */
|
||||
anchor: string;
|
||||
/** Earliest segment start across the session (UTC), or null if none map. */
|
||||
windowStart: Date | null;
|
||||
/** Latest segment end across the session (UTC), or null if none map. */
|
||||
windowEnd: Date | null;
|
||||
rows: GoogleHealthMappedMeasurement[];
|
||||
}
|
||||
|
||||
/**
|
||||
* Map one Google sleep session into per-SEGMENT `SLEEP_DURATION` rows plus its
|
||||
* covered window. The Google sleep payload carries a real per-stage segment
|
||||
* series (each with its own start/end), so one row is emitted PER SEGMENT —
|
||||
* `measuredAt = that segment's END`. The timeline is MEASURED (real onsets), so
|
||||
* these rows are NOT flagged reconstructed — unlike WHOOP, which has no onsets.
|
||||
*
|
||||
* Each segment carries a stage-scoped, INDEXED fieldTag so the several segments
|
||||
* of one stage stay distinct under the `(userId, type, source, externalId)`
|
||||
* dedup key. Unknown stage labels are skipped; a session with no parseable
|
||||
* segment yields nothing.
|
||||
* Each segment's fieldTag keys off the STABLE session anchor plus the segment's
|
||||
* own START instant — `<anchor>:sleep:<segment-start>` — NOT a positional index
|
||||
* and NOT the stage label. This is the fix for the re-score duplication: a
|
||||
* positional index renumbered (and a stage-scoped tag changed) whenever Google
|
||||
* re-scored a night, minting fresh externalIds so the upsert created parallel
|
||||
* duplicate rows the night-total then double-counted. A segment's start instant
|
||||
* is stable per block, and dropping the stage from the key means a mere
|
||||
* re-classification (LIGHT→DEEP on the same block) UPDATES the row in place
|
||||
* rather than orphaning it. Unknown stage labels are skipped; a session with no
|
||||
* parseable segment yields an empty `rows` (and a null window).
|
||||
*
|
||||
* Segment timestamps can arrive OFFSET-LESS (local wall clock); `tz` (the user's
|
||||
* stored zone) anchors them to the correct UTC instant rather than the process
|
||||
* zone — without it a non-UTC user's near-midnight segment END would shift by
|
||||
* their offset. Timestamps carrying an explicit offset/`Z` are honoured as-is.
|
||||
*/
|
||||
export function mapSleepSession(
|
||||
export function mapSleepSessionDetailed(
|
||||
point: GoogleHealthDataPoint,
|
||||
tz?: string,
|
||||
): GoogleHealthMappedMeasurement[] {
|
||||
const segments = readSleepSegments(point);
|
||||
if (segments.length === 0) return [];
|
||||
|
||||
): GoogleHealthSleepSession {
|
||||
const anchor = sleepSessionAnchor(point, tz);
|
||||
const out: GoogleHealthMappedMeasurement[] = [];
|
||||
let segIndex = 0;
|
||||
const segments = readSleepSegments(point);
|
||||
const rows: GoogleHealthMappedMeasurement[] = [];
|
||||
let windowStart: Date | null = null;
|
||||
let windowEnd: Date | null = null;
|
||||
|
||||
for (const seg of segments) {
|
||||
const stage = mapGoogleHealthSleepStage(seg.stage);
|
||||
if (!stage) continue;
|
||||
const mins = minutesBetween(seg.startTime, seg.endTime);
|
||||
if (mins === null || !(mins > 0)) continue;
|
||||
const start = parseLocalInstant(seg.startTime as string, tz);
|
||||
const end = parseLocalInstant(seg.endTime as string, tz);
|
||||
if (!end) continue;
|
||||
out.push({
|
||||
if (!start || !end) continue;
|
||||
if (!windowStart || start < windowStart) windowStart = start;
|
||||
if (!windowEnd || end > windowEnd) windowEnd = end;
|
||||
rows.push({
|
||||
type: "SLEEP_DURATION",
|
||||
value: round2(mins),
|
||||
unit: "minutes",
|
||||
measuredAt: end,
|
||||
fieldTag: `${anchor}:sleep_${stage.toLowerCase()}:${segIndex}`,
|
||||
fieldTag: `${anchor}:sleep:${start.toISOString()}`,
|
||||
sleepStage: stage,
|
||||
});
|
||||
segIndex += 1;
|
||||
}
|
||||
return out;
|
||||
|
||||
return { anchor, windowStart, windowEnd, rows };
|
||||
}
|
||||
|
||||
/**
|
||||
* Flat convenience wrapper — the per-segment rows of one session, without the
|
||||
* window metadata. Retained for callers that only need the rows.
|
||||
*/
|
||||
export function mapSleepSession(
|
||||
point: GoogleHealthDataPoint,
|
||||
tz?: string,
|
||||
): GoogleHealthMappedMeasurement[] {
|
||||
return mapSleepSessionDetailed(point, tz).rows;
|
||||
}
|
||||
|
||||
// ── Workouts (exercise sessions) ───────────────────────────────
|
||||
|
||||
@@ -9,11 +9,13 @@
|
||||
* per-segment series, so the rows lay each block at its true clock time (a
|
||||
* MEASURED timeline, not reconstructed). Upserts as `source = GOOGLE_HEALTH`.
|
||||
*
|
||||
* Each segment carries the `sleepStage` axis and an indexed fieldTag so the
|
||||
* several segments of one stage stay distinct under the
|
||||
* `(userId, type, source, externalId)` dedup key. externalId =
|
||||
* `<session-anchor>:sleep_<stage>:<i>` — a re-scored night overwrites in place.
|
||||
* A 24 h overlap covers Google's after-the-fact re-score.
|
||||
* Each segment carries the `sleepStage` axis and a fieldTag keyed on the STABLE
|
||||
* session anchor plus the segment's own start —
|
||||
* `<session-anchor>:sleep:<segment-start>` — so a re-scored night overwrites in
|
||||
* place instead of minting parallel duplicate rows the night-total would then
|
||||
* double-count. A 24 h overlap covers Google's after-the-fact re-score, and
|
||||
* `replaceStaleGoogleHealthSleep` clears anything an earlier scoring left in the
|
||||
* night's window before the fresh set upserts.
|
||||
*
|
||||
* A per-data-class 403 soft-skips the resource — the sleep bundle is granted
|
||||
* independently of activity / metrics in the Google consent flow.
|
||||
@@ -22,14 +24,16 @@ import {
|
||||
GOOGLE_HEALTH_ACTIVITY_PAGE_SIZE,
|
||||
GOOGLE_HEALTH_DATA_TYPES,
|
||||
fetchDataPoints,
|
||||
mapSleepSession,
|
||||
mapSleepSessionDetailed,
|
||||
} from "./client";
|
||||
import {
|
||||
getValidToken,
|
||||
handleCollectionFetchError,
|
||||
replaceStaleGoogleHealthSleep,
|
||||
upsertGoogleHealthMeasurements,
|
||||
type GoogleHealthMeasurementUpsert,
|
||||
type GoogleHealthResourceSyncOptions,
|
||||
type GoogleHealthSleepReplaceWindow,
|
||||
} from "./sync";
|
||||
import { annotate } from "@/lib/logging/context";
|
||||
import { resolveUserTimezone } from "@/lib/tz/resolver";
|
||||
@@ -63,8 +67,11 @@ export async function syncUserSleep(
|
||||
}
|
||||
|
||||
const readings: GoogleHealthMeasurementUpsert[] = [];
|
||||
const replaceWindows: GoogleHealthSleepReplaceWindow[] = [];
|
||||
for (const point of points) {
|
||||
for (const m of mapSleepSession(point, tz)) {
|
||||
const session = mapSleepSessionDetailed(point, tz);
|
||||
if (session.rows.length === 0) continue;
|
||||
for (const m of session.rows) {
|
||||
readings.push({
|
||||
type: m.type,
|
||||
value: m.value,
|
||||
@@ -74,8 +81,18 @@ export async function syncUserSleep(
|
||||
sleepStage: m.sleepStage ?? null,
|
||||
});
|
||||
}
|
||||
// Clean any stale rows a prior re-score left in this night's window before
|
||||
// the fresh set upserts — so a re-scored night reads its true total rather
|
||||
// than the sum of the old and the re-scored copies.
|
||||
replaceWindows.push({
|
||||
windowStart: session.windowStart,
|
||||
windowEnd: session.windowEnd,
|
||||
keepIds: session.rows.map((m) => m.fieldTag),
|
||||
});
|
||||
}
|
||||
|
||||
await replaceStaleGoogleHealthSleep(userId, replaceWindows);
|
||||
|
||||
const imported = (
|
||||
await upsertGoogleHealthMeasurements(userId, readings, {
|
||||
deferRollup: opts.deferRollup,
|
||||
|
||||
@@ -314,6 +314,63 @@ export interface GoogleHealthMeasurementUpsert {
|
||||
/** Chunk size for the batched `createMany` insert of fresh readings. */
|
||||
const GOOGLE_HEALTH_CREATE_CHUNK = 500;
|
||||
|
||||
/** One just-fetched sleep session's window + the ids that must survive it. */
|
||||
export interface GoogleHealthSleepReplaceWindow {
|
||||
/** Earliest segment start (UTC). Null → nothing to clean for this session. */
|
||||
windowStart: Date | null;
|
||||
/** Latest segment end (UTC). */
|
||||
windowEnd: Date | null;
|
||||
/** The fresh externalIds for THIS session — never soft-deleted. */
|
||||
keepIds: string[];
|
||||
}
|
||||
|
||||
/**
|
||||
* Replace-by-window cleanup for re-scored Google sleep sessions.
|
||||
*
|
||||
* Google re-scores a night after the fact. Before the stable-anchor fix a
|
||||
* re-fetch minted fresh externalIds and left the prior night's rows LIVE, so the
|
||||
* night-total silently double-counted (a 7h35 night read as 10h+). For each
|
||||
* just-fetched session this soft-deletes any LIVE `GOOGLE_HEALTH`
|
||||
* `SLEEP_DURATION` row whose `measuredAt` falls inside the session's
|
||||
* `[windowStart, windowEnd]` but was NOT re-produced by this fetch (`keepIds`
|
||||
* are this session's fresh externalIds). Sleep sessions do not overlap in time,
|
||||
* so a scan bounded to one session's window only ever touches that session's own
|
||||
* rows — a fresh row is protected by `keepIds`, and any leftover (old volatile
|
||||
* key, or a segment Google dropped) is cleared. Rows OUTSIDE every returned
|
||||
* window are never touched, so a night Google did not re-report this tick stays
|
||||
* intact — no data loss, only stale duplicates and re-score orphans go. This is
|
||||
* also the repair path: a full backfill re-fetches history and cleans each night
|
||||
* as it goes. Best-effort — a cleanup failure never fails the user's sync.
|
||||
*/
|
||||
export async function replaceStaleGoogleHealthSleep(
|
||||
userId: string,
|
||||
sessions: GoogleHealthSleepReplaceWindow[],
|
||||
): Promise<number> {
|
||||
let removed = 0;
|
||||
for (const s of sessions) {
|
||||
if (!s.windowStart || !s.windowEnd || s.keepIds.length === 0) continue;
|
||||
try {
|
||||
const res = await prisma.measurement.updateMany({
|
||||
where: {
|
||||
userId,
|
||||
source: "GOOGLE_HEALTH",
|
||||
type: "SLEEP_DURATION",
|
||||
deletedAt: null,
|
||||
measuredAt: { gte: s.windowStart, lte: s.windowEnd },
|
||||
externalId: { notIn: s.keepIds },
|
||||
},
|
||||
data: { deletedAt: new Date() },
|
||||
});
|
||||
removed += res.count;
|
||||
} catch (err) {
|
||||
getEvent()?.addWarning(
|
||||
`google-health: sleep replace-by-window failed: ${err}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
return removed;
|
||||
}
|
||||
|
||||
/**
|
||||
* Write a batch of mapped Google Health readings for one user and (unless
|
||||
* deferred) fold the rollup tier + invalidate status-insight caches once at the
|
||||
|
||||
Reference in new issue
Block a user