mirror of
https://github.com/MBombeck/HealthLog.git
synced 2026-11-11 22:02:16 +00:00
feat(ops): repair walking distances stored a thousand times too small
The unit fix only helps the next import. An account that imported an archive before it carries a decade of days understated 1000x, and the authoritative repair — re-importing the archive on the fixed build — needs an archive the account may no longer have. `scripts/repair-apple-health-distance.ts` is dry-run by default and selects rows only by the stamp the archive fold itself writes: the EXPORT_XML_SOURCE_MAX provenance plus the `stats:` external id minted for that fold. Never by "the value looks small", so a row a native sync has since overwritten is left alone. `--apply` multiplies one account per transaction and writes the audit row in the same transaction, which is what makes a second run a no-op. The archive's own unit is named by the operator (`--unit=mi`) because the discarded attribute is the defect itself, and a row whose repaired value would leave the plausible range is reported rather than written. Refs #944
This commit is contained in:
1 parent
5c7c7afd59
commit
bc7eff8322
5 files changed
+739
No files matched your search
@@ -0,0 +1,121 @@
|
||||
# Apple Health walking-distance repair
|
||||
|
||||
`scripts/repair-apple-health-distance.ts` heals walking distances that the
|
||||
`export.zip` import stored a thousand times too small.
|
||||
|
||||
## The defect
|
||||
|
||||
Apple stamps every quantity `<Record>` in an `export.xml` with the account's
|
||||
own display unit: a metric archive reports `HKQuantityTypeIdentifierDistance-
|
||||
WalkingRunning` in `km`, an imperial one in `mi`. Until this release the record
|
||||
mapping accepted that attribute and never read it, so the day's total was
|
||||
folded as if the number were already metres. A 2.484 km day landed as
|
||||
`2.484 m`, and an imported decade reads as a couple of metres walked. The
|
||||
workout path was never affected — it converted its own distance unit.
|
||||
|
||||
The code fix converts the record's unit before storage, so every import from
|
||||
this release on is correct. This script is for the rows already in the
|
||||
database.
|
||||
|
||||
## Prefer a re-import
|
||||
|
||||
If the account still has its `export.zip`, **re-import it**. The archive fold
|
||||
UPSERTs the same `stats:<identifier>:<day>` rows, so a re-import on the fixed
|
||||
build rewrites every affected day with the right number, needs no flags, and
|
||||
also repairs rows this script deliberately leaves alone (see the criterion
|
||||
below). The script exists for accounts that no longer have the archive.
|
||||
|
||||
## Which rows it touches
|
||||
|
||||
Only rows carrying the stamp the archive importer itself writes:
|
||||
|
||||
| Column | Value |
|
||||
| ------------------------ | ------------------------------------------------------------ |
|
||||
| `type` | `WALKING_RUNNING_DISTANCE` |
|
||||
| `source` | `APPLE_HEALTH` |
|
||||
| `aggregation_provenance` | `EXPORT_XML_SOURCE_MAX` |
|
||||
| `external_id` | `stats:HKQuantityTypeIdentifierDistanceWalkingRunning:<day>` |
|
||||
| `deleted_at` | `NULL` |
|
||||
|
||||
`EXPORT_XML_SOURCE_MAX` is written in exactly one place in the tree — the
|
||||
archive fold — and the `stats:` external id is minted for that same fold, so
|
||||
the pair is a proof of origin rather than a guess. Nothing is ever selected
|
||||
because a value "looks too small".
|
||||
|
||||
Deliberately out of scope:
|
||||
|
||||
- Rows a native iOS sync has since overwritten (`HEALTHKIT_STATISTICS`): their
|
||||
value came from HealthKit's own statistics query and is correct.
|
||||
- Rows whose provenance is `LEGACY_UNKNOWN` or NULL: an archive origin cannot
|
||||
be proven for them. Re-import the archive to repair those.
|
||||
|
||||
## Run a dry-run first
|
||||
|
||||
Always. The default mode reports and never writes:
|
||||
|
||||
```bash
|
||||
# from a source checkout, with DATABASE_URL pointing at the instance
|
||||
pnpm dlx tsx scripts/repair-apple-health-distance.ts
|
||||
```
|
||||
|
||||
It prints one line per account: how many rows would be repaired, the date span
|
||||
they cover, one worked example (`2.484 m -> 2484 m`), and any row whose
|
||||
repaired value would leave the plausibility range.
|
||||
|
||||
This script imports the application dependency graph, so it runs from a source
|
||||
checkout, not through the production image's `healthlog-tsx` launcher.
|
||||
|
||||
## Name the archive's unit
|
||||
|
||||
The stored row keeps no record of the unit its archive was written in — that
|
||||
is the defect — so the operator names it. `--unit=km` is the default and is
|
||||
what the Health app writes in every metric locale; pass `--unit=mi` for an
|
||||
imperial archive:
|
||||
|
||||
```bash
|
||||
pnpm dlx tsx scripts/repair-apple-health-distance.ts --unit=mi
|
||||
```
|
||||
|
||||
Check the dry-run's worked example against what the account's Health app shows
|
||||
for that day before applying. If the two units are mixed across accounts, run
|
||||
the script once per unit with `--apply` and confirm each account's example
|
||||
first.
|
||||
|
||||
## Apply
|
||||
|
||||
```bash
|
||||
pnpm dlx tsx scripts/repair-apple-health-distance.ts --apply
|
||||
```
|
||||
|
||||
Per account, in one transaction: every selected row multiplied by the unit's
|
||||
metre factor, `sync_version` incremented and `updated_at` bumped so paired iOS
|
||||
clients pick the rows up on their next delta sync, and an audit row
|
||||
(`measurement.apple_health_distance.repaired`) recording the unit, the factor
|
||||
and the row count. The DAY/WEEK/MONTH/YEAR rollups and the cached status
|
||||
insights for the touched days are recomputed afterwards.
|
||||
|
||||
A row whose repaired value would leave the plausible range (0–200 000 m) is
|
||||
reported and never written — several of those in one account mean the rows are
|
||||
not the 1000x class and the run should be stopped.
|
||||
|
||||
## It runs once per account
|
||||
|
||||
The audit row is written inside the same transaction as the update, and an
|
||||
account that carries it is skipped. A second run reports the account as already
|
||||
repaired and changes nothing, so the multiplier can never be applied twice.
|
||||
|
||||
## Verify
|
||||
|
||||
```sql
|
||||
SELECT date_trunc('year', measured_at) AS year,
|
||||
round(avg(value)) AS avg_metres_per_day
|
||||
FROM measurements
|
||||
WHERE user_id = '<userId>'
|
||||
AND type = 'WALKING_RUNNING_DISTANCE'
|
||||
AND aggregation_provenance = 'EXPORT_XML_SOURCE_MAX'
|
||||
GROUP BY 1 ORDER BY 1;
|
||||
```
|
||||
|
||||
A plausible daily average is thousands of metres. Single digits mean the rows
|
||||
were not repaired; millions mean a factor was applied twice, which the audit
|
||||
row is there to prevent.
|
||||
@@ -24,6 +24,7 @@ typecheck.
|
||||
- **`assert-deploy.ts`** — post-deploy `/api/version` assertion.
|
||||
- **`rotate-encryption-key.ts`** — AES-256-GCM key rotation; see `docs/ops/encryption-key-rotation.md`.
|
||||
- **`restore-backup.ts`** — restore an off-host AES-GCM backup from an S3-compatible bucket.
|
||||
- **`repair-apple-health-distance.ts`** — repair walking-distance rows the `export.zip` import stored 1000x too small; dry-run by default, see `docs/ops/apple-health-distance-repair.md`.
|
||||
- **`repair-intake-anomalies.ts`** — repair historic medication-ledger defects (duplicate dose-slot rows, implausible `taken_at`); dry-run by default, see `docs/ops/intake-repair.md`.
|
||||
- **`backfill-rollups.ts`**, **`drain-per-sample-cumulative.ts`**, **`backfill-mood-note-column.ts`** — historical data backfills (note: the boot-time `rollup-full-backfill` queue handles new accounts automatically; the CLI is the manual fallback).
|
||||
- **`seed-demo.ts`** + **`seed-demo-v15.sql`** — populate a demo tenant.
|
||||
|
||||
@@ -0,0 +1,138 @@
|
||||
/**
|
||||
* scripts/repair-apple-health-distance.ts — operator repair for walking
|
||||
* distances the `export.zip` import stored 1000x too small (issue #944).
|
||||
*
|
||||
* THE DEFECT (fixed in code from this release): the `<Record>` mapping read
|
||||
* the value but never the record's own `unit` attribute, so a walking-distance
|
||||
* record Apple writes in the account's display unit (`km` on a metric archive,
|
||||
* `mi` on an imperial one) was folded into the day's total as if the number
|
||||
* were already metres. Ten years of history read as a couple of metres a day.
|
||||
*
|
||||
* WHICH ROWS: only the ones carrying the stamp the archive importer writes —
|
||||
* `WALKING_RUNNING_DISTANCE` + `APPLE_HEALTH` + provenance
|
||||
* `EXPORT_XML_SOURCE_MAX` + a `stats:HKQuantityTypeIdentifierDistanceWalking-
|
||||
* Running:<day>` externalId. Never a guess from "the value looks small". Rows
|
||||
* a native iOS sync has since overwritten carry `HEALTHKIT_STATISTICS` and are
|
||||
* correct; rows whose provenance cannot prove an archive origin are left alone.
|
||||
*
|
||||
* If the account still has its `export.zip`, RE-IMPORTING it on the fixed build
|
||||
* is the authoritative repair — the fold UPSERTs the same `stats:` rows with
|
||||
* the right numbers. This script is for the accounts that no longer have it.
|
||||
*
|
||||
* SAFETY:
|
||||
* - Dry-run by DEFAULT: lists the candidate rows per account and exits.
|
||||
* - `--apply` writes, one transaction per account, and records an audit row
|
||||
* in that same transaction. An account carrying that row is skipped, so a
|
||||
* second run finds nothing.
|
||||
* - A row whose repaired value would leave the plausibility range is
|
||||
* reported and never written.
|
||||
* - `--unit=mi` for an imperial archive (default `km`). The stored row keeps
|
||||
* no record of the archive's unit — that is the defect — so the operator
|
||||
* names it; a mismatched unit is why the flag exists rather than a guess.
|
||||
*
|
||||
* RUN (never `pnpm tsx` — the standalone image strips tsx):
|
||||
* pnpm dlx tsx scripts/repair-apple-health-distance.ts # dry run
|
||||
* pnpm dlx tsx scripts/repair-apple-health-distance.ts --apply
|
||||
* pnpm dlx tsx scripts/repair-apple-health-distance.ts --unit=mi --apply
|
||||
*/
|
||||
import "dotenv/config";
|
||||
|
||||
import { prisma } from "@/lib/db";
|
||||
import {
|
||||
applyAppleHealthDistanceRepair,
|
||||
parseArchiveUnit,
|
||||
planAppleHealthDistanceRepair,
|
||||
repairFactorFor,
|
||||
} from "@/lib/measurements/repair-apple-health-distance";
|
||||
|
||||
function readArchiveUnit(): "km" | "mi" {
|
||||
const flag = process.argv.find((arg) => arg.startsWith("--unit="));
|
||||
if (!flag) return "km";
|
||||
const parsed = parseArchiveUnit(flag.slice("--unit=".length));
|
||||
if (!parsed) {
|
||||
throw new Error(`Unsupported --unit value: ${flag}. Use km or mi.`);
|
||||
}
|
||||
return parsed;
|
||||
}
|
||||
|
||||
async function main(): Promise<void> {
|
||||
const apply = process.argv.includes("--apply");
|
||||
const archiveUnit = readArchiveUnit();
|
||||
const factor = repairFactorFor(archiveUnit);
|
||||
|
||||
const plans = await planAppleHealthDistanceRepair(prisma, { archiveUnit });
|
||||
if (plans.length === 0) {
|
||||
console.log(
|
||||
"No export-imported walking-distance rows found — nothing to repair.",
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
console.log(
|
||||
`Archive unit ${archiveUnit} (x${factor} to metres). ` +
|
||||
`${plans.length} account(s) carry export-imported walking distances.\n`,
|
||||
);
|
||||
|
||||
let totalRepairable = 0;
|
||||
let totalUpdated = 0;
|
||||
let totalSkipped = 0;
|
||||
|
||||
for (const plan of plans) {
|
||||
if (plan.alreadyRepairedAt) {
|
||||
console.log(
|
||||
` • ${plan.userId}: already repaired on ` +
|
||||
`${plan.alreadyRepairedAt.toISOString()} — ${plan.candidateCount} row(s) left alone.`,
|
||||
);
|
||||
continue;
|
||||
}
|
||||
|
||||
totalRepairable += plan.repairable.length;
|
||||
const first = plan.repairable[0];
|
||||
const last = plan.repairable[plan.repairable.length - 1];
|
||||
console.log(
|
||||
` • ${plan.userId}: ${plan.repairable.length} row(s)` +
|
||||
(first && last
|
||||
? ` from ${first.measuredAt.toISOString().slice(0, 10)} to ` +
|
||||
`${last.measuredAt.toISOString().slice(0, 10)}`
|
||||
: "") +
|
||||
(plan.outOfRange.length > 0
|
||||
? `, ${plan.outOfRange.length} out of plausible range (skipped)`
|
||||
: ""),
|
||||
);
|
||||
if (first) {
|
||||
console.log(
|
||||
` e.g. ${first.id}: ${first.value} m -> ${first.value * factor} m`,
|
||||
);
|
||||
}
|
||||
for (const row of plan.outOfRange) {
|
||||
console.log(
|
||||
` ! ${row.id} (${row.measuredAt.toISOString().slice(0, 10)}): ` +
|
||||
`${row.value} m x ${factor} leaves the plausible range — not written`,
|
||||
);
|
||||
}
|
||||
|
||||
if (!apply) continue;
|
||||
const outcome = await applyAppleHealthDistanceRepair(prisma, plan, {
|
||||
archiveUnit,
|
||||
});
|
||||
totalUpdated += outcome.updated;
|
||||
totalSkipped += outcome.skipped;
|
||||
console.log(
|
||||
` ✓ repaired ${outcome.updated} row(s), skipped ${outcome.skipped}`,
|
||||
);
|
||||
}
|
||||
|
||||
console.log(
|
||||
apply
|
||||
? `\nDone. ${totalUpdated} row(s) repaired, ${totalSkipped} skipped.`
|
||||
: `\nDry run: ${totalRepairable} row(s) would be repaired. ` +
|
||||
"Re-run with --apply to write.",
|
||||
);
|
||||
}
|
||||
|
||||
main()
|
||||
.catch((err) => {
|
||||
console.error(err);
|
||||
process.exitCode = 1;
|
||||
})
|
||||
.finally(() => prisma.$disconnect());
|
||||
@@ -0,0 +1,248 @@
|
||||
/**
|
||||
* Repair for walking distances the `export.zip` import stored 1000x too
|
||||
* small (issue #944).
|
||||
*
|
||||
* THE DEFECT (fixed in code from this release): `mapAppleHealthEntry()`
|
||||
* accepted the `<Record unit="…">` attribute and never read it, so a
|
||||
* `HKQuantityTypeIdentifierDistanceWalkingRunning` record — which Apple
|
||||
* writes in the account's own display unit, `km` on a metric archive —
|
||||
* was folded into the day's total as if the number were already metres.
|
||||
* Every day of an imported history reads as a couple of metres walked.
|
||||
*
|
||||
* THE CODE FIX converts the record's own unit into the table's `hkUnit`
|
||||
* before storage, so every FUTURE import is right. This module heals what
|
||||
* is already in the database.
|
||||
*
|
||||
* ── The criterion ──────────────────────────────────────────────────
|
||||
*
|
||||
* A row is a candidate only when it carries the stamp the export importer
|
||||
* itself writes, never because its value "looks small":
|
||||
*
|
||||
* type = WALKING_RUNNING_DISTANCE
|
||||
* source = APPLE_HEALTH
|
||||
* aggregationProvenance = EXPORT_XML_SOURCE_MAX
|
||||
* externalId LIKE stats:HKQuantityTypeIdentifierDistanceWalkingRunning:%
|
||||
* deletedAt IS NULL
|
||||
*
|
||||
* `EXPORT_XML_SOURCE_MAX` is written in exactly one place — the archive
|
||||
* fold in `import-apple-health-export.ts` — and the `stats:<HK>:<day>`
|
||||
* externalId is minted by `dailyStatsExternalId()` for that same fold. A
|
||||
* row a native iOS sync has since overwritten carries
|
||||
* `HEALTHKIT_STATISTICS` and its value came from HealthKit's own
|
||||
* statistics query, so it is correct and out of scope. A row whose
|
||||
* provenance is `LEGACY_UNKNOWN` or NULL cannot be PROVEN to have come
|
||||
* from the archive path and is therefore left alone: re-importing the
|
||||
* archive on the fixed build repairs those, and is the authoritative
|
||||
* repair for any account that still has its `export.zip`.
|
||||
*
|
||||
* ── Idempotency ────────────────────────────────────────────────────
|
||||
*
|
||||
* Applying the repair writes one `AuditLog` row per account inside the
|
||||
* same transaction as the update. An account that already carries that
|
||||
* row is skipped, so a second run finds nothing to do — the multiplier
|
||||
* can never be applied twice.
|
||||
*/
|
||||
import type { PrismaClient } from "@/generated/prisma/client";
|
||||
import { auditLog } from "@/lib/auth/audit";
|
||||
import { afterMeasurementMutation } from "@/lib/rollups/after-measurement-mutation";
|
||||
import { validateMeasurementRange } from "@/lib/validations/measurement";
|
||||
|
||||
import { dailyStatsExternalId } from "./apple-health-mapping";
|
||||
import { convertHkValue } from "./hk-units";
|
||||
|
||||
/** The audit action that marks an account's distances as repaired. */
|
||||
export const APPLE_HEALTH_DISTANCE_REPAIR_ACTION =
|
||||
"measurement.apple_health_distance.repaired";
|
||||
|
||||
/** Minted by the same helper the export fold uses, so the two cannot drift. */
|
||||
export const EXPORT_DISTANCE_EXTERNAL_ID_PREFIX = dailyStatsExternalId(
|
||||
"HKQuantityTypeIdentifierDistanceWalkingRunning",
|
||||
"",
|
||||
);
|
||||
|
||||
/**
|
||||
* The unit the affected archive was written in. Apple writes `km` on a
|
||||
* metric account and `mi` on an imperial one, and the stored row keeps no
|
||||
* record of which — the importer discarded the attribute, which is the
|
||||
* defect. The operator names it; `km` is the default because it is what
|
||||
* the Health app writes for every metric locale.
|
||||
*/
|
||||
export type ArchiveDistanceUnit = "km" | "mi";
|
||||
|
||||
export interface RepairCandidateRow {
|
||||
id: string;
|
||||
measuredAt: Date;
|
||||
value: number;
|
||||
}
|
||||
|
||||
export interface AccountRepairPlan {
|
||||
userId: string;
|
||||
/** Candidate rows that would be multiplied. */
|
||||
repairable: RepairCandidateRow[];
|
||||
/**
|
||||
* Candidates whose repaired value would leave the plausibility range
|
||||
* (`validateMeasurementRange`) — reported, never written. A row here is
|
||||
* a sign the account's rows are NOT the 1000x class.
|
||||
*/
|
||||
outOfRange: RepairCandidateRow[];
|
||||
/** Non-null when this account already carries the repair's audit row. */
|
||||
alreadyRepairedAt: Date | null;
|
||||
/** Candidate rows found, whatever the disposition above. */
|
||||
candidateCount: number;
|
||||
}
|
||||
|
||||
/** The DB surface this module needs — the script hands it the real client. */
|
||||
export type RepairClient = Pick<
|
||||
PrismaClient,
|
||||
"measurement" | "auditLog" | "$transaction"
|
||||
>;
|
||||
|
||||
const REPAIRED_TYPE = "WALKING_RUNNING_DISTANCE" as const;
|
||||
|
||||
/** metres per unit of the archive's own length unit. */
|
||||
export function repairFactorFor(unit: ArchiveDistanceUnit): number {
|
||||
const factor = convertHkValue(1, unit, "m");
|
||||
// The two archive units are length units by construction; the shared
|
||||
// table is the single source of the factor either way.
|
||||
if (factor === null) throw new Error(`no metre factor for unit "${unit}"`);
|
||||
return factor;
|
||||
}
|
||||
|
||||
/**
|
||||
* List the repair candidates per account. Read-only — this is what the
|
||||
* script's dry run prints.
|
||||
*/
|
||||
export async function planAppleHealthDistanceRepair(
|
||||
client: RepairClient,
|
||||
options: { archiveUnit: ArchiveDistanceUnit },
|
||||
): Promise<AccountRepairPlan[]> {
|
||||
const factor = repairFactorFor(options.archiveUnit);
|
||||
|
||||
const rows = await client.measurement.findMany({
|
||||
where: {
|
||||
type: REPAIRED_TYPE,
|
||||
source: "APPLE_HEALTH",
|
||||
aggregationProvenance: "EXPORT_XML_SOURCE_MAX",
|
||||
externalId: { startsWith: EXPORT_DISTANCE_EXTERNAL_ID_PREFIX },
|
||||
deletedAt: null,
|
||||
},
|
||||
select: { id: true, userId: true, value: true, measuredAt: true },
|
||||
orderBy: [{ userId: "asc" }, { measuredAt: "asc" }],
|
||||
});
|
||||
if (rows.length === 0) return [];
|
||||
|
||||
const userIds = Array.from(new Set(rows.map((row) => row.userId)));
|
||||
const repaired = await client.auditLog.findMany({
|
||||
where: {
|
||||
action: APPLE_HEALTH_DISTANCE_REPAIR_ACTION,
|
||||
userId: { in: userIds },
|
||||
},
|
||||
select: { userId: true, createdAt: true },
|
||||
orderBy: { createdAt: "asc" },
|
||||
});
|
||||
const repairedAt = new Map<string, Date>();
|
||||
for (const entry of repaired) {
|
||||
if (entry.userId) repairedAt.set(entry.userId, entry.createdAt);
|
||||
}
|
||||
|
||||
const plans = new Map<string, AccountRepairPlan>();
|
||||
for (const row of rows) {
|
||||
let plan = plans.get(row.userId);
|
||||
if (!plan) {
|
||||
plan = {
|
||||
userId: row.userId,
|
||||
repairable: [],
|
||||
outOfRange: [],
|
||||
alreadyRepairedAt: repairedAt.get(row.userId) ?? null,
|
||||
candidateCount: 0,
|
||||
};
|
||||
plans.set(row.userId, plan);
|
||||
}
|
||||
plan.candidateCount += 1;
|
||||
if (plan.alreadyRepairedAt) continue;
|
||||
const candidate: RepairCandidateRow = {
|
||||
id: row.id,
|
||||
measuredAt: row.measuredAt,
|
||||
value: row.value,
|
||||
};
|
||||
if (validateMeasurementRange(REPAIRED_TYPE, row.value * factor) !== null) {
|
||||
plan.outOfRange.push(candidate);
|
||||
} else {
|
||||
plan.repairable.push(candidate);
|
||||
}
|
||||
}
|
||||
return Array.from(plans.values());
|
||||
}
|
||||
|
||||
export interface RepairOutcome {
|
||||
userId: string;
|
||||
updated: number;
|
||||
skipped: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Apply one account's plan: every repairable row multiplied by the
|
||||
* archive unit's metre factor in ONE transaction, together with the audit
|
||||
* row that makes a second run a no-op. Returns what it did.
|
||||
*/
|
||||
export async function applyAppleHealthDistanceRepair(
|
||||
client: RepairClient,
|
||||
plan: AccountRepairPlan,
|
||||
options: { archiveUnit: ArchiveDistanceUnit },
|
||||
): Promise<RepairOutcome> {
|
||||
const factor = repairFactorFor(options.archiveUnit);
|
||||
const skipped = plan.outOfRange.length;
|
||||
if (plan.alreadyRepairedAt || plan.repairable.length === 0) {
|
||||
return { userId: plan.userId, updated: 0, skipped };
|
||||
}
|
||||
|
||||
const ids = plan.repairable.map((row) => row.id);
|
||||
await client.$transaction(async (tx) => {
|
||||
// One statement per chunk rather than one per row: an account with a
|
||||
// decade of history carries thousands of rows and an interactive
|
||||
// transaction has a clock on it. `sync_version` + `updated_at` are
|
||||
// bumped by hand because raw SQL bypasses Prisma's `@updatedAt`, and
|
||||
// the iOS delta feed pages on exactly those two columns.
|
||||
for (let i = 0; i < ids.length; i += 500) {
|
||||
const chunk = ids.slice(i, i + 500);
|
||||
await tx.$executeRaw`
|
||||
UPDATE measurements
|
||||
SET value = value * ${factor},
|
||||
unit = 'm',
|
||||
sync_version = sync_version + 1,
|
||||
updated_at = NOW()
|
||||
WHERE id = ANY(${chunk})`;
|
||||
}
|
||||
await auditLog(APPLE_HEALTH_DISTANCE_REPAIR_ACTION, {
|
||||
client: tx,
|
||||
userId: plan.userId,
|
||||
actorUserId: null,
|
||||
details: {
|
||||
type: REPAIRED_TYPE,
|
||||
archiveUnit: options.archiveUnit,
|
||||
factor,
|
||||
rows: ids.length,
|
||||
skippedOutOfRange: skipped,
|
||||
issue: 944,
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
// Best-effort tail: the DAY/WEEK/MONTH/YEAR rollups and the cached
|
||||
// status insights hold the pre-repair numbers until they are recomputed.
|
||||
await afterMeasurementMutation(
|
||||
plan.userId,
|
||||
plan.repairable.map((row) => ({
|
||||
type: REPAIRED_TYPE,
|
||||
measuredAt: row.measuredAt,
|
||||
})),
|
||||
"repair-apple-health-distance",
|
||||
);
|
||||
|
||||
return { userId: plan.userId, updated: ids.length, skipped };
|
||||
}
|
||||
|
||||
/** Narrowing helper for the script's `--unit` flag. */
|
||||
export function parseArchiveUnit(raw: string): ArchiveDistanceUnit | null {
|
||||
return raw === "km" || raw === "mi" ? raw : null;
|
||||
}
|
||||
@@ -0,0 +1,231 @@
|
||||
/**
|
||||
* Integration coverage for the export-import walking-distance repair
|
||||
* (issue #944).
|
||||
*
|
||||
* The defect stored every archive-imported `WALKING_RUNNING_DISTANCE` row a
|
||||
* thousand times too small. `src/lib/measurements/repair-apple-health-distance.ts`
|
||||
* heals what is already in the database, and everything that matters about it
|
||||
* is a database property: which rows the criterion actually selects, that the
|
||||
* multiply lands in one transaction with its audit row, and that a second run
|
||||
* finds nothing. None of that can be proven against a mock, so this runs
|
||||
* against the testcontainer Postgres (Docker / OrbStack; `pnpm test:integration`).
|
||||
*/
|
||||
import { beforeEach, describe, expect, it } from "vitest";
|
||||
|
||||
import { getPrismaClient, truncateAllTables } from "./setup";
|
||||
import {
|
||||
APPLE_HEALTH_DISTANCE_REPAIR_ACTION,
|
||||
applyAppleHealthDistanceRepair,
|
||||
planAppleHealthDistanceRepair,
|
||||
} from "@/lib/measurements/repair-apple-health-distance";
|
||||
import { settleBackgroundTasks } from "@/lib/logging/background-tasks";
|
||||
import type {
|
||||
MeasurementAggregationProvenance,
|
||||
MeasurementSource,
|
||||
MeasurementType,
|
||||
PrismaClient,
|
||||
} from "@/generated/prisma/client";
|
||||
|
||||
const DISTANCE_PREFIX = "stats:HKQuantityTypeIdentifierDistanceWalkingRunning:";
|
||||
|
||||
let dayCounter = 0;
|
||||
|
||||
async function seedRow(
|
||||
prisma: PrismaClient,
|
||||
userId: string,
|
||||
overrides: {
|
||||
value: number;
|
||||
type?: MeasurementType;
|
||||
source?: MeasurementSource;
|
||||
provenance?: MeasurementAggregationProvenance | null;
|
||||
externalId?: string;
|
||||
},
|
||||
): Promise<string> {
|
||||
dayCounter += 1;
|
||||
const measuredAt = new Date(Date.UTC(2026, 0, dayCounter, 12, 0, 0));
|
||||
const day = measuredAt.toISOString().slice(0, 10);
|
||||
const row = await prisma.measurement.create({
|
||||
data: {
|
||||
userId,
|
||||
type: overrides.type ?? "WALKING_RUNNING_DISTANCE",
|
||||
value: overrides.value,
|
||||
unit: "m",
|
||||
source: overrides.source ?? "APPLE_HEALTH",
|
||||
measuredAt,
|
||||
externalId: overrides.externalId ?? `${DISTANCE_PREFIX}${day}`,
|
||||
aggregationProvenance:
|
||||
overrides.provenance === undefined
|
||||
? "EXPORT_XML_SOURCE_MAX"
|
||||
: overrides.provenance,
|
||||
},
|
||||
select: { id: true },
|
||||
});
|
||||
return row.id;
|
||||
}
|
||||
|
||||
async function readRow(prisma: PrismaClient, id: string) {
|
||||
return prisma.measurement.findUniqueOrThrow({
|
||||
where: { id },
|
||||
select: { value: true, unit: true, syncVersion: true },
|
||||
});
|
||||
}
|
||||
|
||||
async function makeUser(prisma: PrismaClient, tag: string): Promise<string> {
|
||||
const user = await prisma.user.create({
|
||||
data: {
|
||||
username: `distance-repair-${tag}`,
|
||||
email: `distance-repair-${tag}@example.test`,
|
||||
role: "USER",
|
||||
},
|
||||
});
|
||||
return user.id;
|
||||
}
|
||||
|
||||
describe("Apple Health export distance repair — integration", () => {
|
||||
beforeEach(async () => {
|
||||
await truncateAllTables(getPrismaClient());
|
||||
dayCounter = 0;
|
||||
});
|
||||
|
||||
it("selects only rows the archive importer stamped, and repairs them once", async () => {
|
||||
const prisma = getPrismaClient();
|
||||
const userId = await makeUser(prisma, "km");
|
||||
|
||||
const kmDay = await seedRow(prisma, userId, { value: 2.484 });
|
||||
const otherDay = await seedRow(prisma, userId, { value: 7.02 });
|
||||
// Decoys, one per arm of the criterion.
|
||||
const nativeStats = await seedRow(prisma, userId, {
|
||||
value: 8123,
|
||||
provenance: "HEALTHKIT_STATISTICS",
|
||||
});
|
||||
const legacyProvenance = await seedRow(prisma, userId, {
|
||||
value: 3.4,
|
||||
provenance: null,
|
||||
});
|
||||
const manualRow = await seedRow(prisma, userId, {
|
||||
value: 4.1,
|
||||
source: "MANUAL",
|
||||
});
|
||||
const otherType = await seedRow(prisma, userId, {
|
||||
value: 9000,
|
||||
type: "ACTIVITY_STEPS",
|
||||
externalId: "stats:HKQuantityTypeIdentifierStepCount:2026-01-09",
|
||||
});
|
||||
const perSample = await seedRow(prisma, userId, {
|
||||
value: 1.2,
|
||||
externalId: "sample:abc123",
|
||||
});
|
||||
|
||||
const plans = await planAppleHealthDistanceRepair(prisma, {
|
||||
archiveUnit: "km",
|
||||
});
|
||||
expect(plans).toHaveLength(1);
|
||||
expect(plans[0].userId).toBe(userId);
|
||||
expect(plans[0].repairable.map((row) => row.id).sort()).toEqual(
|
||||
[kmDay, otherDay].sort(),
|
||||
);
|
||||
expect(plans[0].alreadyRepairedAt).toBeNull();
|
||||
|
||||
const outcome = await applyAppleHealthDistanceRepair(prisma, plans[0], {
|
||||
archiveUnit: "km",
|
||||
});
|
||||
await settleBackgroundTasks();
|
||||
expect(outcome).toMatchObject({ userId, updated: 2, skipped: 0 });
|
||||
|
||||
expect((await readRow(prisma, kmDay)).value).toBeCloseTo(2484, 6);
|
||||
expect((await readRow(prisma, otherDay)).value).toBeCloseTo(7020, 6);
|
||||
// The delta feed pages on sync_version / updated_at, so a repaired row
|
||||
// has to look mutated to a paired client.
|
||||
expect((await readRow(prisma, kmDay)).syncVersion).toBe(2);
|
||||
expect((await readRow(prisma, kmDay)).unit).toBe("m");
|
||||
|
||||
// Every decoy untouched.
|
||||
expect((await readRow(prisma, nativeStats)).value).toBe(8123);
|
||||
expect((await readRow(prisma, legacyProvenance)).value).toBe(3.4);
|
||||
expect((await readRow(prisma, manualRow)).value).toBe(4.1);
|
||||
expect((await readRow(prisma, otherType)).value).toBe(9000);
|
||||
expect((await readRow(prisma, perSample)).value).toBe(1.2);
|
||||
|
||||
const audit = await prisma.auditLog.findMany({
|
||||
where: { action: APPLE_HEALTH_DISTANCE_REPAIR_ACTION, userId },
|
||||
});
|
||||
expect(audit).toHaveLength(1);
|
||||
expect(JSON.parse(audit[0].details ?? "{}")).toMatchObject({
|
||||
archiveUnit: "km",
|
||||
factor: 1000,
|
||||
rows: 2,
|
||||
issue: 944,
|
||||
});
|
||||
});
|
||||
|
||||
it("is idempotent — a second run finds nothing to repair", async () => {
|
||||
const prisma = getPrismaClient();
|
||||
const userId = await makeUser(prisma, "twice");
|
||||
const rowId = await seedRow(prisma, userId, { value: 2.484 });
|
||||
|
||||
const first = await planAppleHealthDistanceRepair(prisma, {
|
||||
archiveUnit: "km",
|
||||
});
|
||||
await applyAppleHealthDistanceRepair(prisma, first[0], {
|
||||
archiveUnit: "km",
|
||||
});
|
||||
await settleBackgroundTasks();
|
||||
|
||||
const second = await planAppleHealthDistanceRepair(prisma, {
|
||||
archiveUnit: "km",
|
||||
});
|
||||
expect(second).toHaveLength(1);
|
||||
expect(second[0].repairable).toHaveLength(0);
|
||||
expect(second[0].candidateCount).toBe(1);
|
||||
expect(second[0].alreadyRepairedAt).toBeInstanceOf(Date);
|
||||
|
||||
const outcome = await applyAppleHealthDistanceRepair(prisma, second[0], {
|
||||
archiveUnit: "km",
|
||||
});
|
||||
expect(outcome.updated).toBe(0);
|
||||
expect((await readRow(prisma, rowId)).value).toBeCloseTo(2484, 6);
|
||||
expect((await readRow(prisma, rowId)).syncVersion).toBe(2);
|
||||
});
|
||||
|
||||
it("uses the archive's own unit and never writes an implausible value", async () => {
|
||||
const prisma = getPrismaClient();
|
||||
const userId = await makeUser(prisma, "mi");
|
||||
const mile = await seedRow(prisma, userId, { value: 1.543 });
|
||||
const absurd = await seedRow(prisma, userId, { value: 250 });
|
||||
|
||||
const plans = await planAppleHealthDistanceRepair(prisma, {
|
||||
archiveUnit: "mi",
|
||||
});
|
||||
expect(plans[0].repairable.map((row) => row.id)).toEqual([mile]);
|
||||
expect(plans[0].outOfRange.map((row) => row.id)).toEqual([absurd]);
|
||||
|
||||
const outcome = await applyAppleHealthDistanceRepair(prisma, plans[0], {
|
||||
archiveUnit: "mi",
|
||||
});
|
||||
await settleBackgroundTasks();
|
||||
expect(outcome).toMatchObject({ updated: 1, skipped: 1 });
|
||||
expect((await readRow(prisma, mile)).value).toBeCloseTo(2483.217792, 6);
|
||||
expect((await readRow(prisma, absurd)).value).toBe(250);
|
||||
});
|
||||
|
||||
it("plans one entry per account", async () => {
|
||||
const prisma = getPrismaClient();
|
||||
const first = await makeUser(prisma, "a");
|
||||
const second = await makeUser(prisma, "b");
|
||||
await seedRow(prisma, first, { value: 1.5 });
|
||||
await seedRow(prisma, second, { value: 2.5 });
|
||||
await seedRow(prisma, second, { value: 3.5 });
|
||||
|
||||
const plans = await planAppleHealthDistanceRepair(prisma, {
|
||||
archiveUnit: "km",
|
||||
});
|
||||
expect(
|
||||
plans.map((plan) => [plan.userId, plan.repairable.length]).sort(),
|
||||
).toEqual(
|
||||
[
|
||||
[first, 1],
|
||||
[second, 2],
|
||||
].sort(),
|
||||
);
|
||||
});
|
||||
});
|
||||
Reference in new issue
Block a user