Commit Graph
21 Commits
Author SHA1 Message Date
Marc-André Bombeck b49013e6c6 Bound the sleep-timeline delete to the window the re-sync can restore
The Withings arm of the backfill deleted the source's entire
SLEEP_DURATION history while its re-sync fetches only the trailing 30
days regardless of fullSync — every older night was raw data loss with
orphaned rollup buckets on top. The delete is now bounded to the
re-sync window and the touched rollup span is recomputed afterwards;
WHOOP keeps the whole-history clear because its full sync genuinely
re-imports every night.
2026-08-14 02:29:01 +02:00
Marc-André Bombeck 2941d56238 fix(sync): enforce the plausibility domain on every provider write path
Every interactive path, every import, MCP and Telegram checked an incoming
measurement against VALUE_RANGES before storing it. The provider sync writers
did not, so a decode slip or a glitched sensor landed a row the application
itself calls impossible, and every mean, median and personal band derived from
it inherited the impossibility. Filtering the read side hid the row. It kept
landing.

The gate is one module reading the domain the app already declares, applied at
two kinds of seam. reconcileExternalMeasurement is the shared identity writer
for Polar, Oura, WHOOP, the Withings sleep leg, the Apple export importer and
the batch endpoint, so gating there means a provider added later inherits the
refusal rather than having to remember a check it never knew about; the new
rejected_range verdict carries no row, which is what makes the compiler force
every caller to handle it. The five writers that hold Prisma themselves gate at
their entry, before the existence probe decides insert versus overwrite, so an
impossible reading can no longer replace a good stored value.

An out-of-band value is dropped. Not clamped, because a clamp invents a reading
nobody took. Not stored behind a flag, because a quarantined row is still a row
and every consumer would have to learn the flag. Not fatal to the batch, because
one bad sample must not strand the run's other readings behind a held watermark.
Absence reads as absence.

A silent drop, though, is indistinguishable from a provider that sent nothing,
so every drop is tallied on the wide event under measurement.range_rejected with
the metric, the provider and the edge it broke. Counts only: the reading itself
is health data and never enters the log line, and the direction plus the
metric's own declared bound is what a diagnosis actually needs. The tally
accumulates rather than replacing, because a provider gates once per resource
leg and the reconciler once per row. One warning per provider lifts the event to
warn so a drop does not hide inside a green line.

A structural guard enumerates all twenty measurement writers with the reason
each is allowed to write, so a new one has to answer the question before it
ships. scripts/diagnose-implausible-measurements.ts reports the rows already
stored, aggregated per metric and source with no value beyond the bucket's own
extremes. It repairs nothing: whether a stored impossible row is a decode slip
or a band that is too narrow is not a call a script gets to make.
2026-08-07 22:19:43 +02:00
Marc-André Bombeck 527dd6c1fa fix(integrations): escalate a long transient failure streak, and scope a success to its own leg
A sync failure the classifier bucketed as `transient` had no exit. The park
test read `kind === "persistent"` and the persistent streak anchor was only
ever stamped by a persistent failure, so a streak could run for a thousand
attempts with the persistent counter at zero, the state frozen at
`error_transient`, and no ladder it could climb. `transient` is a hypothesis
about retryability, and the classifier defaults every timeout, 429, 5xx and
unrecognised response to it precisely because the wire cannot tell a reboot
from a permanent break. That default is right for one attempt and wrong for a
thousand: an unbroken streak retried hourly for a full day has refuted it by
experiment. Record it as `persistent` from there, which feeds the park ladder
that already exists — 48 h of unbroken failure end to end before an
integration stops attempting.

The counters could not accumulate anyway. `IntegrationStatus` is keyed
(user, integration) while Withings runs four legs on four crons and WHOOP four
resources on four more, and a success cleared the error, every bucket, the
streak anchor and the state whatever leg had recorded them. The ECG tick at
:41 wiped what the sleep tick recorded at :15 — including a reauth park — so
the strike ladder was reset several times an hour and the card painted a green
"connected · 12 minutes ago" over a data class that had stopped arriving. A
success now clears only what it is entitled to clear.

Two columns carry it: `failing_since_at` dates the streak, `failing_leg` names
its owner. The migration backfills the anchor for rows already in an error
state so a connection broken for weeks escalates on its next attempt rather
than restarting its clock.
2026-07-26 14:19:13 +02:00
Marc-André Bombeck e47200505e fix(withings): sync ECG on a schedule and surface write failures
Add an hourly ECG catch-net cron so a watch-only account with no scale pulls its
recordings without a webhook: the queue was registered and bound but never
scheduled, and the handler now grows a cohort arm that walks every connection on
a payload-less tick, mirroring the activity and sleep handlers.

Make the activity and sleep write paths honest. A swallowed activity batch-write
fell through to the success stamp — a green pill over lost data, green forever on
a persistent write bug; it now records a transient failure and refuses the
stamp. A reconcile throw out of either sleep transaction propagated past the
stamp and was swallowed by both callers, leaving neither success nor failure
recorded; the source now records a transient failure and rethrows so the cron
and webhook paths stay honest without changing their warn-and-continue
behaviour.
2026-07-24 20:05:57 +02:00
Marc-André Bombeck 2716f302c7 chore: align hardening tests and formatting 2026-07-20 12:24:15 +02:00
Marc-André Bombeck a8ac376cf7 fix(measurements): reconcile dual identities atomically 2026-07-20 11:55:09 +02:00
Marc-André Bombeck 4591899657 fix(release): harden v1.31 arrival contracts 2026-07-20 00:40:20 +02:00
Marc-André Bombeck b6ded4d1f9 feat(daily): finalise the morning digest when last night's sleep lands
The nightly insight pre-pass runs ~04:30, before last night's sleep has
synced, so the morning digest and health score reference the day without
the current sleep. Close the gap the event-driven way instead of an
on-mount generation.

A day's digest is now provisional until the sleep-dependent output has
been regenerated with last night's sleep, and final after. The lifecycle
is driven by a real marker (User.morningDigestRefreshedOn, the user-local
date the refresh completed), not a heuristic: it flips the instant the
refresh runs, ahead of the snapshot cache that feeds the sleep last-seen
fallback. A day with no sleep device stays honestly provisional and keeps
the sleep-pending note rather than showing a stale final score.

When a completed sleep segment for last night lands from any of the three
sleep transports — Withings sync-sleep, the WHOOP sync, the Apple-Health
measurement batch — a debounced morning-digest-refresh job is enqueued.
"Last night" is judged in the user's profile timezone, so a full-history
re-sync never re-triggers for an old night. The queue singletonKey plus
the marker collapse the many samples of one night, across all sources,
into one refresh per user per local morning; the job re-runs the same
comprehensive pipeline the nightly cron uses (forced, so the content-hash
gate turns a no-change regen into a cheap no-op) and stamps the marker. A
hard generation failure leaves the day provisional and hands recovery to
the existing 45-minute briefing-retry machinery.
2026-07-16 22:57:27 +02:00
Marc-André Bombeck 7386630962 chore(release): v1.28.39 — sync data-loss hardening
Four verified HIGH data-integrity fixes, each in-place per-provider (no
shared helper), from the campaign's data-integrity audit + adversarial
verification:

F1 — drain-per-sample-cumulative buildScanWhere was the only one of four
sibling drains missing deletedAt: null; the nightly cumulative fold
re-summed soft-deleted Apple samples and hard-deleted the legacy-steps
audit tombstones. Added the filter (mirrors consolidate-daily-mean,
dense-intraday-retention, consolidate-legacy-steps).

F2 — the additive late-fold merge had no single-flight guard, so two
overlapping runs (deploy overlap, or the admin endpoint racing the cron)
produced a permanent 2x daily total. Added a pg_advisory_xact_lock per
(user, externalId) plus an in-transaction re-read of the live
contributors so the serialised-second fold contributes zero (a bare lock
alone would still re-add its stale scanned sum). Mirrors the documents
quota advisory-lock pattern.

F3 — Withings sleep externalId keyed on segment START while measuredAt +
the natural key use END; a re-scored segment missed the externalId probe,
the sweep tombstoned the old row, and the re-keyed insert collided on the
natural key and wedged. Added the natural-key rescue (re-key in place),
mirroring the same-file measure path.

F4 — Withings measure sync stamped lastSyncedAt + recordSyncSuccess
unconditionally after a warn-only per-row loop; a transient write failure
stranded the reading behind the 10-min overlap forever. Now holds the
watermark and records failure on a hard row error (mirrors
google-health/fitbit), while a benign P2002 still advances.

Regression test per finding (reverting the three source files fails
exactly four tests). Integration suite green against real Postgres.
2026-07-16 01:45:05 +02:00
Marc-André Bombeck 9c63d3ce20 chore(release): v1.28.25 — every integration held to the same standard
Platform-wide hardening: the failure classes found live this week hunted down
across every integration.

Fitbit carried the same tombstone wedge fixed for Google — probe now matches
soft-deleted rows and the update resurrects them; the comment describing a
partial unique index that never existed is corrected. Sleep segment ids are
stable across re-scoring for Withings (session+segment-start), WHOOP/Polar
(one segment per stage, no positional index) and Oura (run-start keyed), each
with a session-scoped sweep that soft-deletes rows a re-score orphaned — so
legacy and affected nights heal as they are re-read.

One platform rule for deleted rows: source-owned readings resurrect on
re-import (Withings, WHOOP, Oura, Polar, Nightscout, mood webhook, CSV
re-imports, Apple stats day-totals); Apple sample-grain deletions stick per
the paired-client contract, and the mood bulk route's tombstone match is now
a true no-op reported as duplicate. Import counts reconcile against actual
createMany counts.

Google lifecycle: a dead token registers on the cycle's hard-fail ledger, so
an expired connection parks instead of stamping success hourly; backfill and
sleep-repair markers stamp only after a clean pass (pg-boss retries are now
real); failed measurement writes hold the watermark; no-op updates are
skipped; mapping.md tells the current truth.

Scale: doctor-report reads only consumed columns and enabled sections;
exports and backups keyset-paginate; achievements tally via SQL buckets;
series charts day-bucket dense kinds beyond 90 days; BP pairing is a linear
two-pointer; distinct-type scans run server-side; the graded-series cold
fallback is bounded; the fitbit defer tracker loops instead of spreading.
2026-07-11 01:54:12 +02:00
Marc-André Bombeck fb4e3921e2 feat(withings): capture nightly sleep vitals from the summary endpoint
The per-segment sleep sync wrote only the stage timeline; the per-night
summary's vitals — average HR, respiratory rate, SDNN-based HRV, average
SpO2 and the sleep score — were dropped at the fetch. Add a getsummary
fetch that maps each vital to its existing MeasurementType (RESTING_HEART_RATE,
RESPIRATORY_RATE, HEART_RATE_VARIABILITY, OXYGEN_SATURATION, SLEEP_SCORE), so
no migration and no client-contract change: the rows flow to the sync delta
under the WITHINGS source like every other server-side fetch.

Each vital upserts on the stable (userId, type, source, externalId) composite
keyed by the night's session id, so a re-sync overwrites rather than
duplicates. A summary fetch failure is isolated so the stage segments already
written survive. Fractional SpO2 is normalised to percent.
2026-06-20 16:51:51 +02:00
Marc-André Bombeck cbd5c2ba6f fix(withings): key sleep re-sync on the stable externalId, not the shifting measuredAt
With measuredAt now stamped at the segment END, a night that Withings
re-aggregates between syncs shifts its enddate, so the measuredAt-keyed
lookup missed the prior row and the create then collided with the unique
externalId — leaving the night stuck at its stale value. Key the lookup on
the stable externalId and refresh value, measuredAt and stage on update.
Caught by the withings-sleep-sync integration test against real Postgres.
2026-06-15 02:02:12 +02:00
Marc-André Bombeck 7ad0b9c4f4 fix(sleep): stamp Withings sleep segments at the END, not the START
Withings sleep sync stamped each segment's measuredAt with its startdate,
but every reader treats measuredAt as the segment END and resolves the
span as start = end − duration. Each night was therefore rendered one
segment-length earlier than reality. Stamp the enddate so the span lands
at its true clock time.
2026-06-14 23:16:23 +02:00
Marc-André Bombeck cd8849cd51 feat(insights): drop the per-metric assessment cache on fresh measurements
Hang `invalidateStatusInsightsForTypes` off every measurement ingest
path so a new reading of a type drops the cached assessment rows that
reading dirties — the next mount or nightly warm pass then regenerates
them against the new data instead of serving the pre-measurement text
for the rest of the day.

Wired into both `/api/measurements` modes (single + multi-entry batch),
`/api/measurements/batch`, and the three Withings sync paths
(measures + activity + sleep), reusing the `collapseToTypeDayKeys`
result each already computes for its rollup recompute. The call is
fire-and-forget after the user's write commits: idempotent, never a
blocker on the request path, and a failure only warns.
2026-05-31 22:05:31 +02:00
Marc-André BombeckandClaude Opus 4.7 8528c3b4fe chore(release): v1.5.6
Pure-history medication detail page, legacy step consolidation, and the
finish of the outbound-fetch migration.

- /medications/[id] becomes a read-only history surface; every setting
  moves into a new advanced-settings sheet, and the edit trigger becomes
  a two-option picker. The list page is unchanged.
- A pg-boss job folds pre-v1.5.0 granular step rows into one canonical
  daily total per user per day and soft-deletes the originals; migration
  0087 adds a partial index for the converging discovery scan.
- Every remaining server-side fetch routes through safeFetch (Withings,
  Codex OAuth + SSE, bug-report, Umami, GlitchTip, Loki); a new
  healthlog/safe-fetch-required ESLint rule guards the boundary.
- Fixes: intake-edit dialog seeded from real values, consolidation skips
  a unique-constraint day instead of stranding it, avatar upload reads
  the body once under a bounded cap, egress lint rejects absolute URLs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-29 22:07:10 +02:00
Marc-André Bombeck 324d4bb411 chore(release): v1.4.44 — REG-11 dashboard summary fix (iOS Home tile sparkline + value)
iOS-operator-blocking hotfix for REG-11 — Home dashboard tile renders
neither chart nor latest value for BP / Puls / Körperfett when the
most recent reading is older than 7 days. Root cause was in
`/api/dashboard/summary` SQL gates; `latestIn7d` + `sparkBuckets` both
required `measured_at >= sevenDaysAgo`, which returned empty for
sparse accounts, leaving the iOS tile with `latestValue: null` +
`sparkline: []` to render from. Detail-screen
(`/api/measurements/series?kind=…`) escaped because of its broader
window.

Server-side changes:
- `latestIn7d` → `latestEver` (drops 7-day filter; DISTINCT ON returns
  latest reading EVER per type, bounded at N_metrics rows).
- `sparkBuckets` rewritten via ROW_NUMBER() OVER (PARTITION BY type
  ORDER BY bucket_start DESC) so the trailing N daily buckets per
  type render regardless of age, bounded at SPARK_DAYS × N_metrics.
- BP + Pulse metrics.push gated on `latest || allTimeCount > 0` like
  BodyFat already is — accounts with zero readings ever no longer get
  empty placeholder tiles.
- 4 new regression tests in summary/__tests__/route.test.ts cover
  sparse accounts, zero-data accounts, within-7d guards.

Other develop work bundled in this release:
- W4-QoL — copy nits.
- W5-WORKOUTS — user source priority threaded through write-time
  picker + batch route.
- W7-WITHINGS-RELIABILITY — response classifier + sync hardening.
- W8-OPS — ops phase docs.
- B11 — NEXT_PUBLIC_APP_VERSION build-arg routing so /api/version
  cannot drift from CI tag.
- env-check workflow — manifest / example lockstep enforcement.

Resolves PR #194 (squash-merge produced via local -X theirs strategy
because GitHub's UI couldn't auto-resolve develop's post-v1.4.42
contiguous work against the v1.4.42 squash on main; resulting tree is
byte-identical to develop's HEAD).
2026-05-21 22:07:32 +02:00
Marc-André Bombeck 9b6e237f5f chore(release): v1.4.42 — Knip enforcing, queryKey factory closed, iOS Workouts dedup, Withings off-response classification
Six touch-disjoint waves landed on develop before this release:

- W1-KNIP — exports/types tiers flipped to enforcing-mode on main;
  35 unused exports + 52 unused types triaged to 0/0; shadcn surface
  + zod-infer contract types scoped via knip.json ignore-block.
- W2-ZOD-MULTI-ISSUE — `returnAllZodIssues` shared helper +
  `/api/dashboard/widgets` 422 now returns every Zod issue + worker
  audit-ledger breadcrumb. 41 sibling routes documented for v1.4.43.
- W3-QUERYKEY-LONGTAIL — 40 settings/medications/admin/hooks files
  migrated to the queryKeys factory; 9 new factory entries; ESLint
  rule + test-guard scope extended in lockstep.
- W4-ARCH-HYGIENE — BERLIN_DAY_FORMATTER consolidated across nine
  sites (seven status helpers + bucket-series + bp-in-target);
  Suspense double-comment consolidated; doctor-report sanitiser regex
  rewritten to escape-form so diffs are reviewable; pr-detection-
  worker soft-delete filter; offhost-backup DR-intent comment.
- W5-IOS-WORKOUTS — `dedupeWorkoutBatch` write-time cross-source
  dedup helper for the v1.5 iOS HealthKit observer ingest; dashboard
  tile-strip placeholder parity with the live TrendCard chrome.
- W6-WITHINGS-OFF + ENV-CHECK — Withings off-response classifier
  (transient/reauth/persistent) + WithingsApiError typed throw + the
  `pnpm check-env` CLI. APNs `allOrNone: true` closes the AP-2
  silent-disable gap the wave was conceived to close.

No migration, no env-var change, no API contract break for iOS v0.5.4.
The new `/api/dashboard/widgets` 422 envelope is additively extended.

pnpm test --run green at 4815 passing / 1 skipped.
pnpm typecheck, pnpm lint, pnpm knip (no --include flag) green.
2026-05-21 19:33:07 +02:00
Marc-André Bombeck 1114cf7e63 chore(release): v1.4.40 — comprehensive Critical+High architecture closure plus iOS PB30 enablement
Eleven touch-disjoint waves landed on develop plus a six-axis QA
reconcile round before tagging. See CHANGELOG.md for the user-facing
prose; this commit message documents the wave + report inventory for
release-management traceability.

Waves shipped (each with a phase report under .planning/):

- phase-W-POOL-v1440-report.md — p-limit(4) wrapper around the
  analytics 15-way fan-out + pg.Pool max raised 10 -> 20. Closes the
  v1.4.39 empirical-trace finding #1 (Prisma pool starvation).
- phase-W-DELETED-v1440-report.md — 11 reader-tier helpers + 3
  integration contracts close audit Critical Finding #3 (soft-delete
  invisibility) across dashboard, analytics, summaries, rollup tiers,
  and the Coach snapshot path.
- phase-W-INSIGHTS-v1440-report.md — mood-rollup swap on the three
  /api/insights/{features,targets,cards} routes plus the
  compliance-rollup hook gap closure for bulk-projection mints; the
  post-QA reconcile patch closes the six remaining measurement.findMany
  sites in the cluster that still leaked tombstoned rows.
- phase-W-INFRA-v1440-report.md — rollup helper umbrella consolidation
  under src/lib/rollups/* with zero orphan re-exports; lint, typecheck,
  knip, and 93/93 rollup tests green post-restructure.
- phase-W-RSC-v1440-report.md — per-tile Suspense boundaries plus the
  queryKey factory-bypass walker test (extended to comparison/ in the
  reconcile pass).
- phase-W-PRIVACY-v1440-report.md — bilingual /privacy page with 17
  paired-section tests pinning the 9 SB-3 disclosure requirements.
- phase-W-AASA-v1440-report.md — /.well-known/apple-app-site-association
  endpoint pinned to S8WDX4W5KX.dev.healthlog.app with app-ID parity
  assertions guarding against split rotation.
- phase-W-APNS-NOTIFY-v1440-report.md — MEDICATION_REMINDER
  time-sensitive APNs payload (priority 10 + interruption-level)
  scoped via a parameterised test that pins all 6 other event-types
  do not bypass Focus. AP-2 .p8 install gates real delivery.
- phase-W-CONSENT-v1440-report.md — AI consent receipts CRUD with
  append-only invariant pinned in test; revoke-then-regrant leaves
  both rows; idempotent DELETE returns 200 + receipt: null.
- phase-W-WMY-WIRE-v1440-report.md — three previously-dead WMY readers
  (week/month/year) wired through summaries-slice and
  health-score-fast-path with a linear-composability parity test
  pinning the cross-consumer routing contract.
- phase-W-GHOSTS-v1440-report.md — ~1 177 lines of dead code removed
  across 9 atomic commits; startOfUtcDay + wallClockInTz consolidation
  eliminates a sibling-drift risk; knip whitelist trimmed to three
  files pending follow-up.

QA reconcile patches applied per the v1.4.40 QA round (six reports
under .planning/round-v1440-QA-*-findings.md):

- senior C1 — dashboard-suspense-boundaries regex aligned with the
  post-W-INFRA Thread 2 userTimezone-local lift.
- senior H1 — soft-delete filter added to the six remaining
  insights measurement.findMany sites.
- specialist H1 — Promise.allSettled around bulk-projection
  compliance recompute on both /api/medications/intake and
  /api/dashboard/summary.
- ux H1 — compare-toggle migrated to queryKeys.dashboardWidgets
  factory; walker test extended to src/components/comparison/.
- security M1 — consent artefact 64 KB cap enforced via
  Buffer.byteLength (UTF-8 bytes, not UTF-16 code units).

Quality gates green at tag: pnpm typecheck, pnpm lint, pnpm knip
--include files,dependencies,binaries,unlisted, pnpm test --run
(4726 passing / 1 skipped). Tests 4525 -> 4726 unit + 1 skipped over
the v1.4.39 -> v1.4.40 span.

User-facing prose: see CHANGELOG.md.
2026-05-21 12:48:02 +02:00
Marc-André Bombeck 4122c79d66 chore(release): v1.4.39.1 — rollup tier catches Withings + import + admin-restore write paths
Hotfix on top of v1.4.39. Three measurement write paths (Withings
sync, `/api/import`, admin backup restore) were bypassing the
persistent rollup tier's write hook, so the dashboard chart's
`source=rollup` fast-path painted "Noch nicht genug Daten" on the
30 / 90-point range for accounts whose recent data only came through
those paths. The 7-day window kept rendering because the chart's raw
fetch never consults the rollup tier.

Closes the read-coverage gap by wiring `recomputeBucketsForMeasurement`
into every remaining ingest path and re-anchors the boot-time backfill
discovery to per-day coverage so already-deployed instances re-converge
on the next worker boot. The legacy `sum_value IS NULL` branch stays
on the same UNION so the v1.4.39 column-backfill continues working.

Also re-anchors one date-bound test in the medications-intake route
suite so the green CI run no longer depends on the wall clock.

Tests 4647 → 4648 passing; one long-standing skip unchanged.
2026-05-21 09:03:33 +02:00
Marc-André Bombeck 4f09c3f862 chore(release): v1.4.27
v1.4.27 — Mobile capability and maintainer-finding cleanup.

ResponsiveSheet primitive (bottom-sheet on narrow viewports, centred dialog on desktop)
applied to BD, Weight, Glucose, Mood, Activity forms. Maintainer-finding cleanups across
Coach, Insights, Reports, Privacy. GeoLite2 offline geocoding with resilience layer
(graceful fallback when EULA not yet propagated). E2E gap-fill coverage. 450-line
release notes attached to the GitHub Release.
2026-05-15 22:59:10 +02:00
Marc-André Bombeck 17b8d8d2c2 chore(release): v1.4.25
Largest feature delta in the v1.4.x line. Insights expands into seven dedicated metric sub-pages; GLP-1 medication tracking lands end-to-end (drug-level chart with Research Mode acknowledgment, EMA titration ladder, cadence + compliance, side-effect taxonomy, pen/vial inventory); cross-source priority becomes a two-axis resolver; per-user timezone threads through ten analytics surfaces; Withings activity + sleep v2 sync; onboarding rebuilt as a nested-route wizard; Personal Records ship end-to-end; Health Score provenance accordion; Coach runs native first-party prompts across six locales with a 1800-assertion refusal-probe matrix; OpenAPI drift gate hard-fails; multi-arch GHCR image. Migrations 0043 through 0060 are additive and forward-only. PROMPT_VERSION 4.24.0 → 4.25.0 with GROUND RULE 9 (refuses dose recommendations) and GROUND RULE 15 (refuses drug-level estimates with MDR + MDCG 2021-24 cites).

See CHANGELOG.md for the full per-section delta.
2026-05-14 23:49:32 +02:00