Commit Graph
19 Commits
Author SHA1 Message Date
Marc-André Bombeck 3634f6c520 Average pulse by day in the dashboard, insights and series stats
The dashboard summaries, the insights aggregate and the series stats strip
still took a plain mean over pulse readings for their 7-day, 30-day, prior-month
and all-time figures, while the year-ago figure and the chart already weighed
each day once, so the year delta compared two different statistics. They now
go through the day-mean helper: each day is the mean of its hours' means and a
window the mean of its days, in the frame each reader buckets its days by.
Count, min, max, the median and the spread stay over every reading; every other
type is unchanged.
2026-10-04 11:22:17 +02:00
BernhardBuckel 497da7ca8b fix(pulse): make a day of pulse the mean of its hours' means
A watch samples pulse about every 5 to 10 minutes at rest and about every
5 seconds in a workout. The daily value was the plain mean over a day's
readings, so a workout hour of several hundred readings outweighed the other
23 hours: a day with one hard hour and 23 resting hours (60 bpm) read about
128 bpm, not 64.

A day of pulse is now the mean of its per-hour means, in the rollup DAY
bucket, the live per-day read the chart serves, and the series route's day
grain. Windows over several days weigh each day once instead of by its
reading count, so the bias does not return one level up. Reading counts,
min/max, and the regression sums still run over every reading. The chart
tooltip and the data table name the statistic, in all seven locales.

The statistic is limited to pulse; HRV, SpO2 and glucose keep the plain mean
(see src/lib/measurements/day-statistic.ts). Stored DAY rows are rewritten by
the next rollup recompute (admin recompute route), not on read.

Fixes #1107
2026-10-04 10:41:08 +02:00
Marc-André Bombeck 7c5ff89def feat(lint): refuse UTC day keys outside the time zone module
`instant.toISOString().slice(0, 10)` and parsing a key back as UTC
midnight are the idioms behind every day-boundary fix this year: the
evening west of UTC and the small hours east of it land on the
neighbouring day. A new rule, healthlog/no-utc-day-key, refuses them in
src/ outside src/lib/tz and tests, and refuses a disable directive for
itself that is not next-line or gives no reason.

The time zone module now names every legitimate form: userDayKey for a
person's day, dateOnlyKey / dateOnlyAtNoonUtc / dayKeyAsUtcMidnight for
date-only values and day labels, daysBetweenDateKeys / weekdayOfDateKey /
isCalendarDateKey / statedDateKey for key arithmetic and parsing.

Of the 133 sites the rule found, 84 are gone. Most moved onto a named
helper with no change in behaviour; these changed behaviour:

- cycle calendar, insights, Coach snapshot and prediction refresh start
  their lookbacks at the first instant of the user's day
- a model-transcribed date like "June 10, 2026" no longer lands a day
  early in a process east of UTC (documents and lab scans)
- a custom report range covers exactly the picked days in the profile
  zone instead of starting the evening before west of UTC
- date-of-birth pickers cap at the user's today, not the UTC one
- an allergy onset date is stored at noon UTC like other date-only values
- the report's cycle summary, the Coach's usual BP range and the intake
  projector's rollup recompute use the user's day
- vaccination dates render as dates instead of the previous evening

49 remain, each with its reason on the line above: 39 UTC by design (15
file name stamps, rollup and object keys, the UTC-day score engines) and
10 marked baseline. A guard test requires a classified reason on every
directive and holds the baseline to a ceiling that only goes down.
2026-10-01 09:59:57 +02:00
Marc-André Bombeck 3b1314d1b0 Enforce the reminder nag cap, compare the disclaimer version, drop the unread analytics columns
- The daily Coach sweep now charges each ignored day against
  surfaceCount and auto-dismisses at three — the cap the schema comment
  promised since v1.22 but nothing enforced.
- The welcome gate compares the stored disclaimer acknowledgment
  version against the current one, so revised disclaimer text
  re-prompts; /api/auth/me carries the version for the client.
- slope90 and anomalyCount leave the analytics wire: slope90 had no
  reader anywhere (computed on every request, in SQL twice), and the
  slim path could never compute anomalyCount (it always reported 0).
  anomalyCount stays as an in-process field on summarize() for the
  insights feature builder, which is its only real consumer.
2026-08-14 05:19:05 +02:00
Marc-André Bombeck 1a08083436 fix(insights): pair mood correlations on the local logging day
The mood series has been local-day-keyed since v1.32.12, but the metric
partners it joins against (blood pressure, weight, resting pulse) were
still keyed on the UTC calendar day. The join is an exact day-string
match, so for anyone outside UTC an evening reading landed on the next
UTC day and paired with the wrong mood entry — a one-day misalignment
sold as a same-day correlation.

Derive the three mood pairing partners as exact local-day means, keyed in
the user's own timezone via userDayKey (Intl-based, DST-safe). Blood
pressure and resting pulse reuse raw rows already on the request path;
weight rides one additive column on the existing raw-rows query. The
UTC-consistent weight x BP and BP-medication joins are left untouched —
both their sides share one UTC key space and they migrate with the
rollup tier later.
2026-07-24 12:04:40 +02:00
Marc-André Bombeck 86d917c5e2 fix(insights): align cold-fallback slope window to the rollup day boundary
The rollup read path windows the 7/30/90-day regression on
startOfUtcDay(now - N days), while the live-SQL fallback filtered the
REGR_* aggregates on the wall-clock NOW() - INTERVAL bound. An account
crossing a coverage boundary (warm to cold) therefore saw the boundary
day's readings flip in or out of the slope window depending only on
whether the rollup tier was populated, returning a cache-dependent
slope for the same request.

Anchor both cold-fallback paths on date_trunc('day', NOW() AT TIME ZONE
'UTC') - INTERVAL 'N days', the SQL twin of startOfUtcDay, so warm and
cold return identical boundary-day membership. The avg7/avg30/median
windows stay wall-clock since the warm narrow query computes them the
same way.
2026-06-26 13:16:00 +02:00
Marc-André Bombeck 21f6dcfe13 observability(rollups): surface the pending-backfill regression miss
The windowed slope / r2 / sd readers compose from the per-bucket
regression accumulators (migration 0190). Between a deploy and the
boot-backfill completing, a tenant's pre-0190 DAY buckets still carry
NULL accumulators, so composeWindowedRegression returns a full miss and
the slim / comprehensive readers emit a null slope. The null is the
honest answer until the backfill refills the accumulators, but the miss
was passing as a plain rollup read with no signal, against the no-silent-
cap rule.

Add hasPendingAccumulatorBackfill to the read tier and key a new
regression_source annotation off it (unavailable_pending_backfill vs
rollup) on both warm paths. No live-SQL fallback is added; the design
intentionally avoids the heavy scan.
2026-06-22 08:07:58 +02:00
Marc-André Bombeck be37497118 perf(rollups): compose windowed slope/r2/sd from accumulators
The summaries slice (computeFromRollups) and the comprehensive aggregator
(buildFromRollups) each ran a narrow live $queryRaw over the raw
measurements partition on EVERY warm-rollup request to fold slope7/30/90
and r2_7/30/90 via REGR_SLOPE / REGR_R2 - the un-eliminated live scan on
the happy path.

Drop the six REGR_* columns from both narrow queries and compose the
windowed slope / r2 (and the population sd via composeRegression) from
the per-bucket accumulators stored in measurement_rollups. The 90-day
DAY buckets are loaded once (already loaded in comprehensive; one bounded
findMany added in the slim slice), source-collapsed to the canonical
source, then summed over the day-aligned 7/30/90-day sub-windows. A
window whose buckets lack accumulators (pre-backfill) composes to null,
the same shape the empty REGR_* FILTER returned, so a transitional read
never returns a wrong partial regression.

median (PERCENTILE_CONT) and the exact |z|>2 anomaly count stay on the
minimal live query - both are non-composable across buckets. The cold
coverage-miss path keeps the live REGR_* regression. Annotate
slope_source:"rollup" + median_source/anomaly_source:"live" so a
regression to the heavy path stays observable (no-silent-cap rule).
2026-06-22 07:13:06 +02:00
Marc-André Bombeck 49add3f43b perf(analytics): fold the comprehensive aggregate's canonical self-join into one CTE
The warm narrow query and the cold heavy query each referenced the
canonical-source subquery twice — once in the window_stats CTE and once in
the outer body — so the DISTINCT-ON pick over the live 90-day rows ran twice
per rebuild. Extract canonicalMeasurementsCte (the SELECT body without the
alias wrapper), bind it once as WITH cm AS (...), and select FROM cm m from
both window_stats and the outer aggregate. Output is identical; the dominant
self-join runs once.

canonicalMeasurementsFrom now wraps the same body, byte-stable for the
single-reference call sites (pinned by a new parity test).
2026-06-21 00:55:47 +02:00
Marc-André Bombeck 013241ba74 fix(analytics): collapse overlapping sources on the cold BP path
The raw BP pulls in the comprehensive aggregator and the bp-in-target live
fallback read every source's row for BLOOD_PRESSURE_SYS/DIA. BP is a
multi-source metric — a Withings reading mirrored into Apple Health is two
rows for one event — so each shared reading was counted twice in the
in-target denominator and twice in the recency-weighted grade. The rollup
branch of the same fast-path reads source-collapsed measurement_rollups, so a
rollup-warm account and a rollup-cold account returned different BP
percentages and grades for identical data — a web/iOS and cold/warm parity
break.

Route both raw pulls through canonicalMeasurementsFrom so the cold/live path
picks the ladder-winning source per (type, day), matching the warm/rollup
path. Per-event rows of the winning source survive the collapse, so the
downstream 5-minute pairing still sees individual readings.
2026-06-20 16:52:00 +02:00
Marc-André Bombeck 1f4ba5939d style: apply prettier across the tree
One-shot prettier pass so the new format:check gate starts from a clean
baseline. Formatting only — no behaviour change. Generator-owned and
vendored artifacts stay untouched via .prettierignore.
2026-06-19 22:20:31 +02:00
Marc-André BombeckandClaude Opus 4.8 7c494b60cb feat(rollups): collapse overlapping sources across all read paths
Complete the source-aware rollup read path: the slim summaries slice
(rollup + live), the comprehensive aggregator (both paths, including the
dailyByType correlation feed), and the /api/measurements route (rollup
read + the live date_trunc fallback) now resolve overlapping sources to
the source-priority ladder's canonical reading per (type, day) before
aggregating. A dual-source vital (e.g. WHOOP + Apple Watch resting heart
rate) surfaces one canonical series instead of a blend or a doubled
count; cumulative metrics keep summing per canonical source.

The collapse runs in SQL where the aggregate must stay a single
round-trip (buildSourceRankCase emits a per-user ladder-rank CASE;
canonicalMeasurementsFrom restricts a live read to canonical-source rows)
and in application code (collapseRollupRowsBySource) for the row readers,
both driven by the same resolved ladders so live and rollup agree.

Also widens the enum-literal SQL whitelist to accept digits so VO2_MAX
no longer throws on the typed write-hook recompute or the rank CASE.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 10:05:47 +02:00
Marc-André Bombeck 6ce09cc76d feat(insights): stat strip, range band, all-readings subpage, diversity nudge
Add the data-value features to the insights category pages on top of the
density pass:

- Stat strip: lead each metric page with min / max / median / mean,
  read from the already-fetched analytics summaries slice (no new
  network). Median is now computed in both summary producers — exact in
  the live path, a windowed PERCENTILE_CONT in the rollup-tier slim
  slice. The thin HealthKit pages mount it too for parity; no AI
  assessment is added there.
- Range band: shade an Apple-Health-style min-max area around the daily
  mean line where the data supports it (non-cumulative metrics). The
  daily-aggregate read now emits per-day min / max from the rollup
  buckets; the band rides the existing target-range toggle.
- All readings: a dedicated per-metric subpage at /insights/values/<type>
  reusing the measurement list pinned to one type (pagination + inline
  edit/delete). A clear entry button mounts at the foot of each
  category page.
- Diversity nudge: a gentle hint when a metric's readings cluster on one
  weekday or time of day, computed from a bounded, cached timestamp
  window — non-blocking, self-gating.

All new reads stay cache-friendly and off the navigation path.
2026-06-01 20:26:26 +02:00
Marc-André Bombeck 1114cf7e63 chore(release): v1.4.40 — comprehensive Critical+High architecture closure plus iOS PB30 enablement
Eleven touch-disjoint waves landed on develop plus a six-axis QA
reconcile round before tagging. See CHANGELOG.md for the user-facing
prose; this commit message documents the wave + report inventory for
release-management traceability.

Waves shipped (each with a phase report under .planning/):

- phase-W-POOL-v1440-report.md — p-limit(4) wrapper around the
  analytics 15-way fan-out + pg.Pool max raised 10 -> 20. Closes the
  v1.4.39 empirical-trace finding #1 (Prisma pool starvation).
- phase-W-DELETED-v1440-report.md — 11 reader-tier helpers + 3
  integration contracts close audit Critical Finding #3 (soft-delete
  invisibility) across dashboard, analytics, summaries, rollup tiers,
  and the Coach snapshot path.
- phase-W-INSIGHTS-v1440-report.md — mood-rollup swap on the three
  /api/insights/{features,targets,cards} routes plus the
  compliance-rollup hook gap closure for bulk-projection mints; the
  post-QA reconcile patch closes the six remaining measurement.findMany
  sites in the cluster that still leaked tombstoned rows.
- phase-W-INFRA-v1440-report.md — rollup helper umbrella consolidation
  under src/lib/rollups/* with zero orphan re-exports; lint, typecheck,
  knip, and 93/93 rollup tests green post-restructure.
- phase-W-RSC-v1440-report.md — per-tile Suspense boundaries plus the
  queryKey factory-bypass walker test (extended to comparison/ in the
  reconcile pass).
- phase-W-PRIVACY-v1440-report.md — bilingual /privacy page with 17
  paired-section tests pinning the 9 SB-3 disclosure requirements.
- phase-W-AASA-v1440-report.md — /.well-known/apple-app-site-association
  endpoint pinned to S8WDX4W5KX.dev.healthlog.app with app-ID parity
  assertions guarding against split rotation.
- phase-W-APNS-NOTIFY-v1440-report.md — MEDICATION_REMINDER
  time-sensitive APNs payload (priority 10 + interruption-level)
  scoped via a parameterised test that pins all 6 other event-types
  do not bypass Focus. AP-2 .p8 install gates real delivery.
- phase-W-CONSENT-v1440-report.md — AI consent receipts CRUD with
  append-only invariant pinned in test; revoke-then-regrant leaves
  both rows; idempotent DELETE returns 200 + receipt: null.
- phase-W-WMY-WIRE-v1440-report.md — three previously-dead WMY readers
  (week/month/year) wired through summaries-slice and
  health-score-fast-path with a linear-composability parity test
  pinning the cross-consumer routing contract.
- phase-W-GHOSTS-v1440-report.md — ~1 177 lines of dead code removed
  across 9 atomic commits; startOfUtcDay + wallClockInTz consolidation
  eliminates a sibling-drift risk; knip whitelist trimmed to three
  files pending follow-up.

QA reconcile patches applied per the v1.4.40 QA round (six reports
under .planning/round-v1440-QA-*-findings.md):

- senior C1 — dashboard-suspense-boundaries regex aligned with the
  post-W-INFRA Thread 2 userTimezone-local lift.
- senior H1 — soft-delete filter added to the six remaining
  insights measurement.findMany sites.
- specialist H1 — Promise.allSettled around bulk-projection
  compliance recompute on both /api/medications/intake and
  /api/dashboard/summary.
- ux H1 — compare-toggle migrated to queryKeys.dashboardWidgets
  factory; walker test extended to src/components/comparison/.
- security M1 — consent artefact 64 KB cap enforced via
  Buffer.byteLength (UTF-8 bytes, not UTF-16 code units).

Quality gates green at tag: pnpm typecheck, pnpm lint, pnpm knip
--include files,dependencies,binaries,unlisted, pnpm test --run
(4726 passing / 1 skipped). Tests 4525 -> 4726 unit + 1 skipped over
the v1.4.39 -> v1.4.40 span.

User-facing prose: see CHANGELOG.md.
2026-05-21 12:48:02 +02:00
Marc-André Bombeck a550031a84 chore(release): v1.4.38
Robustness sweep, perf hotspots, full localization, coach gate
cascade closure.

- /api/dashboard/summary cold-mount 4.6 s → ~500 ms via DAY-bucket
  sparkline + 60 s response cache.
- Coach API gate inventory test surfaced and closed two orphan
  routes (chat[id] GET+DELETE, chat-message feedback POST).
- Cross-tz fast-path runtime guard falls back to live for users
  more than ±3 h from UTC; proper bucket minting stays v1.5.
- es / fr / it / pl coverage 27 % → 63 % (~3 400 new strings) plus
  three medication-cluster placeholder restorations.
- IANA timezone validation closes the User.timezone self-DoS
  regression the new dashboard SQL would have surfaced.
- Unit suite 4520 → 4524.

See CHANGELOG.md for the full set.
2026-05-17 19:07:03 +02:00
Marc-André Bombeck 98e078d359 v1.4.37.1 — event-loop unblock on the read path
Hotfix on top of v1.4.37. The post-deploy verify caught
/api/analytics cold-mount hits of ~62 s and parallel /api/health
checks of ~58 s on the live instance, far above the v1.4.37
1.5-3 s perf claim. Root cause: the synchronous
ensureUserRollupsFresh(userId) call at the top of each heavy
read path folds the trailing 90-day DAY window when the rollup
watermark trails the newest measurement. On a power-user account
whose iOS step samples push the watermark every few minutes,
each cache miss synchronously paid the full refresh cost on a
Node worker that also serves the rest of the request fan-out
(including /api/health, /api/version, and concurrent iOS calls),
and the event loop was blocked for the duration of the
recompute.

Convert the three call sites (analytics route, comprehensive
aggregator, summaries slice) to fire-and-forget. The downstream
probeRollupCoverage gate falls back to live SQL on partial
coverage so correctness is preserved; the only user-observable
change is one cache window of staleness on the very first
request after a new measurement landed.
2026-05-17 16:37:51 +02:00
Marc-André Bombeck 0e2a99f8e7 v1.4.36 — perf, charts, AI payload trim, UX punch
Heavy-aggregate skip on the rollup-fresh + fully-covered path,
daily chart fetches opt into rollup buckets via source=rollup,
Insights extractFeatures swap drops the prompt payload from
double-digit MB to the low hundreds of kB on power-user accounts,
page-level isLoading gate replaced with per-section early-skeleton
paint, trends-row chart slot pinned for CLS, IntakeHistoryListV2
restored on the medication detail page, About card folded into the
Admin Console with an auto-checking update badge, cumulative metric
tiles read the day's running sum, IP-whois fallback surfaces city
+ country when the ASN lookup misses. Migration 0068 adds an
additive users.insights_exclude_metrics column. Unit suite 4354
passing; typecheck + lint clean.
2026-05-17 13:22:51 +02:00
Marc-André Bombeck 8dc9edb65e v1.4.35 — persistent measurement rollups + partial read-swap
Foundation release for the persistent measurement_rollups cache tier.
Adds the additive table + enum, populates it from every measurement
write hook, lands the Apple-Health-import end-of-job backfill, and
ships the parity-checked read-swap on two reader surfaces.

The comprehensive insights aggregator and the slim analytics
summaries slice now source count / min / max / mean per type from the
DAY rollup buckets via aggregateBuckets, plus the comprehensive
dailyByType correlation feed straight from the bucket means. Slope /
R² / standard deviation / anomaly counts stay on live SQL — they do
not compose across DAY buckets. A defensive parity check against the
live aggregate's COUNT(*) falls back to live SQL when divergence is
detected.

Includes the new scripts/backfill-rollups.ts CLI for the one-shot
fold of historical measurements into all four granularities.

Tests: 4280 unit + 228 integration. tsc + lint clean. Migration
0067_v1434_measurement_rollups is idempotent under re-run.
2026-05-17 09:55:33 +02:00
Marc-André Bombeck f6ca89b963 v1.4.34.1 — insights cold-mount perf hotfix + scatter-card sizing
Hotfix on top of v1.4.34. The Insights page mount was paying ~29 s on
every cold load against accounts populated by the v1.4.34 Apple Health
importer. Root cause: `/api/insights/comprehensive` walked the unbounded
90-day measurement set into JS, then ran per-type `summarize()`, BMI,
blood-pressure classification, target adherence, and four Pearson
correlations from the same pile of rows. On a 100 000+ row account the
route consumed one of the 20 pool connections for the whole 29 s, which
cascaded sibling endpoints into Cloudflare "no available server" 503s.
The scatter-correlation card also surfaced a `width(-1)/height(-1)`
Recharts warning on first paint because its dynamic-mount lacked a
non-zero floor before aspect-ratio resolved.

Changed:
  - /api/insights/comprehensive now reads through a SQL-side aggregator
    (src/lib/insights/comprehensive-aggregator.ts) grouping by metric
    type with Postgres-native AVG / MIN / MAX / REGR_SLOPE / REGR_R2 on
    the measurements (user_id, type, measured_at) index path. Returns
    the same DataSummary-shaped per-type bundle the legacy route
    stitched together; the medication-compliance block keeps its
    bounded Prisma reads. Wrapped through the server cache keyed on
    `${userId}|comprehensive` (60 s TTL). Cold path drops from ~29 s
    to a few hundred ms against fixture data matching the production
    volume that triggered the bug; warm path resolves in single-digit
    milliseconds.
  - Cache-wrap the five remaining read routes the v1.4.34 server-cache
    blueprint left staged but unwired: /api/mood/analytics,
    /api/workouts, /api/bugreport/status, /api/medications,
    /api/dashboard/widgets. The Cloudflare 503 cascade on
    /api/measurements?aggregate=daily resolves as a downstream
    consequence — once the comprehensive route stops monopolising the
    pool, the origin accepts new connections fast enough that
    Cloudflare's circuit-breaker no longer trips.
  - /api/dashboard/chart-overlay-prefs PUT now invalidates the new
    dashboard-widgets cache so the partial-update path doesn't leave
    stale layout state in the cache for up to TTL.

Fixed:
  - Scatter-correlation card no longer logs the Recharts
    `width(-1)/height(-1)` warning on first mount. The wrapper now
    carries a min-h-[180px] floor that matches the loading skeleton so
    ResponsiveContainer always measures a non-zero parent before
    aspect-ratio settles.
2026-05-16 22:26:59 +02:00