Rated mood factors (work / sleep-quality / stress, scored per entry) now
participate in the cross-metric correlation surface. Until now their score
series was captured but read by zero analytics.
Two paths, both on the existing FDR-controlled engine:
- A factor x vital crosstab on the mood insights page and the mood
snapshot: for each factor the user scores, split its days low vs high by
the factor's own median, then run the same Welch test + day floors +
Benjamini-Hochberg correction the tag crosstab uses. Surfaces lines like
"sleep runs ~38 min below baseline on the days you rated work low"
(n-gated, q <= 0.10, never fabricated on thin data).
- Factor channels (FACTOR:<key>) fold into the period-narrative discovery
matrix as both lag source and lag target, so the drivers the coach and
narrative reference can cite a factor relationship.
An inverse factor (higher rating = worse day) gets the documented sign-flip
once, at the series boundary, so "up" always reads as a better day and the
low/high split and phrasing stay honest. Strings land in all six locales;
the surface carries the standing "association, not cause" caption.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add an additive, non-breaking assessment field to GET /api/insights/derived:
{ text, source, updatedAt } | null, keyed to the requested metric id for the
five per-score ids (READINESS, SLEEP_SCORE, RECOVERY_SCORE, STRAIN_SCORE,
STRESS_SCORE). The field is null when the metric is not one of those or when
its status is not "ok". A deterministic text always fills it — composed from
the score's own contributors, so provider-less accounts and the demo are never
empty — and warmer provider prose overrides it once cached, warmed out of band
so the GET never blocks on a model round-trip. de/en, following the v1.12.12
deterministic-narrative model. The batch grid read returns assessment: null;
the detail sheets use the single-metric route.
Introduce a shared buildMetricSignal() helper: a token-lean, signal-rich
descriptor (current, own baseline, signed delta, spread, normal-swing verdict,
normal-range placement, contributors) computed server-side so the model states
the comparison instead of re-deriving it from raw buckets. Thread it into the
generic per-metric assessment snapshot and lead the base prompt from it. Add a
PROMPT_VERSION constant to the base system prompt so prompt changes are
traceable.
Regenerate docs/api/openapi.yaml.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Anthropic and the local provider do not send a JSON mode, so a compliant model
still wraps its { "summary": … } reply in a code fence or prefixes a sentence,
which broke JSON.parse and surfaced the raw fenced string as the assessment.
Add a shared stripJsonFences helper (drop the fence, narrow to the first { …
last }) and apply it as a parse fallback in parseSummaryFromContent. No-op on
already-clean JSON.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The generic metric-status generator read SLEEP_DURATION row-per-stage and
fed the snapshot a single sleep STAGE (~16 min) as the current value, so the
sleep prose reported a fraction of an hour for a full night. Route the
SLEEP_DURATION status/snapshot through reconstructSleepNights — the same
per-night, source-deduped, midnight-aware time-asleep total the dashboard,
series, and targets surfaces use — so the snapshot's series, summary, and
latest value are all the night total in minutes (the registry unit and normal
range are already minutes). Gated to SLEEP_DURATION; no other metric's status
path changes.
The readiness wellness index could surface a contributor value of 0 for an
account that recorded resting heart rate / HRV / respiratory rate but had no
scorable baseline: a non-finite baseline center or today value fell through
clamp100 to a misleading 0 that reads as a real worst-case sub-score. Return
null instead so the client drops the contributor, and document the null-vs-0
contract: null means no usable signal (no baseline, no recent reading,
non-finite input); a finite value — including a genuine 0 — means the baseline
is established and the latest reading really deviates that far.
(cherry picked from commit 0422ac012765f30069a41f662c2b025197ed9a88)
The per-metric assessment cards are tightly grounded but their form had
converged: every card opened the same way, forced a "one doable step"
even when nothing was actionable, and a metric that stayed steady for
weeks produced near-identical paragraphs. Squeeze more value from data
already computed server-side, without touching the grounding floor or any
safety contract.
- Surface FDR-surviving cross-metric correlations on the per-metric card.
The discovery engine already screens behaviour×outcome pairs with a real
Pearson + exact p-value + Benjamini-Hochberg FDR control; that
intelligence only reached the period narrative. A new read-only consumer
runs the same full-matrix discovery and filters to the pairs that
involve the current metric, then feeds the engine's own conservative,
descriptive, never-causal interpretation strings into the prompt as
grounded relations. Best-effort: a correlation hiccup can never block the
generation it only decorates.
- Streak-aware repetition. Derive a steady-run length from the graded
series (consecutive recent weeks within a band of the user's own
baseline) and tell the model when it has already reported "no material
change" N times running, so it acknowledges the continuity in one clause
and pivots to a different facet instead of restating the same level.
- Stop forcing a step; add controlled variety. The closing step is now
conditional — when nothing is genuinely actionable the model affirms and
names one thing to watch rather than manufacturing the filler that lets
platitudes back in. A deterministic per-render variety token (seeded from
user + metric + day, never Math.random / Date.now) rotates the opening
angle so consecutive cards and days don't read identically, and the
phrasing temperature lifts 0.3 → 0.45 while the facts stay pinned by the
snapshot and the forbidden-phrase guards.
- Surface the computed data strength (n + recency) into the prose prompt so
it hedges on thin data instead of guessing what "few" means, matching the
UI confidence badge.
- Add locale-matched few-shot examples (a grounded assessment + a labelled
banned-filler counter-example) to the assessment system prompt, which
most helps weaker local-provider models follow the contract.
The own-baseline grounding, computed-not-hallucinated stats, schema-
enforced output, filler-phrase ban, and correlation-as-association
discipline are all preserved and re-pinned by new guard tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit c9d04cab1523e0e57a3f50c87c9f88684b01b26f)
The tag × metric crosstab summed cumulative metrics (steps, active
energy, sleep duration) per day across every source with no canonical
pick, so once two sources reported the same day — Fitbit + Apple steps,
Fitbit + WHOOP sleep — the per-day total double-counted and inflated the
with/without averages and the Welch delta the surface ranks on. Run the
metric rows through the same per-day source-priority + device-type picker
the analytics steps/sleep path uses before bucketing, keyed by the
metric's priority ladder and the user's Berlin calendar day. Same-source
per-stage rows still sum into the night total; only cross-source twins
collapse. Thread the user's source-priority blob through the read so the
pick honours the configured ladder.
The cross-metric read was a flat `take: 5000` over the 365-day window
ordered `measuredAt desc`, unioned across per-stage sleep + per-sample
pulse/HRV; a multi-source, high-frequency user blew past it and silently
dropped the oldest months, biasing the crosstab and the long-window
correlations toward recent data. Lift the cap to cover the realistic
worst case for the window and annotate a wide event when the cap is hit
so a truncated read is observable rather than silently wrong. The read
stays raw rather than moving to the DAY rollup tier because that tier
buckets on UTC midnight while these aggregates key on the user's Berlin
day, which would skew the day pairing at the boundary.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit aa72ffba6e3bcac9f97be4adc26d689d4f352e98)
Extend the mood-relations surface from mood to a health metric. For each
structured mood tag present on enough days, compare a metric's value on
tag-present vs tag-absent days: active energy on workout days (same day),
sleep length on nights tagged sleep (same day), and next-day recovery
after a tag like alcohol or food (D to D+1 lag). The comparison reuses the
existing statistics engine wholesale — the Welch two-sample t-test, the
per-group day floors, the confidence band, and the Benjamini-Hochberg FDR
control already used by the influence and discovery boards — so no new
statistics code lands. Only pairs that clear p < 0.05 and the FDR control
surface, ranked by adjusted q then absolute delta and capped at eight rows.
The result rides on GET /api/mood/insights as a new additive
`tagMetricCrosstab` field and renders as a confidence-gated, observational
card on the mood insights page, framed as an association in your own data
rather than a cause. The endpoint is server-rendered and not in the OpenAPI
contract, so no contract change is required.
Deprecate the standalone moodLog integration. Mood is now tracked fully
inside HealthLog through entries, structured tags, and rated factors, so
the external webhook and reverse-sync bridge no longer add anything. The
webhook route, the push and pull helpers, and the validation schemas carry
an @deprecated note pointing at the native path; the surface stays
functional this release and is slated for removal in a future major. The
settings card shows a deprecation hint, localised across all six locales.
- Mood discovery footer no longer pairs a mood-only numerator with the
full behaviour-by-outcome denominator. Reword it to surface the mood
relationships out of all day-to-day pairs the engine tested across
every metric, so the count it reports is honest about its scope.
Update all six locales.
- Rank the better-days board on a comparable scale: standardize the tag
mood-point delta into a Cohen's-d effect (delta over the pooled
with/without mood SD, clamped to [0,1]) so it sits alongside a metric
|r| instead of the old |delta|/2 heuristic. Fall back to |delta|/2 only
when the pooled SD is undefined. The pooled SD is ranking-only and is
stripped before the row enters the snapshot payload; the raw delta and
r the cards show are unchanged.
- Normalize the Withings and Whoop resume buttons to the min-h-11 scale
token, dropping the arbitrary min-h-[44px] spelling.
- Give the API token copy button a 44px tap target on small viewports,
scaling back to the tighter icon size from sm up.
- Drop the dead cached prop from InsightStatusCard and every caller; the
card has carried no cached affordance since the badge was removed.
Two changes that turn the descriptive relations surfaces into a
statistically-gated discovery view on the mood page.
Mood as an outcome: the FDR-controlled all-pairs correlation discovery
engine treated mood only as a behaviour (a lag source). Promote it to an
outcome channel too, so the scan also surfaces "behaviour today → next-day
mood" relations (e.g. more daylight today → better mood tomorrow) alongside
the existing "mood today → next-day outcome" direction. The self-pair is
skipped and Benjamini-Hochberg already controls the larger pair family this
opens up. The discovery route now feeds the shared mood daily-mean series
into both roles.
Surface the discovery slice: the engine was computed but never rendered.
The mood Insights page now shows the mood-involving pairs that survive the
false-discovery correction, each phrased observationally with its paired-day
count, r, and BH q-value, plus the honest "X of N tested" footer. The
surface self-fetches the correlation route and degrades to nothing when the
operator has disabled it. Observational only, with the standing disclaimer.
Two new observational surfaces on the mood Insights page, both computed in
the single mood-aggregates engine so the charts and the coach snapshot stay
in sync.
Influence on mood (per tag): for each frequent flat and structured tag,
compare the average daily mood on days the tag is present against the
counterfactual baseline of days it is absent, with the delta and a
confidence band. The band comes from a Welch two-sample t-test reusing the
exact Student-t p-value already behind the correlation cards, not a naive
approximation. Both groups must clear a five-day floor; rows rank by
absolute delta and cap per axis. Sparse tags, single-day tags,
all-same-value days, and zero-delta tags are dropped cleanly — never NaN.
Better days board: one ranked list folding the tag deltas and the
mood x health-metric correlations (sleep, steps, pulse, weight, blood
pressure) into a single effect-size-ranked, confidence-gated surface — the
headline answer to "what goes with my good days?". Near-zero and
under-sampled correlations are excluded; the list is capped and observational
only, with the standing "association, not cause" caption.
Both surfaces ride the mood-status snapshot too so the assessment prose
never contradicts the on-screen relations. New copy in all six locales.
The target reference card carried an inline "Adjust target range" link,
which read as a control buried in a read surface. Lift the action to a
gear button in the category-page header, to the left of the Coach launch
icon, so the two sit as a consistent icon cluster beside the heading.
A small page-scoped provider bridges the two: the reference card
registers its editable target (one per page, except blood glucose which
fans out to several contexts and registers each) and the header gear
opens the per-metric editor sheet the provider owns. Single-target pages
open that target directly; glucose opens the primary context, since
target editing lives only on these pages — there is no separate targets
management surface. The card itself is now a pure read surface.
The gear carries the action's existing label as its accessible name,
clears the 44 px touch-target floor on mobile, and exposes a stable
`data-slot` for end-to-end coverage.
Each Trends-row card carries a one-line caption below its mini-chart.
The first choice is the AI advisor's per-metric sentence, but on a cold
briefing — a web-only account, a pre-briefing cached payload, or the
advisor still generating — that sentence is null and every card fell
back to a static "Awaiting more data" hint, even with a full 30-day
series painted right above it.
Add a deterministic, rule-based descriptor derived from the same series
the mini-chart plots, and apply a three-tier caption precedence:
1. advisor annotation present → the AI sentence (unchanged).
2. else, a computable series → the deterministic descriptor:
direction (rising / falling / stable) plus the first-vs-last
magnitude over the window, e.g. "Rising over 30 days (+8 mmHg)",
"Falling over 30 days (−1.4 kg)", "Stable over the last 30 days".
Mood reads on a categorical 1–5 scale, so it is phrased in plain
terms ("Mood trended slightly higher over 30 days") with no raw
point delta.
3. else (genuinely < 2 points) → the real "Awaiting more data" hint.
The descriptor is observational and neutral by construction: direction
and magnitude only, with no causal, diagnostic, or medical claim and no
value judgement — a rising weight and a rising step count read the same
way here. A per-metric stability floor (the larger of an absolute and a
relative bound) keeps day-to-day noise from reading as a trend.
The descriptor logic lives in a pure, unit-tested helper. The caption
reads the same series the chart shows: mood reuses the existing
mood-analytics cache slot, and numeric metrics take a small dedicated
30-day daily-aggregate read keyed under a new query-key factory entry
that joins the measurement-write invalidation set. New descriptor copy
ships in all six locales. The pending shimmer while the advisor is in
flight is unchanged.
Complete the source-aware rollup read path: the slim summaries slice
(rollup + live), the comprehensive aggregator (both paths, including the
dailyByType correlation feed), and the /api/measurements route (rollup
read + the live date_trunc fallback) now resolve overlapping sources to
the source-priority ladder's canonical reading per (type, day) before
aggregating. A dual-source vital (e.g. WHOOP + Apple Watch resting heart
rate) surfaces one canonical series instead of a blend or a doubled
count; cumulative metrics keep summing per canonical source.
The collapse runs in SQL where the aggregate must stay a single
round-trip (buildSourceRankCase emits a per-user ladder-rank CASE;
canonicalMeasurementsFrom restricts a live read to canonical-source rows)
and in application code (collapseRollupRowsBySource) for the row readers,
both driven by the same resolved ladders so live and rollup agree.
Also widens the enum-literal SQL whitelist to accept digits so VO2_MAX
no longer throws on the typed write-hook recompute or the rank CASE.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Map day-total Banister TRIMP to 0-100 against a personal reference -- the
EWMA-smoothed 75th percentile of the user's own training-day TRIMP over a
42-day chronic window -- instead of the fixed population anchor that pinned a
deconditioned or chronic-condition user near 0 regardless of their effort.
Below a 7-training-day cold-start floor the engine falls back to the
population anchor, recorded as anchor: population with lower confidence; at
or above it the personal anchor takes over. Always writes a score row -- no
new null gate. The anchor is derived from the TRIMP input, never the 0-100
output, so there is no circular feedback; rest-day zeros stay out of the
training-day distribution.
Add a server-internal per-(user, day) strain_trimp_cache (day-total TRIMP +
running EWMA reference + anchor) so the chronic window reads cheap cached
values instead of re-integrating 42 days of HR series each night. The cache
is not a Measurement -- it stays out of every user-facing surface, the
doctor PDF, and the FHIR bundle by construction. No backfill: the nightly
idempotent recompute populates the cache forward and self-heals, falling
back to the population anchor until enough personal history accrues.
Keep the Banister/Morton/Tanaka citations and the doctor-PDF exclusion; the
honesty copy now states the scale is relative to the user's typical effort,
with a general reference during cold start.
The show-all-readings link now carries the originating metric page as a
sanitised from param, so a weight to show-all drill-in returns to weight
rather than the Insights overview. The param is constrained to an internal
/insights/<slug> path so a crafted value cannot redirect off-site.
The Recovery/Stress/Strain crons fire in the small hours (04:45-04:55
Europe/Berlin, ~03 UTC) but scored the CURRENT UTC day, only a few hours
old: Strain missed the just-completed day's workouts and Stress saw a
near-empty intra-day SDNN set and falsely gated as insufficient. Score the
day that just ended (now - 1 day) across all three engines and their
measuredAt / externalId day stamps.
Resolve the previous-day key in one place by extracting the shared score-row
helpers the three engines each duplicated: scoreDayKey / scoreMeasuredAt /
scoreExternalId, the idempotent upsertScoreRow, and the cohort runScoreBatch
tally loop into src/lib/insights/score-row.ts; and loadBaselineProfile into
the baseline engine (the three byte-identical profile loaders + the partial
strain variant now route through it). The derived route should adopt
loadBaselineProfile too.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When a derived wellness score (readiness or recovery) crosses a notability gate
— outside the green band AND confidence at least medium — the insights
generator now appends a SYSTEM CONTEXT block so the briefing can narrate the
shift observationally. Mirrors the GLP-1 plateau detector's append mechanism:
reads the one derived contract, returns null (the common case) at zero token
cost, and instructs the model to frame the signal as an observed pattern, never
a diagnosis or a dose recommendation. Add the readiness/recovery sourceMetric
enum values plus their briefing href + icon + trend-chart mappings.
Derive a daily cardio-load proxy from the per-workout heart-rate series
plus active energy. Each workout's HR series scores via Banister's
gender-weighted exponential TRIMP over the fractional heart-rate reserve
(Tanaka age-predicted HRmax, the user's resting HR); the day's TRIMP totals
map to a 0-100 saturating scale. Days with no usable HR series but a
non-trivial active-energy total fall back to an energy-only proxy, with the
path recorded in the result.
Stored as a COMPUTED STRAIN_SCORE row, idempotent per user per day, gated
when neither a usable HR series nor active energy is present.
Standard: Banister 1991; Morton, Fitz-Clarke & Banister 1990 (TRIMP);
Tanaka, Monahan & Seals 2001 (HRmax).
Apple Watch has no electrodermal sensor, so there is no direct stress
measurement to read — what HealthKit does give is discrete SDNN samples
through the day. Derive an honest HRV-through-day PROXY: a suppressed,
below-baseline intra-day SDNN is a recognised marker of higher sympathetic
load. The provenance and label say "HRV-derived proxy" and never imply a
galvanic/EDA reading.
The engine reuses the shared vitals-baseline math (the same SDNN baseline
the wellness index reads), maps the day's below-baseline suppression to a
0-100 score, and stores it as a COMPUTED STRESS_SCORE row — server-owned, a
client can never POST it. It gates and writes nothing below the minimum
intra-day sample floor or without a usable baseline.
Standard: Kim et al. 2018 (HRV and stress meta-analysis); Thayer et al. 2012
(neurovisceral integration). Idempotent per user per day via the nightly job.
Add a recovery-score engine that turns a user's already-stored nightly
signals into a stored 0-100 score. It reuses the readiness blend — the
resting-heart-rate, HRV (SDNN), sleep and respiratory-rate deviation
from the personal baseline, with the null-redistribution the health
score already uses — so the stored series and the live readiness surface
can never drift. The lineage is the recovery-marker literature (Plews
2013, Buchheit 2014): a higher score reads as more recovered. It is a
descriptive wellness proxy, not a clinical or training-recovery
assessment, and it is only stored when enough components are present —
below that floor no row is written rather than a headline from one
signal.
A nightly job computes each eligible user's score and upserts it as a
COMPUTED RECOVERY_SCORE row, idempotent per user per day on the
externalId day key, so a re-fired tick overwrites in place instead of
duplicating. The score is never computed on a page visit. The queue is
registered in the worker's queue inventory, scheduled, and wired to a
handler; a grep guard pins all four points so it cannot silently never
run.
Add mapping tests for the five event classes (verdict resolution,
severity grading, the value-pinned-to-1 contract, unknown-codepoint
fallback, and that a continuous measurement never gets a classification),
the awareness-card gating + device-verdict framing + disclaimer presence,
and the events route narrowing + empty-state contract. Extend the enum,
chart-token and Coach-gate drift guards for the new event types.
Wire seven previously-deferred HealthKit quantity identifiers through
the full pipeline — enum, mapping, plausibility range, category, list-UI
metadata, FHIR coding, and the generic metric-status assessment:
CARDIO_RECOVERY heartRateRecoveryOneMinute bpm
WRIST_TEMPERATURE appleSleepingWristTemperature celsius
FALL_COUNT numberOfTimesFallen count
SIX_MINUTE_WALK_DISTANCE sixMinuteWalkTestDistance m
STAIR_ASCENT_SPEED stairAscentSpeed m/s
STAIR_DESCENT_SPEED stairDescentSpeed m/s
BREATHING_DISTURBANCES appleSleepingBreathingDisturbances count
Each value rides raw on the wire (identity conversion). Fall count is
cumulative (daily SUM); the stair speeds drain to a daily mean; the rest
store as latest per-workout / per-night spot readings. Wrist temperature
and breathing disturbances are flagged privacy-sensitive. The category
sleep-apnea and walking-steadiness EVENT identifiers stay deferred — they
are events, not quantities.
Each new metric gains an Insights sub-page on the shared scaffold with the
generic assessment card, localised across all six locales. Cardio fitness
(VO2 max) — ingested since v1.4.23 but assessment-less — gains its own
/insights/cardio-fitness page so it picks up the assessment instead of
riding as a bare chart-row on the pulse page.
Normal-range anchors are cited where defensible (six-minute walk 400-700 m,
ATS guideline + population reference); the rest defer to the user's own
baseline rather than fabricate a band.
Migration 0104 appends the enum values additively (IF NOT EXISTS, forward-only).
Build the composite tier of the derived-metrics layer on the shared seam:
- Sleep score — a transparent 0–100 composite from sufficiency, efficiency,
consistency, timing and composition sub-scores over the per-stage sleep
rows, reweighting around any sub-score the data cannot support (a legacy
asleep-only night drops composition gracefully rather than fabricating it).
- Wellness/readiness index — extends the health-score null-redistribution
blend with resting-HR, HRV (SDNN), sleep, respiratory-rate and mood
deviation-from-baseline components; returns insufficient below two present
components so it never headlines from one signal.
- Coincident-deviation flag — counts vitals outside their personal band on
the latest day and fires when two or more coincide, listing the
contributing vitals as possible factors, never a cause or a diagnosis.
Each composite cites its standard, carries coverage + confidence +
provenance, and reweights around missing inputs.
Add an FDR-controlled correlation discovery engine behind
GET /api/insights/correlations: it scans a curated behaviour x outcome
matrix, lag-joins each behaviour day to the next day's outcome, runs Pearson
with the exact Student-t p-value, and applies Benjamini-Hochberg control
across every tested pair so only statistically-defensible patterns surface —
each tagged with n, r, p and the adjusted q, framed descriptive not causal.
Add the score-anatomy detail view at /insights/scores/<metric>: a full
score ring, ranked contributor rows each showing its push/pull and weight
share, and the cited-standard provenance explainer — reused by every
composite, mobile-first and accessible.
Regenerate the OpenAPI contract for the new metric values + the correlation
discovery response, and add the i18n keys across all six locales.
Extend the mood-aggregate engine with two pure helpers over the rows it
already fetches:
- computeTimeOfDayAverages buckets every entry into morning / afternoon /
evening / night in the entry's own timezone (legacy null-tz rows fall
back to UTC). A spread guard keeps the once-a-day logger — whose entries
cluster in a single bucket — from producing a misleading daypart claim:
the pattern is only marked reliable once at least two buckets each clear
the per-bucket sample floor.
- computeMoodStability maps the population standard deviation of the daily
means onto a 0-100 score (higher = steadier) with a descriptive,
non-judgemental band. Sparse loggers below the minimum-days floor get
null rather than a meaningless number.
Both feed the MoodAggregates object that drives the visible mood page and
the LLM snapshot, so the prose and the charts agree. A new time-of-day
narrative takeaway joins the ranked feed, gated on the same reliability
flag plus the existing effect-magnitude bar.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 685b53c0fd994324c270df2b9a5ac964bb442c82)
The stale-while-revalidate path serves the last-good assessment on a cache
miss and returns a terminal payload, so the status hook stopped polling. The
worker warmed a fresh assessment out of band seconds later, but the open card
never picked it up — the stale text stuck until the page remounted.
Add a `revalidating` flag to the read-only miss outcome, set only when
last-good text is served AND a refresh was enqueued (false on the
suppressed-stub branch and when no prior text exists). Thread it through the
generic metric-status generator and the seven specialised generators onto the
wire, type it on the client, and make `nextStatusPollInterval` keep polling on
`preparing || revalidating`. The existing 12-attempt ceiling bounds both
signals, so a persistently failing generation still cannot poll forever.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit c9217534fbdb304c4df1b6eb55fb19e4360b8a10)
A constantly-syncing client drips measurement batches every few minutes.
Every batch deleted the dirtied per-metric assessment cache rows and
re-enqueued a regeneration. The per-(user,scope,locale) enqueue singleton
is only 120 s, so it could not coalesce across batches that arrive minutes
apart, and each delete dropped the row a fresh enqueue would otherwise have
found warm — so a metric card regenerated several times an hour and its
assessment felt regenerated on every visit.
Before evicting and re-enqueuing, check whether the scope's cached
assessment was already (re)generated within a 30 min debounce window. A
scope that is fresh is left intact and not re-enqueued; a genuinely stale
or missing scope still refreshes immediately, so correctness holds. A
timeout stub never counts as fresh, so a scope that recently stalled still
retries. One indexed read per user answers it for every candidate scope at
once.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 024baea664490ef0494add78c87492f01e61c6b9)
The ingest invalidation path mapped a fresh measurement only to the seven
specialised assessment scopes, so a freshly-synced HealthKit metric's generic
assessment card lagged its real-time data until the nightly warm pass —
an asymmetry against the near-real-time specialised cards.
Add a reverse `MeasurementType → metric id` index to the registry and extend
`invalidateStatusInsightsForTypes` so a reading of a type that backs a
registered generic metric also evicts and re-enqueues the matching
`metric:<ID>` scope, using the same single-locale, singleton-debounced
enqueue as the specialised scopes. Unregistered types and the seven
specialised metrics resolve to no generic scope, so the constant sync cannot
fan out to unwanted cards.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 9bc0f116b46950c8ffa79d127d5dcce32ea644a4)
Extend the assessment-card system beyond the seven specialised scopes to
the ~30 HealthKit metric pages, without hand-writing a prompt per metric.
- Add a metric-metadata registry keyed by HealthKit id (display name,
unit, direction, optional normal range, archetype, scope), covering the
data-bearing HealthKit metrics. The seven specialised metrics keep
their own scopes and are excluded from the generic set; an unregistered
id gets no card.
- Add a generic data-driven generator that reuses the existing graded
snapshot, provider runner, stale-while-revalidate cache, and persist
path. One prompt template per archetype (physiological-vital,
activity-fitness, body-composition, mobility-gait,
environmental-exposure) plus a dedicated sleep template, parameterised
by metric metadata. Empty-data metrics return an insufficient marker
with no provider call. Normal ranges use age + sex where the profile
stores them; no ancestry/region ranges.
- Add GET /api/insights/metric-status?metric=<id>: apiHandler-wrapped,
Zod-validated against the closed registry id set (422 on unknown),
read-only with stale-while-revalidate. First insights route with
OpenAPI coverage; regenerate docs/api/openapi.yaml.
- Route the generic metric:<id> scope through the on-demand status queue
worker dispatch.
- Extend the nightly warm pass to the generic metric caches for a user's
data-bearing metrics only (one distinct read gates the set), keeping
de+en warm and never calling a provider for an empty metric.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 41453fbdfee60793c16269b9a9e5f04b6c3ea450)
The write-path regenerate-on-invalidate enqueued a fresh assessment for
both de and en on every dirtying measurement, while the read-path only
ever serves the user's resolved locale. A frequent Apple-Health sync thus
fanned out two provider calls per scope, half for a language the user
never opens. Resolve the user's locale the way the read-path does
(normalizeLocale of user.locale) and enqueue that single locale. The
nightly warm pass still covers both locales for the rare locale switch.
A category open whose today-cache had been evicted dropped to the
preparing skeleton even when a perfectly good assessment from an earlier
day was still on record. Read the most recent non-stub assessment for the
scope and serve it instantly while the worker re-warms the cache; only
fall back to the preparing state when no assessment was ever produced.
The user no longer waits on an open if an assessment ever existed.
The measurement-sync invalidation deleted the dirtied per-scope
assessment rows and waited for the user's next category open (a cache
miss → a worker round-trip while they wait) or the nightly cron to
re-warm them. Enqueue a debounced regenerate for each dirtied scope
instead, coalesced per (user, scope, locale) through the queue's
singleton key so the constant Apple-Health sync collapses a burst of
samples into one regenerate per scope rather than a job storm.
The structured taxonomy tags carry a separate, richer signal than the
free-text tag list: a curated catalog pick with a label key and category,
attached to a mood entry through the tag-link join. The tag-mood takeaway
builder only ever read the flat tags, so the structured tags never reached
the ranked lift/drop pool despite both call sites already passing them.
Merge both axes into one ranked pool. A structured candidate emits its
catalog label key as tagKey, which the feed resolves to the active locale
the same way the weekday key already resolves; flat tags keep their
verbatim label. The lean status snapshot now loads the structured-tag
links too, so the takeaways the model sees match the ones on screen.
The Trends row charts the metrics the daily briefing flags, but only
the legacy BP / weight / mood triple carried an advisor annotation.
Every additive metric (pulse, sleep, steps, HRV, resting heart rate,
active energy, flights, distance, VO2 max, body temperature) rendered
the chart with empty space underneath because the caption only painted
when a slot carried an annotation key.
Give every chart config a standard one-line description (captionKey)
and render it for slots that carry no advisor annotation, so no card
is ever caption-less. The legacy triple keeps its annotation slot. Add
the caption strings across all six locales and regression tests that
pin a caption on every tile.
Surface plain-language takeaways above the mood charts: the weekday dip
and peak, the overall trend direction, the weekend-vs-weekday effect,
the strongest tag-to-mood lift and drop, the in-target share, and the
current logging streak. Each takeaway is threshold-gated — every minimum
sample count and minimum effect size is a named constant, and a takeaway
only renders once its bar is cleared, so the feed never falls back to
filler. The list is ranked by effect size and capped.
The takeaways are pure helpers computed from the existing mood
aggregates, so the same feed reaches both the insights endpoint and the
mood snapshot the model reasons over — what the page shows matches what
the model is sent. Also promote the previously snapshot-only in-target
percentage to a visible headline tile.
The overview Trends row used to render a fixed BP / weight / mood
triple regardless of what the day's data surfaced. It now mirrors the
daily briefing: the row charts exactly the metrics the briefing flags,
in the briefing's priority order, deduped and capped at three. When no
briefing is available — cold mount, a web-only account with no
findings, or a pre-briefing cached payload — selection falls back to
the legacy BP / weight / mood triple so the row never paints empty.
Selection lives in `trend-chart-select.ts` as a pure, unit-tested
mapping from each briefing `sourceMetric` to a chart descriptor
(series, colour, unit, title key). Adherence-context findings
(compliance, GLP-1 plateau) have no standalone trend series and are
skipped so the next finding fills the slot. Annotations stay on the
legacy triple; additive HealthKit metrics render chart-only. The chart
set is derived from the briefing payload already on the page, so there
is no new blocking call.
Add the data-value features to the insights category pages on top of the
density pass:
- Stat strip: lead each metric page with min / max / median / mean,
read from the already-fetched analytics summaries slice (no new
network). Median is now computed in both summary producers — exact in
the live path, a windowed PERCENTILE_CONT in the rollup-tier slim
slice. The thin HealthKit pages mount it too for parity; no AI
assessment is added there.
- Range band: shade an Apple-Health-style min-max area around the daily
mean line where the data supports it (non-cumulative metrics). The
daily-aggregate read now emits per-day min / max from the rollup
buckets; the band rides the existing target-range toggle.
- All readings: a dedicated per-metric subpage at /insights/values/<type>
reusing the measurement list pinned to one type (pagination + inline
edit/delete). A clear entry button mounts at the foot of each
category page.
- Diversity nudge: a gentle hint when a metric's readings cluster on one
weekday or time of day, computed from a bounded, cached timestamp
window — non-blocking, self-gating.
All new reads stay cache-friendly and off the navigation path.
Add the Category -> Tag taxonomy the mood surface was missing and the
free-text note the web form could never write, then surface both on the
Insights mood page.
Schema (migration 0101): a global MoodTagCategory -> MoodTag catalog
plus a MoodEntryTagLink join, seeded with a default five-category set
(i18n-keyed labels, Lucide icons). Purely additive — the legacy flat
MoodEntry.tags JSON column, the GLP-1 quick-tags, and the Telegram /
MoodLog sync are untouched; an entry can carry both axes.
Capture: the mood-logging form and the edit sheet gain a Textarea for
the note (model + API already accepted it) and an icon-chip picker over
the taxonomy catalog (GET /api/mood/tags). Create / update resolve the
picked keys to links; unknown keys drop silently against the catalog.
Insights: a structured-tag breakdown (icon + per-tag avg-mood lift) and
a notes timeline join the existing distribution / weekday / flat-tag
sections. The aggregate extension also fixes the flat-tag axis, which
read the raw JSON string and so never populated.
i18n across all six locales. Tests cover the taxonomy aggregation, the
note round-trip, the structured-tag schema, and the notes timeline.
Lift the mood aggregates the AI status generator already computes —
tag frequency + per-tag average, in-target share, and the
mood × metric Pearson correlations — into a shared
`mood-aggregates.ts` so the prose and the new charts read off one
implementation. `mood-status.ts` now consumes the shared helpers
instead of its own inline copies.
Add a cached `/api/mood/insights` endpoint that returns the
pre-computed shapes (summary, heatmap, distribution, weekday, tags,
correlations) behind a 60-second server cache and a bounded 365-day
read — a cheap read-only path, never a synchronous model call. The
bucket invalidates alongside the mood-analytics cache on a mood write.
Render the shapes on the mood Insights page:
- a calendar heatmap forked from the compliance heatmap, coloured by
the same mood-score bands as the line chart, with an adaptive
window (30 days until ~90 days of history, then 90, then a full
year past 180 days) so the grid is never mostly empty;
- a mood distribution bar chart (share of days per level);
- average mood by weekday with a best/toughest-day callout;
- a tag / trigger breakdown (frequency bars + per-tag average);
- mood vs. sleep / activity / pulse correlation cards reusing the
existing scatter chart.
Add the supporting i18n keys across all six locales.
Opening an Insights category page froze the whole UI: the per-metric
status card fired a cold, uncapped LLM fetch on mount with no client
abort, so the request hung up to the server's 60s ceiling and the
pending/resolve re-renders thrashed the tab strip and ate touch events.
Timeouts were never negatively cached, so every visit re-blocked.
Fix the class holistically:
- Bound every status GET on the client. The shared use-insight-status
hook and the inline medication-compliance query now wrap fetch in an
8s AbortController and stop retrying inline; navigation can no longer
await an uncapped round-trip.
- Make the status routes read-only. On a cache miss the generator
enqueues an out-of-band pg-boss job (insight-status-generate) and
returns a preparing envelope; the client polls a bounded interval
until the worker warms the cache. Models the dashboard snapshot's
ready/preparing pattern. The queue is registered in the worker
allQueues loop + a boss.work handler, with a guard test.
- Fix the nightly warm-pass regression: warm every supported locale
(the client reads against the active UI locale, not the persisted
User.locale) and warm on a cached comprehensive outcome too (its
write still evicts the per-status caches). Add a short-TTL negative
cache on the timeout path so a transient provider stall no longer
makes every subsequent visit re-block or re-enqueue.
- Add loading.tsx for the insights segment + the root so any future
RSC await streams a skeleton instead of blocking the transition.
- Stabilise the advisor regenerate callback so a status-query flip
can't re-render the memoised tab strip mid-gesture.
New i18n key insights.assessmentPreparing across all six locales.
Read the MONTH / YEAR rollup tier for the monthly / yearly slices of the
focused single-metric status snapshots (weight, pulse, bmi,
blood-pressure) via `buildGradedSeriesWithRollups`, so the coarse tiers
finally earn their write amplification. BMI has no tier of its own, so it
scales the WEIGHT tier by 1 / heightCm² (`scaleGradedSeries`) — an exact
linear transform. The multi-metric general overview and the tier-less
mood / adherence series keep their bounded in-memory fold.
Fix the coverage-miss path that returned an always-empty monthly / yearly
slice: the bounded ~90-day recent read can never supply those slices, so
on a cold-tier miss the builder now folds the full history in memory
rather than shipping empty coarse arrays when years of raw history exist.
Excludes tombstoned rows from both reads.
Lift the byte-identical `round`, `normalizeSummaryText`,
`normalizeLocale`, `summarizeSeries`, the `{summary}`-envelope parse, and
the cache-row persist block out of the seven generators into
`status-shared.ts`. Drop the per-medication expectation in the compliance
generator: the prompt returns only `{summary}`, so the per-med cards are
server-side placeholders rather than a dead model-authored branch. Remove
the unused `summarizeGradedSeries` export.
Behaviour is unchanged on the in-memory path; the rollup path is exact
where the tier composes (count / min / max / mean) and the payload stays
under the snapshot char cap.
Hang `invalidateStatusInsightsForTypes` off every measurement ingest
path so a new reading of a type drops the cached assessment rows that
reading dirties — the next mount or nightly warm pass then regenerates
them against the new data instead of serving the pre-measurement text
for the rest of the day.
Wired into both `/api/measurements` modes (single + multi-entry batch),
`/api/measurements/batch`, and the three Withings sync paths
(measures + activity + sleep), reusing the `collapseToTypeDayKeys`
result each already computes for its rollup recompute. The call is
fire-and-forget after the user's write commits: idempotent, never a
blocker on the request path, and a failure only warns.
Pin the snapshot payload budget: a 2-year daily weigher (weight + daily
BP) and a six-type dense account both serialise under the 24 000-char
cap, proving the graded rewrite holds the line on the worst-case
accounts. The stub-skip and timeout-no-persist behaviour is covered by
the per-generator cache-read tests + the shared status-cache suite.
The seven per-card generators embedded the full ~360-day daily array
per metric plus correlation pair arrays, and never touched the
WEEK/MONTH/YEAR rollup tier. A dense daily weigher ran the weight
snapshot ~80-110 KB; a many-type account could push general-status past
100 K tokens.
Embed the shared graded series instead of the full daily array, cap the
embedded correlation/paired arrays to their most recent entries, skip
measurement types with no data in general-status, and apply a generic
24 000-char snapshot budget that sheds yearly -> weekly -> array-tails
-> recent until it fits. The applyPayloadBudget daily buckets stay
in-process to drive the latest-day focus, in-target gates, and
correlation pairing; only the compact shape reaches the prompt.
Measured on a 2-year daily account: weight-status ~15 KB, general-status
~12 KB across six dense types — both well under the cap.
Turn a metric's history into a small, bucketed, AI-ready shape: the
last ~21 days verbatim at daily granularity, the ~10 weeks before that
as ISO-week means, the ~12 months before that as calendar-month means,
and everything older as yearly buckets with a least-squares trend
slope. ~30-45 rows per metric regardless of history depth, never an
individual reading beyond the recent window.
A pure in-memory builder folds raw points; a rollup-backed builder
sources the monthly/yearly slices from the pre-aggregated MONTH/YEAR
tier (readBestGranularityRollups) and the recent/weekly slices from a
bounded ~90-day raw read, so the coarse rollup tier finally earns its
write amplification. The recent/weekly bucketing mirrors the coach's
timeline split, duplicated here so the coach contract stays put.
The per-card status path capped provider generation at 20 s and, on
timeout, persisted the generic no-key fallback as a day-keyed cache row.
The cache-read only checked the day key and text, so that stub looked
like a real assessment and stuck until midnight — hiding the real,
data-driven text for the rest of the day on any account that hit a
single transient stall.
Align the budget with the provider clients' 60 s ceiling, route the
status path through resolveProviderChain + runRawCompletionWithFallback
so a degraded provider cascades like the coach, and stop persisting the
fallback: a timeout or error is a transient miss that serves the
fallback for the current render and re-attempts on the next mount. A
shared cache-read rejects any lingering timeout-stub rows by their
markers, and withTimeout now annotates real errors instead of folding
them silently into the timeout path.
v1.5.0 cut. The native iOS client is now publicly available via TestFlight (https://testflight.apple.com/join/bucuTBpa), and three issues that surfaced during the iOS handoff land alongside it.
## Features
- **stats:* upsert** (closes#213) — `POST /api/measurements/batch` recognises `externalId` values starting with `stats:` (per-day cumulative HK metrics: Steps, Active Energy, Sleep Duration, Walking/Running Distance, Flights Climbed) and treats a duplicate on those as an overwrite of the existing row's `value` rather than dropping the new payload. Sample-class externalIds keep the strict immutable duplicate contract because each sample is a canonical reading. New per-entry status `"updated"` on the response envelope, new `measurement.batch.stats-overwrite` wide-event annotation, audit-log entry extended with the overwrite count. Three integration tests pin the contract.
## Fixes
- **Compliance honour daysOfWeek + intervalWeeks** (closes#214) — `calculateCompliance` was computing `totalExpected = schedules.length * days`, silently ignoring `MedicationSchedule.daysOfWeek` and `intervalWeeks`. Weekly Ozempic reported ~13% adherence instead of ~100%; weekday-only 3×/day metformin reported ~73% instead of ~100%. Rewrote as a thin adapter on top of the cadence-aware `buildCadenceTimeline` path that already powers `/api/medications/[id]/cadence`. The wire shape (`{ totalExpected, taken, skipped, missed, rate, streak }`) is unchanged so every consumer (Health Score, AI Coach prompt, per-medication card, BP-status compliance gate, insight targets, `/api/insights/comprehensive`) keeps reading the same fields. Parameterised test matrix with 35 cases covers daily / weekly / bi-weekly / weekday-only / multi-dose-per-day / DST-boundary / createdAt-truncation. Health-Score regression test pinned.
## Tests
- 8 stale integration assertions retired across `withings-oauth.test.ts`, `withings-oauth-flow.test.ts`, `analytics-bp-aggregate-paged.test.ts`, `analytics-sleep-stages.test.ts`, `apns-dispatch.test.ts` — drift from the v1.4.47.x OAuth fine-grained reason tags, the v1.4.47.2 ES256 PEM verify guard, and the v1.4.49.1 analytics slim-slice annotation rename. Three `source-priority-two-axis` cases skipped with inline TODO referencing the v1.4.49.1 commit and the relocation candidate.
- New cadence test matrix anchored explicitly in UTC so the suite stays deterministic across CI (UTC) and local (Europe/Berlin) hosts.
## Docs
- README rewrite for the v1.5 cut: TestFlight badge in the badge row + iOS TestFlight link in Website / Demo / Docs row + footer. Buy Me A Coffee badge added. Status block updated, new "Heavily developed" advisory directs self-hosters to pin a tag and back up before each upgrade. Tech-Stack flags the iOS app as TestFlight-available. Roadmap promotes v1.5 from "in active development" to "current". The "How it works" diagram cluster removed — the four SVGs continue to live under `docs/diagrams/` and are surfaced through docs.healthlog.dev where they render reliably across themes.
## Final verify
- typecheck clean
- lint clean (one pre-existing warning in `withings/resume/__tests__`)
- pnpm test: 5285 passed / 1 skipped
- pnpm test:integration: 253 passed / 3 skipped
- GHA CI on PR: build amd64 + build arm64 + lint/typecheck/test + integration + e2e + dependency audit + secret scanning + dead-code gate + env check → all green
