mirror of
https://github.com/MBombeck/HealthLog.git
synced 2026-11-09 13:01:00 +00:00
39b0cabb4351652016137c217d6e32adf5e2295e
14
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Marc-André Bombeck
|
900a5c4f1f |
perf: stop keeping folded samples and re-reading duplicates
The nightly folds soft-deleted every raw Apple Health sample they folded into an hourly or daily stats row and kept it for 75 days. Nothing ever read those rows as deletions; they only held a re-upload of the sample off, and they were most of the measurements table and its indexes. Ingest now refuses such a re-upload directly: a raw sample of a dense or mean type, old enough to have been folded, whose hour or day a live stats row covers, is a duplicate with reason folded_window. The batch route and the Apple Health export import apply the same rule. With that in place the dense retention and the daily-mean consolidation delete the raw rows outright, and a new compaction-tombstone-purge job removes the backlog: 5,000 rows a batch, one account at a time under the restore lock, a short pause between batches, resumable because the predicate is idempotent, queued at every boot and continuing itself while it makes progress. A person's own deletions are never in its predicate. The sync feed leaves compaction tombstones out and a restore does not write them back; all three classify through the same function. The batch route answers an exact re-sent sample without a transaction: one read over the batch's sample ids, live or deleted, decides what the reconciler would have decided, at the same point in the loop, so every status, reason, counter and the cross-source merge stay as they were. A batch of duplicates only no longer queues PR detection, a reminder sweep or the detection audit row. Caches are evicted only for a foreground or manual sync or a batch that wrote a hand-entered reading, and marked stale otherwise; the targets cache follows the same split. Also: - POST /api/admin/maintenance/measurements queues VACUUM (ANALYZE) and a REINDEX INDEX CONCURRENTLY per index, largest first, on a connection of its own; it refuses while the purge runs, drops invalid leftover copies and stops at the first failed rebuild. Runbook in docs/ops. - The pure volume queues keep their finished jobs for 96 hours instead of seven days, never less than the failure window plus a day. - work_mem is set per app connection from DATABASE_WORK_MEM (16MB), also when the statement timeout is disabled. - The PR-detection fallback tick re-scans only accounts whose readings or workouts changed since the previous tick. - Type discovery for the status refill walks the live-rows index as a loose index scan instead of grouping every live row. - The host metrics chart polls every five minutes, like its sampler. - The batch route records per-type HealthKit arrivals. |
||
|
Marc-André Bombeck
|
96a3743009 |
Share one server cache between the home page and the API routes
A production build instantiates a module imported from both a Server Component and a route handler once per bundler layer. The cache registry sat in plain module state, so the home page's server render held its own set of Maps and every API route held another. Recording a dose evicted the route handlers' copy; the home page then rendered from a copy nothing had touched and painted the dose as still due until the entry aged out on its own, two or three minutes later. Because that render is dehydrated into the client query cache, it also overwrote the fresh value the browser had already fetched. Measured on `next build && next start`, single process: the page render and the snapshot route reported different cache instances, and after an interactive write the route handlers' copy dropped to zero entries while the page's copy stayed full. The registry now lives on globalThis under a Symbol.for slot, which is the only scope both layers share. db.ts guards its own global assignment with a non-production check because it is only defending against dev reloads; this one has to hold in production, so it is unconditional. |
||
|
Marc-André Bombeck
|
c91558f18f | feat(analytics): replace health score with reference composite | ||
|
Marc-André Bombeck
|
62eb431d66 |
release: v1.32.0 — hardening and scalability (#580)
* release: prepare v1.32.0 * test: stabilize water capture re-entry |
||
|
Marc-André Bombeck
|
9b0d5dc0b7 |
chore(release): v1.28.17 — dashboard edits stick; share scope honoured
Dashboard layout changes take effect immediately: the save now evicts the locale-keyed home-screen snapshot it previously missed, so a re-ordered or toggled tile set no longer reverts to the prior selection for the SWR window. A share link built with a narrowed report-section selection now serves exactly that selection. The stored selection is the grouped export shape; the shared view read it through the flat parser, which dropped every grouped toggle and fell back to the full defaults — an over-share against the owner's choice. The view now folds the grouped shape down faithfully. Documents-only shares were never affected. Hardening: the shared-document download path now sanitises the filename identically to the owner download path, and thumbnail generation re-checks the pixel ceiling after decode so a malformed image that slips past the header sniff still cannot exhaust the worker. |
||
|
Marc-André Bombeck
|
de33f1ad4c |
chore: remove the bug-report integration
Drop the in-app bug-report surface end to end: the user submission page, the feedback intake and admin inbox, the optional GitHub-issue escalation, the operator config, and the dedicated achievement badges. - delete the /bugreport page and the bugreport / feedback / admin-feedback API routes - drop the Feedback model, its enums, and the AppSettings GitHub-issue and enable columns (migration 0205) - remove the admin Feedback section, the Bug Report integration card, and the system-status row - drop the bug-report nav entry and the app-settings provider that only carried its flag - remove the bug-hunter achievement badges and their metric - prune the related i18n keys across every locale and the encrypted-column and key-rotation registries |
||
|
Marc-André Bombeck
|
aaecd141ee |
fix(medications): every intake path converges on the slot, and caches cannot resurrect a pre-write body
The external ingest endpoint was the one intake writer that never attributed to the minted band: it stored a bare row beside the pending slot, so an early take through the API showed as the last intake while the slot kept demanding a dose, the ledger recorded an extra plus a miss, and the day tally counted both. It now resolves the band and converges onto the pending row like every other path, the Telegram confirm and skip buttons do the same preferring the reminder's own slot, and the nightly convergence pass heals the pairs this has already produced — one band match, one live anchor row, both days' rollups recomputed. The cache layer fences in-flight builders on invalidation so a read can neither join a pre-write build nor have a background refresh re-cache a pre-write body as fresh. Ad-hoc rows no longer resolve a different slot through the six-hour radius, a bounded early grace credits a take up to an hour before the window without crossing the previous band, day-scale cadences taken early downgrade the slot-day prompt instead of inviting a double dose, the overdue pill ignores the projector's pending mints, created intakes carry the same plausibility bounds as edits, slot moves keep the recorded injection site and dose, both affected days recompute on a moved toggle, deliberate slot-less skips converge instead of orphaning, the coach reads adherence from the ledger, the cards and quick-add run in the user's timezone with the served next-due as the gate, and schedule-revision, wipe and restore writes invalidate the caches they feed. |
||
|
Marc-André Bombeck
|
f790e14edd |
fix(api,lib): consent before reuse, tighter inputs and caches that tell on themselves
Per-status texts check AI consent before the unchanged-data refresh can re-stamp them, the comprehensive fingerprint folds in the comparison baseline and generation language so toggling either regenerates once, and profile, threshold and source-priority writes evict the analytics, targets and derived caches they feed. CSV export neutralises spreadsheet formula prefixes on free-text cells while numeric columns stay typed, the derived batch validates its type token against the measurement enum and carries a per-user rate limit, and a dose synced from the phone app counts as the interactive write it is. Background cache refresh failures log a structured event instead of vanishing, the warm queue runs visit-triggered jobs alongside the nightly walk, the formatter memo evicts one entry instead of flushing wholesale with its option objects frozen, the targets cold build bounds its fan-out, and time in bed unions every writer's envelope while stages stay with the richest recorder. |
||
|
Marc-André Bombeck
|
b346e78cc7 |
perf(medications): one compliance round trip and a memoised formatter hot path
Intl.DateTimeFormat instances are memoised per locale, timezone and options signature — construction dominated the 365-day ledger build and the hot path drops about fivefold. A batched GET /api/medications/compliance serves every card from one request through the same per-medication cache cells the detail page reads, replacing the per-card fan-out; the list response stays untouched so the due line never waits on a ledger build. The medications and compliance buckets serve stale-while-revalidate, interactive writes evict while background bulk sync marks stale, the per-id route defers ownership and schedule reads until a cache miss, and the card body reserves constant slots for the status and cycle lines so late data stops shifting the layout. |
||
|
Marc-André Bombeck
|
03453aaed2 |
fix(insights,cache): surface revalidation to clients and evict on interactive writes
Stale-served insights responses now carry a revalidating flag; the advisor and comprehensive queries poll on a bounded 25 s cadence until the background warm lands, so an open session converges instead of pinning yesterday's briefing for an hour. Interactive measurement writes (single create, edit, delete, bulk-delete, restore) hard-evict the analytics cache so a manually entered value is reflected on the next read; background batch sync keeps the mark-stale path. The visit-triggered force warm collapses to one job per user instead of one per locale, and the umami script proxy failure now annotates a named action with the failure reason instead of a bare flag. |
||
|
Marc-André Bombeck
|
298e4c34b4 |
perf: collapse the medication compliance hot path and the dashboard fan-out
The medication list fired one compliance request per card every 30 seconds, and each request scanned the medication's full intake history and expanded the recurrence calendar five to six times. The endpoint now mints bands once per request, serves the sub-windows from that single ledger, bounds the event read to a year of selected columns and sits behind a write-invalidated 15-minute server cache; the cards poll at five minutes. The dashboard drops two redundant checklist queries by reading counts from the snapshot, the BMI chart derives from the weight series instead of refetching it, the detail page no longer serialises behind the auth probe and seeds its query from the list row. Expensive analytics reads gain a shared per-user rate limit, and the service worker stops serving cross-version shells, enables navigation preload and trims caches in one pass. |
||
|
Marc-André Bombeck
|
53e87ba345 |
perf: consolidate integration status query, add mood-insights SWR, defer below-fold charts
Settings → Integrations fired five status round-trips on every mount — the consolidated /api/integrations/status alongside the four per-provider /api/<provider>/status queries. Extend the consolidated endpoint to carry every field the four cards render (Withings activity scope, WHOOP/Fitbit backfill state, moodLog webhook secret + entry count) and drive all four cards off the single envelope. The legacy per-provider routes stay for the iOS/test callers; the web cards no longer hit them. Mood-insights cache was a plain 60 s TTL that was hard-evicted on every mood write, so an active logger re-paid the multi-second cold compute on every entry. Add stale-while-revalidate to ServerCache (wrapSwr / cachedSwr + a markStaleByPrefix that collapses the fresh TTL without dropping the value), give the moodInsights bucket a 10-minute stale window, route the read through cachedSwr, and switch the mood-write invalidation from a hard evict to a mark-stale so the prior aggregate serves immediately while a single background recompute warms a fresh one. Defer the eager Recharts on the mood-insights surface. The trajectory forecast card pulled Recharts into the initial chunk of every trajectory-eligible metric page though the main chart is already deferred; split its fan band into trajectory-fan.tsx behind next/dynamic. The three below-fold mood mini-charts (distribution / weekday / time-of-day) were static imports under an already-dynamic hero line chart; defer them with next/dynamic. Each loader paints a skeleton sized to the chart's own band so the chunk arrives without a layout shift; the charts stay Recharts and visually identical. (cherry picked from commit 2b627893ab6b4d7bc457fc18e6e5585b048190e1) |
||
|
Marc-André BombeckandClaude Opus 4.8
|
50e0a65228 |
chore(release): v1.7.0
v1.7.0 — health-record export (PDF + FHIR), flexible medication schedules, first-paint dashboard snapshot, full HealthKit coverage, offline sync delta feed, Coach data clustering. See CHANGELOG. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Marc-André Bombeck
|
affede31a4 |
v1.4.34 — apple-health import + reliability + web-freeze
The final functional web release before the iOS native client lands.
Headline work: a streaming Apple Health `export.zip` importer for
prospective iOS-migrating accounts, a server-side aggregation cache
that collapses the three hottest dashboard reads onto in-process LRU
slots with single-flight coalescing and per-user invalidation, a
broader compliance classifier with a dedicated `early` bucket so
ahead-of-window doses count as compliant, the Coach launch surface
hoisted onto every authed page with a dashboard-hero CTA, and a
trimmer Settings shelf that folds Sources into Targets. The release
also lights up the `cache.<name>.outcome` annotation on the active
wide event so production logs carry the cache hit-ratio signal
without leaking userIds.
### Added
- **Server-side aggregation cache.** A new `ServerCache<T>` primitive
(`src/lib/cache/server-cache.ts`) extends the v1.4.33 Coach snapshot
LRU shape into a reusable per-route layer with TTL expiry,
capacity-bounded LRU eviction, single-flight read coalescing, and
per-instance hit / miss / eviction / stampede counters. Wires the
three hottest dashboard reads — `/api/analytics` (slim + default),
`/api/gamification/achievements`, `/api/medications/intake?scope=compliance`
— and bolts per-user invalidation onto every measurement / mood /
medication / workout / dashboard-widget write endpoint so the next
read paints fresh data instead of waiting out the TTL. Every cache
hit / miss surfaces on the active wide event as
`cache.<name>.outcome` + `cache.<name>.key_hash` so production logs
carry the hit-ratio signal without leaking userIds.
- **Apple Health `export.zip` import.** New endpoint
`POST /api/import/apple-health-export` (synchronous multipart
upload, asynchronous ingest via a dedicated pg-boss
`apple-health-import` worker, per-`MeasurementType` ingestion
stats). Streams the upload straight to disk so a multi-gigabyte
export never lands in V8 heap, hashes the bytes inline for
content-based idempotency, and unpacks `apple_health_export/export.xml`
with a hand-rolled ZIP central-directory walker that handles Zip64
for archives past the 4 GB barrier. The parser folds every
`<Record>`, `<Workout>`, `<Correlation>`, and `<ClinicalRecord>`
into the existing `Measurement` and `Workout` row shapes — spot
rows keyed by `HKMetadataKeyExternalUUID` (or a deterministic
`sample:<sha256>` fallback), cumulative HK types collapsed into
one `stats:<HKType>:<YYYY-MM-DD>` row per user-local day to match
the iOS daily-aggregation convention. Live per-stage progress
surfaces through `GET /api/import/apple-health-export/{jobId}/status`.
An admin variant at `POST /api/admin/import-apple-health-export`
imports on behalf of a target user (cookie-only `requireAdmin()`
gate; Bearer tokens never elevate).
- **`ImportJob` schema model + migration.** New Prisma model captures
the per-upload state machine (`queued | unpacking | parsing |
upserting | done | failed`), the content-hash for idempotency
short-circuits, and the per-`MeasurementType` ingestion counters
the status route surfaces back to the client.
- **`lastSeenByType` on `/api/analytics`.** Both the slim
(`?slice=summaries`) and default slices return a per-type
`{ lastSeenAt, daysAgo }` map. The dashboard trend tiles wire 12
mounts to a new `tileStaleDays()` helper so an "X days / weeks /
months ago" hint paints under any per-metric tile whose last
sample crosses the 7-day floor. Six locales gained additive
week / month plural keys; Polish uses the genitive-plural form
that covers every non-1 count the bucket math can produce.
- **Dashboard "Ask the coach" hero CTA.** A new pill next to the
existing "Hinzufügen" / "Add" launches the Coach drawer directly
from the dashboard hero. The `<CoachLaunchProvider>` hoisted from
the insights layout to the auth shell so the drawer is reachable
from every authed route; the floating action button stays scoped
to `/insights/**` so it cannot distract on the dashboard.
- **Shared achievements query hook.** `useAchievementsQuery()`
centralises the three previous consumers — recent-achievements
card, the `/achievements` mother page, and the unlock notifier —
onto one TanStack queryKey + cache slot so dashboard cold mount
fires the endpoint once instead of twice.
- **Typed authed Cache-Control presets.** New
`src/lib/http/cache-headers.ts` module exports
`NO_STORE_BUT_BFCACHE = "private, max-age=0, must-revalidate"`,
`SHORT_LIVED_PUBLIC`, and an `applyAuthedHeaders()` helper. The
presets land at the framework level via a `next.config.ts`
`headers()` rule that stamps the bfcache-friendly directive on
every authed HTML response, restoring Chromium bfcache eligibility
for in-app back / forward navigation.
- **`scripts/print-bundle-report.mjs`.** Restores the at-a-glance
"top client chunks by parsed size" signal Turbopack dropped from
`next build`; reads `.next/analyze/client.json` (written by
`@next/bundle-analyzer` when `pnpm analyze` runs with `ANALYZE=1`)
and prints a sorted table plus totals.
### Changed
- **Settings sidebar: "Sources" folded into "Targets & Sources".**
Per-metric threshold ranges and per-metric source priority now
share one `/settings/thresholds` page so the same metric's
threshold and device-source preference sit together instead of
one sidebar entry away. `/settings/sources` keeps a
`permanentRedirect` so external bookmarks and docs links follow
through unchanged. Section count: 11 → 10.
- **Insights tab strip: vital pills collapse under a "Vitals"
parent.** Five vital pills (HRV, Resting HR, Oxygen, Body
Temperature, Active Energy) hide behind one parent pill that
opens a popover sub-list. Parent-pill active state mirrors the
URL so spatial orientation survives the collapse; each sub-page
keeps its own URL and bookmark resolves unchanged. Strip
footprint: 14 → 10 entries when every metric has data.
- **Coolify env-var audit.** `mcp__coolify-apps01__env_vars`
inspection captured the section-1 / section-2 duplicates that
have accumulated under apps-01 since v1.3.1. Audit pinned at
`.planning/round-v1434-iwa-coolify-env-audit.md`; no env-var
deletes performed (operator action).
### Fixed
- **Compliance classifier: early intakes count as compliant.** The
classifier picked up a dedicated `early` bucket (window-start
minus three hours through window-start), the `on_time` post-window
tolerance widened from one hour to three hours, and the `late`
band sits between the new `on_time` ceiling and the configurable
`lateMinutes` knob. Heatmap consumers route the new bucket
through the compliant path; the dedicated `early` counter rides
on `DailyComplianceEntry` for downstream consumers that want to
distinguish ahead-of-window from on-window intakes.
- **Compliance heatmap fallback retired.** The v1.4.33 defensive
`looksClassifierBug` fallthrough is gone now that the underlying
classifier widening removes the every-dose-very-late mode the
fallback covered.
### Performance
- **`/api/gamification/achievements` consumer collapse.** Two of
three previous consumers shared the same TanStack queryKey
literal; the third carried a per-user discriminator so the cache
treated it as a fresh slot. Dashboard cold mount fired the
endpoint twice. Collapsing onto the shared hook trims that to a
single request; with the new server-side cache slot warm, the
duplicate-mount worst case rides single-flight coalescing.
- **GHCR build fires once per release tag.** The
`docker-publish.yml` workflow lost its `push.branches: [main]`
trigger; the `:latest` raw-tag enable rule moved onto the tag
ref so each release produces exactly one multi-arch build that
refreshes both the semver tags and the `:latest` alias.
- **NFT-trace warnings silenced.** `next.config.ts` gained an
`outputFileTracingExcludes` block that narrows the Turbopack
tracer away from the `MAXMIND_LICENSE_KEY` env-access path so
the trace-report no longer warns on every dependent route file.
### Refactor
- **Dashboard analytics consumers.** The two cards that previously
decided their own freshness copy now lean on the typed
`tileStaleDays()` helper and the additive `lastSeenByType` field
on the analytics response. Mood and BD-Zielbereich tiles keep the
default `null` because they have no underlying per-type freshness
signal.
### Accessibility
- **Dashboard hero CTA hits the WCAG 2.5.5 touch-target floor.**
The new "Ask the coach" pill carries `min-h-11` on mobile and
the matching `sm:min-h-9` desktop floor so the touch target
stays above 44 px on the Pixel 5 boundary.
- **Insights tab-strip Vitals parent reads correctly to assistive
tech.** Parent pill carries `aria-current="page"` whenever the
current URL matches one of its child vital sub-pages, plus a
`data-slot="insights-tab-strip-group"` hook for visual-regression
testing.
### Internal
- **Two e2e flake windows tightened.** `onboarding-flicker` swapped
its 12-sample 50 ms poll loop for a single Playwright
auto-retrying `toBeHidden({ timeout: 700 })` assertion so the
1-2 ms race window between the analytics-pending shell and
`useAuth().user` resolving collapses to a single retry slot.
`mobile-viewport` dropped `nav a[href]` from the touch-target
sweep (the bottom-nav owns its own WCAG spec) and gated the
44-px floor on `matchMedia('(min-width: 640px)').matches === false`
so the Pixel 5 viewport never tripped into the `sm:` tier during
WebKit render commits.
- **Server-cache observability.** Every `cached()` invocation passes
an `annotate` callback that lands two keys on the wide event:
`cache.<name>.outcome` ∈ `{ hit | miss | stampede }` and
`cache.<name>.key_hash` (non-reversible djb2 32-bit hash). Ops
can grep `cache.analytics.outcome` over any time window to
compute the hit ratio per deployment.
- **AASA followups verified.** The v1.4.33 `/.well-known/apple-app-site-association`
handler serves a direct 200 on every fronting origin; Apple's
CDN has ingested matching bodies for all three domains.
### Web freeze
v1.4.34 marks the last functional release of the web codebase before
the iOS native client launches. Subsequent v1.4.x tags carry security
fixes, dependency bumps, and hotfix-only corrections. New feature work
is paused until the iOS app clears Apple review, at which point a
v1.5.0 version-bump-only release tags the milestone. The Prisma
schema head comment and `.planning/v15-strategic-plan.md` §5
decision-log row pin the freeze trigger in-tree.
|