Related to #585. Investigated the intraday pulse chart's read path
end to end: the ZIP-import raw-sample fold, the live-sync uploaded
10-min bucket fold, their merge, and the chart's render logic. All of
it treats the two ingestion shapes consistently — the chart has no
shape awareness at all, it paints whatever bucket series it receives.
No read-side or display-side defect explains a systematic density
difference between the two ingestion paths.
What was missing is visibility into the write and read sides that
would let a future report actually be diagnosed instead of
re-investigated from scratch. The batch endpoint's wide-event trace
never broke a skip down by reason, and never distinguished an
HR-bucket write from any other entry in the same request. The
intraday route's own trace surfaced only the final bucket count, with
no visibility into which shape (raw sample rows vs uploaded bucket
rows vs folded hourly rows) a day's data actually came from. Both
traces now carry that breakdown, so the next report of "the chart
looks sparser after a given date" is answerable by grep instead of by
re-deriving the whole read path again.
Closes#584. `/insights/pulse` computed hasRestingHr from
RESTING_HEART_RATE presence and used it to swap the ENTIRE primary
chart and the Coach read strip to that series whenever any resting
row existed, while the title, stat strip, capture action, and
"show all values" link kept saying PULSE. A user could see a
resting-only chart under a page that claimed to be pulse throughout.
The chart now always includes PULSE as its first series; when resting
data exists, RESTING_HEART_RATE renders as a second, clearly-labelled
series (the same multi-series pattern the blood-pressure page uses for
systolic/diastolic) instead of replacing it. The Coach read strip
stays pinned to PULSE. The page also drops the resting-calibrated
target band it used to paint behind the chart — that band is
calibrated to steady-state resting readings and would misreport an
expected workout spike in raw PULSE data as "out of target"; the
personal target belongs on the dedicated resting-pulse page against
the clean resting series.
Closes#588. The ZIP extractor decompressed export.xml synchronously
into one in-memory buffer, then wrote it with a blocking writeFileSync
— on the same event loop the web server runs in the default
single-container deployment. A real multi-year export could pin the
Node main thread for minutes, freezing every other request, while
briefly holding roughly 2x the decompressed size in heap. A process
that died mid-extraction never reached the failure path, so the
ImportJob row stayed stuck in "unpacking" with "0 rows imported"
forever and no error ever surfaced.
Extraction now streams through zlib's async inflate into a file write
stream, moving the decompression work off the event loop and bounding
memory to the chunk size. The zip-bomb ceiling moves from
inflateRawSync's maxOutputLength to an equivalent byte-counting
transform on the streamed output.
Orphan-ImportJob reconciliation used to run once, at worker boot.
docker-compose's restart:unless-stopped does bring a crashed worker
back, but a fast restart can land before the stuck row's heartbeat
goes stale or pg-boss stops reporting the job as live — and because
the boot-time pass never re-runs on a healthy worker, that row was
never revisited again. A 15-minute cron now re-runs the same reconcile
so a genuinely abandoned import converges to a visible failure instead
of staying silently stuck.
Marc-André Bombeck