9745 Commits
Author SHA1 Message Date
Marc-André Bombeck 41e75b5bde Merge branch 'fix/v147-review-pdf' into release/v1.47.0 2026-11-03 23:30:26 +01:00
Marc-André Bombeck da48f1af9e Merge branch 'fix/v147-review-wolfi' into release/v1.47.0 2026-11-03 23:30:25 +01:00
Marc-André Bombeck 7af9ebe480 docs(stats): say what the r² clamp does and does not fix 2026-11-03 23:30:21 +01:00
Marc-André Bombeck 0e7134a2ea fix(labs): let a deleted number not hold a text marker's unit, and read the winner's unit after a race
Refs #1313
2026-11-03 23:30:21 +01:00
Marc-André Bombeck 55a3a9b53d fix(medications): keep the rrule anchor for a rule with BYSETPOS
The library drops a period's candidates before DTSTART before BYSETPOS
picks among them, so a start moved into the middle of a week chose a
different day. The property now generates BYSETPOS rules and kept red
without the guard.
2026-11-03 23:30:20 +01:00
Marc-André Bombeck 42cae738f1 build: pin the runner's Node and PostgreSQL client packages
nodejs-22 and postgresql-18-client are installed at exact package
versions, so a rebuild of the same commit runs the same Node and the
same pg_dump; apk upgrade still moves the libraries under them. No
updater reads an apk pin. The image scan does the job instead: Wolfi
records the fixed version for each advisory, so a pinned package with a
published fix turns the HIGH/CRITICAL gate red and names the version.
A guard holds the pin format and keeps the client's minor on the bundled
database's.
2026-11-03 23:27:06 +01:00
Marc-André Bombeck a11ea8444d build: ship a wget for health checks written outside the image
The Wolfi runner has no wget, but health checks outside this repository
call it: compose files copied from here up to v1.46.6 run
'wget --no-verbose --tries=1 --spider .../api/health', and the TrueNAS
catalog app renders 'wget --quiet --spider <url>'. Either replaces the
image's HEALTHCHECK, so such a container never turned healthy and a proxy
that routes on health dropped its traffic.

/usr/local/bin/wget now runs scripts/wget-shim.mjs: the URL, --spider,
-q, -nv, --tries, --timeout/-T, -O - and -O /dev/null, -S; other flags are
ignored. Exit 0 on 2xx/3xx, 1 on any other status, a refused connection
or the timeout. It follows no redirect and refuses any other output file
before sending a request, so it is no download tool. The image's own
HEALTHCHECK and the bundled compose file keep the Node probe.
2026-11-03 23:27:06 +01:00
Marc-André Bombeck 5705184857 test(documents): drop a mock of a module that no longer exists 2026-11-03 23:24:49 +01:00
Marc-André Bombeck 253176d01a fix(documents): index every page of a PDF read by its text layer
"Read with AI" and the index backfill wrote a text-layer read as a complete
`local-pdf` index although it held only the first ten pages (the render
cap), while the local text index reads up to 200. `prepareVisionInput` takes
the number of pages a text-layer read may take, and both index writers ask
for the local index's own cap, so the index is complete.

The `partial` flag on a text read had no reader and is gone; the
extraction and the lab scan report their missing pages themselves.

Refs #1311
2026-11-03 23:24:49 +01:00
Marc-André Bombeck 763d6c600f fix(labs): bound each text-layer read of an unrenderable PDF
`planTextLayerReads` grouped whole pages into the reads the deadline
allows, with no bound on a read's size, while the text path refuses a part
past `OCR_TEXT_READ_MAX_CHARS`. A dense report in a request's three reads
went out as reads of 12 000 characters and more, which ends at the output
limit or loses rows.

`planTextLayerChunks` now cuts a page longer than one read at line
boundaries, keeps a page per read while the deadline allows that many, and
otherwise packs pages into reads of at most twice the surface's target
(8 000 characters for the lab scan, 5 000 for the document extraction).
What the reads cannot take is read from the start, and the pages left over
are reported with the pages without a text layer: `unreadPages` on the lab
scan, `extractionIncomplete` on a document.

Refs #1311
2026-11-03 23:24:38 +01:00
Marc-André Bombeck 119431dc15 fix(documents): fence document text in the summary and the filing suggestion
Both text modes spliced the text after a bare `OCR TEXT:` label, while the
extraction, the lab scan and the label read fence it. A PDF read by its
text layer on a server that cannot render now reaches these prompts, and
the automatic summary reads imported PDFs that way, so text the page does
not show (white on white) could write past its own end. The text is now
fenced with `fenceDocumentText`, the browser-OCR path included, and a marker
forged inside it is scrubbed.

Refs #1311
2026-11-03 23:24:26 +01:00
Marc-André Bombeck b975e86b39 test(clinician-share): start every case on an all-modules-on map
Two cases switch the doctor report off for the rest of their block, and
the blocks that did not set the map themselves inherited it under a
shuffled order, so a record share stopped aggregating.
2026-11-03 22:39:16 +01:00
Marc-André Bombeck 144840bab9 test(documents): type the fixtures the text-layer cases added instead of casting them
The stored bytes are already a Buffer and the MIME literal already fits
its union, so neither needs a cast; the external pick goes through one
helper. The as-never ratchet tightens accordingly.
2026-11-03 22:36:18 +01:00
Marc-André Bombeck 7cb67c2e75 Merge branch 'fix/v147-nightly' into release/v1.47.0 2026-11-03 22:30:50 +01:00
Marc-André Bombeck a598b00081 test(coach): start every snapshot case with no readings
The query-shape case read the measurement mock without setting it and
passed only after a case that left rows behind.
2026-11-03 22:25:43 +01:00
Marc-André Bombeck 1ac69420d5 test: reset to the factory answer instead of casting a new one
A reset mock goes back to the implementation its factory gave it, which
is the fresh session and the open capability these cases want; the cast
replacements grew the test-realism count.
2026-11-03 22:22:32 +01:00
Marc-André Bombeck 9a3140a4d3 test(backup): hold the record-shaped slices at their edges
The health score day, the intraday curve, a medication with its
composition and custom category, an intake row and the cycle's own
symptoms had fields and defaults no slice test met, added since the
floor was set; the backup-validation score had drifted to 40.2 % under
its 41 % floor. It is 54.1 % with these cases, and the floor follows
to 53 %.
2026-11-03 22:15:05 +01:00
Marc-André Bombeck fa439601c4 test(labs): give the unitless-marker refusal its own marker
The refusal case leaned on the marker the case before it mints, and
under a shuffled order it ran first and minted it itself.
2026-11-03 22:13:32 +01:00
Marc-André Bombeck b1cb387062 Merge branch 'fix/v147-pdf-unrenderable' into release/v1.47.0
# Conflicts:
#	.env.production.example
2026-11-03 22:10:14 +01:00
Marc-André Bombeck 3547c4bd2f test: pin the intake history's success badge at the one it now renders
The taken badge moved into one shared component, so the file carries the
token once.
2026-11-03 22:08:05 +01:00
Marc-André Bombeck 394828449a test(documents): read unrenderable PDFs through the routes against Postgres 2026-11-03 22:07:28 +01:00
Marc-André Bombeck df6d20b696 fix(documents): read a PDF the server cannot render by its text layer
On a host whose CPU cannot run the page renderer (x64 without AVX2, arm64
musl without LSE/RCpc), every AI read of a PDF for a provider that only
takes images failed with "Scanning PDFs needs a Claude vision provider",
which says nothing about the cause. Most medical PDFs are produced
digitally and carry every word in their text layer, which pdf.js reads
without rendering.

`prepareVisionInput` now answers a text read (`mode: "text"`) when the
renderer is unavailable on this host, the provider cannot take the PDF
whole and the PDF has a usable text layer. Summary, suggest and the
background summary send that text through their existing text mode;
"Read with AI" and the index backfill write it as the local text index it
is, with no provider call; the automatic index leaves it to its local path.
`planVisionReads` does the same for the extraction and the lab scan: text
reads by page group, with anchored provenance, and the pages without a text
layer reported as `extractionIncomplete` on a document and as `unreadPages`
on a lab scan. A supplement label PDF is read from its text layer too.

A scan on such a host now answers `documents.inbound.pdfRendererUnavailable`
and `labs.ocr.pdfRendererUnavailable` (422): this server's processor cannot
run the page renderer, so a scanned PDF can only be read by a provider that
takes PDF files directly, or as photos. The `pdfNeedsAnthropic` codes are no
longer emitted, since with a working renderer every provider reads a PDF;
the recorded index outcome keeps its `pdf-needs-anthropic` value with
corrected copy.

The extraction path no longer folds a render failure of one document into
the capability error: like `prepareVisionInput` since #776 it reports
"The PDF pages couldn't be rendered for scanning." (`extractFailed`), and
the lab scan answers `labs.ocr.extractFailed`.

Hosts that render, and providers that take PDFs whole, are unchanged.

Refs #1311
2026-11-03 22:07:22 +01:00
Marc-André Bombeck c47f61f02a feat(documents): read a PDF's text layer page by page without rendering
The reader child gains a `page-texts` request: the first pages' text
layers, each read the way a rendered page's is (cut and marked past the
per-page cap), plus the document's page count. Nothing is rendered, so it
works where the page renderer cannot run.

`readPdfTextLayer` judges what that read is worth: a page counts only with
at least 40 characters of its own besides white space, so a scanner's
stamped page number or fax header does not pass for a page of text; a PDF
with no such page is a scan and is reported as one.
2026-11-03 22:07:11 +01:00
Marc-André Bombeck d1ab195b43 Merge pull request #1330 from FloMaetschke/feature/injection-site-in-intake-history 2026-11-03 22:06:35 +01:00
Marc-André Bombeck a503c8855d Merge branch 'feat/v147-wolfi' into release/v1.47.0 2026-11-03 21:56:23 +01:00
Marc-André Bombeck 5d36afb487 build: run the image on a Wolfi runner
The dependency and build stages stay on node:22-trixie-slim, where pnpm
picks the glibc native packages. The runner moves to wolfi-base, pinned
by its index digest: glibc, about fifty packages, and every one of them
known to the image scan, Node and the PostgreSQL client included.

nodejs-22, tzdata and postgresql-18-client come from apk, so the pgclient
stage and its pin guard go. The builder writes its Node module ABI and
glibc version out and the runner refuses to build on another ABI or an
older glibc. Wolfi's npm is major 12 and runs no install scripts unless
package.json allows them, so the Prisma CLI install names the three
packages that have one, and the build checks that the schema engine file
is there before asking the CLI about it. npm is installed for that step
and removed in the same layer; the check that nothing is left asks a
fresh shell, whose command cache cannot remember the deleted binary.
2026-11-03 21:55:30 +01:00
Marc-André Bombeck 2c33c2a0a0 test(integration): stop cases reading what another case left behind
- the AI-state cases take their baseline in a world of their own
  instead of from the S0 case;
- the live-window case drops a rollup tier the rollup case built;
- the job readers start each case on an empty queue, and the
  no-schema case starts pg-boss again after it;
- the server move case reseeds the server it emptied;
- the extremes cases rebuild the tier one of them breaks on purpose;
- the first Google Health backfill seeds its own history;
- the batch budget warms the process caches as the reads do;
- the HOT-update count is read as a difference: since Postgres 15 the
  transaction view also shows an earlier transaction's unflushed counts;
- the MANAGE matrix checks its effect set after all cases rather than
  in a case of its own that a shuffle could run first.
2026-11-03 21:35:57 +01:00
Marc-André Bombeck d7046c9737 test(integration): keep the stateful walks in order, give the rest their own rows
The migration before-and-after files, the dump directory, the geo
backfill's three passes, the settings trip, the backup parity steps,
the lab conversion record and the pulse method's dry run before its
rebuild are each one sequence by design; they now say so and keep their
order with shuffle: false. The CSV import's zone and crash cases wrote
into the account the lab and medication walks count, and now use
accounts of their own.
2026-11-03 21:35:57 +01:00
Marc-André Bombeck 713806fe9b test(parity): keep each route walk in order under a shuffled run
A parity file is one account walked through a route's cases, each
request reading what the earlier ones stored, and the recorded files
pin that sequence. The nightly shuffle reordered the steps and every
snapshot after the first moved. Keep each walk in order with
shuffle: false, wrapping the files whose top-level blocks build on each
other, and note the rule for order-bound files in the testing guide.
2026-11-03 21:35:56 +01:00
Marc-André Bombeck 899a054e64 Merge branch 'feat/v147-mcp-reading-id' into release/v1.47.0 2026-11-03 21:33:50 +01:00
Marc-André Bombeck 66709222b2 docs(api): describe the lab reading id next to the analyte id 2026-11-03 21:33:18 +01:00
Marc-André Bombeck 52e6bcd480 test(mcp): prove a report's lab link lands on that report's reading
Two draws of one analyte a year apart and two reports on the same day, against
Postgres: each report lists its own reading id, and fetching it returns that
reading with that report in its links, not the latest one.
2026-11-03 21:33:17 +01:00
Marc-André Bombeck ba27e728cf feat(mcp): fetch one lab reading by its own id
A link to a lab result named its analyte, and fetch lab:<analyte> answers
with the latest reading of that marker. Following a link from a report that
holds an earlier draw therefore landed on a later reading, whose links named
a different report and visit.

fetch labresult:<id> now returns that one reading (value or text result,
unit, reference range, status, date) with its own links, behind the labs
module and narrowed to the caller's live rows. Every link to a lab result
uses this id, each reading in get_labs carries it, and fetch lab:<analyte>
names the reading it quotes in metadata.readingId. The analyte view and
search are unchanged.

Refs #1323
2026-11-03 21:33:16 +01:00
Marc-André Bombeck 691c2e7301 refactor(labs): shape one reading from one shared select
The labs block and a single-reading read now build a reading from the same
select and the same shaping, so one reading fetched by its id renders exactly
like the latest reading of a marker.
2026-11-03 21:33:14 +01:00
Marc-André Bombeck cd344125d5 fix(labs): let a text-only marker take the unit of its first number
A marker minted from a text result ("SEE NOTE") has an empty unit, and a
marker with a reading has its unit locked, so the first number to arrive,
a unitless ratio included, was refused against a unit nothing had chosen.
While every reading is text the unit has nothing to contradict, so the
first number sets it. A marker that already holds a number keeps its unit.

Refs #1313
2026-11-03 21:19:28 +01:00
Marc-André Bombeck 377e0d0d3a build: copy pg_dump and pg_restore from the PostgreSQL image
postgresql-client-18 from the PGDG repository brought Perl and Debian's
client wrapper scripts into the runner for two binaries the app calls by
name. A pgclient stage on postgres:18.6-trixie, pinned by digest, now
hands over pg_dump, pg_restore and the libpq they link; the Debian
libraries libpq needs (Kerberos, LDAP, SASL) still come from apt so the
upgrade and the image scan see them. The runner registers libpq with the
loader and fails the build when either binary has an unresolved library;
the major 18 version checks stay.

The client minor follows the bundled database's: a guard holds the two
pins together, and the Dockerfile's Dependabot watch now ignores a
PostgreSQL major the same way the compose watch does.
2026-11-03 21:18:44 +01:00
Marc-André Bombeck cf7905dc92 refactor(medications): hold recurrence.ts at its size baseline 2026-11-03 20:58:46 +01:00
Marc-André Bombeck 227c80878c test: give every case its own mock state under a shuffled order
Cases leaned on what an earlier case set up or left behind: a once-value
the code under test never consumed, an implementation swapped in and
kept by clearAllMocks, a payload only another block's setup provided,
the zone resolver's per-user cache, module state in a hoisted object,
and a scope set one case discovered for the next. Each now resets or
states what it reads.
2026-11-03 20:58:38 +01:00
Marc-André Bombeck 4f7802ba28 test(browser): leave no state behind for the next case
Await the form's explicit unmount: an unmount left running overlaps the
next render's act, and React then commits nothing for the rest of the
file. Return to the newest history entry after stepping back, so the
next case's push adds an entry instead of replacing the forward one.
2026-11-03 20:58:38 +01:00
Marc-André Bombeck 54d0647a6d fix(stats): keep r² within [0, 1] when y sits far from zero
The centred sums lose their last digits when every y is close to one
large value, and a two-point fit came out at r² 1.00000004. Clamp the
ratio to 1 and the y sum of squares at 0, and pin the two shrunk
counterexamples the nightly property run found.
2026-11-03 20:58:37 +01:00
Marc-André Bombeck 270c6ca619 refactor(medications): keep the rrule start alignment in its own module
recurrence.ts sits on the source-size ratchet; the helper and its reasoning
read on their own.
2026-11-03 20:57:46 +01:00
Marc-André Bombeck d8743e9111 chore: drop the trivyignore entries for npm's bundled packages
Both covered copies inside the npm CLI of the Node base image. The
runner removes npm, so neither CVE matches anything in the image; a scan
of the built image with an empty ignore file reports neither.
2026-11-03 20:53:29 +01:00
Marc-André Bombeck 3d0ae05192 docs: say the published image renders documents on a Raspberry Pi 4
The image runs the glibc canvas build now, which needs no ARMv8.2
features, so the Pi 4 gets thumbnails and scanned-PDF rendering back
without a code change: the gate only ever closed for musl on arm64. The
gate's comment, the env example and the getting-started prerequisites
say so, and keep NATIVE_CANVAS as the override. The x64 AVX2 gate stays:
the glibc x64 build rendered a PDF page under an emulated core without
AVX2, but it still carries AVX2 code and that one run does not cover
every path.
2026-11-03 20:53:28 +01:00
Marc-André Bombeck de7858100f build: run the image on Debian trixie instead of Alpine
Every stage now starts from node:22-trixie-slim, pinned by its multi-arch
index digest, so the native modules load their glibc builds. The arm64
musl Skia build needs ARMv8.2 (LSE atomics, RCpc loads) and dies with
SIGILL on an ARMv8.0 core such as the Raspberry Pi 4's Cortex-A72; the
glibc build runs there (#1311).

The runner keeps what it had: tzdata, TZ, the build-time OS upgrade, the
nextjs user (uid 1001, now in group nodejs gid 1001), the backup and
GeoLite2 directories and every build-time assertion. pg_dump and
pg_restore of major 18 come from the PGDG apt repository, whose signing
key is fetched by node and checked against a pinned SHA-256 before apt
trusts it; PATH names the major 18 directory so the binary does not
depend on Debian's version-picking wrapper. The canvas hoist takes the
-gnu binary package by the pinned version and still fails the build on a
mismatch. bookworm-slim was the other candidate; on the day of the
switch it carried a CRITICAL zlib finding that trixie-slim does not.
2026-11-03 20:53:20 +01:00
Marc-André Bombeck 65e35761ec build: probe container health with Node's fetch instead of wget
The probe now needs nothing but the node binary the image already runs,
which frees the base image from carrying wget. The image HEALTHCHECK and
the compose healthcheck use the same one-liner: a non-2xx answer, a
refused connection or the 4 s abort exit 1, inside Docker's 5 s timeout.
The boot-storm guard reads the abort from the fetch and refuses a probe
that names wget or curl.
2026-11-03 20:53:20 +01:00
Marc-André Bombeck 775933b7d5 Merge branch 'feat/v147-dbroles' into release/v1.47.0 2026-11-03 20:52:17 +01:00
Marc-André Bombeck 3d1a6224ba test: drop vi imports the removed schema-mirror mocks left unused 2026-11-03 20:49:31 +01:00
Marc-André Bombeck df1e19f1c2 docs: describe the app's database role, its limits and how to opt out 2026-11-03 20:38:46 +01:00
Marc-André Bombeck c7ff22ab00 chore(env): whitelist DB_APP_ROLE and document it beside the database settings 2026-11-03 20:37:48 +01:00
Marc-André Bombeck 485161a657 feat(db): run the app as a role that cannot change the schema
After the migrations the entrypoint ensures healthlog_app, a login role
with row access to the app's schema, MAINTAIN on measurements and
ownership of the pgboss schema, and starts the app with a DATABASE_URL
for it in place of the owner's. The role cannot alter or drop tables,
create roles, schemas or functions in the app's schema, read server
files or run programs, or write the migration history.

The password is an HMAC of a fixed label under API_TOKEN_HMAC_KEY, so a
separate worker container derives the same one, and it is sent as a
SCRAM verifier. Every boot re-applies the grants in one transaction
under an advisory lock, and drops as the owner the indexes the
maintenance job may no longer drop. When the owner cannot create roles,
the server is older than Postgres 17, or the role cannot log in, the
boot says why and the app stays on the owner as before. DB_APP_ROLE=off
skips it.
2026-11-03 20:36:37 +01:00