The edit seed rounded every value to two decimals, so a TSH of 0.004 opened
as 0. Values at or above 1 keep the two-decimal display precision; below 1
the seed keeps three significant figures, written without exponent
notation.
The matcher stored the registry entry's key, so LDL and HDL markers were
recorded as total-cholesterol. The key now names what the marker is (the
catalog slug, or a registry-only key for an analyte with no seed), and the
registry resolves ldl and hdl to the cholesterol entry for the conversion as
before. A key already written in the old form is rewritten by the same fill
step that brings a stale key in line with its name, the first time a reading
needs it. The OpenAPI descriptions say what the key is and that it follows
the marker's name and unit.
The single-reading edit cleared sourceValue and sourceUnit whenever the body
carried a value, and the web edit sheet always sends one, so a date or note
edit erased the "converted from" hint. It is now cleared only when the value
differs from the stored one. The sheet also opened a converted reading at its
stored precision (100.90191999999999); it now starts from the value as the lab
surfaces show it and leaves an untouched value out of the request, so the
stored full-precision number is not replaced by its rounded display.
A converted reading's bounds move with its value, but the printed range text
stays in the unit the report used, and every reader preferred that text: the
list showed "Report range 3.9 to 5.5" beside 100.9 mg/dL, and the doctor
report and the FHIR referenceRange.text did the same. The text is still
stored verbatim. The range resolver now takes the reading's sourceUnit (as a
required field, so no reader can forget to select it) and, for a converted
reading, leaves the window's text empty so every surface states the converted
bounds in the marker's unit, and hands the printed text over separately with
its unit. The lab DTO carries it as printedReferenceText, and the note beside
a reading shows it as "as printed in mmol/L: ...".
Slice 2 of phase 3 of #1095. A reading that stated a unit other than its
marker's was refused. When the marker's analyte is known and the registry has the
pair, the three write paths now convert it instead: POST /api/labs, the OCR
commit and the document commit.
The converted value is stored at full precision in the marker's unit, with what
the report printed in the existing source_value and source_unit columns. The
report's own reference window is in the printed unit, so it is converted with the
value; otherwise the verdict would compare a converted value with an unconverted
window. A pair the registry does not have is still refused as before (the 422
now says why), as is a marker whose analyte is unknown.
The marker's key is filled the first time a reading needs it. The same scan
committed again is still a duplicate: a converted row holds the number in the
marker's unit, so the duplicate check also matches on what the report printed.
Editing the value of a converted reading clears the conversion record, because
the number is then the person's own.
The lab result DTO and the OpenAPI schemas (labs and the OCR commit) carry
converted, sourceValue and sourceUnit.
Refs #1095
Slice 1 of phase 3 of #1095. A Biomarker row keeps only its name, so nothing
said which analyte it was. This adds the explicit alias list in the catalog the
issue settled on, including the registry-only keys (urea, blood urea nitrogen,
uric acid, total calcium, total bilirubin), and sets biomarkers.analyte_key
(from migration 0385) from it.
matchAnalyteKey equals a marker's folded name against the localised names of the
catalog seeds the registry knows and against the alias list. It matches the
whole name only, so "Calcium (ionisiert)" is not "Calcium"; a name two analytes
claim matches neither; and the marker's own unit must be one the analyte reaches,
so a marker called "Glucose" in a unit the registry cannot convert stays
unlinked.
The key is set when a marker is created (the biomarker route, the mint used by
lab writes, the legacy backfill) and, for an existing marker, by
ensureAnalyteKey the first time a reading needs it. It never overwrites a key
that is set. There is no boot job for the existing markers: the conversion that
uses the key fills it on first use, and a sweep can be added later if wanted.
The key is returned on the biomarker API and in the OpenAPI schema.
Refs #1095
MCP `search` and `fetch` link a metric as `/insights?metric=<id>` and a lab
value as `/labs?analyte=<name>`. Neither page read its query, so every link
opened the overview or the full list.
`/insights` now resolves the id server-side, in whichever spelling the
result carries (a Coach source key such as `hrv` or `bp`, a signal or
metric-status key such as `GRIP_STRENGTH`, a measurement type, or a
sub-page slug), and redirects to the metric's page, or to its value list
when no page focuses it. An id it cannot place renders the overview as
before.
`/labs` asks for the newest reading stored under the exact analyte name
and opens its marker's page; a name with no reading keeps the list and
drops the query.
"Read with AI" and the lab scan used to run their model calls inside the
request. At the default response time a read of three calls outlasted the
100 s a proxy in front of the server commonly allows, and the person saw a
cut connection for a read the server was still doing.
Both now run as a background run (`DocumentAiRun`, migration 0366):
- `POST /api/documents/inbound/{id}/index` with `Prefer: respond-async`
answers 202 with a run id once every quick refusal has passed. Without
the header it answers exactly as before, which the iPhone app relies on.
A second request while a read of the same document is queued or running
attaches to that run.
- `POST /api/labs/ocr/extract` always answers 202. Only the web client
calls it.
- `GET /api/ai-runs/{id}` serves the run: the same body the synchronous
route answers with, or the failure code and status it would have given.
Scoped to the caller; another account's run is a 404.
The worker (`document-ai-run`, a long queue with a one-hour expiry, one
job per run under `lockedPass`) claims the run, asks the capability again
for the record and re-checks the wire for the provider it picks, sets the
run's deadline from the person's AI response time and refuses a read that
cannot fit the job's expiry. Sync route and worker run one body per kind,
so the two paths cannot drift. Budget is reserved at enqueue and settled
once by whoever ends the run.
Input and result are sealed with labelled AES-256-GCM, registered for key
rotation; the input is dropped when the run ends. A five-minute reaper
fails a run no worker took within fifteen minutes (and hands its
reservation back), fails a run past its deadline, and deletes a finished
run after an hour. Runs are outside backups and inside the data wipe.
The web client polls through the query-key factory and says, calmly, that
the read continues in the background, and when no worker has picked it up.
A numeric lab reading was stored with its value as received and the marker's
unit stamped on it, so 5.6 mmol/L against a marker kept in mg/dL became
5.6 mg/dL and read as far below range. Both the manual/API write and the
scan commit did this; nothing compared the two units.
- POST /api/labs returns 422 labs.unit.mismatch, with the marker's unit and
the reading's unit in meta, and writes nothing. The unit is compared only
when one was sent: on the biomarkerId path it may be omitted, and the
marker's unit is then the stated one. A qualitative reading has no unit.
- The scan commit skips such a row with reason unit_mismatch, so the outcome
reads partial or failed instead of a clean save, and the toast names the
analytes concerned rather than saying "duplicates".
- The extract response carries the matched marker's unit. The review screen
shows a calm badge on a row in another unit, starts it unticked like a
duplicate, and treats it as a blocking error on the unit field if it is
ticked anyway.
- A small lab-specific normaliser forgives spelling only: spacing, the micro
prefix, the case of the litre, and a unit printed in capitals for a short
list where capitals cannot mean another unit. Case is otherwise significant,
so mIU/L and MIU/L, or g/L and G/L, are never merged.
The document-commit path already refuses a mismatch and is left as it is.
Readings stored wrongly before this cannot be identified, because the
original unit was never kept; they are not touched.
Adds EN and DE strings; the other locales are left to the maintainers. The
OpenAPI spec and the error-code catalogue carry the new code.
Fixes#1094
Range bars and the lab list, detail rows, chart tooltip and reference
ranges printed numbers with a plain `String()`, so a German reader saw
"1.8 mIU/L" and "61.3 kg" under a German interface. The same was true
of the mean, GMI, eA1C and the advanced indices on the glucose panel.
`RangeBar` now prints value, bounds and delta through the locale number
formatter and takes an optional `formatValue` for surfaces with their
own precision. Lab values keep their trim rule (whole numbers bare, at
most two decimals) in `formatLabValue`, which now hands the digits to a
caller-supplied locale formatter; `useLabNumber` wires it to the
reader's locale for every lab surface. Inputs are unchanged.
The scan commit and the review screen looked for an existing reading in
the UTC day of the scanned date. A hand-entered morning draw east of UTC
sits on the previous UTC day, so scanning the same report wrote the
reading a second time. Both checks now compare calendar days: a date-only
reading's stated date, or the user's day of a hand-entered instant.
Local lab OCR reads a report photo on the device with tesseract.js, whose
defaults load the worker, the WebAssembly core and the language data
from cdn.jsdelivr.net. The production CSP allows workers and fetches only
from this origin, so the engine could not load; where it did load (dev),
the browser of anyone scanning a report contacted a third-party CDN.
The worker, the three LSTM core builds and the deu and eng best_int
language data (about 17 MB) are now copied from pinned packages into
public/tesseract/ at prebuild and predev, and createWorker is pointed at
them with the worker started from its URL rather than a blob. A test
holds the options against the staged file list.
Reading a stored document (suggest, summary, extract, index, chat,
reindex), scanning a lab report and turning medication text into a
schedule now answer under documentAi, labsOcr and medicationExtract. The
operator's master switch and the new reading-documents switch stop the
work and the egress; a module off for the record, a delegate inside
somebody else's record, and a missing receipt refuse with the capability
envelope. Existing no-provider codes keep their status.
The document pick and the labs pick re-check the capability for the
provider they picked, immediately before anything is sent, so a job
enqueued before a switch flipped finds no provider and falls back to the
provider-free text layer. Consent follows the document rule for all
three: any provider that leaves the machine needs an ai_extraction or
ai_full receipt, answered for the provider actually picked.
The fenced Coach chat and attachments answer under coach and documentAi.
Uploads are always stored and indexed; the background summary is only
enqueued while documentAi is open. The capability probes report the
capability beside the provider answer and keep a read offered when only
the receipt is missing. medications/extract keeps its 503 for no provider
and adds ai.provider.none, and its budget refusal gains the
coach.budget.exceeded code.
A self-hoster scanning a stack of lab reports ran into a fixed six scans
an hour with no way to raise it. LABS_OCR_LIMIT_PER_HOUR now sets the
per-user ceiling (default 6, clamped to 1-1000, falling back to the
default when unset or not a number), the same shape as
DOCUMENT_AI_LIMIT_PER_HOUR. It is on the compose whitelist, in the
production example, the env manifest and the AI provider guide.
The route also charged the slot before it had looked at the request, so
a malformed body, an unsupported or oversized file, a PDF that could not
be rendered or a scan the daily budget refused still used up the hour.
Those failures hand the slot back now; a scan that reached the provider
keeps it. The 429 carries meta.retryAt. Refs #985
The free-text reference-range parser decided what a printed window meant from
an alternation of German and English words. The app ships six locales, and a
lab prints in the language of the country it stands in, so `jusqu'à 5,0`,
`hasta 5,0`, `fino a 5,0` and `do 5,0` all fell through to `low: null,
high: null`. The string was stored, the reading was then judged against the
biomarker's own band or against nothing at all, and no surface anywhere said
that the lab had printed a window the app could not read.
The vocabulary is TABULATED here, not derived from `messages/<locale>.json`
the way an app-owned word list would be. These words are prose a lab printed
on paper; the app owns no string they could be read out of, and the language
of a report has nothing to do with the language of the UI. So the table is
hand-written, keyed by the shipped `Locale` union so that adding a locale
fails to compile until someone has answered how that language prints a range,
and every locale's words are tried against every input at once. The cost is
written down at the table: a report in a language absent from it stays
text-only.
Three things had to change beside the word lists. Introducing words are now
stripped only on a second pass, because Italian "da", Polish "od" and French
"de" also state a floor on their own and stripping first read "da 3,5" as a
bare number. Input and table are diacritic-folded and their apostrophes
normalised, so a table row written "jusqu'a" meets a report's "jusqu'à" and
the curled apostrophe a word processor emits. And one-letter separators ("a",
"e", "y") are only accepted with whitespace on both sides, so "3.5e5" is not
read as a window. Decimal commas already worked and are pinned by the new
cases.
An unreadable window now has a name. `isUnreadableRange` separates "the report
stated nothing" from "the report stated something nobody could read" — the two
were indistinguishable at every call site, which is what let the French case
pass unnoticed. The manual form already showed the reader that the text was
kept but not read; the two ingest paths now count it into their wide event, so
an unsupported notation surfaces as a number rather than as silence.
The German and English lists are the previous alternation split word for word,
and the existing suite covering them passes unchanged.
A lab runs its own method on its own device, so the window a physician
evaluates a value against is the one printed next to it on that report,
not a catalog band averaged over every lab. Reading a report pulled the
value and dropped that range: the extraction asked for it, the commit
handed it to the biomarker resolver, and the resolver only uses it when
it mints a NEW marker. For every analyte the user already had — the
normal case after a first panel — the printed range was read and then
thrown away.
Add `sourceReferenceLow` / `sourceReferenceHigh` / `sourceReferenceText`
to `LabResult` and carry the range from the report to the reading. The
text is stored whether or not numbers can be read out of it, because
"bis 5,0" and "negativ" state a window too; a range that could not be
parsed is kept verbatim, never replaced with invented bounds.
`resolveEffectiveReferenceRange` is the one place the precedence lives:
the source window wins for that single reading, and the catalog stays
the net for readings without one. A printed string with no derivable
bound does not displace the catalog. Every surface that judges a
reading routes through it — the API verdict, the doctor report, the
insights blocks and per-marker assessment, the coach snapshot, the MCP
read and the FHIR `referenceRange` — so a value cannot read in range on
one surface and out on another.
The API keeps `referenceLow` / `referenceHigh` as the window in force,
so a client that reads only those two renders the right range and the
right verdict with no change of its own; the catalog band, the source
window, its origin and a divergence flag ride alongside for clients
that want to show both.
Read side: the chart draws the source window as a second, visually
distinct band with the reading it came from named beneath it, the
reading rows name the printed range and the catalog band where the two
disagree, and the doctor-report PDF prints the report's own string with
a footnote naming the saved range. Entry side: one optional field on
the lab form, folded away until asked for, and the printed range is
editable on the scan review screen.
The range parser covers the notations German and English reports
actually print — two-sided, one-sided, comma or dot decimals, thousands
separators, trailing units — reads units without ever converting them,
and refuses to guess on anything ambiguous.
Two directions, folding the vault and labs pipelines into one linking path
without weakening either manual flow.
Vault document → lab values (auto-stage). After the per-document auto-index
job transcribes a stored document, a lab-looking document is run through the
same inbound extraction the manual "Extract" button uses, but the facts land
PENDING for the existing human review — nothing is committed. Gated on both
the documents and labs modules plus the auto-AI-read consent gate the
auto-index external path already uses; a text-structuring pass over the
transcription the index just produced, so no second vision call is spent. The
staging is a guarded STORED→EXTRACTED transition, and a keyword heuristic (or
an explicit LAB_RESULT kind) keeps an arbitrary photo from burning a call.
Everyone without the opt-in keeps the manual extract button.
Lab scan → vault. A lab report scanned in the labs area is now also filed
into the Documents vault (kind LAB_RESULT) through the existing upload —
encrypted at rest, EXIF-stripped, thumbnailed, sha256-deduped — and the
committed labs are cross-linked back to it with the same APPROVED-observation
provenance the manual vault → labs path produces. Text-mode scans keep the
image on-device and file nothing. Idempotent: a re-commit inserts no rows and
links nothing new, and the vault dedupe means no duplicate document.
Owner-scoped and module-gated on both sides; the cross-link is best-effort so
it never fails the authoritative write.
Add the clinical-signals wave's new measurement types and wire them through
the signal registry so FHIR, correlation eligibility, ingestion, and the
measurement-list surfaces derive from one definition each:
- MeasurementType: PHQ9_SCORE, GAD7_SCORE, GRIP_STRENGTH, PAIN_NRS,
WAIST_CIRCUMFERENCE, WAIST_TO_HEIGHT (+ AssessmentInstrument enum and the
MentalHealthAssessment model that backs the screener scores).
- Registry: the six measurement/score signals plus the longevity lab panel as
kind:"biomarker" entries; the FHIR golden fixture is regenerated in lockstep.
Grip-strength and waist-to-height carry a null LOINC (NEEDS-VERIFY) and emit a
local text concept until a stable code is confirmed.
- FHIR: add the survey observation category for the screener / pain signals.
- Labs: extend the biomarker catalogue with ApoB, Lp(a), fasting insulin, and
the omega-3 index; add LOINC codes for those plus GGT and hs-CRP; add a
computed HOMA-IR helper.
- Exhaustive maps updated to keep the enum coverage, category, list-meta, and
personal-record drift guards green.
Resolve a vision-capable provider, gate with consent + a 6/hour rate bucket +
a reserved per-day token budget, then transcribe an uploaded photo or PDF into
structured proposed rows. The raw upload stays in memory only and is never
persisted. The commit route writes only the rows a human confirmed, resolving
or minting a biomarker per row with source OCR and skipping commit-time
duplicates. The capability probe drives whether the UI shows the scan entry.
Extracted text is treated as untrusted; the review screen is the safety
boundary.
Add first-class qualitative lab findings (positive / negative / borderline
serology) alongside the existing numeric path. A LabResult is now EITHER a
numeric reading (value + unit, optional reference bounds) OR a qualitative one
(valueText, e.g. "negativ"), never both, never neither.
- schema: make `value` nullable, add `value_text`; migration 0181 is additive
(DROP NOT NULL + ADD COLUMN), every existing numeric row is untouched.
- validation: numeric-XOR-qualitative refine on create + update; unit/bounds
apply to the numeric path only; valueText capped at 120 chars. The single
resource PUT rejects a cross-type edit.
- serialise: carry valueText through both DTOs; a qualitative row reports the
neutral `unknown` range verdict (no number to classify).
- FHIR: a qualitative result emits valueCodeableConcept — SNOMED qualifier
value for confidently-known terms (Negative 260385009, Positive 10828004,
Not detected 260415000, Detected 260373001), text-only for borderline /
grenzwertig and any unrecognised term. The raw text always rides .text.
- UI: a numeric/qualitative toggle in the lab form; list, history, detail,
chart, sparkline, and the doctor-report PDF all render qualitative readings
as their text and never plot or divide on a null value.
- i18n: new labs.form keys across all six locales.
- openapi: response carries a nullable value + valueText; regenerated.
One-shot prettier pass so the new format:check gate starts from a clean
baseline. Formatting only — no behaviour change. Generator-owned and
vendored artifacts stay untouched via .prettierignore.
The catalog GET eagerly threw on the first un-decryptable context row,
500ing the whole list. The list serializer now decrypts fail-soft (null
on a bad-key/malformed row) via decryptContextSoft, bounding the
fail-closed blast radius to one marker; the single-resource GET stays the
authoritative throwing read for the plaintext context.
A PUT that moves a single reference bound past the row's stored other
bound slipped through the schema refine (which only fires when both bounds
arrive together). Both the lab-result and biomarker routes now merge the
effective bounds and 422 on low > high. Editing catalog-resolved fields
(analyte/unit/panel/range) on a linked reading is also rejected rather than
silently no-oped. Shared isInvertedRange/effectiveBound helpers with tests.
The edit sheet loaded the decrypted note from the single-resource GET and
sent note:null on a load miss, silently wiping a note that simply could
not be read. The submit path now omits the note key entirely in that case
(server preserves the stored ciphertext), disables the textarea, and shows
a hint. Decision encoded in a pure resolveNoteForUpdate helper with tests.
A free-text lab write resolves or mints a user-scoped biomarker by
(userId, lower(analyte)) so no LabResult row ever persists unlinked. An
existing marker is reused as-is (its unit/range stay authoritative); a
miss mints one from the reading's own spelling. Every reading is now
editable and detail-navigable, and the boot-time backfill becomes a pure
pre-upgrade migration. Closes the unlinked-row dead-end and the contract
hole in one path.
Add a user-scoped Biomarker catalog so a marker is defined once — name,
unit, lower/upper bound, optional encrypted context, panel — and every
later reading just links it. New /api/biomarkers CRUD plus a structured
/api/labs create path: a biomarkerId resolves the unit + reference range
server-side, so the list and detail DTOs carry the resolved values and
the client never recomputes the range. Lab reads include the linked
marker and fall back to the legacy free-text fields for unlinked rows.
Add a lab-result restore endpoint for the delete-Undo affordance, a
shared serialiser, a static common-panel seed list, and tests for
catalog validation, structured-entry resolution, and the context codec.
Reference-range classification + low/high range rendering was implemented three
times (validations, PDF core, lab list). Extract a single classifier and a
structure-only range formatter that takes a per-surface number formatter, so
each surface keeps its own number formatting while sharing the canonical
classification. Behaviour is unchanged per surface and now covered by one test.
Marc-André Bombeck