0475 and 0480 added their columns with a bare ALTER TABLE, which waits
for its table lock behind every open transaction while every write to
the table queues behind it. Both now run in a DO block with a five
second lock_timeout and carry the boot's retry marker; the effect is
the same and both stay safe to run again.
Neither has shipped in a release. A database that already applied them
is unaffected: prisma migrate deploy and migrate status ignore the
checksum of an applied migration (checked against a database migrated
with the old files), so nothing is re-run and nothing is reported.
The lock-timeout guard now refuses, from 0475 on, any statement that
takes a table lock outside such a block.
The backup carried neither the import ledger nor the tombstones of
deleted documents, so after a restore onto a fresh instance a token
importer stored every imported document the person had deleted on
purpose again.
The document filing section now carries documentImportKeys: the
ledger's rows plus the own and alias keys of the documents waiting in
Recently deleted, which still do not travel as documents. The restore
writes them back, minus any key a restored live document holds, and
counts them in the restore summary and the preview. DocumentImportKey
moves from the excluded models to the two-ended ones, and the round
trip, the column audit and the parity goldens carry a ledger row.
The boot discovery for the documents 0479 marked took the first thousand
by id and skipped those whose account has the documents module off.
Such an account keeps its marks, so with enough of them it held every
place of every boot and the accounts after it never got their retry.
The marks are walked by id past the last row read, an account found
with the module off is left out of the query from then on, and the
boot stops at a thousand documents enqueued rather than read.
Refs #1277
The index-all backfill billed the read and wrote the index in one try:
an index write that failed after the read was settled again in the
catch, handing the reservation back twice. The read is settled once,
and a failed write charges nothing back.
The backfill also checked consent and the operator's switches once, at
the pick, then read up to two hundred documents. The wire is asked
again before every read; a refusal hands that document's reservation
back and ends the walk with no-consent or unavailable.
The extraction merged facts across reads on a key whose only date was
the fact's own effective date. Facts without one collapsed across reads
of different reports, so the same value on two days staged once. An
undated fact is now keyed on its read's report date, falling back to
the document's, as the lab scan keys a row.
Refs #1257
Without Prefer: respond-async the extract route ran the read with no
schedule, so a twenty-page report read a page at a time, with a retry
each, could make forty model calls in one HTTP request. The iPhone app
sends no Prefer header and reads the document detail from the answer,
so the request keeps its 200 and its body.
An extraction without a run deadline now plans at most three reads (six
calls): a longer document has its pages or text shared out between
them, and what does not fit is reported as extractionIncomplete on the
document. The background run keeps reading it a page at a time.
Refs #1257
A long document read in parts can end with a part cut at the provider's
output limit, a part never read, or more facts than one review takes.
That was told once, as a notice when the run finished; a review opened
later looked complete.
InboundDocument.extractionIncomplete (migration 0483) is written with
the facts by both stagings, the extract route or run and lab
auto-staging. The document detail publishes it and the review shows the
same note whenever it is opened. The disaster-recovery backup carries
it; an older file restores it as false.
Refs #1257
End course tombstones the pending placeholders at or after its instant.
Moving or removing the end clears the instant, but the tombstoned rows
kept their unique slots, so the today projector treated them as present
and minted nothing: the later dose of that day was neither listed,
reminded nor missed.
The window's one writer now revives the placeholders the End stamped
with its instant once that instant is gone, when the slot is inside the
courses as they stand and no live row holds it.
Refs #1280
canStartCourse answered true on the day a course was ended with End
course, while a course starting that day shares its last day and the
start refused it with 422 overlap. The start is now offered from the
day after the last course's end, the first day validation and the
engine accept.
A course ended at an instant is over for every rule that asks about
today, whatever the zone it is read from: after a move west the course
rows read CURRENT while the medication's own status read ENDED. The
course rows, the coach summary, the overlap rule and the previous-end
read take the instant-aware reading.
Refs #1280
calculateCompliance and buildTimelineForWindow built the cadence
engine's context field by field and left out endedAt, so a dose slot
after End course counted as missed in the streak and the timeline while
the rate left it out.
Both now take the context from one builder, endedAt is required on
ComplianceMedicationContext and CadenceEngineContext, and the course
anchor guard holds every builder of either type to naming it.
Refs #1280
POST /api/medications/{id}/courses/end resolved the caller with
requireAuth(), so a delegate managing the record got 403 while the
medication PUT that used to carry the end date admits MANAGE, and the
reads still served canEndCourse true. The action now resolves the record
at MANAGE over medications and files the previous end (C4) with the
count of tombstoned placeholders.
The medication reads narrow canEndCourse and canStartCourse to the
caller: End needs MANAGE, starting a course stays owner-only. The list
cell stays per record and the narrowing runs after it. Create and edit
answers pass the grant they were resolved under instead of null.
Registered in the sharing surface guard, the MANAGE handler matrix and
the activity feed's verb lines.
Refs #1280
The ledger stamped last_failure_at, last_ok_at, next_retry_at and
updated_at with the database's NOW(), while every reader compares them
with the app's clock: the runner's skip hints and the background bench,
the admin provider-health card, the server-health tri-state and the
support report. A backoff or pause therefore sat wherever the skew
between the two clocks put it. With the app clock pinned, a five-minute
backoff lasted until the database's clock plus five minutes, months away,
or was over before it began.
Both writes now bind the instant they took as a parameter
(`$N::timestamp`, the naive UTC columns), the same one-clock rule the
summary reads follow. The escalating pause arithmetic is unchanged.
The pause integration test pins the app clock years behind and ahead of
the database's and checks the stored instants, a backoff that lifts after
five minutes, a pause that benches background work for its full length,
and a success the server health reads as recent.
Refs #1279
The batch golden lost a card's request about one run in eight. When the
batched call fails, every pending card falls back to its own call at once,
and each failure is recorded on the provider health ledger without being
awaited. Background work leaves a provider out from its third failure in a
row on, so a card that read the ledger after two of its siblings' writes
had landed was benched and never reached the model. Which card, if any,
was scheduling.
The stored notes were the same either way, so only the request list moved.
The parity harness now runs on a ledger that records and benches nothing;
the bench keeps its own tests.
Refs #1279
The GET contract run requires every list operation to answer at least
once with a row, and the new list of deleted documents had nothing to
list. The seed now ingests a second document and tombstones it the way
the delete route does.
The medication reads now carry `endedAt` and `canEndCourse`, and every
course its `endedAt`; the row dumps carry the new columns. Additions only.
Refs #1280
Ending the running course writes the same course rows the other course
routes do, so it follows the same data-layer reasoning: the medications
module gates the surface, not the row store.
Refs #1280
"End course" now posts to `POST /api/medications/{id}/courses/end`, which
ends the running course at that moment: its last day becomes today on the
person's clock and the instant is stored as `endedAt`, so a dose later today
is not due, not reminded and never missed, while one taken earlier still
counts. Pending placeholders already minted for later slots are tombstoned.
A medication without dates gets the course its window describes, ended now.
End stays its own verb instead of `PUT { endsOn }`: an end date set through
the PUT or the course editor keeps meaning the whole day, and the PUT keeps
accepting it on any course, since moving an end is a legitimate edit.
The list and detail reads serve `canEndCourse`, true exactly while a course
is running, and the action refuses with 409 `medications.course.notRunning`
when it is false, so no client can reopen an ended course or invent a
one-day course from a planned one by ending it. The web controls render End
from the served flag; the contributed tests keep their cases against it.
Refs #1280
A course's end date is a calendar day and counts in full: the doses of that
day stay due, reminded and expected until local midnight. That is right for
an end date set in the editor and wrong for "End course", whose dialog
promises that nothing more is due.
A course now carries `endedAt` (migration 0482, additive, nullable), the
instant it was ended on its `endsOn` day, projected onto the medication next
to `startsOn` / `endsOn` by the one course writer. From that instant:
- the schedule engine emits no slot, so a dose later that day is not due,
not reminded, not projected onto the today list and never expected by
compliance, while a dose earlier that day stays inside the course;
- the course reads ENDED (`courseStatusAt` / `courseStatusOn`), so the dose
actions, `remindersActive` and the reminder gate stop at once, follow-ups
of an earlier dose included;
- every engine context builder, compliance context, dose history, Coach
snapshot, doctor report and the adherence storyline read it, through one
select and one picker for the medication window.
Any write that moves a course's `endsOn` clears the instant, so an end date
chosen in the editor keeps meaning the whole day. Backups carry it in both
purposes and a restore projects it again. `courseStatusAt` moves next to the
other course rules.
Refs #1280
Drives the status-card path the way an idle instance runs it: a data sync
every minute dirtying five cards, the production enqueue with its queue
name, singleton key and retry options, the queue's consumer, the card's
single call and fallback, the status chokepoint, the chain walker with the
health ledger's in-memory twin, the gateway client and the note store's
negative windows. Only the wire, pg-boss, the clock and the unrelated
Postgres reads are faked; the wire answers every request with reasoning
and finish_reason "length".
Against the code before this series the hour counted 151 requests, 150 of
them sent with "none", none charged to the day's ledger, and 300 when the
reasoning lesson did not survive from one job to the next (the pattern
behind the reported log). Now it counts six, charges all of them, sends
"none" once, and logs the pause once per window.
Refs #1279
pg-boss gives a send that names no retry options the queue's defaults:
two retries, at once, without a backoff. A job that reached a model and
failed then ran again immediately, twice, against the provider that had
just failed it. Two model-bound queues still relied on those defaults:
the Coach memory refresh send and the nudge cron. Both now ask for the
same two retries, a minute apart and backed off.
ai-job-retry-bound-guard holds the rule: every send to a queue whose jobs
reach a model names retryLimit (at most 3) and, unless it is 0,
retryBackoff with a retryDelay of at least 30 s; every such cron carries
the same in its schedule options. The queues come from the model-run
sites the expiry guard already keeps complete (now a shared helper), plus
the job files that call a provider directly, which the new guard keeps
complete itself. A planted case runs the checker over sources shaped like
the tree before this change. The queue golden records the nudge cron's
new options.
Refs #1279
The content-index backfill capped a run at 200 documents, but counted
only the ones it indexed. Against a provider that failed every call (a
model that reasons and never answers) the cap was never reached, so the
walk transcribed every un-indexed document in turn, one provider call
each, until the job's time ran out after more than two hours, and the
refunded failures never reached the day's token ceiling.
The cap now counts provider calls, the walk stops with reason
provider-failing after three failed calls in a row (one document the
model cannot read is still walked past), and a failed read charges what
its reply reported spending. The per-document read moves into its own
function.
Refs #1279
A status note whose generation stalled or failed opened a fixed five-minute
negative window, and only the read path honoured it. The ingest
invalidator counted only a stored note as fresh, so every data sync
re-enqueued each note whose last generation had just failed, and the
worker generated it again whatever the window said. With a model that
never answers, every sync bought another round of calls per note.
The window now grows with the generations in a row that ended without a
note (failure_streak, migration 0480): five minutes, doubling per failure,
up to two hours. A written or re-dated note resets it. A failure inside an
open window neither moves nor counts it.
runStatusCompletion waits the window out before any provider work,
whichever path selected the note again (a sync, a warm pass, a queued
job), and the ingest invalidator no longer enqueues a note inside its
window. A derived-score assessment that fails now opens the same window.
The person's own regenerate closes the record's windows first, and saving
the AI provider settings closes them and forgets the provider's health
rows, so a fixed setup does not sit out a pause the old one earned.
The window code moves to status-negative-window.ts; the status parity
golden records that the mood note screened earlier in the same minute is
no longer generated again inside its window.
Refs #1279
The document picker showed a document deleted in HealthLog as "Deleted
in HealthLog" and would not let it be ticked, so a person who deleted an
imported document by mistake met a dead end exactly where they would look
for it again.
Such a row can now be ticked, and the import sends revive: true for it.
A document still in Recently deleted is restored through the shared
restore, nothing is fetched, and the row reads "Restored from Recently
deleted". One the purge already removed is fetched and stored again, and
only once it is stored does the ledger forget its key, so a failed fetch
leaves the deletion standing. Without revive the route answers "deleted"
as before, and the token upload has no such switch: a Paperless workflow
or the import script re-sending a deleted document is still refused,
whether it is in Recently deleted or long purged.
The search names the document a deleted key still points at (documentId,
null once purged), so the row can say "In Recently deleted" rather than
"Deleted in HealthLog". The outcome gains "restored", sent only in answer
to revive. The row logic moves to picker-rows.ts.
Refs #1275
Deleting a document only stamps a tombstone and the purge removes it
after 30 days, but the only way back was the undo toast, gone a few
seconds after the delete. Recently deleted lists the deleted documents
of the record with type, filing date, where an import came from, when
each was deleted and how long it has left, and offers Restore, Restore
all and Delete now.
GET /api/documents/inbound/deleted lists them (owner-only, capped at 200
with the list-cap meta). POST /api/documents/inbound/{id}/purge removes
one deleted document for good through the same function the nightly
purge runs, so its import keys and aliases still move into the ledger
and an importer re-sending it is still answered "deleted"; a live
document answers 409 documents.inbound.notDeleted. The usage read gains
deletedCount, 0 under a grant. The restore route and bulk restore
share one restore, which now also clears ledger entries for the
document's own keys.
The vault header shows a quiet "Recently deleted (n)" link only while
something is in it, and the delete toasts and confirmations say that
deleted documents stay there for 30 days. The purge's queue facts and
grace window move to document-purge-queue.ts so the worker registry
still loads none of the work.
Refs #1275
The health ledger already counted every failed call and paused a provider
that kept failing, with a pause that grows. A paused provider only moved to
the tail of the chain, though, and the tail is still walked: a chain whose
only entry answered every call with reasoning and no answer was called by
every background job regardless.
A walk for background work (surface "job") now leaves out every provider
the ledger has benched until its window lifts; the local floor only once
it is paused, past a brief backoff. One ai.chain.background_paused line
per window says so. A person's own call still reaches the provider at the
tail and reports its own error. The ledger is in Postgres, so the bench
holds across workers and restarts.
The status, briefing and document-index paths refunded a failed call in
full, so background spend on replies without an answer never reached the
day's token ceiling. A failed chain now carries the tokens its hops
reported (tokensSpent) and those paths charge them.
The two clients share one answerless step (askAgainUnreasoned), which
keeps both inside the source size ratchet; its baselines go down.
Refs #1279
The #1267 rule asks a reply that reasoned without answering once more with
reasoning_effort "none", and asks every later call that way for an hour.
That is what Gemma 4 on Ollama needs. gpt-oss behind vLLM accepts only
low / medium / high and ignores "none", and a GLM chat template reads an
effort it does not know as its maximum, so for those models the repeat and
every later "none" were wasted requests or longer thoughts.
When a reply sent as "none" comes back without an answer as well, the
Local and gateway clients now learn that too: later calls go out as the
call asks, with the thinking room #1276 keeps, and an answerless reply is
final. One call costs two requests at most and one after that, and the
log gets one line for each lesson instead of one per call.
A failed answerless call now carries what it cost on its error, and a
repeat that answers reports both requests, so the day's token ledger can
charge them. The learned lessons are pinned on globalThis, so the Coach's
route handlers and the worker's jobs share one copy per process instead
of learning the same thing twice.
Refs #1279
Two guards with the same file name froze the same failure from two sides:
one stopped a runtime import of pdf-parse, the other pinned one pdfjs-dist
in package.json, the lockfile and the checkout, and allowed only the shared
loader to import it. They are one guard now. It strips comments before it
matches, so prose naming a package does not count, walks every runtime
extension and skips __mocks__ as well as __tests__, treats a type-only
import of pdfjs-dist outside the loader as an import, and keeps the import
shapes each side's self-test proved, including the near-miss name that must
not match.
Refs #1277
Two pairs of tests ran a text read and a rasterization in one process,
one file per order. The pdfjs-one-version pair stays; the other pair and
its shared helper go. The kept pair now first asserts that nothing in the
process has claimed the pdf.js worker yet, so a file that lands in a shared
process fails instead of passing on an order it never ran, and it mocks the
failure signal the text read can emit.
Refs #1277
The text read and the rasterizer each opened pdf.js on their own: the text
read with bare options, the rasterizer through src/lib/documents/pdfjs.ts
with the standard font, CMap and wasm directories. The loader now serves
both, so a CID font with a predefined CMap is mapped to characters on the
text path too, and there is one set of options to keep in step.
The loader is typed with pdf.js' own declarations instead of a hand-written
shape. That drops isEvalSupported, which pdf.js 6 no longer has and which did
nothing. Only the render slice the DOM-typed declarations cannot describe
(the native canvas factory) keeps a named local type in the rasterizer.
The run joiner with its named steps moves into the loader as joinTextRuns,
and readPdfText is its one page walk; the second layout copy that mimicked
the previous library's tabs and blank lines is gone. extractPdfText keeps
its outcomes and page cap on top of it, and a failed read still says what
pdf.js threw (documents.local_extract.failed). A test draws a word in two
fonts into a real PDF and reads it back whole.
Refs #1277
LifecycleManageBody takes the medication (id, name, active, course
status) instead of one prop per field. The call site in
medication-detail-tabs.tsx shrinks to one line, so the course status
reaches the controls without growing a file the size ratchet holds.
While two pdf.js versions broke each other, a local text read after a
page render failed and was recorded as decrypt-error, and a scanned PDF
that had to be rasterized for the provider was recorded as raster-failed.
Neither has a content index, and on an instance without an AI provider
nothing offers to index it again.
Migration 0479 clears the attempt time of exactly those PDFs and keeps
their reason, a state no index attempt leaves behind. A new boot
discovery, document-index-retry, enqueues the upload's auto-index job for
each such row, so it runs the same provider-first, local-fallback decision
under the same consent rules; a deferred import stays local because its
row says so. The attempt writes its time back, so every document drops out
after one try, and one that really cannot be read records its reason again
and stays settled. An account with the documents module off keeps its
marks until the module is on.
An integration test seeds both kinds of damage, runs each migration's
statement and only what a boot runs on its own, and sees a preview and an
index appear, while a blank page, a broken ciphertext and a file that is
not a PDF settle after one retry and are not found by the next boot.
Refs #1277
While two pdf.js versions broke each other, every PDF preview rendered
after a local text read failed and was recorded as raster-failed, which
the thumbnail backfill treats as final. Migration 0478 deletes those
failure rows, so the boot-time thumbnail backfill finds each document
again and renders it once. A stored failure cannot tell the clash from a
page that really renders blank; such a page fails once more and settles
with one attempt. Only raster-failed is cleared: the clash surfaced as a
failed rasterization, which the thumbnail path maps to that reason alone.
Refs #1277
The lockfile must resolve exactly one pdfjs-dist, the version package.json
pins, and no package may depend on it except the app; every lockfile line
naming pdfjs-dist is either understood or reported. The checkout must
resolve that version from the working directory, and no production module
may import pdfjs-dist except src/lib/documents/pdfjs.ts, nor pdf-parse at
all. Run against the previous lockfile it reports the 5.4.296 dependency
and the old override; an extra import in the rasterizer fails it as well.
Refs #1277
Nothing imports pdf-parse any more, so its pdfjs-dist 5.4.296 and the
older @napi-rs/canvas that came with it leave the lockfile. The override
that kept that transitive copy past a renderer advisory has nothing left
to move and goes too. The standalone tracing stops naming pdf-parse; its
pdfjs-dist glob now matches the one copy the app pins. The Dockerfile and
canvas guard comments describe the second copy as history.
Refs #1277
Under Node, pdf.js runs its worker in the server process and keeps it on
a global, and whichever copy of pdf.js opens a document first owns that
global until a restart. The local text read went through pdf-parse, which
pins its own pdfjs-dist 5.4.296; the rasterizer uses the app's 6.4.299. A
text read first made every later rasterization fail with "The API version
does not match the Worker version" (thumbnails raster-failed, the vision
read of a scan 422), and a rasterization first made every later local text
read fail, which left the document out of the search index with nothing in
the logs.
src/lib/documents/pdfjs.ts is now the one place pdf.js is loaded and
opened, with one set of options (asset directories, no eval, no system
fonts). The text read walks the pages with getTextContent and lays the
text out the way pdf-parse did with the options it was called with: line
breaks, tabs between far-apart items, a blank line after each page, the
200-page cap. Text extracted from the PDF fixtures and from generated lab
tables, encrypted files and doctor reports is identical before and after.
A failed text read now emits documents.local_extract.failed with the
error pdf.js threw, and annotates the index event with it, instead of
returning a bare error.
The report tests read PDF text through the same reader. Two tests, one per
order, run a text read and a rasterization in one process; both failed
before this change.
Refs #1277
End and its confirmation now live in EndCourseControl, which renders only
while the course runs. When End stops being offered, the open dialog and
its state unmount with it, so the state no longer has to be reset during
render.
The course-status prop uses the shared CourseStatus type instead of a
spelled-out union, and the End confirmation's open state is dropped when
End stops being offered, so a dialog closed by a refetch cannot pop up
by itself once a later course offers End again.
Ending sends today as the end of the medication's latest course. While
the PUT was refused this did nothing; now that it succeeds, pressing End
on a course that had already ended reopened it up to today (doses due
again, the days between counted as missed), and on a course that had not
started yet it became a one-day course today.
LifecycleManageBody now takes the server-resolved courseStatus and shows
End only while the course is CURRENT (a medication with no dates is
CURRENT). Pausing stays available whatever the course.
"End medication" sent new Date().toISOString(), a full instant, against
an endsOn that only accepts YYYY-MM-DD, so every press answered 422 and
the toast said the medication could not be ended. The instant's date was
also the UTC one, a day off around midnight for anyone not on UTC.
The button now sends the person's calendar day in their timezone
(userDayKey), the same day key the medication card uses.
The reasoning separation grew three files past their size ratchet. The Local
client's reasoning collector and the new separation helpers move to
local-reasoning.ts, the gateway client's thinking room and reply separation
become two private methods, and the Coach's error-code resolver moves out of
chat-bubble.tsx into coach-error-key.ts, still re-exported from
message-thread.tsx. Code motion apart from comments that said when a branch
was added rather than what it does. The baselines record the shorter files.
Refs #1276
The lab-scan read asks every provider for no reasoning. On an
OpenAI-compatible gateway or a local server that "none" is a request the
model behind it may ignore, so these wires now add the low thinking budget on
top, the same rule the file already holds for a model that always thinks:
the answer keeps its full allowance either way.
Refs #1276
Every 90-day series in the seed is a mean-reverting walk, and around a
settled target that is flat noise. A 7-day trend line over the last 30
days of blood pressure drew a straight line, which is the view the demo
exists to show.
A busy stretch three to four weeks back now lifts blood pressure and
pulse and lets them settle again, the mood dips with it, and weekend
readings sit a touch higher. Blood pressure also pulls back towards its
target harder, so a seed no longer ends several mmHg above the band its
stored texts describe; today's reading settles on the days after the
stretch. The two stored texts that summarise the last weeks mention the
stretch.
An integration test runs the real seed and holds the 7-day mean to a
visible swing and today's reading to the settled side of it.
Marc-André Bombeck