A status note whose generation stalled or failed opened a fixed five-minute
negative window, and only the read path honoured it. The ingest
invalidator counted only a stored note as fresh, so every data sync
re-enqueued each note whose last generation had just failed, and the
worker generated it again whatever the window said. With a model that
never answers, every sync bought another round of calls per note.
The window now grows with the generations in a row that ended without a
note (failure_streak, migration 0480): five minutes, doubling per failure,
up to two hours. A written or re-dated note resets it. A failure inside an
open window neither moves nor counts it.
runStatusCompletion waits the window out before any provider work,
whichever path selected the note again (a sync, a warm pass, a queued
job), and the ingest invalidator no longer enqueues a note inside its
window. A derived-score assessment that fails now opens the same window.
The person's own regenerate closes the record's windows first, and saving
the AI provider settings closes them and forgets the provider's health
rows, so a fixed setup does not sit out a pause the old one earned.
The window code moves to status-negative-window.ts; the status parity
golden records that the mood note screened earlier in the same minute is
no longer generated again inside its window.
Refs #1279
The document picker showed a document deleted in HealthLog as "Deleted
in HealthLog" and would not let it be ticked, so a person who deleted an
imported document by mistake met a dead end exactly where they would look
for it again.
Such a row can now be ticked, and the import sends revive: true for it.
A document still in Recently deleted is restored through the shared
restore, nothing is fetched, and the row reads "Restored from Recently
deleted". One the purge already removed is fetched and stored again, and
only once it is stored does the ledger forget its key, so a failed fetch
leaves the deletion standing. Without revive the route answers "deleted"
as before, and the token upload has no such switch: a Paperless workflow
or the import script re-sending a deleted document is still refused,
whether it is in Recently deleted or long purged.
The search names the document a deleted key still points at (documentId,
null once purged), so the row can say "In Recently deleted" rather than
"Deleted in HealthLog". The outcome gains "restored", sent only in answer
to revive. The row logic moves to picker-rows.ts.
Refs #1275
Deleting a document only stamps a tombstone and the purge removes it
after 30 days, but the only way back was the undo toast, gone a few
seconds after the delete. Recently deleted lists the deleted documents
of the record with type, filing date, where an import came from, when
each was deleted and how long it has left, and offers Restore, Restore
all and Delete now.
GET /api/documents/inbound/deleted lists them (owner-only, capped at 200
with the list-cap meta). POST /api/documents/inbound/{id}/purge removes
one deleted document for good through the same function the nightly
purge runs, so its import keys and aliases still move into the ledger
and an importer re-sending it is still answered "deleted"; a live
document answers 409 documents.inbound.notDeleted. The usage read gains
deletedCount, 0 under a grant. The restore route and bulk restore
share one restore, which now also clears ledger entries for the
document's own keys.
The vault header shows a quiet "Recently deleted (n)" link only while
something is in it, and the delete toasts and confirmations say that
deleted documents stay there for 30 days. The purge's queue facts and
grace window move to document-purge-queue.ts so the worker registry
still loads none of the work.
Refs #1275
Under Node, pdf.js runs its worker in the server process and keeps it on
a global, and whichever copy of pdf.js opens a document first owns that
global until a restart. The local text read went through pdf-parse, which
pins its own pdfjs-dist 5.4.296; the rasterizer uses the app's 6.4.299. A
text read first made every later rasterization fail with "The API version
does not match the Worker version" (thumbnails raster-failed, the vision
read of a scan 422), and a rasterization first made every later local text
read fail, which left the document out of the search index with nothing in
the logs.
src/lib/documents/pdfjs.ts is now the one place pdf.js is loaded and
opened, with one set of options (asset directories, no eval, no system
fonts). The text read walks the pages with getTextContent and lays the
text out the way pdf-parse did with the options it was called with: line
breaks, tabs between far-apart items, a blank line after each page, the
200-page cap. Text extracted from the PDF fixtures and from generated lab
tables, encrypted files and doctor reports is identical before and after.
A failed text read now emits documents.local_extract.failed with the
error pdf.js threw, and annotates the index event with it, instead of
returning a bare error.
The report tests read PDF text through the same reader. Two tests, one per
order, run a text read and a rasterization in one process; both failed
before this change.
Refs #1277
The summaries slice and the comprehensive aggregator took their JS windows
(the DAY-bucket bounds, the rollup slope anchors, lastSeen staleness) from
the process clock and their SQL windows (avg7, avg30, the month before, the
median, the 90-day canonical pick, the cold-path slope evidence) from
NOW(), the database's clock. One response mixed two clocks, so a reading
on a window's edge fell in or out with whichever host answered, and with
the process clock pinned the analytics parity walk changed its avg7 with
the hour it ran at.
Each read now takes one `now` and binds it into its statements
(`request-clock-sql.ts`): `$N::timestamp`, which keeps the driver's
offset-free UTC string as written in any session zone. The canonical-pick
helper takes its window as an instant plus an interval instead of reading
NOW(); the series and calendar readers anchor it on their own `since`, the
BP-in-target read on its 365-day start. The window-mean and live-slope
columns the two aggregates each spelled out now come from one builder.
The streak's local day series read its bounds through `::timestamptz`,
which follows the session zone: outside a UTC session today's reading fell
out of the streak. It reads them as UTC first.
The parity walks for analytics, the dashboard summary and the
comprehensive overview pin a fixed day instead of today: their records are
laid out from the anchor day, so the chronotype's work and free nights and
the weight-by-weekday points moved with the weekday the walk ran on. The
snapshots are unchanged. A new suite pins the clock to four times of day,
far from the database clock, with readings ten minutes either side of
every edge, on both read paths, in a UTC and an Auckland session.
Since the lab staging reads a long report in parts, one auto-index could
make the transcription plus up to twenty reads of two calls each at the
person's response timeout, hours past pg-boss's fifteen-minute default,
which failed the job and retried it beside the reads still running. The
staging now takes the run's deadline as a schedule and plans only the
reads that fit before it. The index job is long, with an expiry above the
calls that run before the plan can say no, a stop on jobDeadline and a
lock per document; the document run hands the same staging its own
deadline, which moves the run's expiry as the reads are planned. The
document summary, one call and its reasoning repeat, gets an expiry above
that worst case.
ocrReadCallMs in the read core is the one per-call timeout both deadlines
divide by. The expiry guard now names every importer of the document and
lab read entry points with the queues that reach it.
The runtime guard's planted cases stripped every file's comments again on
each rescan, which took over a second alone and timed out under a full
parallel run; the stripped text is now memoised.
Refs #1274
A least-squares slope per day over readings minutes apart explodes: three
blood-pressure readings in one sitting, the only ones of the week, fitted
thousands of mmHg per day, and the dashboard tile printed seven days of it
as the 7-day trend. The 30-day fallback had the same flaw.
A window now earns a slope only when its readings fall on at least three
UTC days whose first and last lie at least half the window apart (4 days
for the 7-day window, 15 for the 30-day one). The rule lives once in
window-confidence.ts and gates every path that publishes a windowed trend:
trendSlope (summarize and its callers), the rollup composition and the
live REGR_SLOPE fallback of the summaries slice and the comprehensive
aggregator, and the per-bucket slope of folded week, month and year
buckets. Slope, direction and confidence are present or absent together.
The fit itself stays over individual readings, as the window means do.
The parity snapshots change only where the fixture falls short of the
rule: a weekly body-fat series has two days in its 7-day window, and a
ten-day oxygen-saturation series spans nine days of its 30-day window.
A status run (a metric card, the batched assessment, a derived score, a
workout line, a Coach memory pass) is one model call plus the one repeat
the Local and gateway clients send when a thinking model answered with
reasoning alone, each with the person's response timeout. The run's outer
bound was a single call, so the repeat was cut off, and the open card
stopped polling after 48 s, so a note from a model that needs minutes
never reached the page. The bound is now the per-call timeout times
STATUS_MODEL_CALLS, and the card's poll ceiling follows the published
response timeout (fast for the first 48 s, every 15 s after), only while
the route reports the note in flight; the fallback text stays meanwhile.
The status queue is exclusive, so the longer poll cannot queue a second
generation beside a running one.
The morning refresh is one comprehensive generation of up to five calls
in a row, with no outer bound and pg-boss's fifteen-minute expiry, so a
large setting had pg-boss fail and retry it while it was still running.
It now runs behind the same bound as the warms, and the job carries an
expiry derived from that worst case. The per-metric note, the workout line
and the Coach memory refresh get expiries derived the same way.
queue-runtime.ts gains a bounded class for these jobs: model time read off
model-time.ts, an expiry above it, carried on every send. A new guard
names every caller of runStatusCompletion and generateComprehensiveInsight
with the queues that reach it, and holds each such queue to bounded above
its runs or long under a run lock.
Refs #1274
Extracting lab values from a stored document still sent the whole text
in one call. A multi-page report produced more JSON than one call's
output ceiling, the answer was cut mid-fact, the parse failed twice, and
the read came back as 422 "Couldn't read the document. Try a clearer
copy." The lab scan had already learned to read the same report a page
or a part at a time, but each surface carried its own copy of the read,
so the document path never got it. It also ran under the shared 60 s
timeout, dropped every fact past 120 (or every fact when one did not
match the schema), and charged a flat reservation.
The read now lives once, in lib/documents/ocr-read.ts and ocr-plan.ts:
the per-read limits, the corrective retry, the salvage of a cut
answer's complete rows, the wire re-check and budget reservation before
each read, and the planning of a PDF a page at a time and text in parts
at line boundaries. The lab scan and the document extraction (the
extract route, its background run and the lab auto-staging) both read
through it, and the background run plans its reads on the same deadline
as a lab scan. The per-read budget moves to a shared metered budget.
The document path validates the answer fact by fact, keeps the lab
scan's row cap (400), and says why nothing came back: 422
documents.inbound.outputLimit when the reads stopped at the output
limit, 429 when the budget refused them, extractFailed only for answers
that were not the schema. A run whose reads were cut or not reached
reports `incomplete` on its result.
A guard holds the class: the output ceiling is set only in the core,
both surfaces read through runOcrReads, both runs share the deadline,
and a lab read and a document read go out under the same limits.
Refs #1257
step-up-elevation-guard timed out under load in three separate gates
and passed every time on its own. One of its tests imported
@/lib/auth/step-up inside the test body to read the fresh-factor set,
and that module reaches the database client and the generated Prisma
client: the import alone ran against the test timeout, and how long it
took depended on the machine, not on the code under test.
The fresh-factor definitions (StepUpMethod, FRESH_FACTOR_METHODS,
isFreshFactorMethod) move to step-up-factors.ts, which imports nothing;
the guard imports it statically and the test is synchronous. Two more
guards did the same: next-image-optimizer-guard loaded next.config and
its plugins per test, and proxy-relative-redirect-guard loaded the
redirect helper with next/server. Both import statically now.
guard-dynamic-import-guard.test.ts parses every file of the guard
project with the TypeScript parser and fails on any import() call, so
a guard that needs a value imports it with the file or reads the
source; a test that must load late belongs to the isolated project.
A full backup carries the account's own mood tags, rated factors and
tag groups, and the JSON import left them out: a mood entry imported
into another account landed without every link to a tag the person had
made.
Before the entries are written, every key they or the file name is
resolved once. A catalogue tag or one the account owns is used as it
is; an own tag whose label the file states differently is kept, never
overwritten, and named in keptMoodTagKeys (keptMoodCategoryKeys for a
group). A tag the file defines and the account lacks is created in its
group, the group created the same way, within the account's caps of 50
tags and 12 groups. A key another account of the server holds comes
over under a key derived from the account and the file's key, so a
second import of the same file finds the copy instead of minting
another. The links are written through that resolution, own rated
factors included, inside the entry's transaction; anything unresolved
is still dropped and named. The import card names kept keys in all
nine locales.
The field-coverage guard now holds the backup's customMoodTags and
customMoodTagCategories to the import schema. The round trip moves a
backup to another server (the same seeded catalogue, none of the
account's rows), imports it into a fresh account and compares the
re-export, own tags, group and rated factor included; a second case
imports into another account of the same server, and a third keeps a
renamed own tag.
Postgres returns rows that tie on an ORDER BY in no particular order,
and not the same one twice. The medication reads ordered by columns two
rows share all the time: every read that hands schedules on included
them with no order at all, medications were listed by createdAt alone
(one import writes many in a millisecond), intakes by scheduledFor (a
twice-daily medication puts every morning dose on one instant), and
"the latest intake" lookups by takenAt. A client read the same account
in a different order from one request to the next, and a latest-intake
lookup picked either of two rows.
Every findMany / findFirst on a medication table, nested includes
included, now ends its order on the id in the direction of its last key,
and schedules are read by window, then id (MEDICATION_SCHEDULE_ORDER, the
order the backup already wrote). medication-read-order-guard.test.ts
holds the tree to it. medication-read-order.test.ts writes tying rows in
the opposite order to the one each read promises, through the medication
list and today's intakes, and fails without the tie-breakers.
The parity suite's own state reader had the same gap: it sorts rows by
content with the ids blanked, and rows that tie then kept whatever order
the unordered read returned, so their <id:N> labels swapped under a
loaded run. Its reads are ordered by id, which for a cuid is the order
the rows were written in.
The JSON import's schema left the route, so its entry in the inline
schema strictness register goes; the schema module says why its objects
are not strict.
A restore never runs beside a backup pass or another large job: the
heavy lane puts its delivery back in the queue for a minute at a time.
The job's own row says queued for the whole wait, and the console said
the restore starts "as soon as the background worker picks it up",
which read as a stuck worker while a backup ran.
The status routes now ask the queue. A delivery the lane put back sits
in created with a start_after ahead of now, and the active row of a
heavy queue is what holds the lane. The job view carries
waiting: "backup" | "busy" | null, and the console says it waits for
the running backup, or for another large job, and since when it has
been queued, in all nine locales. A queue that cannot be asked answers
no wait and is reported at warn.
The job view and its reads move to backup-restore-view.ts, which keeps
the worker module under its size baseline.
The JSON import read six measurement fields and six mood fields while
the full backup and the JSON export carry twice that. A sleep night came
back as stageless rows, a range reading as a single value, an ECG alert
without its verdict, and a mood entry without its note, its level-A
values, its zone, its tag and factor links or its day context.
The import now reads valueMin/valueMax, sleepStage,
rhythmClassification and deviceType on a reading, and note, a1 to a5,
tz, factors, structuredTags and context on a mood entry. Each mood entry
is written with its links and context in one transaction, the way the
bulk mood ingest writes one. A link to a tag the importing account does
not have is dropped and named in droppedTagKeys / droppedFactorKeys, and
the import card says which. The schema moves to
src/lib/import/json/schema.ts so the guards can read it.
json-import-field-coverage.test.ts runs the JSON export and both backup
variants on an account with every column filled in: every field they
write is either read by the import or named with the reason it is not
(row ids, source, deletion records, ciphertext, sync bookkeeping), and a
reason for a field no exporter writes fails as stale. The round trip
imports each file into a fresh account and compares the re-export field
for field. The published contract is held to the runtime schema.
PATCH /api/tokens/{id} sets mirrorToAppleHealth on a measurement token:
cookie session only, the caller's own tokens only, revoked ones included,
any other token refused with tokens.mirror.not_measurement_token. It
stamps mirrorChangedAt and audits only when the value changes. The mint
accepts the setting (off by default), the token list returns it, and
GET /api/tokens/measurements counts the EXTERNAL readings no token
reaches.
Measurement upserts in /api/sync/changes carry appleHealthMirror. A new
measurementMirror domain delivers { id, appleHealthMirror } for the live
readings of every token whose setting changed past the cursor, keyed on
(token.mirrorChangedAt, measurement.id). Switching the setting writes the
token row only, so the readings keep their updatedAt and syncVersion and
nothing keyed on them recomputes. The domain is additive inside the v2
cursor and is left out of the retention expiry, since its watermark is a
setting's age, not a deletion's.
A reading sent with a measurements:write token keeps its EXTERNAL source
and now points at the token that carried it (measurements.ingest_token_id).
Every stored row answers a server-resolved appleHealthMirror: true for
WITHINGS, IMPORT and MANUAL, the token's choice for EXTERNAL, false
otherwise. The plain list accepts appleHealthMirror=true so a client can
page exactly those rows.
Migration 0476 adds the columns and links existing EXTERNAL readings only
where one measurement token provably sent them; 0477 builds the index
concurrently. Backups carry the link and a restore keeps it only for a
token the account still holds. A guard holds every EXTERNAL writer to
setting the link.
Forty-two guard files carried their own comment stripper, block comments
first in one non-greedy pass, so a /* inside a line comment opened a
block that hid the code after it, and the stricter copies hid code
behind a /* inside a string, which every e2e route glob has. They all
read through the shared stripComments now, and it scans the way the
parser does: strings, template literals and regular expressions keep
every character, a comment inside a template expression goes, and line
numbers are kept. A parity test strips every source file both ways,
with the scanner and with the TypeScript parser, and fails on the first
that differs.
No guard reports anything new. Each one fails when the stripper returns
nothing; three of them passed on an empty file and assert that the code
they ban is read at all now.
The JSON import refused the app's own files: the full backup writes an
empty note, tag list or glucose context as null and the JSON export as
an empty string, and the import schema read either as a value of the
wrong type. The file is now read in the exporter's terms before it is
validated: the export's envelope is opened, every null or empty field
is treated as absent, a backup row carrying a deletion is left out and
counted as skipped, and a unit the app converts on every other way in
is converted to the stored one.
The round trip also showed the portable backup dropping what a reading
itself says: glucose context, sleep stage, range bounds, rhythm and
device. They ride the portable file now, and a portable restore puts
them back. Sync bookkeeping stays out.
An integration test exports an account three ways (full backup, JSON
export, measurement CSV), imports each into a fresh account and compares
what the import carries.
The create, its mirror replay and the edit answered the written row
without the read-model fields the declared resource carries (reminder
state, stock and runway, actionability, course status, composition,
links), while the detail read answered them. They answer the read model
now, built from the row they just wrote (medicationDetail), so a client
reads back exactly what GET /api/medications/{id} would say.
A dose write that converged onto an existing slot answered the slot
upsert's narrow row, without userId, medicationId, updatedAt,
autoRecorded, autoMissed or attributionSource. The upsert reads and
returns the whole event row, the same shape a write that inserted its own
answers. The parity walks drop their contract skips and are re-pinned.
Twelve routes wrote their <surface>.validation-failed row with a bare
prisma.auditLog.create, so they skipped the two decisions auditLog makes:
an owner's own refused request stays on the wide event
(meta.rejected_attempt) instead of becoming a row, and a delegate's names
the delegate as the actor. The bulk medication and mood ingests, the
layout, overlay, device, health-sync, metric and ingest routes all go
through auditLog now, and a structural guard keeps raw writes to the one
writer and the briefing failure marker. The unit tests assert the auditLog
call, and the bulk walks are re-pinned without the owner's refusal row.
Work a request starts and does not wait for runs inside the request's
context, so an annotate({ action }) from it landed on the request's event
whenever it ran before the event was emitted. The measurement tail's
status re-warm logged a hand-entered reading as
insights.status.invalidate.skipped on some requests and as
measurement.create on others.
runDetached starts such work so that its action is dropped for the event
it was detached from; meta, warnings and the auth it reads stay as they
were, and an event the work opens for itself keeps its own action.
fireAndForget now takes a function and starts it detached, and every
trackBackgroundTask handle on a request path is built with runDetached,
which a structural guard holds.
POST /api/medications/intake/bulk keeps authentication, its own bulk bucket, the body with its size and schema refusals, the payload schema and the HTTP mapping. The ingest, one 846-line function, moves to lib/medications/intake-bulk/ in the order it always ran: the ownership read, the Apple mirror gate and the batch pre-reads (ingest.ts), each entry's refusals, site, write bookkeeping and P2002 answer (entry.ts), its slot attribution (entry-attribution.ts) and its slot, replay, probe or standalone write (entry-write.ts), then the audit row, the dose days, the snooze, the tails, the arrival record and the checkpoint. The two identical canonical-slot write blocks of the loop are one function now. The route falls from 1044 to 231 lines; no function in the new modules is over 70.
The parity walk recorded before the move (medication-intake-bulk.parity) is unchanged. The per-entry external-id markers name the module that classifies each entry, the batch-surface and key-scope legs read the moved code, and the intake-tracking allowlist and the dense-read guard follow it.
A '/*' inside a line comment opens a block for every guard that strips
block comments first, and the code after it disappears from that guard's
view. Four server comments and one new one did this; reword them.
GET and POST /api/medications/intake keep authentication, the query and body parsing with their refusals, the 404 and the HTTP mapping. The today list moves to lib/medications/intake-today.ts and the cached compliance buckets to intake-compliance.ts. The status update, one 432-line callback, moves to lib/medications/update-intake-status/ in the order it always ran: the event lookup, the record-only rule, the delegated refusals and the injection site (admission.ts), the take in place or onto another slot (take.ts), then the skip or snooze, the audit row, the rollups and the tails (update-intake-status.ts). The route falls from 634 to 149 lines; no function in the new modules is over 70.
The parity walk recorded before the move (medication-intake-status.parity) is unchanged. The sharing and activity-verb guards read the status-update modules beside the route, and the dense-read guard keeps the moved code in scope.
POST and GET /api/medications/{id}/intake keep authentication, the intake and ownership guards, the body and query parsing with their refusals, the delegated outcome-change refusal and the HTTP mapping. The dose write, one 607-line function, moves to lib/medications/record-intake/ in the order it always ran: the injection-site check and the slot attribution (attribution.ts), the replay and dedup answers and the slot, probe or standalone write (write.ts), then the inventory, the audit row, the rollup, the lifecycle and the tails (record-intake.ts). The history list moves to list-intake-events.ts. The route falls from 767 to 200 lines; no function in the new modules is over 75.
The parity walk recorded before the move (medication-intake.parity) is unchanged. The activity-verb guard now reads the awaited audit actions of a delegable write's handler modules beside the route, for this route and for the medication create that moved before it, and holds each module to its route's imports; the sharing guard, the intake-tracking allowlist and the dense-read guard follow the moved code.
GET /api/measurements/series keeps authentication, the query and its
refusal; readSeries (lib/measurements/series/read-series.ts) picks the
reader in the old order (sleep nights, paired blood pressure, the bucketed
read of a dense kind, the raw rows) and adds the direction, change,
reference lines and stat strip. The kinds, units and point shape live in
kinds.ts. The route falls from 752 to 95 lines.
The parity walk recorded before the move is unchanged. The guards on the
verdict resolver, the spread, the pulse day mean and the unit literals read
the new files, the last two with their scope widened to the series folder.
GET, PUT and DELETE /api/medications/{id} keep authentication, the ownership guard, the body and the HTTP mapping. The read moves to lib/medications/read-medication.ts; the edit, one 886-line callback, moves to lib/medications/update-medication/, split along its stages in the order they always ran: the preflight checks and refusals, the schedule normalisation, the transaction (era archive, tombstones, the row update, the pause era, the course write) and the commit tail with its audit row. The delete moves to delete-medication.ts. The route falls from 1270 to 100 lines; no function in the new modules is over 115.
The parity walk recorded before the move (medication-detail.parity) is unchanged. The sharing guard reads the edit and delete modules for the handler body, the intake-tracking allowlists name the modules that now read the schedule and the open slots, the published-DTO leg reads the read module, and the dense-read guard keeps the moved code in scope.
GET keeps authentication, the module gate and the AI capability, and hands
the rest to readWorkoutDetail (lib/workouts/detail.ts): the stored row, the
canonical winner of its cluster, the enrichment reads and the body, as
named stages. DELETE keeps authentication and the HTTP answers; the
ownership check, the synced-source refusal and the transactional removal
with its records, audit, cache and detection tail move to
lib/workouts/delete-manual-workout.ts.
The parity walk recorded before the move is unchanged; the guards that name
the canonical-identity reads and the track readers point at the service.
The route keeps authentication, the slice refusal, the summaries slice, the
cache read and the headers. buildAnalyticsResponse (446 lines) moves to
lib/analytics/response/default-body.ts and calls one module per block, in
its old order: the blood-pressure in-target windows, the glucose summaries
and clinical panel, the correlation hypotheses with their stored patterns,
the Health Score with its Rest Mode framing, and the sleep stages. The
route falls from 761 to 155 lines.
The parity walk recorded before the move is unchanged; the guards on the
glucose window and the bounded reads follow the code.
Marc-André Bombeck