The writer added in v1.38.6 compared the process's live heap usage against
80 % of V8's heap limit and aborted above it. That reading is not the
backup's footprint; it is the whole process's, garbage included. A
long-running app sits at 400 MB of largely collectable heap, so on a 1 GB
container (a 419 MB budget) the weekly pass aborted the first flush of every
account: four out of four in seven seconds, the smallest of them a demo
record whose entire stored copy is 1.2 MB, each told its record was too large
and its operator to buy memory.
Drop the gauge. What the writer holds is bounded by its shape and needs no
reading to prove: the three unbounded tables are pulled a page at a time,
every other section is released as soon as its JSON is in the sink, and at
most one flush of text is pending. The one copy that does grow with the
record is the stored blob, so the bound moves there — counted in the
ciphertext bytes actually produced, against a fifth of this process's heap
limit, which is what two copies of the answer plus a live set have to fit
inside. The message names both numbers.
A pass that wrote nothing for anybody now fails its job instead of reporting
`ok: true` with `backed: 0`, so the failing-queue panel and the backups page
read the same thing an operator would. A pass where some accounts were
written keeps passing with the failures as counts.
Measured under --max-old-space-size=450 from a heap deliberately dirtied
first: 445 000 measurements peak at 28 MB held by the backup, a one-row
record at under 1 MB.
`DataBackup.data` holds every weekly disaster-recovery snapshot and every
uploaded pack, AES-256-GCM at rest like everything else, under a column
called `data`. The rotation registry was keyed on the `*Encrypted` naming
convention, so the script never read a backup, reported zero rows
remaining, and the runbook read that zero as permission to retire the
previous key. Following the documented procedure destroyed every stored
backup — the copy an operator reaches for precisely when nothing else
works.
Register it, and register the idempotent-replay response cache the same
sweep turned up. The backup column walks in bounded batches (one row is a
whole compressed account) and re-seals the ciphertext without reading the
plaintext, so both stored envelopes come back byte-identical and so will
any later one. The cache is marked disposable: a row it cannot read is
dropped rather than counted as an error, because keeping it only
guarantees the next replay gets a body it cannot parse.
Running the script against a seeded database rather than reading it turned
up two more. It died on the mood day-context note, whose table is keyed on
the entry rather than on an `id`, and abandoned every column after it —
the backups included. And the document index's verbatim text rotated
without ever being named in the summary.
Make the naming assumption stop carrying weight. A new guard traces the
encryption helpers through their wrappers, walks every Prisma write
payload, and fails when a column receives ciphertext without being
registered. It asserts an anchored, non-zero match count so a matcher that
stops matching fails instead of passing. A second check pins each model's
primary key against the schema.
The script now ends by naming the registered columns it did NOT walk and
exiting non-zero when any is missing: a zero that means "nothing left" and
a zero that means "never looked" must not print the same. The runbook's
retire-the-key step was rewritten around that signal, and tells an
operator who already rotated on an older release how to get their backups
back.
The compression that landed last release was real and was measured in the
wrong place. On a development machine with a multi-gigabyte heap the pass
finished; the container the job runs in is capped at 1 GB, which gives V8
a 524 MB limit, and there the same pass died of memory about fifteen
seconds in. Because the job shares the application process, that was not
one failed backup — it was a restart, and every signed-in session on the
host went with it.
Reproduced before touching anything, under `--max-old-space-size=450`
against a seeded account of 445 000 measurements, 30 000 mood entries and
60 000 intake events: `FATAL ERROR: Reached heap limit` at 34 s. The same
fixture doubled to 890 000 measurements now completes inside a 296 MB
limit, holding 16 MB of live heap where the old writer held 168 MB.
The document is written incrementally now. `buildFullBackupPayload` takes
a `deferBulk` mode that declares the three unbounded tables instead of
reading them, and `streamFullBackupJson` walks the payload in key order,
pulls those tables through page by page, and releases every other section
as soon as its JSON reaches the sink. `packBackupBlobStreaming` feeds that
into gzip and an AES-256-GCM stream, so nothing between the database rows
and the stored value ever exists whole. Both writers go through the same
per-row serialisers, and the round-trip suite pins their output byte for
byte rather than trusting that they agree.
The intake and mood reads are ordered totally now. A keyset cursor needs
it, and a partial order also let two exports of one account differ over a
tie, which every multi-dose day has.
The streamed ciphertext is a new shape at rest, because GCM only produces
its tag once the last block is in and a leading tag is what makes the old
layout unstreamable. It carries a marker no older writer can produce, so
the two families are disjoint by construction rather than by sniffing.
All three stored shapes — plain, compressed, streamed — restore, proved
both ways.
The writer also watches its own live heap and stops at 80 % of V8's
limit. An account too large for its host is now one failed backup,
counted in the run's meta, instead of a restart for everybody on it.
The stored column is the ceiling that is left: `data_backups.data` is a
single `text` value, so the finished artifact has to exist as one string,
and that is the only part of the job still growing with the record. The
operator runbook says so, with the number at which it starts to matter.
The correction was inserted mid-bullet, which left the off-host sentence
trailing after it and the paragraph unwrapped. The off-host sentence goes
back with the technical description it belongs to, and the correction
reads last.
A `z.object()` without `.strict()` strips an unknown key and parses
clean, so a caller that misspells a field name gets a 200 and loses the
value with nothing to tell it. The worst of them was the Coach
question-dismiss endpoint, where an omitted `question` means "dismiss
all": a typo in that key wiped every pending question and answered as
though one had been dismissed.
Twenty-four inline schemas across twenty-two endpoints are strict now —
the settings toggles, the restore and bulk-delete endpoints, message
feedback, the admin user update, the onboarding acknowledgements, the
provider chain, the chart overlay preferences, the medication extractor
and the two client-side reporters.
The strip stays where it carries weight. Sync ingest, device
registration and the layout PUTs are built out of fields that shipped as
additive and optional, on the promise that a server which does not know
one ignores it; an app running ahead of the install it talks to is
routine here rather than exceptional, and refusing would turn a dropped
field into a refused batch — a year of history lost to one row, a
dashboard that will not save, a phone that quietly stops registering for
push. A read filter discards nothing when a stray query parameter rides
along, the OAuth endpoints are required by RFC 6749 and RFC 7591 to
ignore parameters they do not recognise, and an import file comes from
another version or another tool by definition. Each is registered with
its reason in a new inventory test, so a schema cannot join them without
someone making the call. That test parses the tree rather than grepping
it: fourteen of these schemas are written `z\n .object({…})` and a
`z.object(` sweep never saw them.
`sanitiseZodIssues` renders `unrecognized_keys` instead of passing it
through. Zod leaves `path` empty and puts the names in `issue.keys`, so
the sanitised issue named nothing at all and the caller was told only
that its body had failed. The names now ride a bounded `keys` field and
the message is rebuilt from them, capped at ten keys of 64 characters
and restricted to identifier characters, so a caller-supplied key cannot
echo back verbatim through the error path or into the audit ledger. The
names survive `stripValuesFromMessage` because a field name is
structure, not user-typed content.
The entry said the weekly pass finishes again. It was written from a
measurement taken on a development machine with a multi-gigabyte heap,
and it does not hold on a small container: capped at 1 GB the runtime
gets a 524 MB heap, and the pass still runs out of memory there. It gets
roughly three times further than before, which is a real improvement and
is now described as one, but it does not complete, and the failure takes
the whole process down rather than failing as a job.
Building the record incrementally instead of whole is the fix and has not
landed. Until it does, an instance backing up a large record needs more
than 1 GB, and the entry says so.
The column-reader sweep read every `field:` key as a write, so
`where: { ticketHash }` counted as writing `ticketHash`. All twelve columns it
reported were a lookup key or a cron's discovery predicate being filtered on,
and a review tool that has cried wolf twelve times is not consulted on the
thirteenth. It also reported only columns that already had a write, which made
a column with no write at all the one thing it could not see.
Mask each file, bracket-match every Prisma argument, and tag its contents as
write, projection or query so nesting resolves innermost-wins. Follow a payload
assembled before the call: a const handed to `update:`, a spread merged into a
`create:`, a `rows.push({...})` builder filling a typed array. Credit raw SQL
against the `@map` column name. Report a column nothing touches as its own
category, and split the output so a write with no reader reads differently from
a query predicate, a relation's foreign key and an annotated column.
Twelve false alarms become nineteen findings that hold up. Four are worth
acting on and are left for a decision rather than annotated away: the glucose
unit preference and an import job's export date have readers and no writer, and
the provider-health status code and the known-device first-seen stamp have the
opposite.
The matcher underneath is a test now, checked by breaking the fix three ways.
Carries the night's audit work: the weekly backup ran out of memory
rather than out of time and now completes, two dashboard reads stopped
scanning the largest tables, six webhook verbs compare a secret behind a
rate limit, three global channel switches actually stop delivery,
thirty-five settings reach the container, four guarded helpers are used
where their naive twins were, and three guards now see what they claim.
The weekly pass can stop and leave no trace where anyone looks. The
handler's own error never runs when the job is killed rather than thrown;
`recordError()` increments an unnamed counter; and the admin status page's
failing-queue panel looks back 72 hours, which for a weekly queue means a
permanent failure is on screen three days in seven. The backups page
itself showed an ordinary row with an ordinary timestamp, because a copy
made six weeks ago reads exactly like Sunday's until somebody subtracts
the dates.
The listing now carries two facts that fail at different times: the age of
the newest scheduled copy, which never leaves any retention window, and
the terminal state of the last scheduled run, which is the only place the
reason is written down. `readLastQueueRun` takes the newest run rather
than a window, so a weekly queue's outcome stays readable until the next
one replaces it, and it unwraps the value pg-boss stores a thrown error
under, so the page says "job timed out" instead of raw JSON. The
failing-queue reader gets the same unwrap.
Manual uploads are excluded from the cadence judgement by the shape of the
input, so an ad-hoc upload cannot make a dead cron look alive, and a fresh
instance is not called overdue for having no copy yet.
The listing also stops reading every blob into the web process to measure
it; `octet_length` answers that in the database, which on an instance with
large records was its own way of running out of memory.
Four advisories landed against the URI parser that arrives under ajv,
including the copy the connector SDK pulls, all of them host confusion
or request forgery through percent-encoded schemes, skipped IDN
canonicalisation and malformed authorities. The floor set yesterday was
one patch below the fix. A query-string parser needs 6.16.0 for an
array-limit bypass and a denial of service.
The audit reads no known vulnerabilities after the bump.
The inventory walked a flat, hand-written array of route trees, so its
coverage grew only when somebody remembered to grow it. It had stopped
growing: the coach, environment, mental-health, inbound-document and MCP
trees were all outside it, and so was the doctor-report export route the
header already named as a doctorReport surface. Every route in the first
four turned out to be gated, so nothing leaked — but that was the authors'
care rather than the inventory's, and a green run claimed more than it
covered.
The trees are declared per module key as an exhaustive Record<ModuleKey, …>:
a new registry key does not compile without an entry, one test asserts every
key is declared, another asserts every declared tree exists, and a third
discovers an API directory named after a module key on disk and fails when
it is not declared. An empty array is now a claim with a reason beside it
(glucose and recovery ride the measurement engine), not a silence.
Walking the newly covered trees classifies every route: coach, environment,
mental-health, the document vault and /api/export/health-record all gate
directly. The seven /api/mcp routes are the credential surface behind the
remote endpoint and go on the exempt list with the reason written out — the
endpoint itself lives outside /api and carries the gate, so a credential
minted while the module is off is inert. That refusal is added to the
behavioural list so deleting its proof fails here.
/api/export/health-record joins /api/fhir as a key-pinned tree: both serve
the whole-record aggregate, so doctorReport is the only correct key.
The Withings entrypoints, the Telegram bot webhook and the Coolify deploy
hook limited POST and left the verification verbs unlimited. GET and HEAD
compare the same secret and answer 200 against 401, which is an oracle for
guessing it and a free load channel besides; timingSafeEqual protects the
timing channel, not the status code. The project's rule is that the
per-source limit runs before secret verification, and it was being applied
per handler rather than per file.
Each verification verb now runs the same limit its POST sibling used, on the
same bucket. Telegram and the deploy hook grow the small local helper the
Withings routes already share. HEAD re-shapes the limiter's JSON envelope
into a bodyless 429 and keeps the rate-limit headers.
The new guard walks every webhook route from the filesystem, resolves which
local helpers reach a comparison and which reach the limiter, and fails when
an exported verb reaches a comparison without the limiter in front of it. It
asserts a non-zero match count so an empty match set cannot read as green.
WHOOP already passed: it exports POST only.
The admin AI key now goes to the API its base URL names, seven code
scanning findings are closed at their source, and the guard that was
meant to hold the image's two dependency pins in step can see scoped
packages for the first time. Two accessibility markers and one audit
read no longer race their own page.
The operator card offered a free-text base URL with an OpenAI placeholder
and no word about what else it takes, so an Anthropic endpoint looked
unsupported even though the field accepts that host. One line beside the
existing hints, in all seven locales, plus the changelog entry.
The account-deletion fix landed after v1.38.2 was tagged, so its entry
moves out of that section and into this one where it belongs. Also in
this release: every measurement source carries a display name in all
seven languages, with a test that reads the enums out of the schema.
Settings, Sources listed GOOGLE_HEALTH, OURA, POLAR and STRAVA by their
internal name: the catalogue was written for seven sources and every
provider added since fell through to the raw value. The measurement list
carried the same fall-through in its source filter and its table column,
and the medication intake history read "via APPLE_HEALTH" for a dose
synced from Apple Health.
The three catalogues move into one module keyed by the enum, so an
omission is a type error rather than a raw value on screen, and the sleep
discrepancy tooltip drops the private brand-name map it had grown to work
around the gap.
The published contract still told a client that a Bearer token can never
clear the step-up on account deletion or the record wipe, which is what
an app generates its client from. Describe the elevation arm on both:
which factors qualify, that a password proof does not, that the proof
rides the X-Step-Up header, and that it is spent only when the erasure
runs. The 403 a narrow-scope token gets was never published either;
it is named now, ahead of the second-factor question it precedes.
The mint endpoint lists the surfaces an elevation unlocks, so both
routes join that list.
Also note on the record wipe that API tokens are on its own delete list:
a client that wipes over Bearer signs itself out, which is worth knowing
before the retry rather than after it.
No assistant prefill on Anthropic JSON calls, Claude 5 and Fable count as
vision-capable, the briefing cache carries its language and is never
served to a reader of another one, and job paths resolve a missing
account language to the instance default.
The Anthropic vision check listed the Claude 3 and 4 families by prefix,
so a key pinned to claude-sonnet-5, claude-opus-5 or a Fable model was
told the provider cannot read images, and lab and document OCR refused
the upload even though the model handles image and PDF blocks.
Replace the per-release prefix list with a rule: any claude-<tier>-<major>
slug from the 4 family upward, plus every claude-fable-* model, counts as
vision-capable. Claude 3 handling is unchanged. The Settings preset list
gains claude-sonnet-5 and claude-opus-5 alongside the existing entries.
The Anthropic client steered JSON replies by appending an assistant turn
containing an opening brace and gluing the brace back onto the reply.
Current Claude models (the 4.6 / 4.7 / 4.8 family, Opus 5, Sonnet 5,
Fable 5 and 5.1) reject an assistant prefill with HTTP 400, so every
JSON surface on the provider (briefing, status cards, document and lab
extraction) failed for a self-hoster on a current model while plain chat
kept working.
Drop the prefill on every model. The request now carries a plain
instruction on the last user turn to reply with a single JSON object and
nothing else, and the reply is narrowed to the outermost object body via
a shared extractJsonObject helper, so a fenced or prefixed answer still
parses. The insights stripJsonFences helper delegates to the same
function rather than keeping a second copy. The tool-use path is
unchanged.
Reported independently by @sweidinger.
Marc-André Bombeck