The probe decided on the eight oldest values alone. An install
reinstalled under a fresh key long ago holds old rows sealed under a key
that is gone and all current data under the key it runs with; it works
today, and the check would refuse it with 503 on every request.
When every oldest value fails, the newest values of the same columns are
now sampled in reverse order. One that opens makes the verdict
inconclusive (mixed): the process serves, warns and probes again at the
next start. Only old and newest values that all fail are a mismatch. A
newest-only open never seals; sealing stays with the oldest-first rule.
Every probe query now runs in its own transaction under a
transaction-local statement_timeout (2 s), so a large or locked table
ends that column with an error and the probe as incomplete instead of
holding up the boot. The newest order is the exact mirror of the oldest,
so an index on the timestamp serves both directions.
The module comment lists the three conditions that still yield a
mismatch.
At a first start there is no recorded key yet, so an operator facing a
refusal they are certain is false has no row to delete. With
ENCRYPTION_KEY_CHECK=warn (default enforce) the check still runs and logs
its full outcome, /api/health names a mismatch as a warning field without
failing, requests are served, and no key is recorded unless every key id
passes.
The variable is on the compose whitelist, in both example files, the env
manifest, the Unraid and Portainer templates and the published-env guard.
The log block, the mismatch page and the self-hosting pages offer it as the
last resort after the three real fixes, and say that it disables a safety
check.
apiHandler answers 503 encryption.key_mismatch before auth, but the remote
MCP endpoint and the OAuth authorize, token and register routes wrap in
withBackgroundEvent instead and never checked. An MCP write tool could seal
new rows under the wrong key, and those rows would open under it at the
next boot.
The refusal moves into one helper used by apiHandler, /mcp (same envelope)
and the OAuth routes (RFC 6749 temporarily_unavailable with the error code
as an extension member). A structural test fails when a route outside
apiHandler, or a server action, lacks the refusal in every exported handler
and is not on a short allowlist of routes that never touch the database.
The boot key check recorded whatever key was configured whenever its probe
found nothing to try. It only read string columns, so an install whose
ciphertext sat in Bytes columns (notes, documents, ECG, symptoms) read as
empty; a probe that ran out of time did the same; and rows written by a
process that served under a wrong key could open on the next boot and get
that key recorded as the right one.
The probe now reads every registered column, string and Bytes, in all four
stored layouts, and tries the oldest values first, ordered by the row's
updatedAt (else createdAt, else primary key). A key is recorded only when
the oldest value that opens is preceded by at most one that does not, or
when no column holds a single value under the key id. A time-out, an
unreadable table, oversized values or mixed results are inconclusive: the
process serves, records nothing and checks again at the next start.
When the first insert loses a race, the stored record is read back and must
open under this process's key, else the outcome is a mismatch.
A rotation that walked every column without errors removes the record of
each previous key id that no longer holds any value. The refusal log block
and the self-hosting pages explain how to reset the check for a key id.
The boot check decided on the first value it found under a key id, so
one damaged or foreign-sealed row could take a correctly keyed server
down with 503 on every request. The probe now samples up to five
values, one per column first. One value that opens clears the key; a
mismatch needs at least two values and none opening. A single value
that does not open is inconclusive: the server keeps serving, writes no
canary, logs a warning and probes again at the next start.
Integration setup truncates encryption_key_canaries between tests.
Every configured key id now has a canary in encryption_key_canaries,
checked at boot. A key that cannot open its canary, or existing data
written before the canary existed, puts the process in a refusal state:
every API route answers 503 encryption.key_mismatch, /api/health reports
reason encryption_key_mismatch, the pages render one explanation, and
neither the queue producer nor the worker starts. The process stays up
so the reason is visible instead of hiding in a crash loop.
The admin Encryption page gains a "Back up your encryption key" step:
key id and fingerprint (never the key), where the key lives per
platform, a check of a pasted copy that is compared and discarded, and
a confirmation bound to the active key id and fingerprint. The overview
gets a tile and admins get a dashboard banner while it is due.
HEALTHLOG_PLATFORM picks the platform the step opens on. Adds the
TrueNAS self-hosting page and key backup sections in the docs.
Marc-André Bombeck