Deleting an account, deleting a managed profile, "Delete all data" and the
admin wipe now write an off-host deletion request in the same transaction
as the deletion. A new job (offhost-backup-purge, kicked straight away and
run nightly as a backstop) lists the bucket once, deletes every
<date>/user-<id>.json.enc object of each pending account and removes the
request only when nothing of the account is left. A refused delete keeps
the request with the bucket's answer and is retried; the off-host card
shows how many deletions are outstanding and why. A copy the nightly run
was writing while the account was deleted is taken back after the upload.
The off-host card also reads the bucket's lifecycle rule and says plainly
when there is none (every nightly copy kept forever) or when the
credential may not read it.
Every stored backup now records the key ids its content was written
under, and every off-host upload records the same per key. The admin
encryption view lists, per key, the stored copies that need it with the
oldest date, and when the last off-host copy needing it expires. The
"safe to drop" badge now requires that no backup still needs a retired
key, instead of only that every row was rotated.
Migration 0358 adds data_backups.inner_key_ids, the key-use table and the
deletion requests.
A backup carries every encrypted column as the stored ciphertext, and a
restore writes it back verbatim. Key rotation re-seals the stored copy's
envelope but cannot reach inside it, so a copy taken before a rotation
still needs the retired key. Without it the restore went through and
left rows no reader can open.
The restore, the restore preview, the admin upload and the monthly
restore drill now collect the key ids of every inner ciphertext value,
hold them against ENCRYPTION_KEYS and decrypt one value per key. A
missing key, or an id configured with different key material, is
refused before anything is deleted, with the key id named. The restore
status and the preview say which key to put back.
The restore also checks, inside its transaction, that it added no
reference from the account's rows to rows another account owns. The
check is derived from the schema's foreign keys rather than written per
section, so the tag and category upserts keyed on a global key are
covered along with every link table. The illness parent update is
scoped to the owner as well.
The off-host listing now follows every page: the first page held only
the oldest dates, so the drill could read an old object and report the
chain as stale.
A snooze that had already run out before the send is history, not a
choice about the reminded slot, so it no longer keeps a row from being
repaired. The "no due date" arm matched nothing, because a reminder
cannot be created without a cadence; it now covers a rule whose last
occurrence was the one sent, which is open unless it was done or
skipped since, while a course whose window closed before the send stays
closed through the end-date guard. The start-page step reads a stored
item list only when it is a list, so a malformed layout can no longer
fail the migration.
Renaming an overdue booster check-up on the web still moved it to next
year. The edit sheet sends the first due date as the browser's local
midnight on every save, a booster row stores the dose instant plus N
days, and the route read the different instant as a new first due date.
The route now compares the first due date by calendar day in the
profile timezone and drops a resent same-day value from the write. The
edit sheet sends only the fields that changed. Re-enabling a reminder
that stays due after it is sent keeps the slot it was on instead of
scheduling from now; a short-cycle reminder is still scheduled from
now. Moving the notify hour later on the day a reminder was already
sent carries the last-sent cursor along, so the slot does not go out a
second time at the new hour.
The start page read every reminder due before the end of the day, so a
daily weigh-in at 20:00 or a twice-daily blood-pressure course showed
"check-up due" from the morning on and took a pinned place on the rail.
`holdsOpenAfterReminder` is now the one definition of a reminder that
stays due after it is sent: an interval longer than seven days, a rule
whose next gap is longer than seven days, a rule with no occurrence
left, or no cadence; never an appointment. The tick uses it to decide
whether a delivered reminder rolls on, and the start page uses it to
show such a reminder for its whole due day and keep it under the cap.
A short-cycle reminder shows only once its time has come and is not
pinned.
The tick also skips a reminder already sent for its current slot
outside its notify hour before the auto-resolve read. The eventful
satisfy worker still resolves it the moment a reading or lab result
lands, and the poll still runs for it once a day, in the notify hour;
the weekly hold stays after that poll so the safety net is daily.
The first cut kept only free-text check-ups due after their reminder
and let every reminder with a measurement type roll on. A fortnightly
PHQ-9 or GAD-7 reminder is a measurement reminder, and each one still
jumped two weeks forward the morning it was sent without the
questionnaire being filled.
The rule is now the cycle length. A reminder whose next slot is more
than seven days away, or that has no cadence, stays due after its
reminder until it is satisfied (a matching reading, a lab result or
"done"), skipped or snoozed, and is sent again at most once a week. A
reminder on a weekly or shorter cycle rolls on as before. Appointments
stay one-shot. A reading that lands on an open measurement reminder
still sets lastSatisfiedAt and schedules the next cycle from itself.
The 0355 repair drops its "no measurement type" condition for the same
cycle rule, so the fortnightly questionnaires in the reported shape
are put back on the day they were reminded.
The snoozed fixture sat inside a week of the send, so the one-week clause already excluded it and dropping the snooze guard from the migration left the suite green. Snooze it further out, a minute after the send, so only the snooze cursor tells it apart from a roll-on.
The start-page item choice stores the enabled kinds, so a list saved before a kind existed reads as that kind switched off. Say so next to the kind list, where the next kind will be added.
Every edit of a reminder recomputed its due date from now, so fixing
the label or the location of an open, overdue check-up moved it to its
next slot, the same "gone until next year" the reminder tick caused.
The due date is now recomputed only when the edit changes when the
reminder recurs: the interval, the rule, the first due date (compared
by value, since the edit forms resend every field), or switching a
disabled reminder back on. A notify-hour edit keeps the due day and
moves the hour on it, together with a snooze pinned to that slot.
The start page's short list left out things a reminder had just told
the person about. A check-up counted only once its due hour had
passed, a booked visit was read one at a time and dropped out at its
start time, which is when its reminder fires, and both sat below doses,
sync problems and the Coach check-in under a three-item cap, so a busy
morning pushed them off.
The check-up item now covers every check-up due today or overdue and
names up to three of them. Visits are read for the whole local day,
also once started, and for the next 48 hours: everything booked today
shares one item, the next day with a visit gets a second, and the
today / tomorrow / day-after wording follows the calendar rather than
an hour count. A due check-up and today's visits always keep a place
under the cap; the displayed order is unchanged. A Coach cadence label
is translated here as it is on the checkups page.
The digest's wire shape is unchanged: visits reach it only as
`upcoming_visit` items, which may now appear twice.
The reminder tick rolled every delivered reminder on to its next slot.
For a check-up (a reminder with no measurement type, made by the person
or suggested by the Coach) that filed an open task under next month or
next year the moment it was reminded, although nobody had done it. The
start page and the check-up card then showed a yearly check-up due
today as scheduled for next year, and a check-up with no cadence
dropped off every surface.
A check-up now keeps its due date after the reminder until it is
marked done, skipped or snoozed, and is reminded again at most once a
week while it stays open. A new `last_notified_at` column records the
last delivered reminder; a repeat is held while that send was for the
current slot and fewer than seven calendar days have passed, so a
snooze, a skip or a completion opens a fresh slot that fires on its own
day. The per-day claim stays the guard against two sends in one day.
Measurement reminders keep rolling on (the next reading satisfies them
and their next slot comes round soon), as do check-ups on a weekly or
shorter cadence, whose next slot is the next reminder anyway.
Appointment reminders stay one-shot.
Migration 0355 adds the column and puts back check-ups the old tick
rolled on: a live, enabled check-up whose most recent reminder claim is
followed within ten minutes by its last write, carrying a due date more
than a week past that send (or none, for a one-shot), not done, skipped
or snoozed since. Its due date returns to the instant of that send.
Anything else is left as it is. The same migration adds `upcoming_visit`
to start-page item choices stored before visits were one of them, which
otherwise read as visits switched off.
The due phrase on the check-up card and the checkups page counted a
check-up one day past its date as "Overdue by 1 days", and the same
counted form read wrongly in German. One day over now has its own
phrase in all seven languages. Both overdue phrases keep the warning
colour through a shared `isOverdueDue` helper, so a surface no longer
compares against a single key.
Every upload that sends a file counts towards the token's hourly limit,
workflow re-sends included, and a spent lookup allowance makes the
script wait rather than skip the wait. Note the 20 extra source ids a
document may collect, and name the settings section in the environment
example as the app shows it.
The runbook now says that key rotation converts copies still stored as one
value into pieces a slice at a time, that a backup skips an account whose
copy is being restored, that two backups of one account run one after the
other, what a reader answers when its copy is replaced mid-read, and that
migration 0352 cannot be undone by going back to v1.39.1.
Key rotation re-sealed a single-value backup in place, which holds the
stored string, its bytes, the plaintext and the new ciphertext at once; a
copy can be over a hundred megabytes and the in-app rotation walked 25 of
them per page, enough to run a 1 GB container out of memory. Rotation now
converts each such copy into pieces under the active key instead, one row
at a time. The ~hlgcm1. stream of v1.38.6 to v1.39.1 is read a megabyte
at a time with its tag checked at the end, inside one transaction, so a
copy that fails the check is left untouched; the copy keeps its date. The
scan reads only the first characters of each value. Measured on a copy of
a 2.6 million-reading account (126 MB stored): converted in 25 s with the
heap flat at 57 MB under a 200 MB limit, where re-sealing in place ran out
of heap.
The weekly and manual backup leave an account alone while a restore of
its current copy is queued or running, and count it as skipped, not
failed. A reader whose copy is replaced mid-read answers 409
backup_changed, "replaced by a newer one while it was being read", rather
than the undecryptable error; a restore rolls back with the same code.
Pieces are sealed with a fixed associated-data label, so a value the same
key sealed for another purpose never opens as a piece; rotation re-seals
under the same label. A row holding both a single value and pieces, which
only an older release writing after a downgrade produces, is refused with
a message saying so, and the next weekly run replaces it. Rotation treats a
row deleted between its read and its write as gone rather than an error.
The drill section said the drill needs nothing beyond the uploader's
GetObject and ListBucket, but the documented uploader grant is PutObject,
GetObject and AbortMultipartUpload. The drill lists the bucket to find the
newest object, so ListBucket is an addition, not something already granted.
The weekly-backup section still described one stored value capped at a
fifth of the heap and advised more memory. It now says how a copy is stored
and checked, that its size no longer depends on memory, what
BACKUP_MAX_STORED_MB limits and how to raise it, how older copies are read,
and how key rotation handles both forms. It also states that a 512 MB
container is below the app's own working set during a backup.
A stored backup was one value in data_backups.data. v1.39.1 wrote it a
piece at a time but still joined the pieces into that one value, and one
value has to pass through the app whole, so a copy was capped at a fifth of
the heap: 105 MB in the default 1 GB container. An account with 1.75
million readings is past that and could not be backed up at all (#1031).
A copy is now kept as ordered pieces of about a megabyte in the new
data_backup_chunks table (migration 0352). Each piece is sealed on its own
with AES-256-GCM, and its sealed header names the copy (a random stream id
kept on the row), its position and whether it is the last. A reader checks
every piece before it releases a byte, so a piece that is missing, moved,
altered, taken from another copy, or a copy cut short is refused with
backup.payload.undecryptable before a restore deletes anything, and the
second read during the restore checks each piece again.
The write is one transaction, so the previous copy stays whole and readable
until the new one commits. Pieces are written through the model API: a raw
query with a large parameter is kept alive by Prisma's query plan cache,
which left 160 MB of heap behind per hundred one-megabyte inserts.
Restore, preview, download, the restore job and the admin list read both
forms; copies written before this change keep their single value and still
restore. The heap-derived limit is gone. The one limit left is on storage,
BACKUP_MAX_STORED_MB (default 2048), and its message names the setting. Key
rotation covers the pieces, the wipe and account deletion take them with
their backup, and the backup plan lists the table as not carried.
Measured on an account of 2.6 million readings in a 1 GB container with web
and worker in one process: backup 87 s, 97 MiB in 96 pieces, preview 20 s,
download 25 s, restore 264 s with every reading back exactly, peak 567 MiB,
no restart.
A 429 on the lookup no longer waits for the Retry-After; the keyed upload
answers the same question. The source key goes in the query string and the
form. The closing summary and the guide point to Read with AI for deferred
documents instead of Index all.
Refs #1038
A document uploaded with aiRead=defer is now indexed locally by any
automatic path, because the indexer reads the marker from the row rather
than from the job. Index all for search leaves such documents out and no
longer counts them. Only an explicit Read with AI or Generate summary ends
the hold.
Uploads that send a body keep their slot of the hourly limit, duplicates
included; handing it back let a token send the same bytes under id after
id for free. One document takes at most 20 further source ids, beyond
which the upload answers 409 documents.inbound.sourceAliasLimit. The
query-string key check is metered on the lookup allowance it shares with
GET /api/documents/inbound/source.
Refs #1038
Name the Paperless-ngx webhook fields as the workflow editor shows them,
note that Paperless does not follow redirects, and date the double-brace
placeholders correctly: only {{doc_id}} needs 3.0. Say what a dry run
does not check, which re-sends count against which allowance, that Index
all leaves held-back documents alone, and that a workflow can hold AI
reading back with aiRead=defer.
A weekly or uploaded backup written by v1.39.x is a single ~hlgcm1. stream
in data_backups.data. Its key id sits behind the stream marker, which the
string codec's parser does not read, so key rotation filed every such row
as legacy, failed to decrypt it and counted it as an error. On a host that
ran v1.39.x that was every weekly copy, and the rotation never finished
clean.
Rotation now reads the key id of a streamed value and re-seals it as a
stream under the active key, without touching what it holds.
A current disaster-recovery backup writes aggregationProvenance for every
measurement, null included. The restore treated null like a missing field
and marked every Apple Health stats: row without a provenance as
LEGACY_UNKNOWN, which is meant only for backups too old to carry the field.
The nightly daily-mean fold writes exactly such rows, so on a large Apple
Health account a restore changed tens of thousands of readings it was
supposed to put back as they were (#1031).
Only an absent field now takes the legacy path.
Ask HealthLog whether a document is already held before downloading it
from Paperless-ngx or Papra, and send the source key in the query string.
Pin Paperless API version 9 (Paperless-ngx 2.16 and later), apply --since
locally for Paperless too, fall back to the file name for a blank title,
stop with a clear message when the HealthLog address redirects, and say
only that storage is full when the figures are withheld. The guide names
the minimum versions and the shared OTHER namespace.
Refs #1038
A keyed upload that matched a stored document by its bytes never recorded
its key, so deleting that document let the next import run store it again.
Such keys are now kept as aliases of the document, the source-key check
reads them, and the purge carries them into the ledger with the row's own.
A source key may ride the query string and is then answered before the
body is read or the hourly allowance charged; an upload answered as a
duplicate hands its slot back. GET /api/documents/inbound/source answers
whether a key is held, under the same documents:write scope, so an
importer can skip a download.
aiRead=defer is stored on the row. The summary catch-up, the summary job
and lab auto-staging skip such documents until an explicit Read with AI
or Generate summary clears the marker; backup and restore carry it.
A documents:write caller no longer sees the quota figures on a full vault.
Refs #1038
A delegate who may keep the illness journal but not read the visits
would be refused by GET /api/body-sites on every open of the condition
form, and each refusal writes an access-denied row. The field now asks
only when the grant covers the visits' section, and the condition page
shows the site as text rather than a link in that case.
Refs #1025
A condition (illness episode) can carry where on the body it sits, in the
person's own words and encrypted at rest, plus an optional side (LEFT,
RIGHT, BOTH). Same enum, codec and reasoning as the procedure's site.
- migration 0354: two nullable columns on illness_episodes, reusing the
laterality type from 0349
- create and edit accept bodySite and laterality; on PATCH an absent key
keeps the stored value, so an edit from the shipped iPhone app, which
never sends either field, leaves them untouched
- GET /api/body-sites: the sites the record holds across visits and
conditions, folded after the decrypt and bounded per kind; with site
(and laterality) the visits and conditions there, one hop to their
documents, lab results and visits. Read in the profile section;
conditions only under an illness grant with the module on, and links
into ungranted sections come back as unlabelled placeholders
- the link service reads document and visit links from the condition's
end of the two existing tables
- /checkups gains a Body sites tab; the visit and condition forms share
one body-site field pair that suggests sites already used; a
condition's page links its site into the view
- the doctor report, the clinician page and the MCP doctor-visit summary
name the site on conditions; the Coach illness block carries it
sanitised, and MCP get_illness_recovery returns it fenced
- backup and restore carry both fields in both file kinds; the site is
registered as ciphertext and rotates with the key
- i18n in all seven locales, OpenAPI regenerated
Refs #1025
Document tokens, the Paperless-ngx workflow recipe, the import script for
Paperless-ngx and Papra, scheduling for Papra, storage, and
DOCUMENT_UPLOAD_LIMIT_PER_HOUR.
Refs #1038
A single dependency-free Node 22 script that copies tagged documents from
Paperless-ngx (REST, original files) or Papra (API key, /file) into the
vault through a document token. Serial, one file in memory, waits on 429
Retry-After, and resumable through the server's source-key dedup. Supports
--since, --kind / --kind-map, --dry-run with sizes, and holds back automatic
AI reading unless --ai-read is given.
Refs #1038
A narrow `documents:write` token, minted cookie-only at
`POST /api/tokens/documents` and accepted by the vault upload route alone.
A scoped caller draws on its own hourly bucket per token
(`DOCUMENT_UPLOAD_LIMIT_PER_HOUR`, default 120) and gets a receipt
`{ id, duplicate }` instead of the stored row.
Uploads take optional `sourceSystem` / `sourceId` fields. The source key is
unique per account across live and deleted documents, and the tombstone
purge carries it to a small ledger, so re-sending a document the owner
deleted answers 200 `deleted: true` and stores nothing. `aiRead=defer`
indexes locally and skips the summary and lab staging.
The detail sheet shows where an imported document came from; backup and
restore carry both fields; Delete All Data clears the ledger.
Refs #1038, #938
A commit that reaches main without a pull request, such as an asset
uploaded through the web UI, was never analysed until the weekly run.
Run on push to main as well. Pull-request runs still cancel their
superseded heads; push runs are grouped per commit and never cancelled,
so back-to-back merges each keep their own analysis.
Bump @simplewebauthn/server 13.3.3 -> 14.0.2 and @simplewebauthn/browser
13.3.0 -> 14.0.0, and keep React on an exact pin at 19.3.0 with react-dom
and both type packages moved alongside.
SimpleWebAuthn 14 changes one default the server relied on: the verifier
used to accept every algorithm the library knew (EdDSA, ES256, ES512,
PS256/384/512, RS256/384/512, RS1) and now accepts only the three it
offers. Pass both lists explicitly so a registration that succeeded
before still succeeds, and so the offered list stays EdDSA/ES256/RS256
on a runtime that would otherwise prepend ML-DSA-44. The option JSON of
all four ceremonies, and verification against stored credentials, match
the 13.x snapshots unchanged.
Every existing passkey test mocks @simplewebauthn/server, so a library
upgrade could change the options a browser or the iOS app receives, or
the algorithms the verifier accepts, without any test noticing. Drive
the real library with a software authenticator: snapshot the option
JSON of all four ceremonies, register and authenticate with each of the
ten algorithms the verifier has accepted, and verify an assertion
against a credential row stored the way earlier releases wrote it.
Marc-André Bombeck