Papra organization ids are checked against org_ plus 24 characters
before anything is sent. A 403 user.not_in_organization and a 400
server.invalid_request.params read as no such organization instead of a
refused key; the error body is read for its code and never passed on.
The connection test also reads the tags, so a key without tags:read
fails at save (permissionMissing) instead of on the first filter. Name
searches quote each word, so a leading dash is not a negation, and the
date range goes to Papra as date:>= / date:<= with the per-row check
kept as a net.
The ingest path takes the capability from its caller, so the upload and
the import route each read getAiCapability("documentAi") in their own
source, where the AI route inventory classifies them.
A person stores one connection per system (base address, Papra
organization id, API token encrypted at rest) and can search it by
document name with tag and date filters, then import picked documents
one per request through the same ingest path as an upload: source-key
answer before any download, content dedup, deleted stays deleted, quota,
provenance, and the person's own AI reading setting. An import can link
the document to the visit, condition or vaccination it was picked for.
The operator lists the instances in DOCUMENT_SOURCE_ORIGINS; unset, the
feature is off. Only listed origins are contacted, through safeFetch with
the origin pinned, redirects refused, timeouts and byte caps. The routes
are cookie-only and owner-only. The token is registered for rotation,
removed by Delete all data, and not carried in backups.
The first cut kept only free-text check-ups due after their reminder
and let every reminder with a measurement type roll on. A fortnightly
PHQ-9 or GAD-7 reminder is a measurement reminder, and each one still
jumped two weeks forward the morning it was sent without the
questionnaire being filled.
The rule is now the cycle length. A reminder whose next slot is more
than seven days away, or that has no cadence, stays due after its
reminder until it is satisfied (a matching reading, a lab result or
"done"), skipped or snoozed, and is sent again at most once a week. A
reminder on a weekly or shorter cycle rolls on as before. Appointments
stay one-shot. A reading that lands on an open measurement reminder
still sets lastSatisfiedAt and schedules the next cycle from itself.
The 0355 repair drops its "no measurement type" condition for the same
cycle rule, so the fortnightly questionnaires in the reported shape
are put back on the day they were reminded.
Every edit of a reminder recomputed its due date from now, so fixing
the label or the location of an open, overdue check-up moved it to its
next slot, the same "gone until next year" the reminder tick caused.
The due date is now recomputed only when the edit changes when the
reminder recurs: the interval, the rule, the first due date (compared
by value, since the edit forms resend every field), or switching a
disabled reminder back on. A notify-hour edit keeps the due day and
moves the hour on it, together with a snooze pinned to that slot.
The start page's short list left out things a reminder had just told
the person about. A check-up counted only once its due hour had
passed, a booked visit was read one at a time and dropped out at its
start time, which is when its reminder fires, and both sat below doses,
sync problems and the Coach check-in under a three-item cap, so a busy
morning pushed them off.
The check-up item now covers every check-up due today or overdue and
names up to three of them. Visits are read for the whole local day,
also once started, and for the next 48 hours: everything booked today
shares one item, the next day with a visit gets a second, and the
today / tomorrow / day-after wording follows the calendar rather than
an hour count. A due check-up and today's visits always keep a place
under the cap; the displayed order is unchanged. A Coach cadence label
is translated here as it is on the checkups page.
The digest's wire shape is unchanged: visits reach it only as
`upcoming_visit` items, which may now appear twice.
The reminder tick rolled every delivered reminder on to its next slot.
For a check-up (a reminder with no measurement type, made by the person
or suggested by the Coach) that filed an open task under next month or
next year the moment it was reminded, although nobody had done it. The
start page and the check-up card then showed a yearly check-up due
today as scheduled for next year, and a check-up with no cadence
dropped off every surface.
A check-up now keeps its due date after the reminder until it is
marked done, skipped or snoozed, and is reminded again at most once a
week while it stays open. A new `last_notified_at` column records the
last delivered reminder; a repeat is held while that send was for the
current slot and fewer than seven calendar days have passed, so a
snooze, a skip or a completion opens a fresh slot that fires on its own
day. The per-day claim stays the guard against two sends in one day.
Measurement reminders keep rolling on (the next reading satisfies them
and their next slot comes round soon), as do check-ups on a weekly or
shorter cadence, whose next slot is the next reminder anyway.
Appointment reminders stay one-shot.
Migration 0355 adds the column and puts back check-ups the old tick
rolled on: a live, enabled check-up whose most recent reminder claim is
followed within ten minutes by its last write, carrying a due date more
than a week past that send (or none, for a one-shot), not done, skipped
or snoozed since. Its due date returns to the instant of that send.
Anything else is left as it is. The same migration adds `upcoming_visit`
to start-page item choices stored before visits were one of them, which
otherwise read as visits switched off.
Every upload that sends a file counts towards the token's hourly limit,
workflow re-sends included, and a spent lookup allowance makes the
script wait rather than skip the wait. Note the 20 extra source ids a
document may collect, and name the settings section in the environment
example as the app shows it.
The runbook now says that key rotation converts copies still stored as one
value into pieces a slice at a time, that a backup skips an account whose
copy is being restored, that two backups of one account run one after the
other, what a reader answers when its copy is replaced mid-read, and that
migration 0352 cannot be undone by going back to v1.39.1.
Key rotation re-sealed a single-value backup in place, which holds the
stored string, its bytes, the plaintext and the new ciphertext at once; a
copy can be over a hundred megabytes and the in-app rotation walked 25 of
them per page, enough to run a 1 GB container out of memory. Rotation now
converts each such copy into pieces under the active key instead, one row
at a time. The ~hlgcm1. stream of v1.38.6 to v1.39.1 is read a megabyte
at a time with its tag checked at the end, inside one transaction, so a
copy that fails the check is left untouched; the copy keeps its date. The
scan reads only the first characters of each value. Measured on a copy of
a 2.6 million-reading account (126 MB stored): converted in 25 s with the
heap flat at 57 MB under a 200 MB limit, where re-sealing in place ran out
of heap.
The weekly and manual backup leave an account alone while a restore of
its current copy is queued or running, and count it as skipped, not
failed. A reader whose copy is replaced mid-read answers 409
backup_changed, "replaced by a newer one while it was being read", rather
than the undecryptable error; a restore rolls back with the same code.
Pieces are sealed with a fixed associated-data label, so a value the same
key sealed for another purpose never opens as a piece; rotation re-seals
under the same label. A row holding both a single value and pieces, which
only an older release writing after a downgrade produces, is refused with
a message saying so, and the next weekly run replaces it. Rotation treats a
row deleted between its read and its write as gone rather than an error.
The drill section said the drill needs nothing beyond the uploader's
GetObject and ListBucket, but the documented uploader grant is PutObject,
GetObject and AbortMultipartUpload. The drill lists the bucket to find the
newest object, so ListBucket is an addition, not something already granted.
The weekly-backup section still described one stored value capped at a
fifth of the heap and advised more memory. It now says how a copy is stored
and checked, that its size no longer depends on memory, what
BACKUP_MAX_STORED_MB limits and how to raise it, how older copies are read,
and how key rotation handles both forms. It also states that a 512 MB
container is below the app's own working set during a backup.
A stored backup was one value in data_backups.data. v1.39.1 wrote it a
piece at a time but still joined the pieces into that one value, and one
value has to pass through the app whole, so a copy was capped at a fifth of
the heap: 105 MB in the default 1 GB container. An account with 1.75
million readings is past that and could not be backed up at all (#1031).
A copy is now kept as ordered pieces of about a megabyte in the new
data_backup_chunks table (migration 0352). Each piece is sealed on its own
with AES-256-GCM, and its sealed header names the copy (a random stream id
kept on the row), its position and whether it is the last. A reader checks
every piece before it releases a byte, so a piece that is missing, moved,
altered, taken from another copy, or a copy cut short is refused with
backup.payload.undecryptable before a restore deletes anything, and the
second read during the restore checks each piece again.
The write is one transaction, so the previous copy stays whole and readable
until the new one commits. Pieces are written through the model API: a raw
query with a large parameter is kept alive by Prisma's query plan cache,
which left 160 MB of heap behind per hundred one-megabyte inserts.
Restore, preview, download, the restore job and the admin list read both
forms; copies written before this change keep their single value and still
restore. The heap-derived limit is gone. The one limit left is on storage,
BACKUP_MAX_STORED_MB (default 2048), and its message names the setting. Key
rotation covers the pieces, the wipe and account deletion take them with
their backup, and the backup plan lists the table as not carried.
Measured on an account of 2.6 million readings in a 1 GB container with web
and worker in one process: backup 87 s, 97 MiB in 96 pieces, preview 20 s,
download 25 s, restore 264 s with every reading back exactly, peak 567 MiB,
no restart.
A 429 on the lookup no longer waits for the Retry-After; the keyed upload
answers the same question. The source key goes in the query string and the
form. The closing summary and the guide point to Read with AI for deferred
documents instead of Index all.
Refs #1038
A document uploaded with aiRead=defer is now indexed locally by any
automatic path, because the indexer reads the marker from the row rather
than from the job. Index all for search leaves such documents out and no
longer counts them. Only an explicit Read with AI or Generate summary ends
the hold.
Uploads that send a body keep their slot of the hourly limit, duplicates
included; handing it back let a token send the same bytes under id after
id for free. One document takes at most 20 further source ids, beyond
which the upload answers 409 documents.inbound.sourceAliasLimit. The
query-string key check is metered on the lookup allowance it shares with
GET /api/documents/inbound/source.
Refs #1038
Name the Paperless-ngx webhook fields as the workflow editor shows them,
note that Paperless does not follow redirects, and date the double-brace
placeholders correctly: only {{doc_id}} needs 3.0. Say what a dry run
does not check, which re-sends count against which allowance, that Index
all leaves held-back documents alone, and that a workflow can hold AI
reading back with aiRead=defer.
Ask HealthLog whether a document is already held before downloading it
from Paperless-ngx or Papra, and send the source key in the query string.
Pin Paperless API version 9 (Paperless-ngx 2.16 and later), apply --since
locally for Paperless too, fall back to the file name for a blank title,
stop with a clear message when the HealthLog address redirects, and say
only that storage is full when the figures are withheld. The guide names
the minimum versions and the shared OTHER namespace.
Refs #1038
A keyed upload that matched a stored document by its bytes never recorded
its key, so deleting that document let the next import run store it again.
Such keys are now kept as aliases of the document, the source-key check
reads them, and the purge carries them into the ledger with the row's own.
A source key may ride the query string and is then answered before the
body is read or the hourly allowance charged; an upload answered as a
duplicate hands its slot back. GET /api/documents/inbound/source answers
whether a key is held, under the same documents:write scope, so an
importer can skip a download.
aiRead=defer is stored on the row. The summary catch-up, the summary job
and lab auto-staging skip such documents until an explicit Read with AI
or Generate summary clears the marker; backup and restore carry it.
A documents:write caller no longer sees the quota figures on a full vault.
Refs #1038
A condition (illness episode) can carry where on the body it sits, in the
person's own words and encrypted at rest, plus an optional side (LEFT,
RIGHT, BOTH). Same enum, codec and reasoning as the procedure's site.
- migration 0354: two nullable columns on illness_episodes, reusing the
laterality type from 0349
- create and edit accept bodySite and laterality; on PATCH an absent key
keeps the stored value, so an edit from the shipped iPhone app, which
never sends either field, leaves them untouched
- GET /api/body-sites: the sites the record holds across visits and
conditions, folded after the decrypt and bounded per kind; with site
(and laterality) the visits and conditions there, one hop to their
documents, lab results and visits. Read in the profile section;
conditions only under an illness grant with the module on, and links
into ungranted sections come back as unlabelled placeholders
- the link service reads document and visit links from the condition's
end of the two existing tables
- /checkups gains a Body sites tab; the visit and condition forms share
one body-site field pair that suggests sites already used; a
condition's page links its site into the view
- the doctor report, the clinician page and the MCP doctor-visit summary
name the site on conditions; the Coach illness block carries it
sanitised, and MCP get_illness_recovery returns it fenced
- backup and restore carry both fields in both file kinds; the site is
registered as ciphertext and rotates with the key
- i18n in all seven locales, OpenAPI regenerated
Refs #1025
Document tokens, the Paperless-ngx workflow recipe, the import script for
Paperless-ngx and Papra, scheduling for Papra, storage, and
DOCUMENT_UPLOAD_LIMIT_PER_HOUR.
Refs #1038
A narrow `documents:write` token, minted cookie-only at
`POST /api/tokens/documents` and accepted by the vault upload route alone.
A scoped caller draws on its own hourly bucket per token
(`DOCUMENT_UPLOAD_LIMIT_PER_HOUR`, default 120) and gets a receipt
`{ id, duplicate }` instead of the stored row.
Uploads take optional `sourceSystem` / `sourceId` fields. The source key is
unique per account across live and deleted documents, and the tombstone
purge carries it to a small ledger, so re-sending a document the owner
deleted answers 200 `deleted: true` and stores nothing. `aiRead=defer`
indexes locally and skips the summary and lab staging.
The detail sheet shows where an imported document came from; backup and
restore carry both fields; Delete All Data clears the ledger.
Refs #1038, #938
A medication's units per dose, a schedule's override and a restored
backup now accept any value above 0 and at most 100 with at most four
decimal places, which is exactly what the Decimal(10,4) column stores.
Before, only whole numbers 1-100 and the fractions 1/4, 1/3, 1/2, 2/3,
3/4 passed, so one and a half tablets could not be recorded. A restore
now refuses a value no write route would accept.
The medication editor keeps its buttons and adds an Other field that
reads 1.5, 1,5, 1 1/2 and 1½, holds the step while the text does not
read, and previews the value it will save. The supply tab's packaging
dialog uses the same reader; it refused every non-whole value before,
so a medication at half a tablet could not have its packaging saved.
Cards and the supply tab show a whole number plus a fraction as 1½.
The rule, the input reader and the display format live in one module
without imports so the editor and the validators share them.
Refs #1034
A per-medication trackIntake flag (migration 0351, default on for every
existing and new row). With it off the dose, dates and schedule stay stored,
but nothing derives a due dose from them: the reminder worker, the manual
reminder check, the today projector, the auto-miss pass, the low-stock pass,
the doses card, the today list, the outstanding-dose badge, compliance and
its heatmap, the doctor report, the Coach, the BP-status gate, insights,
targets, achievements, the efficacy view and the MCP schedule tool all skip
it.
The medication wire serves such a medication with schedules: [] and the
stored rows in an additive recordedSchedules field, so a client that arms
local reminders from schedules stays silent without an update. Switching
tracking off or back on archives a schedule revision at that instant, the
untracked stretch with an empty payload, so it never counts as missed and
the projector does not put a dose from before the switch back on the card.
A structural guard freezes the set of schedule readers: each one filters on
the flag, derives from dueSchedules, shapes the wire with scheduleWireFields,
or is allowlisted with the reason it derives nothing due. The flag travels
through backup, restore and the medication CSV export.
An uploaded file whose encrypted copy passes the size one stored backup may
have on this host (a fifth of the heap limit) stopped the store with
BackupBlobTooLargeError, which the route let through as a 500 and reported
to the error tracker. It is now a 413 with the numbers and the remedy in the
message and meta.errorCode backup.upload.too_large, and the refusal is
audited like the others. The store rolls back, so nothing is kept.
A restore commits its transaction and then rebuilds the chart tiers. A
worker that stopped during that rebuild left the job running with a stale
heartbeat, and the boot sweep and the admission check treated it like any
interrupted job: the sweep queued it again, so the second run deleted and
re-wrote the account and lost whatever was written in between, and a second
interruption closed it with "rolled back, nothing was changed", which was
false.
The job now records `committedAt` (added to migration 0347, which has not
shipped) and awaits that write before the rebuild starts. The sweep and the
admission check close a committed job as `failed_after_commit`, never queue
it again, and evict the account's caches. The rebuild itself is left to the
nightly and boot passes that already rebuild missing tiers.
The module comment now says what an interrupted restore does (restarted
once, then failed) and what happens to other writes while a restore runs:
new rows are written at once, updates to rows the restore already deleted
wait on their lock until it commits or rolls back.
A visit can be filed as a procedure or surgery (new EncounterKind
PROCEDURE) and carry a body site in the person's own words, encrypted at
rest, plus an optional side (LEFT, RIGHT, BOTH). An existing visit
switches to the new kind by an ordinary edit, keeping its date, reason,
outcome and links.
- migration 0349: enum value, laterality type, two nullable columns
- GET /api/encounters/procedures: the DONE procedures, newest first,
searched after the decrypt over site, side and reason, with body-site
choices and a total computed over the whole history
- create and edit accept bodySite and laterality on any kind
- /checkups gains a Procedures tab with a body-site filter; the visit
form offers the site for a procedure and keeps it in view on any
visit that carries one
- backup and restore carry both fields; the body site is registered as
ciphertext and rotates with the key
- MCP get_visits narrows by kind and reads the whole record for
procedures when no window is named; the Coach visit block names a
procedure's site
- i18n in all seven locales, OpenAPI regenerated
Refs #1025
A bridge such as Tasker or Home Assistant could only send sleep stages
through the batch route, in HealthKit's terms: the category identifier,
a stage codepoint from 0 to 5, and a value it had to work out in
minutes. It can now post one segment to `POST /api/measurements` with
`type: "SLEEP_DURATION"`, a `sleepStage` of `IN_BED`, `ASLEEP`, `AWAKE`,
`CORE`, `DEEP` or `REM`, and either `startDate` + `endDate` or
`measuredAt` (the end) + `value` (minutes).
The row is the one the batch route writes for the same segment, and the
dedupe is the batch route's: the opposite-source same-reading check,
then one reconcile over both unique identities. Without an `externalId`
the server derives one from the stage and both instants, so a re-post
answers 200 `duplicate`, a corrected start answers 200 `updated`, and a
new segment answers 201 `inserted`. The narrow `measurements:write`
token is accepted and still writes only MANUAL rows. Every other body
on this route behaves as before.
Requested in #972.
`/api/auth/me` has reported the resolved zone, the stored one or the
instance default when the stored value is unusable, while
`/api/user/profile` returned the raw column. A client reading the
profile could cut days differently from the server, and sending the
value back in a PATCH was refused.
Both reads, and the profile PATCH answer, now go through one
`resolveStoredTimezone`. The field stays a string and always holds a
valid IANA name. The web account editor already reads the resolved
zone from `/api/auth/me` and saves through the validating timezone
route, so it shows the zone in use and can only write a valid one.
The nutrient batch docs told a client to log and drop an `upsert_failed`
entry. The route means the opposite: a write group failed on the
server's side and nothing was stored. The docs now say to send the
entry again on a later sync, and keep the three validation skips
terminal.
A retry under the same Idempotency-Key used to replay the cached
failure, because the 200 carrying `upsert_failed` was cached like any
other. Both the nutrient and the cycle bulk routes now mark that
response `no-store`, as the measurement batch already does for
`failed`, so the retry writes again.
The cycle bulk route already answered its validation refusal with
`cycle.bulk.invalid` and an over-cap batch with `cycle.bulk.too_large`.
Both are now pinned by tests and named in the route's 422 response.
The validation 422 from `POST /api/measurements/batch` now carries
`meta.errorCode: "measurement.batch.invalid"`, beside the existing
`too_large` and `source_not_permitted` codes, so a client can file the
rows as permanently refused instead of holding its anchor on a guess.
A `value_out_of_range` entry now also carries `convertedValue`, the
reading after conversion into the stored unit, and `range`
(`min`, `max`, `unit`), the band it was checked against. Both appear
only on that skip. A fraction sent where a percent was expected is now
visible in the response instead of looking like a sensor glitch.
Status codes, the envelope and every existing field are unchanged.
Marc-André Bombeck