docs: describe the app's database role, its limits and how to opt out

docs/ops/deploy.md

@@ -140,6 +140,22 @@ An INVALID index whose migration is already recorded applied is not touched at
 boot. The boot names it on stderr with `WARNING — index … is INVALID`; rebuild
 it with `REINDEX INDEX CONCURRENTLY "<index>";` when the database is quiet.
 
+### The app's database role
+
+The steps above run as the owner `DATABASE_URL` names: the dump, the
+migrations and, after them, the setup of `healthlog_app`, the role the app
+then connects as ([database-role.md](../self-hosting/database-role.md)). The
+app never runs a migration. A migration that creates a table needs nothing
+more: the setup grants the role every table it lacks before the app starts.
+
+On the first start of the release that brings the role, the setup hands the
+existing `pgboss` tables to it, which takes each table's lock for a moment.
+Stop the old container (and a separate worker) before the new one starts, as
+for any update with migrations; a lock that does not come within 15 seconds
+leaves that start on the owner with a notice, and the next start tries again.
+A web container and a worker container that start together wait for each
+other on an advisory lock and derive the same password.
+
 ### v1.46.0 notes
 
 - **Postgres 18.** The compose file moves the bundled database from

docs/ops/disaster-recovery.md

@@ -147,6 +147,14 @@ different.
    Sign in. The app's own key check runs at start and refuses to serve on a
    key that does not open the data.
 
+The restore runs as the owner `DATABASE_URL` names, and the restored tables
+belong to it. The app connects as `healthlog_app`
+([database-role.md](../self-hosting/database-role.md)): the start in step 5
+grants that role the restored tables and hands it the queue schema before the
+app runs. A dump carries grants to `healthlog_app`; `db-restore` leaves them
+out (`--no-owner --no-privileges`). Restoring with `pg_restore` by hand, pass
+the same two flags, or it fails on a database without the role.
+
 When the dumps are still in the server's `backups` volume (the database
 volume was lost, the backups volume was not), step 4 is
 `docker compose run --rm db-restore latest`.

docs/ops/measurement-maintenance.md

@@ -214,7 +214,7 @@ So an index is retired in two halves (v1.43.8, migration 0416, for
    exactly as the schema does, and `prisma migrate dev` / `migrate diff` see
    no drift and generate nothing.
 2. The index name goes into `RETIRED_MEASUREMENT_INDEXES` in
-   `src/lib/jobs/measurement-maintenance.ts`. Every run of this job, the
+   `src/lib/jobs/retired-measurement-indexes.mjs`. Every run of this job, the
    nightly 03:40 check included, drops each name still present with
    `DROP INDEX CONCURRENTLY IF EXISTS`, waiting at most 60 seconds for its
    locks. A run that runs out of time leaves the index (possibly marked
@@ -242,6 +242,16 @@ SET lock_timeout = '60s';
 DROP INDEX CONCURRENTLY IF EXISTS "measurements_user_id_type_measured_at_idx";
 ```
 
+When the app connects as its restricted role
+([database-role.md](../self-hosting/database-role.md)), the job may vacuum and
+rebuild (`MAINTAIN`) but not drop: `DROP INDEX` belongs to the table's owner.
+A run then skips the drops, of retired indexes and of the invalid copy a
+failed rebuild leaves, names what it found under
+`maintenance_drops_left_for_boot` with a warning, and the next start drops
+each as the owner with `DROP INDEX CONCURRENTLY`, waiting at most five
+seconds. Until then a failed rebuild's copy holds its disk; restart the
+container, or drop it by hand as the owner with the statement above.
+
 For the next large index, follow the same two halves.
 `src/__tests__/migration-lock-timeout-guard.test.ts` refuses a
 `DROP INDEX CONCURRENTLY` in any migration and requires every retired name to

docs/self-hosting/database-role.md

@@ -0,0 +1,112 @@
+# The app's database role
+
+The container connects to Postgres under two names. The user in
+`DATABASE_URL` (on the bundled compose file, `healthlog`, which the official
+Postgres image makes a superuser) owns the schema: the entrypoint uses it for
+the dump before an update, the migrations and the restore. The app itself,
+web and worker, connects as `healthlog_app`, a role the entrypoint sets up
+after the migrations on every start. Nothing to configure: the first start of
+a release with this change creates it.
+
+The point is the blast radius of a bug. If a dependency of the app were ever
+made to run code inside the container, the database connection it found would
+be one that can read and write rows, not one that can drop tables, rewrite the
+schema or reach the server's files.
+
+## What the role may and may not do
+
+| May                                                                       | May not                                                         |
+| ------------------------------------------------------------------------- | --------------------------------------------------------------- |
+| read, insert, update and delete rows in every table of the app's schema   | create, alter or drop tables, indexes, types or functions there |
+| use and advance its sequences                                             | write `_prisma_migrations` (it may read it)                     |
+| `VACUUM`, `ANALYZE` and `REINDEX` the `measurements` table (`MAINTAIN`)   | create roles, schemas or databases                              |
+| own the `pgboss` schema, where the job queue keeps and changes its tables | act as a superuser: no `COPY … PROGRAM`, no `pg_read_file`      |
+| create temporary tables                                                   | become the owner (`SET ROLE`)                                   |
+
+The queue schema is the role's because pg-boss creates and migrates its own
+tables, functions and per-queue partitions while the app runs. Nothing of the
+app's data lives there. The start also takes `CREATE` on the app's schema
+away from `PUBLIC`, which Postgres 15 and newer already do for a new database
+but a database first created on an older major still allows.
+
+Its password is derived from `API_TOKEN_HMAC_KEY`, so a separate worker
+container ([scaling.md](scaling.md)) arrives at the same one without a new
+variable, and it is stored as a SCRAM hash. The owner's URL is replaced, not
+kept beside the role's, so it is not in the app's environment.
+`docker compose exec app …` still starts with the container's configured
+environment, so the operator scripts (`reset-password.mjs` and the others)
+run as the owner as before.
+
+Each start re-applies the grants, so a table the new migrations created is
+covered before the app starts, and a restored database is put right on the
+start after the restore.
+
+## What the start log says
+
+When it is in place:
+
+```text
+HealthLog: the app connects as healthlog_app: rows and the queue, no schema changes (DB_APP_ROLE=off turns this off)
+```
+
+When it is not, one notice names the reason, and the app runs as the
+`DATABASE_URL` user, exactly as every earlier release did:
+
+```text
+HealthLog: notice: the app connects as healthlog, the user DATABASE_URL names, not as the restricted role healthlog_app: <reason>. …
+```
+
+The reasons:
+
+- **The user may not create roles.** A managed database (RDS, Cloud SQL,
+  Supabase and the like) or a database server where you gave HealthLog an
+  account without `CREATEROLE`. Either leave it, or create the split yourself
+  (below) and set `DB_APP_ROLE=off`.
+- **Postgres older than 17.** The maintenance job's `VACUUM` and index rebuild
+  need the `MAINTAIN` privilege, which Postgres 17 introduced; before it only
+  the table's owner may run them. The bundled database is Postgres 18; see
+  [postgres-major-upgrade.md](../ops/postgres-major-upgrade.md) for one you
+  run yourself.
+- **The role cannot log in.** A `pg_hba.conf` that lists users by name needs a
+  line for `healthlog_app` beside the one for the owner.
+- **A role named `healthlog_app` exists that HealthLog did not create.** The
+  start leaves a role it does not recognise alone.
+- **Setting it up failed**, most often a lock it waited 15 seconds for: the
+  first start hands the existing queue tables to the role, and another
+  container still using them holds it up. The next start tries again.
+
+None of these stops the start.
+
+## Turning it off
+
+Set `DB_APP_ROLE=off` in `.env` and recreate the container. The app then
+connects as the `DATABASE_URL` user. Use it when you manage database roles
+yourself, including when `DATABASE_URL` already names a restricted role of
+your own; then that role needs what the table above lists, and the
+migrations need a user that owns the schema.
+
+The role stays in the database when you turn it off and does no harm. To
+remove it, as the owner:
+
+```sql
+REASSIGN OWNED BY healthlog_app TO healthlog;
+DROP OWNED BY healthlog_app;
+DROP ROLE healthlog_app;
+```
+
+## Index drops at start
+
+`DROP INDEX` belongs to a table's owner; `MAINTAIN` does not include it. The
+measurement maintenance therefore no longer drops a retired index or the
+invalid copy a failed rebuild leaves; it names them on its run and the next
+start drops them as the owner, waiting at most five seconds for each
+([measurement-maintenance.md](../ops/measurement-maintenance.md)).
+
+## Dumps and restores
+
+The nightly dump runs as the role and reads everything: the app's schema
+through its grants, the queue schema as its owner. A dump carries the grants
+to `healthlog_app`; `db-restore` leaves them and the ownership out
+(`--no-owner --no-privileges`), restores as the owner, and the next start
+grants again. A restore of your own with `pg_restore` should do the same, or
+it fails on a database without the role.

docs/self-hosting/scaling.md

@@ -129,6 +129,15 @@ looks healthy. Deploy both from the same image tag
 (`HEALTHLOG_IMAGE_REF` is shared by `app` and the `app-worker` block)
 and restart the worker together with the web containers.
 
+## Database role
+
+Both containers run the entrypoint, and both set up `healthlog_app`, the
+database role the app connects as
+([database-role.md](database-role.md)). They take turns on an advisory lock
+and derive the same password from `API_TOKEN_HMAC_KEY`, which the shared
+environment block gives both, so a worker that starts after the web container
+changes nothing the web container relies on.
+
 ## Healthchecks
 
 - `app` (web) healthcheck: `wget /api/health` every 30s.