Moving the database to a newer Postgres major

From v1.46 the compose file runs Postgres 18. Installs from v1.45 or earlier ran 16, and a major version cannot open the data directory of an older one, so the compose file carries a one-time step that moves the database across on the first start after the update. Nothing needs doing by hand on a stack that runs the project's docker-compose.yml.

Installs that bring their own Postgres (Unraid, an existing database server) are not touched: HealthLog runs on 16 and 18 alike, and the move there is yours to schedule with the steps under By hand.

What happens on the first start

db-upgrade is a short-lived container next to db. It starts after the new Postgres 18 database is up and before the app, and it exits in a second when there is nothing to move. When the old pgdata volume holds a Postgres 16 database and the new one is empty, it:

checks there is room on the disk for a second copy;
starts Postgres 16 inside its own container on the old volume, reachable only from inside that container;
streams a dump into Postgres 18 in a single transaction;
compares the row count of every table between the two;
gathers the planner statistics, which a restore does not carry;
stops Postgres 16 and leaves a marker file on the old volume.

The app starts once that succeeded, applies the release's migrations and serves as before. On a typical personal record the whole move takes seconds; a database of a few gigabytes takes a few minutes. Watch it with docker compose logs -f db-upgrade.

If any step fails, the new database stays empty, the app does not start, and the log says what happened. A move cut short by a restart or a full disk is run again from scratch on the next docker compose up -d.

The old pgdata volume is read and never changed, apart from the marker file. That is the way back.

Going back

Stop the stack, put the two lines of the db service back the way v1.45 had them, and start it with the v1.45 image:

db:
  image: postgres:16.15-alpine
  volumes:
    - pgdata:/var/lib/postgresql/data

docker compose down
HEALTHLOG_IMAGE_REF=:v1.45.0 docker compose up -d

Anything written after the move lives only in the Postgres 18 volume.

Afterwards

Keep the old volume until the new database has run for a while. When you are sure, remove it (docker volume ls shows its full name, prefixed with the compose project):

docker volume rm <project>_pgdata

db-upgrade then finds nothing to move. A later release drops the service and the pgdata line.

Starting over

When the move has run and the Postgres 18 volume is empty again (deleted by hand, say), db-upgrade refuses to import the old state a second time, since that would quietly bring back the database as it was on the day of the move. To do exactly that on purpose, remove the marker and start again:

docker compose run --rm --entrypoint rm db-upgrade \
  /var/lib/postgresql/data/HEALTHLOG_MOVED_TO_POSTGRES_18
docker compose up -d

By hand

For a Postgres you run yourself, or a compose file you have changed so much that the step does not fit, the move is a dump and a restore. Take the encryption key's copy out of its safe place first: the dump is unreadable without it, and this is the moment you would find out.

# 1. Stop the app so nothing writes while the dump runs (and app-worker, if
#    you run the worker role in its own container).
docker compose stop app

# 2. Dump the database in Postgres' custom format.
docker compose exec -T db pg_dump -U healthlog -Fc healthlog \
  > healthlog-pg16-$(date +%F).dump

# 3. Stop the old database. Its volume stays untouched.
docker compose stop db

Point the db service at Postgres 18 and a fresh volume. From 18 on, the image keeps its data under /var/lib/postgresql/18/docker and expects the volume mounted one level up, at /var/lib/postgresql; mounted at .../data it refuses to start with "there appears to be PostgreSQL data in: /var/lib/postgresql/data".

# 5. Start the new database and restore the dump into it.
docker compose up -d db
docker compose exec -T db pg_restore -U healthlog -d healthlog --no-owner \
  < healthlog-pg16-<date>.dump

# 6. Gather the planner statistics. A restore brings the rows but not the
#    statistics, and until autovacuum gets round to every table the planner
#    guesses at their sizes, which can turn a sub-second page into one that
#    takes many seconds.
docker compose exec -T db vacuumdb -U healthlog -d healthlog --analyze-in-stages

# 7. Start the app and confirm it.
docker compose up -d app
curl -s http://localhost:3000/api/version

The start log should say there are no pending migrations beyond the release's own: the dump carried the migration history with it.

If you had run CREATE EXTENSION pg_stat_statements; for query statistics, the dump carries it and the compose command: still loads the library, so nothing needs doing there.

pg_upgrade --link is faster on a very large database, but it needs both majors' binaries in one container; for a personal health record the dump is seconds to minutes and has nothing to set up.

5.1 KiB Raw Blame History