mirror of
https://github.com/MBombeck/HealthLog.git
synced 2026-11-07 04:12:04 +00:00
docs: describe the Postgres 18 move and the versions HealthLog runs on
This commit is contained in:
Marc-André Bombeck committed
1 parent
d4d91994e8
commit
27c8533f40
13 files changed
+140
-73
No files matched your search
@@ -87,7 +87,7 @@ curl -s http://localhost:3000/api/version # returns version + buildSha + built
|
||||
|
||||
`SESSION_COOKIE_SECURE=false` is the one setting plain-HTTP self-hosts must set: without it the session cookie carries the `Secure` flag, the browser drops it over HTTP, and login silently fails. Leave it unset (or `true`) only when a TLS reverse proxy serves HTTPS — and then set `NEXT_PUBLIC_APP_URL` and `APP_URL` to the public URL first (see the [self-hosting guide](https://docs.healthlog.dev/self-hosting/)).
|
||||
|
||||
Persistent state lives in exactly one named volume (`pgdata`, the Postgres data dir); the app container is stateless, so backing up the database backs up everything.
|
||||
Persistent state lives in exactly one named volume (`postgres-data`, the Postgres data); the app container is stateless, so backing up the database backs up everything. An install from v1.45 or earlier moves its Postgres 16 database into it on the first start ([details](docs/ops/postgres-major-upgrade.md)).
|
||||
|
||||
NAS and homelab users have ready-made paths: [Unraid Community Applications](docs/self-hosting/unraid.md), [Portainer app templates](docs/self-hosting/portainer/README.md) and [TrueNAS](docs/self-hosting/truenas.md) (the Community train app is pending catalog review).
|
||||
|
||||
@@ -105,7 +105,7 @@ The App Store listing carries a small price, which exists to cover Apple's yearl
|
||||
|
||||
## Tech stack
|
||||
|
||||
Next.js 16 (App Router, React Server Components), TypeScript strict, PostgreSQL 16 with Prisma, pg-boss for jobs, Tailwind CSS 4 + shadcn/ui, Recharts, WebAuthn passkeys, Vitest + Playwright, multi-stage Alpine Docker image. The OpenAPI 3.1 contract for native clients is generated from the Zod schemas at [`docs/api/openapi.yaml`](docs/api/openapi.yaml).
|
||||
Next.js 16 (App Router, React Server Components), TypeScript strict, PostgreSQL 18 with Prisma, pg-boss for jobs, Tailwind CSS 4 + shadcn/ui, Recharts, WebAuthn passkeys, Vitest + Playwright, multi-stage Alpine Docker image. The OpenAPI 3.1 contract for native clients is generated from the Zod schemas at [`docs/api/openapi.yaml`](docs/api/openapi.yaml).
|
||||
|
||||
## Documentation
|
||||
|
||||
|
||||
@@ -81,7 +81,7 @@
|
||||
<text x="600" y="360" text-anchor="middle" fill="#6272a4" font-size="11">pre-aggregated buckets</text>
|
||||
|
||||
<rect x="510" y="392" width="180" height="56" rx="10" fill="#44475a" stroke="#ff79c6" stroke-width="1.5" />
|
||||
<text x="600" y="417" text-anchor="middle" font-weight="600">PostgreSQL 16</text>
|
||||
<text x="600" y="417" text-anchor="middle" font-weight="600">PostgreSQL 18</text>
|
||||
<text x="600" y="435" text-anchor="middle" fill="#bd93f9" font-size="11">encrypted at rest</text>
|
||||
</g>
|
||||
|
||||
|
||||
|
Before Width: | Height: | Size: 11 KiB After Width: | Height: | Size: 11 KiB |
@@ -1,6 +1,6 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1200 620" font-family="ui-sans-serif, -apple-system, Segoe UI, Roboto, sans-serif" role="img" aria-labelledby="title desc">
|
||||
<title id="title">HealthLog self-hosting topology</title>
|
||||
<desc id="desc">Internet traffic terminates at a reverse proxy (Caddy, Traefik or Nginx) for TLS, then reaches the Next.js app on port 3000. The app shares a Postgres 16 database with the pg-boss worker process for reminders, insights, backups and the Apple Health import. GHCR ships the multi-arch image; Coolify watches the digest and pulls; off-host S3-compatible storage is optional for backups.</desc>
|
||||
<desc id="desc">Internet traffic terminates at a reverse proxy (Caddy, Traefik or Nginx) for TLS, then reaches the Next.js app on port 3000. The app shares a Postgres 18 database with the pg-boss worker process for reminders, insights, backups and the Apple Health import. GHCR ships the multi-arch image; Coolify watches the digest and pulls; off-host S3-compatible storage is optional for backups.</desc>
|
||||
|
||||
<rect x="0" y="0" width="1200" height="620" fill="#282a36" />
|
||||
|
||||
@@ -47,7 +47,7 @@
|
||||
<!-- Postgres (green) -->
|
||||
<g>
|
||||
<rect x="760" y="250" width="200" height="80" rx="10" fill="#44475a" stroke="#50fa7b" stroke-width="1.5" />
|
||||
<text x="860" y="276" text-anchor="middle" fill="#f8f8f2" font-size="14" font-weight="600">PostgreSQL 16</text>
|
||||
<text x="860" y="276" text-anchor="middle" fill="#f8f8f2" font-size="14" font-weight="600">PostgreSQL 18</text>
|
||||
<text x="860" y="296" text-anchor="middle" fill="#bd93f9" font-size="11">26 Prisma models</text>
|
||||
<text x="860" y="312" text-anchor="middle" fill="#6272a4" font-size="11">+ pg-boss queues</text>
|
||||
</g>
|
||||
|
||||
|
Before Width: | Height: | Size: 8.1 KiB After Width: | Height: | Size: 8.1 KiB |
@@ -10,7 +10,7 @@ copy.
|
||||
| ------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `01-data-flow.svg` | Sources (Withings, Apple Health export.zip, iPhone HealthKit, manual, Nightscout) → ingest endpoints → Measurement + MeasurementRollup → reads (rollup probe / live SQL fallback / cache) → surfaces (dashboard, insights, Coach, doctor PDF). |
|
||||
| `02-coach-pipeline.svg` | User question → snapshot builder → prompt assembler → provider chain (ChatGPT OAuth / OpenAI / Anthropic / local / admin-shared) → response parser → cited reply with mini-charts. |
|
||||
| `03-self-hosting-topology.svg` | Internet → reverse proxy (TLS) → Next.js app (port 3000) ↔ pg-boss worker ↔ Postgres 16, plus GHCR image pull, optional Coolify autodeploy, optional S3 off-host backup. |
|
||||
| `03-self-hosting-topology.svg` | Internet → reverse proxy (TLS) → Next.js app (port 3000) ↔ pg-boss worker ↔ Postgres 18, plus GHCR image pull, optional Coolify autodeploy, optional S3 off-host backup. |
|
||||
| `04-source-priority.svg` | Three sources for the same metric on the same day → resolver picks one canonical row per (cumulative vs point) ladder. Losing rows stay in the audit trail. |
|
||||
| `05-security-model.svg` | Three concentric layers — auth perimeter (passkey + password fallback), session perimeter (HttpOnly cookie + Postgres sessions), encrypted core (AES-256-GCM versioned keys). Side rails: rate limiter, audit log, HMAC tokens, CSP/HSTS, SSRF guard. |
|
||||
|
||||
|
||||
@@ -129,6 +129,26 @@ An INVALID index whose migration is already recorded applied is not touched at
|
||||
boot. The boot names it on stderr with `WARNING — index … is INVALID`; rebuild
|
||||
it with `REINDEX INDEX CONCURRENTLY "<index>";` when the database is quiet.
|
||||
|
||||
### v1.46.0 notes
|
||||
|
||||
- **Postgres 18.** The compose file moves the bundled database from
|
||||
Postgres 16 to 18. The first `docker compose up -d` with the new compose
|
||||
file runs `db-upgrade`, which copies the database into a new
|
||||
`postgres-data` volume, checks every table's row count, and only then lets
|
||||
the app start; the old `pgdata` volume stays as it was, as the way back.
|
||||
Pulling only the new image with the old compose file keeps 16, which this
|
||||
release still runs on. Details, rollback and the steps for a database you
|
||||
run yourself: [postgres-major-upgrade.md](postgres-major-upgrade.md).
|
||||
- Migration `0485` drops `preview_features_json` from `users`, the same
|
||||
pattern as `0430` in v1.44.0 below: do not run the previous image beside
|
||||
the new schema. The sequence above stops the old container first; a
|
||||
web/worker split stops both roles before the first new container starts.
|
||||
- The other migrations add columns and one index (built concurrently).
|
||||
Three touch rows: `0476` links API-written measurements to their token on
|
||||
accounts that have exactly one measurement token, and `0478`/`0479` give
|
||||
PDFs whose preview or text read failed one more attempt. On a prod-sized
|
||||
copy all of them together finished in under a second.
|
||||
|
||||
### v1.45.0 notes
|
||||
|
||||
A single-container install (`HEALTHLOG_PROCESS_TYPE=all`, the shipped
|
||||
|
||||
@@ -1,32 +1,97 @@
|
||||
# Moving the database to a newer Postgres major
|
||||
|
||||
The compose file ships Postgres 16 and stays there for now, because a major
|
||||
version change is the one database update that cannot happen by itself on
|
||||
`docker compose pull`. HealthLog runs on Postgres 18 as well: the full
|
||||
integration suite runs against 18 (`INTEGRATION_PG_IMAGE=postgres:18-alpine
|
||||
pnpm test:integration`), and the TrueNAS app starts new installs on 18.
|
||||
From v1.46 the compose file runs Postgres 18. Installs from v1.45 or earlier
|
||||
ran 16, and a major version cannot open the data directory of an older one,
|
||||
so the compose file carries a one-time step that moves the database across
|
||||
on the first start after the update. Nothing needs doing by hand on a stack
|
||||
that runs the project's `docker-compose.yml`.
|
||||
|
||||
## Why changing the image line is not enough
|
||||
Installs that bring their own Postgres (Unraid, an existing database server)
|
||||
are not touched: HealthLog runs on 16 and 18 alike, and the move there is
|
||||
yours to schedule with the steps under [By hand](#by-hand).
|
||||
|
||||
Minor releases (16.14 to 16.15) share the on-disk format. Majors do not, and
|
||||
the official image refuses to start a data directory written by an older
|
||||
major. From 18 on, the image also keeps its data one level deeper, under
|
||||
`/var/lib/postgresql/18/docker`, and expects the volume mounted at
|
||||
`/var/lib/postgresql` instead of `/var/lib/postgresql/data`. With the old
|
||||
volume and only the image line changed, the `db` container exits at once with
|
||||
"there appears to be PostgreSQL data in: /var/lib/postgresql/data". Nothing is
|
||||
lost, but the stack stays down until the image line goes back to 16.
|
||||
## What happens on the first start
|
||||
|
||||
So the move is a dump from 16 and a restore into a fresh 18 volume.
|
||||
`db-upgrade` is a short-lived container next to `db`. It starts after the new
|
||||
Postgres 18 database is up and before the app, and it exits in a second when
|
||||
there is nothing to move. When the old `pgdata` volume holds a Postgres 16
|
||||
database and the new one is empty, it:
|
||||
|
||||
## Steps
|
||||
1. checks there is room on the disk for a second copy;
|
||||
2. starts Postgres 16 inside its own container on the old volume, reachable
|
||||
only from inside that container;
|
||||
3. streams a dump into Postgres 18 in a single transaction;
|
||||
4. compares the row count of every table between the two;
|
||||
5. gathers the planner statistics, which a restore does not carry;
|
||||
6. stops Postgres 16 and leaves a marker file on the old volume.
|
||||
|
||||
Take the encryption key's copy out of its safe place first: the dump is
|
||||
unreadable without it, and this is the moment you would find out.
|
||||
The app starts once that succeeded, applies the release's migrations and
|
||||
serves as before. On a typical personal record the whole move takes seconds;
|
||||
a database of a few gigabytes takes a few minutes. Watch it with
|
||||
`docker compose logs -f db-upgrade`.
|
||||
|
||||
If any step fails, the new database stays empty, the app does not start, and
|
||||
the log says what happened. A move cut short by a restart or a full disk is
|
||||
run again from scratch on the next `docker compose up -d`.
|
||||
|
||||
The old `pgdata` volume is read and never changed, apart from the marker
|
||||
file. That is the way back.
|
||||
|
||||
## Going back
|
||||
|
||||
Stop the stack, put the two lines of the `db` service back the way v1.45 had
|
||||
them, and start it with the v1.45 image:
|
||||
|
||||
```yaml
|
||||
db:
|
||||
image: postgres:16.15-alpine
|
||||
volumes:
|
||||
- pgdata:/var/lib/postgresql/data
|
||||
```
|
||||
|
||||
```bash
|
||||
# 1. Stop the app so nothing writes while the dump runs. If you run the
|
||||
# worker role in its own container, stop it too: docker compose stop app app-worker
|
||||
docker compose down
|
||||
HEALTHLOG_IMAGE_REF=:v1.45.0 docker compose up -d
|
||||
```
|
||||
|
||||
Anything written after the move lives only in the Postgres 18 volume.
|
||||
|
||||
## Afterwards
|
||||
|
||||
Keep the old volume until the new database has run for a while. When you are
|
||||
sure, remove it (`docker volume ls` shows its full name, prefixed with the
|
||||
compose project):
|
||||
|
||||
```bash
|
||||
docker volume rm <project>_pgdata
|
||||
```
|
||||
|
||||
`db-upgrade` then finds nothing to move. A later release drops the service
|
||||
and the `pgdata` line.
|
||||
|
||||
## Starting over
|
||||
|
||||
When the move has run and the Postgres 18 volume is empty again (deleted by
|
||||
hand, say), `db-upgrade` refuses to import the old state a second time, since
|
||||
that would quietly bring back the database as it was on the day of the move.
|
||||
To do exactly that on purpose, remove the marker and start again:
|
||||
|
||||
```bash
|
||||
docker compose run --rm --entrypoint rm db-upgrade \
|
||||
/var/lib/postgresql/data/HEALTHLOG_MOVED_TO_POSTGRES_18
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
## By hand
|
||||
|
||||
For a Postgres you run yourself, or a compose file you have changed so much
|
||||
that the step does not fit, the move is a dump and a restore. Take the
|
||||
encryption key's copy out of its safe place first: the dump is unreadable
|
||||
without it, and this is the moment you would find out.
|
||||
|
||||
```bash
|
||||
# 1. Stop the app so nothing writes while the dump runs (and app-worker, if
|
||||
# you run the worker role in its own container).
|
||||
docker compose stop app
|
||||
|
||||
# 2. Dump the database in Postgres' custom format.
|
||||
@@ -37,56 +102,31 @@ docker compose exec -T db pg_dump -U healthlog -Fc healthlog \
|
||||
docker compose stop db
|
||||
```
|
||||
|
||||
4. In `docker-compose.yml`, change the `db` service:
|
||||
|
||||
```yaml
|
||||
image: postgres:18.6-alpine # or the current 18 minor
|
||||
volumes:
|
||||
- pgdata18:/var/lib/postgresql
|
||||
```
|
||||
|
||||
and add the new volume next to `pgdata` in the top-level `volumes:` block:
|
||||
|
||||
```yaml
|
||||
volumes:
|
||||
pgdata:
|
||||
pgdata18:
|
||||
```
|
||||
4. Point the `db` service at Postgres 18 and a fresh volume. From 18 on, the
|
||||
image keeps its data under `/var/lib/postgresql/18/docker` and expects the
|
||||
volume mounted one level up, at `/var/lib/postgresql`; mounted at
|
||||
`.../data` it refuses to start with "there appears to be PostgreSQL data
|
||||
in: /var/lib/postgresql/data".
|
||||
|
||||
```bash
|
||||
# 5. Start the new database and wait until it reports healthy.
|
||||
# 5. Start the new database and restore the dump into it.
|
||||
docker compose up -d db
|
||||
docker compose ps db
|
||||
|
||||
# 6. Restore the dump into it.
|
||||
docker compose exec -T db pg_restore -U healthlog -d healthlog --no-owner \
|
||||
< healthlog-pg16-<date>.dump
|
||||
|
||||
# 7. Gather the planner statistics. A restore brings the rows but not the
|
||||
# 6. Gather the planner statistics. A restore brings the rows but not the
|
||||
# statistics, and until autovacuum gets round to every table the planner
|
||||
# guesses at their sizes, which can turn a sub-second page into one that
|
||||
# takes many seconds.
|
||||
docker compose exec -T db vacuumdb -U healthlog -d healthlog --analyze-in-stages
|
||||
|
||||
# 8. Start the app (and app-worker, if you run it) and confirm it.
|
||||
# 7. Start the app and confirm it.
|
||||
docker compose up -d app
|
||||
curl -s http://localhost:3000/api/version
|
||||
```
|
||||
|
||||
The start log should say there are no pending migrations: the dump carried the
|
||||
migration history with it. Sign in and open a few pages before you go on.
|
||||
|
||||
## Afterwards
|
||||
|
||||
Keep the old `pgdata` volume and the dump until the new database has run for a
|
||||
few days. Going back means restoring the old `image:` and `volumes:` lines;
|
||||
the old volume still holds the database as it was before step 2. When you are
|
||||
sure, remove it (`docker volume ls` shows its full name, prefixed with the
|
||||
compose project):
|
||||
|
||||
```bash
|
||||
docker volume rm <project>_pgdata
|
||||
```
|
||||
The start log should say there are no pending migrations beyond the release's
|
||||
own: the dump carried the migration history with it.
|
||||
|
||||
If you had run `CREATE EXTENSION pg_stat_statements;` for
|
||||
[query statistics](query-statistics.md), the dump carries it and the compose
|
||||
|
||||
@@ -81,7 +81,9 @@ explicitly in `.env` (see the commented example there).
|
||||
|
||||
### Using an existing or shared Postgres
|
||||
|
||||
HealthLog can live in a database on a Postgres instance you already run.
|
||||
HealthLog can live in a database on a Postgres instance you already run,
|
||||
version 16 or newer; the compose file ships 18, and every release is tested
|
||||
on both.
|
||||
It stores every time as UTC and pins each connection it opens to UTC
|
||||
(`TimeZone=UTC`, sent with the libpq `options` startup parameter), so the
|
||||
server's or the database's own `timezone` setting does not matter: an
|
||||
@@ -105,9 +107,12 @@ ALTER DATABASE healthlog SET timezone TO 'UTC';
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
Compose starts two services:
|
||||
Compose starts two services and one short-lived step:
|
||||
|
||||
- `db` — PostgreSQL 16 with a named volume for persistence.
|
||||
- `db` — PostgreSQL 18 with a named volume for persistence.
|
||||
- `db-upgrade` — moves a Postgres 16 database from v1.45 or earlier into
|
||||
the new one, once, and exits; on a new install it has nothing to do. See
|
||||
[Moving to a newer Postgres major](../ops/postgres-major-upgrade.md).
|
||||
- `app` — HealthLog itself, listening on `localhost:3000`. The image
|
||||
runs `prisma migrate deploy` on boot so the schema lands
|
||||
automatically; subsequent restarts no-op when the database is
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# HealthLog on Portainer
|
||||
|
||||
HealthLog ships a Portainer **app template** that deploys the app and its
|
||||
PostgreSQL 16 database as one stack from the project `docker-compose.yml`.
|
||||
PostgreSQL 18 database as one stack from the project `docker-compose.yml`.
|
||||
|
||||
## Add the template
|
||||
|
||||
@@ -39,7 +39,7 @@ curl -s http://<host>:3000/api/version
|
||||
|
||||
## Notes
|
||||
|
||||
- Persistent state lives in one named volume (`pgdata`, the Postgres data dir);
|
||||
- Persistent state lives in one named volume (`postgres-data`, the Postgres data);
|
||||
the app container is stateless. Back up the database to back up everything.
|
||||
- Pin a release tag (`HEALTHLOG_IMAGE_REF=:vX.Y.Z`) for deliberate upgrades, or
|
||||
track `:latest`. Skim the
|
||||
|
||||
@@ -174,7 +174,7 @@ sized for the web traffic alone leaves the worker's jobs queueing for
|
||||
the same connections.
|
||||
|
||||
Plan total Postgres slots as **container_count × budget**.
|
||||
A stock Postgres 16 container ships with `max_connections = 100`, so
|
||||
A stock Postgres container ships with `max_connections = 100`, so
|
||||
the safe envelope is:
|
||||
|
||||
| Web replicas | Worker replicas | Total containers | Pool slots used | Headroom under 100 |
|
||||
|
||||
@@ -213,7 +213,7 @@ the menu next to Custom App, Install via YAML, using these values:
|
||||
| User | runs as uid `1001` by default; no extra capabilities, `no-new-privileges` is fine |
|
||||
| Health check | `GET /api/health`, `200` when ready |
|
||||
| Start period | at least 60 s, with more retries than the image's 5 × 30 s: migrations run at every boot |
|
||||
| Database | PostgreSQL 16 or 18, no extensions, in its own container with its own dataset |
|
||||
| Database | PostgreSQL 18 (16 works too), no extensions, in its own container with its own dataset |
|
||||
| Volumes | none for the app; the database gets its own dataset |
|
||||
|
||||
| Variable | Required | Notes |
|
||||
|
||||
@@ -69,10 +69,12 @@ curl -s http://tower:3000/api/version
|
||||
Use this if you prefer the per-app Unraid UI. You install PostgreSQL yourself,
|
||||
then add HealthLog pointed at it.
|
||||
|
||||
### Step 1 — install PostgreSQL 16
|
||||
### Step 1 — install PostgreSQL
|
||||
|
||||
1. In **Apps** (Community Applications), search for a **PostgreSQL** container
|
||||
(for example jj9987's `postgresql`). Install version 16.
|
||||
(for example jj9987's `postgresql`). Install version 18; an existing 16
|
||||
works too. Moving a 16 to 18 later is a dump and restore, see
|
||||
[Moving to a newer Postgres major](../ops/postgres-major-upgrade.md#by-hand).
|
||||
2. Set:
|
||||
- `POSTGRES_USER` = `healthlog`
|
||||
- `POSTGRES_PASSWORD` = your generated password
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
<!--
|
||||
HealthLog — Unraid Community Applications template.
|
||||
|
||||
CA templates describe ONE container. HealthLog needs PostgreSQL 16, so
|
||||
CA templates describe ONE container. HealthLog needs PostgreSQL (18, or 16), so
|
||||
install a PostgreSQL container FIRST (CA has community PostgreSQL templates),
|
||||
create a `healthlog` database + user, then point DATABASE_URL below at it.
|
||||
Full walkthrough: docs/self-hosting/unraid.md.
|
||||
@@ -26,7 +26,7 @@
|
||||
cycle, and medications on one encrypted timeline. AES-256-GCM at rest, no
|
||||
vendor cloud, no telemetry.
|
||||
|
||||
REQUIRES a separate PostgreSQL 16 container — install one first (CA has
|
||||
REQUIRES a separate PostgreSQL 18 (or 16) container — install one first (CA has
|
||||
community PostgreSQL templates), create a database and user, then set
|
||||
DATABASE_URL below. Step-by-step guide:
|
||||
https://github.com/MBombeck/HealthLog/blob/main/docs/self-hosting/unraid.md
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
# `prisma/`
|
||||
|
||||
The database layer. HealthLog runs PostgreSQL 16 via Prisma 7. This directory holds the single schema file and the forward-only migration history; the generated client is emitted elsewhere.
|
||||
The database layer. HealthLog runs on PostgreSQL 18 (the compose default) or 16 via Prisma 7. This directory holds the single schema file and the forward-only migration history; the generated client is emitted elsewhere.
|
||||
|
||||
## Contents
|
||||
|
||||
|
||||
Reference in new issue
Block a user