tom/HealthLog
1
0 Fork 0
mirror of https://github.com/MBombeck/HealthLog.git synced 2026-11-10 05:12:14 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(environment): air quality, pollen and UV per day, locations sealed at rest

Browse Source 
Fetch the Open-Meteo air-quality feed (CAMS, domains=auto) with the weather:
17 hourly variables folded into local days by fixed rules (means and maxima
only with 18 of 24 hours, ozone as the highest 8-hour running mean, UV only
with the midday hours), a value the feed did not cover stored as null, never
as zero. Requests are cut into 90-day chunks, newest first, and chunks before
mid 2022 ask for the eight pollutant and index variables only. An air-quality
failure never costs the weather row; the day stays unfetched and the nightly
gap fill (up to 90 days per account, newest first) asks for it again.

Every Open-Meteo request asks an instance-wide budget kept in Postgres first
(500 a minute, 4000 an hour, 8000 a day, weighted by the hosted pricing
rule). A refused request is not sent and the run ends as done with
budget_blocked. OPENMETEO_AIR_QUALITY_URL and ENVIRONMENT_AIR_QUALITY_DISABLED
are honoured; the account switch lives on PATCH /api/environment/preferences.

The home, each dated location period and each stored day keep their coarse
location sealed as one value under healthlog/environment-location/v1. The
writers write the sealed value only, the readers open it and fall back to the
readable columns for rows the free-text encryption backfill has not reached,
and the backfill now seals all three and clears the readable columns. The
backup carries the location readable in a portable file and sealed in a
disaster-recovery file; a restore seals either under the receiving key.

Three correlation channels join the environment family (PM2.5, the ozone
8-hour high, the day's highest pollen count), read through the averaged
same-day window and left empty while air quality is off. The Coach and MCP
gain get_environment: module-gated, clamped to the lookback limit, no
coordinate or place name anywhere, with provenance and the attribution
lines. The settings section gets the switch, the coverage and the newest
day's values; the dashboard shows a quiet chip on a day with high pollen,
a warm night or very poor air. Copernicus and Open-Meteo are credited next
to the values.
This commit is contained in:
Marc-André Bombeck committed 2026-10-08 10:02:39 +02:00
1 parent aca497d9ac
commit 3c98d95a40
73 files changed
+6061 -385
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/api/openapi.yaml
+168 -5
View File
@@ -21901,9 +21901,10 @@ paths:
tags:
- Environment
summary: Change the environment switches
description: Turns the air-quality part of the module on or off for this account (on by default). Off means the nightly
fetch skips the air-quality feed and the overview leaves the fields out; days already stored are kept.
Module-gated. Strict body.
description: "Turns the air-quality part of the module on or off for this account (on by default). Off means the nightly
fetch skips the air-quality feed, and the overview, the Coach, MCP and the correlations leave the air-quality
values out; days already stored keep theirs. Turning it on queues a refresh, and the nightly gap fill catches up
on older days. Module-gated. Strict body: an unknown key is a 422."
requestBody:
required: true
content:
@@ -21921,7 +21922,6 @@ paths:
"403": *a20
"422": *a4
"429": *a5
"501": *a9
/api/environment:
get:
tags:
@@ -53899,6 +53899,7 @@ components:
- get_cycle
- get_correlations
- get_metric_table
- get_environment
- show_result
- snapshot
labelKey:
@@ -54011,9 +54012,10 @@ components:
- illness
- cycle
- correlations
- environment
- snapshot
description: "The data domain a step, table or chip is about: a measurement-backed scope source, or a domain read as a
whole (labs, illness, cycle, correlations, the full snapshot)."
whole (labs, illness, cycle, correlations, the environment, the full snapshot)."
CoachMethod:
type: object
properties:
@@ -54097,6 +54099,7 @@ components:
- get_cycle
- get_correlations
- get_metric_table
- get_environment
domain:
$ref: "#/components/schemas/CoachStepDomain"
window:
@@ -54835,6 +54838,7 @@ components:
- get_cycle
- get_correlations
- get_metric_table
- get_environment
domain:
$ref: "#/components/schemas/CoachStepDomain"
window:
@@ -79093,12 +79097,171 @@ components:
attribution:
type: string
description: Upstream attribution string. Display it wherever the observations are shown.
airQuality:
type: object
properties:
enabled:
type: boolean
description: The account's switch (`PATCH /api/environment/preferences`).
operatorDisabled:
type: boolean
description: True when the operator turned air quality off for the instance; the account switch then has no effect.
days:
type: integer
minimum: -9007199254740991
maximum: 9007199254740991
description: Stored days the air-quality feed returned hourly values for.
latestDate:
anyOf:
- type: string
- type: "null"
description: Newest such day (YYYY-MM-DD).
domain:
anyOf:
- type: string
- type: "null"
description: "The model domain of that day: `cams_europe` (about 11 km, with pollen) or `cams_global` (about 45 km, no
pollen)."
required:
- enabled
- operatorDisabled
- days
- latestDate
- domain
additionalProperties: false
description: v1.42 — the air-quality part of the module.
latestDay:
anyOf:
- type: object
properties:
date:
type: string
description: YYYY-MM-DD of the newest stored day.
tempMin:
anyOf:
- type: number
- type: "null"
tempMax:
anyOf:
- type: number
- type: "null"
apparentMax:
anyOf:
- type: number
- type: "null"
description: Daily feels-like maximum, °C.
airQuality:
anyOf:
- type: object
properties:
pm25Mean:
anyOf:
- type: number
- type: "null"
description: PM2.5 daily mean, µg/m³.
pm10Mean:
anyOf:
- type: number
- type: "null"
description: PM10 daily mean, µg/m³.
no2Mean:
anyOf:
- type: number
- type: "null"
description: NO2 daily mean, µg/m³.
o3Max8h:
anyOf:
- type: number
- type: "null"
description: Ozone, highest 8-hour running mean, µg/m³.
eaqiMax:
anyOf:
- type: number
- type: "null"
description: European air-quality index, daily maximum.
uvIndexMax:
anyOf:
- type: number
- type: "null"
dustMax:
anyOf:
- type: number
- type: "null"
description: Dust, µg/m³.
pollen:
type: object
properties:
alder:
anyOf:
- type: number
- type: "null"
birch:
anyOf:
- type: number
- type: "null"
grass:
anyOf:
- type: number
- type: "null"
mugwort:
anyOf:
- type: number
- type: "null"
olive:
anyOf:
- type: number
- type: "null"
ragweed:
anyOf:
- type: number
- type: "null"
required:
- alder
- birch
- grass
- mugwort
- olive
- ragweed
additionalProperties: false
description: Daily maxima, grains/m³; null where not covered (outside Europe, before mid 2022).
required:
- pm25Mean
- pm10Mean
- no2Mean
- o3Max8h
- eaqiMax
- uvIndexMax
- dustMax
- pollen
additionalProperties: false
- type: "null"
description: Null while air quality is off for the account or the instance, and for a day whose air-quality part was not
fetched yet. Null values inside mean not covered, never zero.
required:
- date
- tempMin
- tempMax
- apparentMax
- airQuality
additionalProperties: false
- type: "null"
description: v1.42 — the newest stored day, usually yesterday or the day before (the archive settles late). No forecast.
Null when nothing is stored.
attributions:
type: array
items:
type: string
description: v1.42 — every attribution line, weather first; the air-quality lines (Open-Meteo, Copernicus) while air
quality is on. Display them wherever the values are shown.
required:
- home
- travel
- context
- lastFetchFailure
- attribution
- airQuality
- latestDay
- attributions
additionalProperties: false
description: "The module overview: home, travel overrides, a count of stored observations, and whether the background
fetch is healthy."
docs/self-hosting/environment.md
+110
View File
@@ -0,0 +1,110 @@
# Weather, air quality and pollen
The environment module stores, for each day, the weather and the outdoor air
at a coarse location: temperature, sunshine, daylight, rain, pressure,
humidity, and since v1.42 fine particles, ozone, nitrogen dioxide, the
European and US air-quality indices, UV, Saharan dust and six kinds of
pollen. Those days sit beside the person's own data: the correlations look
for patterns between them and sleep, heart-rate variability, blood pressure,
mood and symptoms, the Coach and connected MCP clients can read them, and the
dashboard shows a quiet note on a day with high pollen, a warm night or very
poor air.
The module is on by default. Nothing leaves the host until a person sets a
home location in Settings → Location & weather.
## What leaves the host
Three hosted Open-Meteo APIs, all keyless:
| Host | What for | Variable to self-host it |
| -------------------------------- | ------------------------------------ | --------------------------- |
| `archive-api.open-meteo.com` | daily weather (ERA5 reanalysis) | `OPENMETEO_BASE_URL` |
| `air-quality-api.open-meteo.com` | hourly air quality and pollen (CAMS) | `OPENMETEO_AIR_QUALITY_URL` |
| `geocoding-api.open-meteo.com` | the city search in settings | `OPENMETEO_GEOCODING_URL` |
A request carries the coordinates rounded to one decimal (about 11 km, the
size of a town), a date range and a timezone. No account id, no name, no
health value. Open-Meteo is open source; an operator who wants no egress at
all can run it and point the three variables at their own instance (a host
on the LAN is allowed).
## The data, and what it is not
- Weather comes from the ERA5 reanalysis on a grid of 9 to 25 kilometres. It
settles over a few days, so the newest stored day is usually yesterday or
the day before. Nothing here is a forecast.
- Air quality comes from the CAMS models: hourly on about 11 kilometres in
Europe, about 45 kilometres elsewhere. The feed starts in 2013; pollen, UV,
dust and aerosol depth only from about mid 2022, and pollen only in Europe.
A value the feed did not cover is stored as empty, never as zero.
- Hourly values become a day by fixed rules: means and maxima only for a day
with at least 18 of 24 hours, ozone as the highest 8-hour running mean,
UV only when the midday hours are all there.
- These are modelled outdoor conditions for an area, not anyone's personal
exposure. Every surface that shows them says so, and none of them says the
weather caused anything: a pattern is described as something that occurred
together.
## Switches
- **Per account:** Settings → Location & weather → Air quality and pollen.
On by default. Off stops the air-quality requests for that account and
leaves the air-quality values out of every surface; the days already
stored keep theirs.
- **For the whole instance:** `ENVIRONMENT_AIR_QUALITY_DISABLED=1` turns the
air-quality part off for every account (`1`, `true`, `yes` or `on`). The
weather keeps working. Settings then says the operator turned it off.
- The whole module is a normal module switch (Settings → Modules), and an
operator can make it unavailable like any other module.
All four variables are on the compose `environment:` whitelist.
## Request budget
The hosted APIs are free for non-commercial use up to 600 calls a minute,
5,000 an hour and 10,000 a day per instance, where a request for many
variables or many days counts as several calls. HealthLog keeps its own
counter in Postgres, shared by the web process and the worker, and stays
below 500 a minute, 4,000 an hour and 8,000 a day. A request the counter
refuses is not sent; the nightly run ends early and continues the next night.
For scale: a full two-year backfill of one account is about 150 calls, the
nightly refresh about 3 per account.
## Nightly fetch and catching up
The `environment-fetch` job runs at 02:10. For each account with a home it
refreshes the last seven days (weather and air quality together), then fills
up to 90 stored days that do not have air quality yet, newest first. An
instance that upgrades to v1.42 with two years of weather per account
catches up on air quality in about nine nights without any action. The
backfill button in settings fetches both for the range it is given.
## Locations are encrypted
Since v1.42 the home, each dated location period and each stored day keep
their coarse location sealed with the instance's encryption key (one value
per row, `{ lat, lon, label }`, under the label
`healthlog/environment-location/v1`). The boot-time encryption backfill seals
rows written by older releases and clears their readable columns; the
readable columns are dropped in a later release once the backfill reports
zero rows everywhere. Key rotation (`scripts/rotate-encryption-key.ts`, or the
in-app rotation) covers all three.
What the person sees is unchanged: the settings page and
`GET /api/environment` answer with the readable location, opened on the
server. The Coach and MCP read (`get_environment`) carry no coordinate and no
place name at all, only how many days were spent away from home.
Backups follow the usual split: a portable export carries the location
readable, a disaster-recovery backup carries it sealed, and a restore of
either seals it under the receiving host's key.
## Attribution
Open-Meteo data is CC BY 4.0, and CAMS data asks for the notice "Contains
modified Copernicus Atmosphere Monitoring Service information" (modified,
because the hourly values are folded into days). The settings page lists
both lines under the values, and the Coach and MCP read return them with
every result.