A Nightscout self-host on host networking with
NIGHTSCOUT_PRIVATE_ORIGINS=http://localhost:1337 worked before and must
keep working unchanged; an operator who lists an exact loopback origin
with its port has made the same deliberate decision as one who lists a
LAN origin. The never-grantable floor is now the unspecified address,
link-local and the metadata endpoint only. Loopback (127/8, ::1,
localhost) is grantable when listed exactly, still resolved and pinned
at connect time with redirects forbidden, and still refused when not
listed.
::1 is answered before the embedded-IPv4 unwrap, which read it as the
IPv4-compatible spelling of 0.0.0.1 and refused it as unspecified. The
pinned Nightscout transport test runs the real floor again. Each refused
range keeps its own test; loopback moves to the grantable-when-listed
set with a test that an unlisted loopback is still refused, and a
Nightscout grant of http://localhost:1337 is asserted to parse.
Refs #947
Lift the Nightscout allowlist parser and verdict into
src/lib/private-origin-policy.ts so a second operator grant can reuse
one grammar instead of a second parser. The Nightscout validator keeps
its exported names and behaviour; its tests are unchanged.
Add the floor both precedents lacked: an entry whose host is a literal
address in 127/8, 0/8, 169.254/16, ::, ::1 or fe80::/10 is not a grant,
and the operator-approved pinned dispatcher drops an answer in those
ranges even when the name is listed. RFC1918, CGNAT and IPv6 unique-
local stay grantable; that is what the list is for. Loopback inside the
container is the app itself, and on host networking every admin port of
the host; link-local is the cloud-metadata endpoint. No relay lives on
either.
The pinned Nightscout transport test resolves its listed name to a
loopback server on purpose, so it stubs the grantable-address predicate
for that harness only.
Refs #947
Integration syncs (Fitbit, Oura, Polar, WHOOP, Strava, Nightscout) now
mark the per-user analytics cells stale and sweep the workouts cache at
their sync tails — imported rows used to stay invisible to the cached
dashboard, analytics and workout readers until the TTL lapsed. The WHOOP
per-resource webhook/cron driver and the Polar workouts leg carry the
same call, matching the posture Google Health and the workout batch
route already had.
Illness, cycle and encounter write routes hard-evict the analytics
bucket through the new invalidateUserHealthContext helper: Rest Mode,
the cycle context and the Today-rail appointment feed all read from
cached cells under that bucket, and an interactive write must be true on
the very next read.
Every interactive path, every import, MCP and Telegram checked an incoming
measurement against VALUE_RANGES before storing it. The provider sync writers
did not, so a decode slip or a glitched sensor landed a row the application
itself calls impossible, and every mean, median and personal band derived from
it inherited the impossibility. Filtering the read side hid the row. It kept
landing.
The gate is one module reading the domain the app already declares, applied at
two kinds of seam. reconcileExternalMeasurement is the shared identity writer
for Polar, Oura, WHOOP, the Withings sleep leg, the Apple export importer and
the batch endpoint, so gating there means a provider added later inherits the
refusal rather than having to remember a check it never knew about; the new
rejected_range verdict carries no row, which is what makes the compiler force
every caller to handle it. The five writers that hold Prisma themselves gate at
their entry, before the existence probe decides insert versus overwrite, so an
impossible reading can no longer replace a good stored value.
An out-of-band value is dropped. Not clamped, because a clamp invents a reading
nobody took. Not stored behind a flag, because a quarantined row is still a row
and every consumer would have to learn the flag. Not fatal to the batch, because
one bad sample must not strand the run's other readings behind a held watermark.
Absence reads as absence.
A silent drop, though, is indistinguishable from a provider that sent nothing,
so every drop is tallied on the wide event under measurement.range_rejected with
the metric, the provider and the edge it broke. Counts only: the reading itself
is health data and never enters the log line, and the direction plus the
metric's own declared bound is what a diagnosis actually needs. The tally
accumulates rather than replacing, because a provider gates once per resource
leg and the reconciler once per row. One warning per provider lifts the event to
warn so a drop does not hide inside a green line.
A structural guard enumerates all twenty measurement writers with the reason
each is allowed to write, so a new one has to answer the question before it
ships. scripts/diagnose-implausible-measurements.ts reports the rows already
stored, aggregated per metric and source with no value beyond the bucket's own
extremes. It repairs nothing: whether a stored impossible row is a decode slip
or a band that is too narrow is not a call a script gets to make.
Six provider syncs returned a bare count, which narrowed their failures away:
a Nightscout window in which every SGV row was refused by a constraint
returned the same 0 as a quiet night, and an Oura or Polar run whose
collections all failed returned 0 too. The routes then answered 200 with a
count and the settings card drew a green tick over it.
Every sync now reports what it imported AND whether any part of it did not
land, and the route resolves the outcome server-side so the web card and the
native client cannot disagree. Google Health keeps its 502 for a run that
failed and wrote nothing, but a run that failed after writing some resources
is now a partial rather than a discarded result. The labs OCR commit carries
the same resolved outcome.
Introduce a recorded-marker primitive on the integration ledger so a provider
that records-then-rethrows a sync failure is not recorded a second time at the
poll-cohort boundary, while an unrecorded escape (an Oura or Polar write-path
throw, or any future path) lands on the ledger exactly once, correctly
classified and redacted by the source that owns it. Without the marker a
boundary-blind recorder would double-count the failure streak and could persist
a Nightscout token-bearing URL into the stored error.
Give the poll-cohort factory a required per-provider failure recorder and wire
one for Oura, Polar, Nightscout, and Strava.
Nightscout now counts only freshly inserted readings instead of insert plus
re-confirm (the inflated count fired the reminder trigger every tick), records a
transient failure and skips the success stamp when some rows fail to write, and
emits the measurement-arrival spine for its inserts. Correct the docstring that
promised a far-history backfill that does not exist: the hourly window imports
about 48 hours and nothing deeper.
Strava now enqueues its history backfill at connect time, mirroring the WHOOP
and Fitbit callbacks, so deep history imports within the hour instead of waiting
for the next worker restart.
Platform-wide hardening: the failure classes found live this week hunted down
across every integration.
Fitbit carried the same tombstone wedge fixed for Google — probe now matches
soft-deleted rows and the update resurrects them; the comment describing a
partial unique index that never existed is corrected. Sleep segment ids are
stable across re-scoring for Withings (session+segment-start), WHOOP/Polar
(one segment per stage, no positional index) and Oura (run-start keyed), each
with a session-scoped sweep that soft-deletes rows a re-score orphaned — so
legacy and affected nights heal as they are re-read.
One platform rule for deleted rows: source-owned readings resurrect on
re-import (Withings, WHOOP, Oura, Polar, Nightscout, mood webhook, CSV
re-imports, Apple stats day-totals); Apple sample-grain deletions stick per
the paired-client contract, and the mood bulk route's tombstone match is now
a true no-op reported as duplicate. Import counts reconcile against actual
createMany counts.
Google lifecycle: a dead token registers on the cycle's hard-fail ledger, so
an expired connection parks instead of stamping success hourly; backfill and
sleep-repair markers stamp only after a clean pass (pg-boss retries are now
real); failed measurement writes hold the watermark; no-op updates are
skipped; mapping.md tells the current truth.
Scale: doctor-report reads only consumed columns and enabled sections;
exports and backups keyset-paginate; achievements tally via SQL buckets;
series charts day-bucket dense kinds beyond 90 days; BP pairing is a linear
two-pointer; distinct-type scans run server-side; the graded-series cold
fallback is bounded; the fitbit defer tracker loops instead of spreading.
One-shot prettier pass so the new format:check gate starts from a clean
baseline. Formatting only — no behaviour change. Generator-owned and
vendored artifacts stay untouched via .prettierignore.
Polar and Oura response classifiers were byte-identical copies of the
status-driven verdict, the typed API error, and the invalid_grant probe.
Extract classifyHttpStatus, an IntegrationApiError base carrying the vendor
label, and a vendor-scoped isOAuthInvalidGrant into a shared module; Polar
and Oura now re-export under their existing names and types so call sites
and instanceof checks are unchanged.
Collapse the triplicated classifyXFailure mapping (Polar, Oura, Nightscout)
onto one toFailureKind helper. Behaviour is identical: same classifications,
same failure kinds, same error types.
The entries URL carries the access token as ?token=<secret>; a SafeFetchError
embeds the full target URL in its message, and the sync feeds err.message into
recordSyncFailure, which the status endpoint returns as lastError. Strip token
and api-secret query values to =REDACTED before the message reaches the ledger,
so the secret can never surface in the status response.
Route the sync write through mapSgvEntryToMeasurement so the
entry-to-measurement shape has one source of truth instead of an
inline copy plus an unused parallel mapper that could drift.
Pull continuous glucose readings from a self-hosted Nightscout instance
and store them as immutable BLOOD_GLUCOSE mg/dL measurements
(source=NIGHTSCOUT), giving the glucose panel real CGM density.
- client over safeFetch: public-host pin by default, relaxed only on the
per-user private-network opt-in; redirect pinned manual + timeout
- credentials encrypted at rest on User; URL is the connect marker
- sync maps SGV entries first-write-wins on (userId,type,source,externalId)
- connect/disconnect/status routes: apiHandler + requireAuth + rate limit;
connect validates by a live test fetch with clear errors
- hourly poll-cohort job registered in the all-queues registry
- settings card with URL, token, and the private-network toggle
- i18n across all six locales
Marc-André Bombeck