mirror of
https://github.com/MBombeck/HealthLog.git
synced 2026-11-08 12:32:43 +00:00
Marc-André Bombeck
c986efab30
- attempt-number-semantics: name the real bucket column
(consecutiveFailuresByKind) and drop a drifted line citation
- backup-restore: show the smoke-test response inside the standard
{ data, error } envelope
- deploy: the manual migration step now uses the bundled Prisma CLI
via node (the image strips pnpm); WHOOP vars are already on the
shipped compose whitelist
- encryption-key-rotation: point at the encrypted-column registry
instead of a five-model list several releases behind
- env-check: the CI gate shipped (env-check.yml); the partial-APNs
example now shows the synthetic all-or-none row and the non-zero exit
- password-reset: the admin API reset route exists; only the
self-service route does not
- tls-cert-pin: stop asserting client-repo pin-set details that cannot
be audited from this repository
- drop v141-followup-issues.md: a v1.4.1 punch list whose items are
fixed, removed, or superseded; keeping it under docs/ops invites
re-fixing shipped work
55 lines
2.5 KiB
Markdown
55 lines
2.5 KiB
Markdown
# `attemptNumber` audit-row vs alert-signal semantics
|
||
|
||
Two numbers describe a failing integration sync, and they can diverge.
|
||
On-call operators reading an audit-log row alongside an admin alert
|
||
should expect the alert number to be smaller than (or equal to) the
|
||
audit number, never the other way around.
|
||
|
||
## The two numbers
|
||
|
||
`integrations.sync.failed` audit rows write
|
||
`attemptNumber = transient + reauth_required + persistent` — the **sum**
|
||
of all three failure buckets. This is the historical "consecutive
|
||
failures" running total carried by the legacy
|
||
`IntegrationStatus.consecutiveFailures` column (dropped in migration
|
||
`0077`, v1.4.47). Auditors looking at a single row see one integer
|
||
that captures how many failures the bucket ledger has accumulated since
|
||
the last success.
|
||
|
||
The persistent-failure alert dispatcher in
|
||
`src/lib/integrations/status.ts` (the v1.4.43 W14 block) uses
|
||
`Math.max(...Object.values(buckets))` — the
|
||
**max** across the three buckets. A 3-deep persistent streak pages even
|
||
when the transient bucket is empty, and a flapping integration that
|
||
ping-pongs between transient and reauth_required never trips a
|
||
single-bucket threshold even if its sum looks alarming.
|
||
|
||
## When the two diverge
|
||
|
||
A typed-failure-streak edge case: integration fails 2× transient, then
|
||
2× reauth_required, then 1× persistent. Audit row shows
|
||
`attemptNumber: 5`. Alert signal is `max(2, 2, 1) = 2`, which is below
|
||
the default threshold of 3, so no page fires. From the audit log this
|
||
looks like a sustained 5-failure outage; from the alert dispatcher it
|
||
looks like three small bumps that never crossed a single-bucket
|
||
ceiling.
|
||
|
||
Both readings are correct for their purpose. The audit log records what
|
||
happened end-to-end; the alert dispatcher only fires when a single
|
||
failure mode persists, because mixed-bucket streaks usually indicate a
|
||
transient network blip layered on top of an unrelated reauth prompt —
|
||
two problems, neither of which is the "this integration is broken" page
|
||
we want to wake someone up for.
|
||
|
||
## Runbook implication
|
||
|
||
If you see an audit row with `attemptNumber >= threshold` but no alert
|
||
was dispatched, check the per-bucket counts on
|
||
`IntegrationStatus.consecutiveFailuresByKind` (DB column
|
||
`consecutive_failures_by_kind`) for that user/integration pair. The
|
||
sum may be high while every individual bucket sits below the threshold.
|
||
This is working as designed; manually investigate only if a single
|
||
bucket is at or above the threshold and `alertedAt` is unset (broken
|
||
dispatcher) or set within the last 24 hours (rate-limited, see the
|
||
`ALERT_REPEAT_WINDOW_MS` constant).
|