mirror of
https://github.com/MBombeck/HealthLog.git
synced 2026-11-15 15:02:04 +00:00
fix(integrations): log a failed sync as a signal, not an audit row
The audit log is the trail of what a person or an admin did to an account. Every failed provider sync wrote a row there, burying that trail, and kept the provider's error message in plain text while the status row seals it. A failed sync now emits an integrations.sync.failed warning with the same counts and no message.
This commit is contained in:
1 parent
537a73d3b2
commit
689796db33
6 files changed
+125
-90
No files matched your search
@@ -1,18 +1,24 @@
|
||||
# `attemptNumber` audit-row vs alert-signal semantics
|
||||
# `attempt_number` signal vs alert semantics
|
||||
|
||||
Two numbers describe a failing integration sync, and they can diverge.
|
||||
On-call operators reading an audit-log row alongside an admin alert
|
||||
should expect the alert number to be smaller than (or equal to) the
|
||||
audit number, never the other way around.
|
||||
Operators reading an `integrations.sync.failed` log line alongside an admin
|
||||
alert should expect the alert number to be smaller than (or equal to) the
|
||||
logged number, never the other way around.
|
||||
|
||||
Since v1.43.7 a failed sync is a `warn` signal in the log store, not an
|
||||
audit-log row: the audit log records what a person or an admin did to an
|
||||
account, and a provider answering 503 is neither. The provider's message is
|
||||
not on the signal; it stays encrypted on the integration's status row and
|
||||
shows in the integration's settings.
|
||||
|
||||
## The two numbers
|
||||
|
||||
`integrations.sync.failed` audit rows write
|
||||
`attemptNumber = transient + reauth_required + persistent` — the **sum**
|
||||
`integrations.sync.failed` signals carry
|
||||
`attempt_number = transient + reauth_required + persistent` — the **sum**
|
||||
of all three failure buckets. This is the historical "consecutive
|
||||
failures" running total carried by the legacy
|
||||
`IntegrationStatus.consecutiveFailures` column (dropped in migration
|
||||
`0077`, v1.4.47). Auditors looking at a single row see one integer
|
||||
`0077`, v1.4.47). An operator looking at a single line sees one integer
|
||||
that captures how many failures the bucket ledger has accumulated since
|
||||
the last success.
|
||||
|
||||
@@ -27,14 +33,14 @@ single-bucket threshold even if its sum looks alarming.
|
||||
## When the two diverge
|
||||
|
||||
A typed-failure-streak edge case: integration fails 2× transient, then
|
||||
2× reauth_required, then 1× persistent. Audit row shows
|
||||
`attemptNumber: 5`. Alert signal is `max(2, 2, 1) = 2`, which is below
|
||||
the default threshold of 3, so no page fires. From the audit log this
|
||||
2× reauth_required, then 1× persistent. The last signal shows
|
||||
`attempt_number: 5`. The alert number is `max(2, 2, 1) = 2`, which is below
|
||||
the default threshold of 3, so no page fires. From the log this
|
||||
looks like a sustained 5-failure outage; from the alert dispatcher it
|
||||
looks like three small bumps that never crossed a single-bucket
|
||||
ceiling.
|
||||
|
||||
Both readings are correct for their purpose. The audit log records what
|
||||
Both readings are correct for their purpose. The signal records what
|
||||
happened end-to-end; the alert dispatcher only fires when a single
|
||||
failure mode persists, because mixed-bucket streaks usually indicate a
|
||||
transient network blip layered on top of an unrelated reauth prompt —
|
||||
@@ -43,7 +49,7 @@ we want to wake someone up for.
|
||||
|
||||
## Runbook implication
|
||||
|
||||
If you see an audit row with `attemptNumber >= threshold` but no alert
|
||||
If you see a signal with `attempt_number >= threshold` but no alert
|
||||
was dispatched, check the per-bucket counts on
|
||||
`IntegrationStatus.consecutiveFailuresByKind` (DB column
|
||||
`consecutive_failures_by_kind`) for that user/integration pair. The
|
||||
|
||||
Reference in new issue
Block a user