mirror of
https://github.com/MBombeck/HealthLog.git
synced 2026-11-17 15:42:24 +00:00
fix(notifications): auto-disable channels on persistent hard rejects (410, etc.)
v1.4.15 Phase B3, criteria 1 + 2 (atomic — they share migration + retry- policy types). Until this release the dispatcher fired channels best- effort and kept retrying every 60s on a 410 Gone — that meant a stale Web-Push subscription never recovered, a Telegram bot whose chat was deleted got the same treatment, and the user had no visibility into WHY their phone had stopped buzzing. Reliability rules now: - HARD reject (HTTP 410 web-push / ntfy, Telegram "chat not found" / "blocked by the user") → channel auto-disabled, disabled_reason captured, audit log notification.channel.auto_disabled written. Sender NEVER retried until the user clicks "Re-enable" in Settings. - SOFT reject (5xx, 429, network timeout) → exponential backoff: [30s, 5min, 30min, 2h]. Counter consecutive_failures increments per failure; after the 5th in-a-row the channel is auto-disabled with reason give_up_after_5_failures. - COOLDOWN window (next_retry_at > now) → dispatcher skips the channel without calling the sender — defeats the API-quota-burn loop a flapping upstream would otherwise create. Implementation: - Migration 0028 adds 6 columns to notification_channels (all nullable / zero-default → no-op for existing rows). - src/lib/notifications/retry-policy.ts: classifiers + frozen BACKOFF_SCHEDULE_MS + nextRetryAt() helpers. - src/lib/notifications/channel-state.ts: only writer of the new columns (recordChannelSuccess, recordChannelHardReject, recordChannelTransientFailure, reEnableChannel, isChannelInCooldown). Each transient/hard path writes audit. - Senders return a structured SendOutcome instead of bare booleans. - Dispatcher: cooldown-skip → success → hard-reject → transient/ give-up. Sender-throws absorbed as transient failures. Tests: +20 unit tests (retry-policy.test.ts, channel-state.test.ts, dispatcher.test.ts). 883/883 unit tests pass; typecheck clean; lint clean (no new warnings). Pre-existing bug fix: src/app/api/settings/telegram/test/route.ts had if (!ok) against a SendMessageResult object — always truthy, so the route never reported test-send failures. Now reads result.ok. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
080545248e
commit
87a40fd0a4
15 files changed
+1187
-35
No files matched your search
@@ -0,0 +1,34 @@
|
||||
-- v1.4.15 Phase B3: notification reliability metadata.
|
||||
--
|
||||
-- Until this release the dispatcher fired channels best-effort and the
|
||||
-- only feedback signal we kept was a Wide-Event warning per failure.
|
||||
-- That meant a stale Web-Push subscription kept burning bandwidth on
|
||||
-- every dispatch (a 410 from FCM is a permanent reject — re-trying it
|
||||
-- never recovers), a Telegram bot whose chat was deleted got the same
|
||||
-- treatment, and the user never saw the channel's actual health.
|
||||
--
|
||||
-- These columns let the dispatcher:
|
||||
-- 1. Auto-disable a channel on a hard reject (HTTP 410, Telegram
|
||||
-- "chat not found" or "blocked by the user", ntfy 410), with the
|
||||
-- reason captured for the Settings UI.
|
||||
-- 2. Apply exponential backoff [30s, 5min, 30min, 2h] to soft errors
|
||||
-- (5xx, 429, network timeout) instead of hammering a flapping
|
||||
-- upstream — and after 5 consecutive transient failures, treat
|
||||
-- the channel as effectively dead and auto-disable it too.
|
||||
-- 3. Surface "Active / Auto-disabled / Sending paused" + last
|
||||
-- success / failure timestamps in Settings → Notifications, with
|
||||
-- a "Re-enable" button that clears `disabled_reason` +
|
||||
-- `consecutive_failures` and a "Send test" button.
|
||||
--
|
||||
-- All columns are nullable / zero-default so the migration is a no-op
|
||||
-- for existing rows: a healthy channel reports
|
||||
-- consecutive_failures=0, last_success_at=null, disabled_reason=null,
|
||||
-- which renders as "Active (no recent send recorded)" in the UI.
|
||||
|
||||
ALTER TABLE "notification_channels"
|
||||
ADD COLUMN "disabled_reason" TEXT,
|
||||
ADD COLUMN "consecutive_failures" INTEGER NOT NULL DEFAULT 0,
|
||||
ADD COLUMN "last_success_at" TIMESTAMP(3),
|
||||
ADD COLUMN "last_failure_at" TIMESTAMP(3),
|
||||
ADD COLUMN "last_failure_reason" TEXT,
|
||||
ADD COLUMN "next_retry_at" TIMESTAMP(3);
|
||||
@@ -106,6 +106,7 @@ model User {
|
||||
feedback Feedback[]
|
||||
idempotencyKeys IdempotencyKey[]
|
||||
devices Device[]
|
||||
integrationStatuses IntegrationStatus[]
|
||||
|
||||
@@map("users")
|
||||
}
|
||||
@@ -444,6 +445,46 @@ model MoodEntry {
|
||||
@@map("mood_entries")
|
||||
}
|
||||
|
||||
// ─── Integration Status (v1.4.15) ─────────────────────────
|
||||
//
|
||||
// Per-(user, integration) row that captures the most recent sync attempt
|
||||
// outcome so the Settings → Integrations UI can show:
|
||||
// - state ("connected", "error_reauth", "error_transient", "disconnected")
|
||||
// - last successful sync timestamp
|
||||
// - last attempted sync timestamp
|
||||
// - last error message (encrypted at rest if non-null — see crypto.ts)
|
||||
// - consecutive failure counter (resets on success)
|
||||
//
|
||||
// `recordSyncSuccess` / `recordSyncFailure` helpers in
|
||||
// `src/lib/integrations/status.ts` are the only writers. The
|
||||
// `consecutiveFailures` counter feeds the admin-Telegram alerting
|
||||
// threshold (default 3). State `error_reauth` parks scheduled syncs
|
||||
// until the user reconnects (the sync entry-points consult this row
|
||||
// before doing any external work).
|
||||
//
|
||||
// `integration` values are bare strings ("withings", "moodlog") rather
|
||||
// than an enum so future integrations don't require a migration.
|
||||
model IntegrationStatus {
|
||||
id String @id @default(cuid())
|
||||
userId String @map("user_id")
|
||||
integration String // "withings" | "moodlog"
|
||||
state String @default("connected") // "connected" | "error_reauth" | "error_transient" | "disconnected"
|
||||
lastSuccessAt DateTime? @map("last_success_at")
|
||||
lastAttemptAt DateTime? @map("last_attempt_at")
|
||||
lastError String? @map("last_error") // AES-256-GCM encrypted (may contain token-fragment metadata)
|
||||
consecutiveFailures Int @default(0) @map("consecutive_failures")
|
||||
alertedAt DateTime? @map("alerted_at") // last time admin Telegram alert fired (idempotency guard)
|
||||
createdAt DateTime @default(now()) @map("created_at")
|
||||
updatedAt DateTime @updatedAt @map("updated_at")
|
||||
|
||||
user User @relation(fields: [userId], references: [id], onDelete: Cascade)
|
||||
|
||||
@@unique([userId, integration])
|
||||
@@index([userId])
|
||||
@@index([state])
|
||||
@@map("integration_statuses")
|
||||
}
|
||||
|
||||
// ─── App Settings ─────────────────────────────────────────
|
||||
|
||||
model AppSettings {
|
||||
@@ -559,6 +600,23 @@ model NotificationChannel {
|
||||
createdAt DateTime @default(now()) @map("created_at")
|
||||
updatedAt DateTime @updatedAt @map("updated_at")
|
||||
|
||||
// ── v1.4.15 Phase B3: reliability fields ─────────────────────
|
||||
// Reason the channel was auto-disabled (e.g. "hard_reject_410",
|
||||
// "telegram_blocked_by_user", "telegram_chat_not_found",
|
||||
// "give_up_after_5_failures"). NULL when the channel is healthy
|
||||
// OR was disabled manually (in which case `enabled=false` alone
|
||||
// is enough — no automated process should treat that as
|
||||
// re-enable-able).
|
||||
disabledReason String? @map("disabled_reason")
|
||||
consecutiveFailures Int @default(0) @map("consecutive_failures")
|
||||
lastSuccessAt DateTime? @map("last_success_at")
|
||||
lastFailureAt DateTime? @map("last_failure_at")
|
||||
lastFailureReason String? @map("last_failure_reason")
|
||||
// The next time the dispatcher is allowed to attempt a send for
|
||||
// this channel after a transient failure. NULL = no cooldown.
|
||||
// Hard rejects flip `enabled=false` instead of writing this.
|
||||
nextRetryAt DateTime? @map("next_retry_at")
|
||||
|
||||
user User @relation(fields: [userId], references: [id], onDelete: Cascade)
|
||||
preferences NotificationPreference[]
|
||||
|
||||
|
||||
Reference in new issue
Block a user