mirror of
https://github.com/MBombeck/HealthLog.git
synced 2026-11-09 13:01:00 +00:00
Marc-André Bombeck
ba4774e48a
The health ledger already counted every failed call and paused a provider that kept failing, with a pause that grows. A paused provider only moved to the tail of the chain, though, and the tail is still walked: a chain whose only entry answered every call with reasoning and no answer was called by every background job regardless. A walk for background work (surface "job") now leaves out every provider the ledger has benched until its window lifts; the local floor only once it is paused, past a brief backoff. One ai.chain.background_paused line per window says so. A person's own call still reaches the provider at the tail and reports its own error. The ledger is in Postgres, so the bench holds across workers and restarts. The status, briefing and document-index paths refunded a failed call in full, so background spend on replies without an answer never reached the day's token ceiling. A failed chain now carries the tokens its hops reported (tokensSpent) and those paths charge them. The two clients share one answerless step (askAgainUnreasoned), which keeps both inside the source size ratchet; its baselines go down. Refs #1279
101 lines
3.6 KiB
TypeScript
101 lines
3.6 KiB
TypeScript
import { annotate } from "@/lib/logging/context";
|
|
|
|
/**
|
|
* Race an awaited promise against a hard timeout.
|
|
*
|
|
* The seven `*-status.ts` routes await `provider.generateCompletion()`
|
|
* and need an upper bound so a stalled provider does not pin the
|
|
* InsightStatusCard behind React-Query's retries. `withTimeout` resolves
|
|
* with the wrapped promise's value when it lands inside the budget, or
|
|
* the `fallback` value otherwise.
|
|
*
|
|
* Returns a typed envelope so the call site can distinguish a real
|
|
* timeout from an upstream error from a normal value, and react
|
|
* differently to each — a timeout/error must NOT be persisted as a
|
|
* day-long cache entry the way a real assessment is.
|
|
*
|
|
* The budget matches the provider clients' own 60 s ceiling. The
|
|
* earlier 20 s cap fired below the providers' floor on cold starts and
|
|
* model warm-ups, converting healthy-but-slow generations into the
|
|
* generic fallback; aligning the two lets a slow-but-valid round-trip
|
|
* land instead of being discarded.
|
|
*/
|
|
export interface TimeoutEnvelope<T> {
|
|
/** True when the budget expired before the upstream settled. */
|
|
timedOut: boolean;
|
|
/** True when the upstream rejected before the budget expired. */
|
|
errored: boolean;
|
|
value: T;
|
|
/** What the upstream rejected with, when `errored`. */
|
|
error?: unknown;
|
|
}
|
|
|
|
export function withTimeout<T>(
|
|
start: () => Promise<T>,
|
|
ms: number,
|
|
fallback: T,
|
|
/**
|
|
* v1.9.0 — fired once when the budget expires before the upstream settles.
|
|
* `withTimeout` cannot cancel the underlying promise, so a caller that owns
|
|
* a cancellation handle (e.g. an `AbortController`) passes it here to cut
|
|
* the detached work off — preventing a late-resolving promise from running
|
|
* side effects the caller has already moved past.
|
|
*/
|
|
onTimeout?: () => void,
|
|
): Promise<TimeoutEnvelope<T>> {
|
|
let timer: ReturnType<typeof setTimeout> | null = null;
|
|
let settled = false;
|
|
|
|
return new Promise<TimeoutEnvelope<T>>((resolve) => {
|
|
timer = setTimeout(() => {
|
|
if (settled) return;
|
|
settled = true;
|
|
annotate({
|
|
action: { name: "insights.status.provider_timeout" },
|
|
meta: { timeout_ms: ms },
|
|
});
|
|
onTimeout?.();
|
|
resolve({ timedOut: true, errored: false, value: fallback });
|
|
}, ms);
|
|
|
|
start()
|
|
.then((value) => {
|
|
if (settled) return;
|
|
settled = true;
|
|
if (timer) {
|
|
clearTimeout(timer);
|
|
timer = null;
|
|
}
|
|
resolve({ timedOut: false, errored: false, value });
|
|
})
|
|
.catch((error: unknown) => {
|
|
if (settled) return;
|
|
settled = true;
|
|
if (timer) {
|
|
clearTimeout(timer);
|
|
timer = null;
|
|
}
|
|
// Upstream errored before the timeout fired. Surface it as a
|
|
// distinct `errored` envelope and annotate the reason so a real
|
|
// provider failure is observable in the wide events instead of
|
|
// being silently swallowed into the timeout path.
|
|
const reason =
|
|
error instanceof Error ? error.message : String(error ?? "unknown");
|
|
annotate({
|
|
action: { name: "insights.status.provider_error" },
|
|
meta: { reason: reason.slice(0, 240) },
|
|
});
|
|
resolve({ timedOut: false, errored: true, value: fallback, error });
|
|
});
|
|
});
|
|
}
|
|
|
|
/**
|
|
* Status-path provider budget, aligned with the provider clients' own
|
|
* 60 s ceiling. A round-trip that exceeds this is treated as a transient
|
|
* miss — the caller returns the deterministic fallback for this render
|
|
* WITHOUT persisting it as the day's assessment, so the next mount
|
|
* re-attempts a real generation.
|
|
*/
|
|
export const STATUS_PROVIDER_TIMEOUT_MS = 60_000;
|