mirror of
https://github.com/MBombeck/HealthLog.git
synced 2026-11-09 13:01:00 +00:00
Marc-André Bombeck
5ee1b3e28f
The response-timeout setting reached the briefing, the status notes and the Coach, which threaded it by hand. Document reads, document extraction, lab OCR and lab staging called the provider without a timeout, so every client fell back to its own 60 s literal and a slow self-hosted model was cut off however high the setting stood. The setting now rides on the provider instance. resolveProvider, resolveProviderChain and resolveProviderForTest stamp the record's aiResponseTimeoutSeconds on everything they return, and each wire client reads its ceiling through callTimeoutMs: the setting, else the surface's own timeoutMs, else 60 s. A call site no longer has to pass it, so a new one cannot forget it. The connection test runs under the same ceiling a read does. The Coach nudge tick and the reaction line keep their short ceilings (timeoutPolicy: "surface-ceiling"): the tick runs accounts one after another under a shared budget, the reaction line's claim lease is shorter than the largest setting, and both have deterministic copy that stands in. /api/auth/me publishes the effective value as ai.provider.responseTimeoutMs, so a client can size its own request timeout from it instead of guessing. Refs #1090
92 lines
3.5 KiB
TypeScript
92 lines
3.5 KiB
TypeScript
import type { AIProvider, CompletionParams } from "./types";
|
||
|
||
/**
|
||
* The provider timeout a call gets when neither the surface nor the person set
|
||
* one. Every client used to carry its own copy of this literal.
|
||
*/
|
||
export const PROVIDER_DEFAULT_TIMEOUT_MS = 60_000;
|
||
|
||
/**
|
||
* Resolve the upstream provider timeout (ms) for a non-chat generation surface.
|
||
*
|
||
* The per-user `User.aiResponseTimeoutSeconds` setting (Settings → AI) is the
|
||
* operator's lever for slow self-hosted / local backends whose first request
|
||
* loads the model. This helper is the one place that converts the stored
|
||
* seconds into milliseconds, falling back to the surface's budget default when
|
||
* the user has not set a value.
|
||
*
|
||
* Semantics: a positive stored value wins (seconds → ms), and any unset /
|
||
* non-positive value yields the surface default. The setting is bounded to
|
||
* 10–600 s at write-time, so no read-time clamp is needed here.
|
||
*/
|
||
export function resolveEffectiveTimeoutMs(
|
||
aiResponseTimeoutSeconds: number | null | undefined,
|
||
budgetDefaultMs: number,
|
||
): number {
|
||
return aiResponseTimeoutSeconds != null && aiResponseTimeoutSeconds > 0
|
||
? aiResponseTimeoutSeconds * 1000
|
||
: budgetDefaultMs;
|
||
}
|
||
|
||
/**
|
||
* The timeout one provider call actually runs under. Every client reads its
|
||
* `safeFetch` ceiling from here and nowhere else.
|
||
*
|
||
* The person's setting rides on the provider instance (`responseTimeoutSeconds`,
|
||
* stamped by `bindResponseTimeout` when the provider is resolved for a record),
|
||
* so a call site cannot forget it: whatever a surface passes as `timeoutMs` is
|
||
* only the default the setting overrides. A surface that is deliberately
|
||
* latency-bounded and has a deterministic fallback opts out with
|
||
* `timeoutPolicy: "surface-ceiling"`, and the structural guard keeps that set
|
||
* frozen.
|
||
*/
|
||
export function callTimeoutMs(
|
||
params: Pick<CompletionParams, "timeoutMs" | "timeoutPolicy">,
|
||
responseTimeoutSeconds: number | null | undefined,
|
||
): number {
|
||
const surfaceMs = params.timeoutMs ?? PROVIDER_DEFAULT_TIMEOUT_MS;
|
||
if (params.timeoutPolicy === "surface-ceiling") return surfaceMs;
|
||
return resolveEffectiveTimeoutMs(responseTimeoutSeconds, surfaceMs);
|
||
}
|
||
|
||
/**
|
||
* Stamp the record owner's response-timeout setting onto a resolved provider.
|
||
* Called by every exported resolver in `provider.ts` on its way out, which is
|
||
* the only way a provider reaches a call site.
|
||
*/
|
||
export function bindResponseTimeout<P extends AIProvider>(
|
||
provider: P,
|
||
aiResponseTimeoutSeconds: number | null | undefined,
|
||
): P {
|
||
provider.responseTimeoutSeconds =
|
||
aiResponseTimeoutSeconds != null && aiResponseTimeoutSeconds > 0
|
||
? aiResponseTimeoutSeconds
|
||
: null;
|
||
return provider;
|
||
}
|
||
|
||
/**
|
||
* Room a browser leaves on top of the server's model-call budget for what
|
||
* surrounds the calls: the upload, decrypting and rasterising the original,
|
||
* writing the result, the response itself.
|
||
*/
|
||
export const CLIENT_ABORT_MARGIN_MS = 30_000;
|
||
|
||
/**
|
||
* The request timeout a browser uses for a route that waits on `modelCalls`
|
||
* sequential model calls, given the `ai.provider.responseTimeoutMs` the
|
||
* account payload published. Never recomputes the setting: an absent or
|
||
* malformed value (a payload from before the field existed) reads as the
|
||
* server's own default.
|
||
*/
|
||
export function clientAbortMs(
|
||
responseTimeoutMs: number | null | undefined,
|
||
modelCalls: number,
|
||
): number {
|
||
const perCall =
|
||
typeof responseTimeoutMs === "number" && responseTimeoutMs > 0
|
||
? responseTimeoutMs
|
||
: PROVIDER_DEFAULT_TIMEOUT_MS;
|
||
return perCall * modelCalls + CLIENT_ABORT_MARGIN_MS;
|
||
}
|