Service readiness

A cross-repo rule. Every Wisent service exposes a health route, and every deployment script in this organization treats one as a verdict.

Rule 1 — liveness and readiness are different questions

GET /health answers whether a process is running. That is all it is allowed to mean, and a service may say so in its own body.

GET /readyz answers whether the product works: it exercises the dependency the product cannot function without — a credential redemption, a database round trip, a broker handshake — and returns 503 naming what failed. It carries no secret; the name of the failing dependency is the whole payload.

Deploy checks, uptime monitors and release qualification read /readyz. Reading /health proves a process exists and nothing else.

Owning implementation: brama — GET /readyz redeems one capability per configured provider; see its README section "Primary interfaces".

Rule 2 — an authorization failure is never dressed as capacity

A refused credential, an expired capability, a mismatched authorization id and a revoked grant are authorization failures. They are 503 authorization_error with retryable: false, and the body names the operator action.

They are never 429, never capacity_error, never retryable: true. Capacity means come back later; authorization means come back after somebody changes something. A caller told to retry an identity that does not match will retry until its budget runs out, and an operator handed a capacity code will go looking at subscriptions and quotas.

Rule 3 — a retry policy may not outlive the thing it retries

A supervisor, dispatcher or client that retries a failure must be able to stop. Anything a retry cannot reach — an item in a trash, a missing launcher variable, a credential the provider refuses after issuing it fresh, an unregistered workload key — is a state somebody must change, and repeating the call is not progress toward changing it.

So: classify by whether repetition can help, not by which HTTP family the status belongs to. restart: on-failure on a process whose configuration is wrong is an infinite loop; retrying a refused credential once per request buys two upstream round trips and a log line per attempt. When a failure is unreachable by retry, retire the candidate, name the action that unblocks it, and say that retrying will not help.

Owning implementations: skarbiec answers 410 Gone for a trashed item and 409 for a legacy envelope rather than 503 infra_down; brama retires a subscription credential that was refused with a token the provider had just issued.

Why this exists

On 2026-08-11 the Brama gateway on charless-mac-mini refused every capability redemption for a full day while GET /health answered ok and every failed request returned 429 capacity_error, retryable: true. Five separate credential defects were each diagnosed from zero during that day — a regenerated workload key, a single-use capability spent by model discovery, a migration that dropped grants, dangling grants that aborted that migration, and a mismatched authorization id — because the two surfaces a person consults first, the health check and the error code, both failed in the reassuring direction.

On 2026-08-13 the same shape appeared three more times in one session, which is why Rule 3 exists: a trashed vault item answered 503 infra_down, retryable: true and told callers to wait for infrastructure that was fine; a supervised gateway restarted 3,869 times against a launcher variable no restart could supply; and a subscription refused with a freshly issued token was retried once per request, 275,468 times on one host. Every one of them was a state an operator had to change, reported as weather.

The general shape, worth carrying beyond these three rules: when a check and an error message are both wrong in the comforting direction, the time to diagnose stops being bounded by the defect and starts being bounded by how long someone believes them.