> ## Documentation Index
> Fetch the complete documentation index at: https://routerdocs.hivenet.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Error codes
> Understand Hivenet Router error responses, HTTP statuses, retry behavior, backend classification, response headers, and debugging workflows.
Hivenet Router’s client-facing API uses structured errors so applications can distinguish authentication, quota, routing, capacity, timeout, and backend failures.
A typical response is:
```json theme={null}
{
"error": {
"code": "no_agents_available",
"message": "All agents for this model are currently offline or unhealthy",
"source": "router"
}
}
```
Use:
```text theme={null}
error.code
```
for programmatic decisions.
Use the HTTP status for broad error handling and `message` for diagnostics. Do not make application logic depend on the exact message text.
## Response fields
| Field | Type | Description |
| --------------- | ------ | ---------------------------------------------------------------------------------------- |
| `error.code` | String | Machine-readable Hivenet Router error category |
| `error.message` | String | Human-readable diagnostic information |
| `error.source` | String | Coarse indication of whether Hivenet Router or an inference backend produced the failure |
### Error source
Possible values are:
| Source | Meaning |
| --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `router` | Hivenet Router produced the error during parsing, authentication, authorization, quota admission, routing, queueing, forwarding, or timeout handling |
| `backend` | The inference engine or external provider produced the underlying failure, or Hivenet Router attributed an unstructured forwarding failure to that path |
The `router` value can describe an error generated by either the router process or Hivenet Router agent code.
The `backend` value is also intentionally coarse. It does not identify which engine, provider, or machine produced the error.
An error message can contain text returned by the inference backend.
Treat it as operational data. Do not display it directly to an end user without considering whether it exposes model, infrastructure, validation, or internal service details.
## Response-format scope
The structured envelope is used by known client-facing `/v1/*` routes, including:
* authentication failures
* model discovery errors
* inference validation errors
* model-access denials
* quota failures
* routing and capacity failures
* classified backend errors
Not every HTTP error produced by the complete server currently uses this envelope.
### Administration endpoints
Many `/admin/*` validation responses use the simpler form:
```json theme={null}
{
"error": "version is required"
}
```
Some administration errors also include additional fields:
```json theme={null}
{
"error": "stale version",
"current_version": "rev_00000042"
}
```
See [Admin endpoints](/use-the-api/admin-endpoints) for their endpoint-specific responses.
### Unknown routes
A request to a route that Hivenet Router has not registered may receive an ordinary HTTP `404` rather than a structured Hivenet Router error.
For example:
```text theme={null}
POST /v1/responses
```
is not supported and does not enter the inference handler.
Clients should therefore handle:
* structured Hivenet Router errors
* ordinary HTTP errors
* connection failures
* truncated streams
## Error-code summary
| Code | Typical HTTP status | Source | Meaning |
| ---------------------------- | ------------------: | --------------------- | --------------------------------------------------------------------------------------------------------- |
| `request_invalid` | `400` | `router` | Malformed, empty, incomplete, or unsupported request |
| `context_length_exceeded` | `400` | `backend` | Prompt and requested output exceed the backend context limit |
| `invalid_parameter` | `400` | Usually `backend` | Backend rejected one or more request fields |
| `input_too_long` | `400` | `router` | Policy input-token or image cap exceeded |
| `unauthorized` | `401` | `router` | Client or administrator credentials are missing or invalid |
| `model_forbidden` | `403` | `router` | The client key’s explicit allowlist does not permit the model |
| `model_not_found` | `404` | `router` | The model is not registered or is deliberately hidden from the caller |
| Not guaranteed | `413` | Server | The request body exceeds `HIVENET_ROUTER_MAX_REQUEST_BYTES` |
| `rate_limit_exceeded` | `429` | `router` | RPM, serverless per-key occupancy, ITPM, or OTPM limit exhausted, or strict per-model quota entry missing |
| `token_limit_exceeded` | `429` | `router` | Daily token admission or output accounting rejected the request |
| `concurrency_limit_exceeded` | `429` | `router` | Model occupancy or in-flight budget remained full, or the healthy serving pool breached `shed_if` |
| `no_agents_available` | `503` | `router` | The policy chain found no eligible agent |
| `no_capacity` | `503` | `router` | Eligible agents exist but their declared request slots are full |
| `agent_disconnected` | `503` | `router` | The selected agent could not be reached |
| `backend_unavailable` | `503` | `backend` | Backend was unreachable, loading, overloaded, or temporarily unavailable |
| `queue_full` | `503` | `router` | The router’s global pending-request channel remained full |
| `backend_error` | `500` or `502` | `router` or `backend` | Unexpected internal, backend, provider, or response-format failure |
| `request_timeout` | `504` | `router` | The router-side request deadline expired |
The status shown is the normal mapping. Some internal failures explicitly return HTTP `500` with `backend_error`, while forwarded backend or provider failures normally map it to HTTP `502`.
## Request and authorization errors
### `request_invalid`
```text theme={null}
HTTP 400
```
Common triggers include:
* malformed JSON
* empty request body
* missing `model`
* model ID longer than 256 characters
* an inference engine used with an unsupported capability
* request parsing failure inside a Hivenet Router agent
Example:
```json theme={null}
{
"error": {
"code": "request_invalid",
"message": "model field is required",
"source": "router"
}
}
```
Do not retry the same payload.
Correct the request before sending it again.
### `context_length_exceeded`
```text theme={null}
HTTP 400
```
The inference backend determined that the request exceeded its effective context window.
The total may include:
* system or developer instructions
* conversation history
* tool definitions
* tool results
* multimodal content
* requested maximum output
Example:
```json theme={null}
{
"error": {
"code": "context_length_exceeded",
"message": "This model's maximum context length is 32768 tokens",
"source": "backend"
}
}
```
Reduce:
* prompt size
* retained conversation history
* tool definitions
* retrieved documents
* `max_tokens`
* `max_completion_tokens`
Do not retry the unchanged request against another agent serving the same model.
### `invalid_parameter`
```text theme={null}
HTTP 400
```
The backend rejected a field or value such as:
* `temperature`
* `top_p`
* `max_tokens`
* `max_completion_tokens`
* `reasoning_effort`
* roles or content blocks
* tools or tool choice
* structured-output options
* model-specific parameters
Example:
```json theme={null}
{
"error": {
"code": "invalid_parameter",
"message": "temperature must be between 0 and 2",
"source": "backend"
}
}
```
Do not retry the same request.
Inspect the backend message and compare the request with the exact model and engine configuration.
### `unauthorized`
```text theme={null}
HTTP 401
```
This covers both missing and invalid credentials.
Hivenet Router deliberately returns the same response for both cases:
```json theme={null}
{
"error": {
"code": "unauthorized",
"message": "unauthorized",
"source": "router"
}
}
```
The response includes:
```http theme={null}
WWW-Authenticate: Bearer realm="hivenet-router"
```
Possible causes include:
* missing `Authorization` header
* incorrect bearer key
* client sending `x-api-key` instead
* expired static or dynamic key
* disabled dynamic key
* removed key
* using a key hash instead of the raw key
* using an administrator key on a client route
* using a client key on an administrator route
Do not retry until the credential or header is corrected.
### `model_forbidden`
```text theme={null}
HTTP 403
```
The key uses an explicit model allowlist and the requested model is not included.
```json theme={null}
{
"error": {
"code": "model_forbidden",
"message": "your API key does not have access to model: restricted-model",
"source": "router"
}
}
```
Do not retry the same key and model combination.
Use an allowed model or update the key’s access configuration.
A model missing from `quota.per_model` does not produce `model_forbidden`.
That strict-enumeration case produces `rate_limit_exceeded`.
### `model_not_found`
```text theme={null}
HTTP 404
```
For inference, this normally means no agent currently registers the exact model ID.
```json theme={null}
{
"error": {
"code": "model_not_found",
"message": "No agents registered for this model",
"source": "router"
}
}
```
The model-detail endpoint also returns this code when the model exists but is not visible to the caller:
```text theme={null}
GET /v1/models/{model}
```
This prevents one tenant from discovering another tenant’s model catalog.
Check:
```bash theme={null}
curl \
-H "Authorization: Bearer " \
http://localhost:8080/v1/models \
| jq -r '.data[].id'
```
Retry only after:
* correcting the model ID
* registering an agent
* restoring the agent
* changing the key’s access
## Request-size rejection
### HTTP `413`
Hivenet Router limits request bodies under `/v1/*` to:
```text theme={null}
10485760 bytes
```
by default, which is 10 MiB.
A request above `HIVENET_ROUTER_MAX_REQUEST_BYTES` is rejected before it reaches the normal inference handler. That means the response body is not guaranteed to use Hivenet Router’s structured error envelope or a stable `error.code`.
Handle this case using the HTTP status:
```text theme={null}
413 Request Entity Too Large
```
Reduce the request body, split large embedding or reranking batches, or change the configured byte limit. Set:
```bash theme={null}
export HIVENET_ROUTER_MAX_REQUEST_BYTES=0
```
to disable the built-in limit, while accounting for any smaller reverse-proxy or ingress limit.
Do not retry the unchanged request.
## Quota errors
### `rate_limit_exceeded`
```text theme={null}
HTTP 429
```
This code covers request-rate admission, serverless per-key admission, and a missing strict per-model quota declaration.
#### RPM bucket exhausted
```json theme={null}
{
"error": {
"code": "rate_limit_exceeded",
"message": "rate limit exceeded",
"source": "router"
}
}
```
The response includes:
```http theme={null}
X-RateLimit-Remaining-Requests: 0
```
The RPM limiter refills continuously rather than resetting at the beginning of each wall-clock minute.
Retry with bounded exponential backoff and jitter.
Hivenet Router does not currently return a:
```http theme={null}
Retry-After
```
header, so the client must choose its own retry timing.
#### Serverless per-key limits
For a `mode: serverless` policy, this code also reports:
* `per-key occupancy share exceeded, please retry`
* `input token rate exceeded, please retry`
* `output token rate exceeded, please retry`
These responses include `Retry-After: 1`. Occupancy and input-token limits reject the current request. Output tokens are charged after a response finishes, so an exhausted output bucket rejects a subsequent request.
#### Missing per-model quota declaration
A key using `quota.per_model` must enumerate every permitted model.
An undeclared model returns:
```json theme={null}
{
"error": {
"code": "rate_limit_exceeded",
"message": "no quota declared for model restricted-model on this API key",
"source": "router"
}
}
```
This is a configuration error, not temporary RPM exhaustion.
Do not retry until the key has a complete entry such as:
```yaml theme={null}
quota:
per_model:
restricted-model:
requests_per_minute_per_replica: 0
tokens_per_day: 0
```
### `token_limit_exceeded`
```text theme={null}
HTTP 429
```
Hivenet Router can reject a token budget at two stages.
#### Before routing
The router checks whether this worst case fits:
```text theme={null}
estimated input tokens
+ requested maximum output
```
A request may therefore be rejected even when it would probably generate a shorter answer.
When output was reserved in the request, the message is:
```json theme={null}
{
"error": {
"code": "token_limit_exceeded",
"message": "insufficient daily token budget for requested max_tokens",
"source": "router"
}
}
```
The response can include a nonzero remaining value:
```http theme={null}
X-RateLimit-Remaining-Tokens: 250
```
The request may succeed before the UTC reset when you reduce:
* prompt size
* `max_tokens`
* `max_completion_tokens`
#### After non-streaming inference
The backend may complete the request before Hivenet Router knows the actual output-token count.
When the completion no longer fits:
* the response body is discarded
* Hivenet Router returns `429`
* the prompt-token deduction remains
* actual usage and the selected agent are preserved in the audit record
The response includes:
```http theme={null}
X-RateLimit-Remaining-Tokens: 0
```
A `token_limit_exceeded` response does not always mean that the backend performed no work.
For a post-response rejection, inference has already completed.
#### Streaming responses
Once a stream has started, Hivenet Router cannot replace the HTTP `200` response with a later JSON `429`.
Streaming output is accounted for after delivery. When it exceeds the remaining budget, Hivenet Router records the quota event in metrics, but the client retains the already delivered response.
## Admission errors
### `input_too_long`
```text theme={null}
HTTP 400
```
This response is produced before routing when a policy's `max_input_tokens` or `images_max` cap is exceeded:
```json theme={null}
{
"error": {
"code": "input_too_long",
"message": "input is 140000 tokens, over the model limit of 131072",
"source": "router"
}
}
```
Reduce the prompt or image count. Retrying an unchanged request will not help.
### `concurrency_limit_exceeded`
```text theme={null}
HTTP 429
Retry-After: 1
```
This response is used when either:
* the request would exceed the model's token-weighted occupancy budget or `max_inflight` limit and capacity did not become available within `HIVENET_ROUTER_ADMIT_PARK_TIMEOUT`; or
* aggregate healthy-pool engine pressure breached a configured `shed_if` threshold before queueing.
```json theme={null}
{
"error": {
"code": "concurrency_limit_exceeded",
"message": "server at capacity for this model, please retry",
"source": "router"
}
}
```
Retry with a short randomized delay. See [Admission control](/routing/admission-control) for footprint and budget calculation.
## Routing and capacity errors
### `no_agents_available`
```text theme={null}
HTTP 503
```
This is broader than “all agents are offline.”
It means the full local policy chain ended without an eligible agent and no more specific model or capacity reason was available.
Possible causes include:
* agents offline or unhealthy
* unhealthy inference backends
* capability mismatch
* static `match` filters
* dynamic `exclude_if` gates
* agents excluded after earlier failed attempts
* a model-specific policy that matches no agents
* all fallback steps exhausted
Messages can include:
```text theme={null}
All agents for this model are currently offline or unhealthy
```
or a more general policy-exhaustion explanation.
Check:
```bash theme={null}
curl \
-H "Authorization: Bearer " \
http://localhost:8080/admin/health \
| jq .
```
Then inspect:
```bash theme={null}
curl \
-H "Authorization: Bearer " \
http://localhost:8080/admin/routing-table \
| jq .
```
Retry with backoff when the failure is caused by a temporary fleet or health condition.
A policy or capability mismatch requires configuration changes instead.
### `no_capacity`
```text theme={null}
HTTP 503
```
Healthy, filter-compatible agents exist, but every eligible agent has reached its declared concurrency capacity.
```json theme={null}
{
"error": {
"code": "no_capacity",
"message": "All agents for this model are at maximum capacity",
"source": "router"
}
}
```
Before returning the error, Hivenet Router may:
* wait in the per-model capacity queue
* retry selection after a slot becomes available
* advance through fallback steps
Retry after a short randomized delay.
A sustained rate indicates that you should investigate:
* agent `--capacity`
* fleet size
* request duration
* engine queues
* context and output lengths
* per-model wait-queue depth
### `agent_disconnected`
```text theme={null}
HTTP 503
```
The selected agent could not be reached over the libp2p request path.
Possible causes include:
* stale peer connection
* unreachable announced address
* firewall or NAT change
* agent restart
* lost network path
* process termination
Hivenet Router normally handles this internally first.
It grants one connection reset and redial attempt per agent without consuming the policy step’s normal try budget. Further failures can cause another agent or fallback step to be selected.
The client may therefore receive a later error such as:
* `no_agents_available`
* `no_capacity`
* `request_timeout`
rather than the original `agent_disconnected`.
When the code does reach the client, retry with backoff and inspect the agent connection path.
### `queue_full`
```text theme={null}
HTTP 503
```
The router’s global pending-request channel stayed full for five seconds.
```json theme={null}
{
"error": {
"code": "queue_full",
"message": "Request queue is full, please retry",
"source": "router"
}
}
```
This is controlled by:
```text theme={null}
--queue-size
```
It is distinct from the per-model capacity wait queue.
When the per-model queue is full, the routing session advances through fallback steps. That condition does not directly return `queue_full`; the final response is normally `no_capacity` or `no_agents_available`.
Retry with exponential backoff and investigate:
* request arrival rate
* `--queue-size`
* `--max-concurrent`
* agent capacity
* backend latency
* router CPU and memory
### `request_timeout`
```text theme={null}
HTTP 504
```
The router’s end-to-end deadline expired.
The deadline can include:
* waiting for processor concurrency
* waiting in the per-model capacity queue
* policy selection
* retries
* agent redial
* agent forwarding
* inference
* provider fallback
Messages vary according to where the deadline expired:
```text theme={null}
Request expired in queue
Request timed out waiting for an available agent
Request deadline exceeded while contacting agent
Request timed out before provider fallback could be attempted
Request timeout
```
A timeout is an ambiguous result.
The backend may have received or even completed the request before the client received the timeout. Hivenet Router does not deduplicate retries by `X-Request-ID`.
Retry only when the application operation is safe to repeat.
For systematic timeouts, inspect the request path before increasing:
```text theme={null}
--request-timeout
```
## Backend errors
### `backend_unavailable`
```text theme={null}
HTTP 503
```
For native Chat Completions, this can represent:
* backend connection failure
* model still loading
* backend HTTP `429`
* backend HTTP `503`
* temporary engine overload
Example:
```json theme={null}
{
"error": {
"code": "backend_unavailable",
"message": "backend unreachable: connection refused",
"source": "backend"
}
}
```
Hivenet Router normally treats this as retryable inside the routing session.
It can try:
* another agent
* another local fallback step
* an external provider fallback
The final client-visible error may therefore be `no_agents_available` rather than `backend_unavailable`.
Retry with backoff when the backend condition is temporary.
### `backend_error`
```text theme={null}
HTTP 500 or 502
```
This is the catch-all for failures that Hivenet Router cannot classify more specifically.
Possible causes include:
* inference backend HTTP `500` or `502`
* unexpected backend response
* invalid response JSON
* provider fallback failure
* unstructured error from a transparent proxy endpoint
* storage or authentication-provider failure
* another internal processing error
Example:
```json theme={null}
{
"error": {
"code": "backend_error",
"message": "provider openai: upstream request failed",
"source": "backend"
}
}
```
A local agent backend error is normally retried inside the policy chain.
An external provider fallback failure returns `backend_error` directly because no routing step remains after that fallback.
Retry once only when:
* the request is safe to repeat
* the failure appears transient
* retrying will not worsen an overloaded backend
Repeated errors with the same input require investigation.
## How native Chat Completions errors are classified
The Hivenet Router agent classifies non-success responses from the native:
```text theme={null}
POST /v1/chat/completions
```
handler before returning them to the router.
### JSON error types
The classifier recognizes both:
```json theme={null}
{
"message": "Prompt is too long",
"type": "context_length_exceeded"
}
```
and:
```json theme={null}
{
"error": {
"message": "Prompt is too long",
"type": "context_length_exceeded"
}
}
```
Known type mappings are:
| Backend type | Hivenet Router code |
| ------------------------- | ------------------------- |
| `context_length_exceeded` | `context_length_exceeded` |
| `invalid_request_error` | `invalid_parameter` |
| `invalid_parameter` | `invalid_parameter` |
A recognized type takes precedence over the HTTP-status fallback.
### HTTP and text fallback
When the error type is not recognized:
| Backend response | Hivenet Router code |
| ---------------------------------------------------------------- | ------------------------- |
| `400` or `422` with context and length, window, or token wording | `context_length_exceeded` |
| Other `400` or `422` | `invalid_parameter` |
| `429` or `503` | `backend_unavailable` |
| Other non-success status | `backend_error` |
For this native path, Hivenet Router extracts a message from the backend JSON where possible.
Otherwise, it uses the raw body, truncated to 512 bytes.
A backend HTTP `429` is classified as `backend_unavailable`, not as Hivenet Router’s `rate_limit_exceeded`.
`rate_limit_exceeded` describes Hivenet Router client-key quota admission.
## Errors from transparent proxy endpoints
These paths use the agent’s generic transparent proxy:
```text theme={null}
POST /v1/messages
POST /v1/messages/count_tokens
POST /v1/embeddings
POST /v1/rerank
```
The agent initially forwards the backend’s status and body without applying the native Chat Completions classifier.
When the router receives a non-success response:
1. It checks whether the body already contains a Hivenet Router structured error.
2. If it does, that code and source are preserved.
3. Otherwise, the router wraps the body as `backend_error`.
For example, an ordinary Anthropic-format backend error:
```json theme={null}
{
"type": "error",
"error": {
"type": "invalid_request_error",
"message": "max_tokens is required"
}
}
```
does not match Hivenet Router’s own envelope:
```json theme={null}
{
"error": {
"code": "...",
"message": "...",
"source": "..."
}
}
```
The final client response can therefore become:
```text theme={null}
HTTP 502
backend_error
```
rather than preserving the backend’s original HTTP `400`.
Backend classification is currently more precise for native Chat Completions than for Messages, token counting, embeddings, and reranking.
Inspect the backend log when one of the transparent endpoints returns a broad `backend_error`.
## Internal routing retries
A backend or agent error is not necessarily returned to the client immediately.
Hivenet Router distinguishes several categories.
### Request-level errors
These stop the routing session immediately:
```text theme={null}
request_invalid
context_length_exceeded
invalid_parameter
token_limit_exceeded
```
Trying the same request on another agent would normally produce the same outcome.
### Connection failures
A stale or disconnected agent connection receives one free redial attempt per request.
That redial does not consume the policy step’s `max_tries` budget.
### Retryable forwarding failures
Other agent and backend errors can:
1. mark the selected agent as failed in the current policy step
2. select another eligible agent
3. continue until `max_tries` is reached
4. advance through the fallback chain
5. use provider fallback where configured
Because of this, the client sees the final routing outcome rather than every internal attempt.
For example:
```text theme={null}
backend_unavailable
→ another agent tried
→ fallback step exhausted
→ client receives no_agents_available
```
Use audit logs, metrics, traces, and backend logs when the final error does not reveal the first failure.
## Streaming error behavior
A structured JSON error can be returned only before response headers and body have started.
After an SSE stream begins:
* the HTTP status is already `200`
* some output may already have reached the client
* Hivenet Router cannot replace the stream with a JSON error envelope
A failure during streaming can therefore appear as:
* a stream ending early
* an incomplete SSE event
* a connection reset
* a client-side parsing error
The client should treat an incomplete stream as a failed or partial operation, even when the initial HTTP status was `200`.
Use the request ID to inspect the associated audit record, router logs, agent logs, and trace.
## Error response headers
Hivenet Router can add the following headers.
| Header | When present | Meaning |
| -------------------------------- | ---------------------------------------- | ---------------------------------------------------------- |
| `X-Request-ID` | Every router response | UUID used for logs, audit records, and support correlation |
| `traceparent` | When a valid trace span exists | W3C trace context for Tempo or another tracing backend |
| `WWW-Authenticate` | `401` responses | Bearer authentication challenge |
| `X-RateLimit-Remaining-Requests` | Finite RPM quota | Remaining request-bucket capacity |
| `X-RateLimit-Remaining-Tokens` | Finite daily token budget | Remaining token budget known at that stage |
| `Retry-After` | `concurrency_limit_exceeded` | Retry delay in seconds; the current value is `1` |
| `Retry-After` | Serverless per-key `rate_limit_exceeded` | Retry delay in seconds; the current value is `1` |
Unlimited quota values are omitted rather than represented as negative numbers.
Hivenet Router does not provide a general:
```text theme={null}
Retry-After
```
header for every retryable error. It is set on admission-capacity rejection.
Backend error headers are not guaranteed to be preserved when Hivenet Router converts a backend response into a structured error.
## Correlate an error
Every router response includes:
```http theme={null}
X-Request-ID: 8b9e4a6d-dfb4-4b95-bada-319e7e5b878a
```
Log this value in the calling application.
Search it in Loki:
```logql theme={null}
{job="hivenet-router", log_type="audit"}
| json
| request_id =
"8b9e4a6d-dfb4-4b95-bada-319e7e5b878a"
```
The audit record can show:
* tenant
* dynamic key ID
* model
* status
* error code
* selected agent
* latency
* token counts
* trace ID
Use the trace ID in Tempo for the complete distributed request path.
## Diagnose by status
### HTTP `400`
Check:
* request JSON
* required model and message fields
* context length
* model-specific parameter support
* tools and structured-output schemas
* backend logs
### HTTP `401`
Check:
* raw bearer key
* credential expiration
* dynamic key state
* correct client or administrator key type
* reverse-proxy header forwarding
### HTTP `403`
Check the explicit model allowlist.
### HTTP `404`
Check:
* endpoint path
* base URL construction
* exact model ID
* client model visibility
* registered agents
A plain `404` may mean an unsupported route. A structured `model_not_found` refers to model discovery or routing.
### HTTP `413`
Reduce the request body or split a large batch. Check both `HIVENET_ROUTER_MAX_REQUEST_BYTES` and any reverse-proxy or ingress body limit.
### HTTP `429`
Inspect:
```text theme={null}
error.code
error.message
X-RateLimit-Remaining-Requests
X-RateLimit-Remaining-Tokens
```
Distinguish:
* RPM exhaustion
* undeclared per-model quota
* insufficient worst-case token budget
* exhausted daily token budget
### HTTP `503`
Inspect the specific code.
| Code | First place to look |
| --------------------- | --------------------------------------------------- |
| `no_agents_available` | Agent health, capability, policy filters, and gates |
| `no_capacity` | Agent load, declared capacity, and queues |
| `agent_disconnected` | libp2p address and network path |
| `backend_unavailable` | Inference backend health and startup |
| `queue_full` | Router concurrency and pending-request pressure |
### HTTP `504`
Check:
* global queueing
* per-model queueing
* policy retries
* connection resets
* backend queue depth
* model loading
* prompt and output size
* router request timeout
## Client retry guidance
Before retrying, determine both:
* whether the error is temporary
* whether repeating the application operation is safe
| Code | Default client action |
| ---------------------------- | ---------------------------------------------------------------------- |
| `request_invalid` | Correct the request |
| `context_length_exceeded` | Reduce context or output |
| `invalid_parameter` | Remove or correct the rejected field |
| `input_too_long` | Reduce input text or image count |
| `unauthorized` | Correct or refresh credentials |
| `model_forbidden` | Change model or access configuration |
| `model_not_found` | Correct model or wait for deployment state to change |
| HTTP `413` | Reduce the request body or split the batch |
| `rate_limit_exceeded` | Back off for RPM exhaustion; change configuration for undeclared model |
| `token_limit_exceeded` | Reduce request size or wait for the UTC budget reset |
| `concurrency_limit_exceeded` | Honor `Retry-After` and retry with jitter |
| `no_agents_available` | Back off and inspect fleet or policy state |
| `no_capacity` | Retry after a short randomized delay |
| `agent_disconnected` | Retry with backoff after checking connectivity |
| `backend_unavailable` | Retry with exponential backoff |
| `queue_full` | Retry with exponential backoff |
| `backend_error` | Retry at most selectively and investigate repeated failures |
| `request_timeout` | Retry only when duplicate execution is acceptable |
Hivenet Router does not use `X-Request-ID` as an idempotency key.
Sending the same ID again creates another inference operation.
Official SDKs and frameworks may add their own automatic retries. Account for those retries when setting:
* request quotas
* application deadlines
* maximum attempts
* concurrency
* audit expectations
## Production handling
A production client should:
1. Parse structured errors when the body matches the Hivenet Router envelope.
2. Preserve the HTTP status and `X-Request-ID`.
3. Handle ordinary HTTP errors when no structured envelope exists.
4. Distinguish temporary capacity failures from request defects.
5. Apply bounded retries with exponential backoff and jitter.
6. Avoid retrying unsafe or ambiguous operations automatically.
7. Log the code, status, source, request ID, and model.
8. Avoid logging credentials or complete sensitive prompts.
9. Alert on sustained error rates rather than individual transient failures.
10. Use audit records and traces for detailed investigation.
## Next steps
Understand queueing, concurrency, latency, throughput, and benchmarking behavior.
Handle Hivenet Router errors from Python, JavaScript, SDKs, and direct HTTP clients.
See how retries, fallback steps, capacity, and policy exhaustion affect the final response.
Configure the request and token quotas behind `429` responses.
Diagnose fleet health, capacity, routing state, and dynamic key errors.
Correlate error codes with tenants, models, agents, request IDs, and traces.