Error codes - Hivenet Router

Hivenet Router’s client-facing API uses structured errors so applications can distinguish authentication, quota, routing, capacity, timeout, and backend failures. A typical response is:

{
  "error": {
    "code": "no_agents_available",
    "message": "All agents for this model are currently offline or unhealthy",
    "source": "router"
  }
}

Use:

error.code

for programmatic decisions. Use the HTTP status for broad error handling and message for diagnostics. Do not make application logic depend on the exact message text.

Response fields

Field	Type	Description
`error.code`	String	Machine-readable Hivenet Router error category
`error.message`	String	Human-readable diagnostic information
`error.source`	String	Coarse indication of whether Hivenet Router or an inference backend produced the failure

Error source

Possible values are:

Source	Meaning
`router`	Hivenet Router produced the error during parsing, authentication, authorization, quota admission, routing, queueing, forwarding, or timeout handling
`backend`	The inference engine or external provider produced the underlying failure, or Hivenet Router attributed an unstructured forwarding failure to that path

The router value can describe an error generated by either the router process or Hivenet Router agent code. The backend value is also intentionally coarse. It does not identify which engine, provider, or machine produced the error.

An error message can contain text returned by the inference backend.Treat it as operational data. Do not display it directly to an end user without considering whether it exposes model, infrastructure, validation, or internal service details.

Response-format scope

The structured envelope is used by known client-facing /v1/* routes, including:

authentication failures
model discovery errors
inference validation errors
model-access denials
quota failures
routing and capacity failures
classified backend errors

Not every HTTP error produced by the complete server currently uses this envelope.

Administration endpoints

Many /admin/* validation responses use the simpler form:

{
  "error": "version is required"
}

Some administration errors also include additional fields:

{
  "error": "stale version",
  "current_version": "rev_00000042"
}

See Admin endpoints for their endpoint-specific responses.

Unknown routes

A request to a route that Hivenet Router has not registered may receive an ordinary HTTP 404 rather than a structured Hivenet Router error. For example:

POST /v1/responses

is not supported and does not enter the inference handler. Clients should therefore handle:

structured Hivenet Router errors
ordinary HTTP errors
connection failures
truncated streams

Error-code summary

Code	Typical HTTP status	Source	Meaning
`request_invalid`	`400`	`router`	Malformed, empty, incomplete, or unsupported request
`context_length_exceeded`	`400`	`backend`	Prompt and requested output exceed the backend context limit
`invalid_parameter`	`400`	Usually `backend`	Backend rejected one or more request fields
`input_too_long`	`400`	`router`	Policy input-token or image cap exceeded
`unauthorized`	`401`	`router`	Client or administrator credentials are missing or invalid
`model_forbidden`	`403`	`router`	The client key’s explicit allowlist does not permit the model
`model_not_found`	`404`	`router`	The model is not registered or is deliberately hidden from the caller
`model_redirected`	`307`	`router`	Another site serves the model: repeat the request at the `Location` URL (Cross-site redirect)
Not guaranteed	`413`	Server	The request body exceeds `HIVENET_ROUTER_MAX_REQUEST_BYTES`
`rate_limit_exceeded`	`429`	`router`	RPM, serverless per-key occupancy, ITPM, or OTPM limit exhausted, or strict per-model quota entry missing
`token_limit_exceeded`	`429`	`router`	Daily token admission or output accounting rejected the request
`concurrency_limit_exceeded`	`429`	`router`	Model occupancy or in-flight budget remained full, or the healthy serving pool breached `shed_if`
`no_agents_available`	`503`	`router`	The policy chain found no eligible agent
`no_capacity`	`503`	`router`	Eligible agents exist but their declared request slots are full
`agent_disconnected`	`503`	`router`	The selected agent could not be reached
`backend_unavailable`	`503`	`backend`	Backend was unreachable, loading, overloaded, or temporarily unavailable
`queue_full`	`503`	`router`	The router’s global pending-request channel remained full
`model_waking`	`503`	`router`	A model scaled to zero on this site did not wake within the hold budget, too many requests already wait for it, or it woke but the admission gate had no room for a released request before its hold budget ended (Hold at zero)
`backend_error`	`500` or `502`	`router` or `backend`	Unexpected internal, backend, provider, or response-format failure
`request_timeout`	`504`	`router`	The router-side request deadline expired
`stream_truncated`	`200` (in-stream frame)	`router`	An upstream stream was cut short mid-generation after the response started

The status shown is the normal mapping. Some internal failures explicitly return HTTP 500 with backend_error, while forwarded backend or provider failures normally map it to HTTP 502.

Request and authorization errors

`request_invalid`

HTTP 400

Common triggers include:

malformed JSON
empty request body
missing model
model ID longer than 256 characters
an inference engine used with an unsupported capability
request parsing failure inside a Hivenet Router agent

Example:

{
  "error": {
    "code": "request_invalid",
    "message": "model field is required",
    "source": "router"
  }
}

Do not retry the same payload. Correct the request before sending it again.

`context_length_exceeded`

HTTP 400

The inference backend determined that the request exceeded its effective context window. The total may include:

system or developer instructions
conversation history
tool definitions
tool results
multimodal content
requested maximum output

Example:

{
  "error": {
    "code": "context_length_exceeded",
    "message": "This model's maximum context length is 32768 tokens",
    "source": "backend"
  }
}

Reduce:

prompt size
retained conversation history
tool definitions
retrieved documents
max_tokens
max_completion_tokens

Do not retry the unchanged request against another agent serving the same model.

`invalid_parameter`

HTTP 400

The backend rejected a field or value such as:

temperature
top_p
max_tokens
max_completion_tokens
reasoning_effort
roles or content blocks
tools or tool choice
structured-output options
model-specific parameters

Example:

{
  "error": {
    "code": "invalid_parameter",
    "message": "temperature must be between 0 and 2",
    "source": "backend"
  }
}

Do not retry the same request. Inspect the backend message and compare the request with the exact model and engine configuration.

`unauthorized`

HTTP 401

This covers both missing and invalid credentials. Hivenet Router deliberately returns the same response for both cases:

{
  "error": {
    "code": "unauthorized",
    "message": "unauthorized",
    "source": "router"
  }
}

The response includes:

WWW-Authenticate: Bearer realm="hivenet-router"

Possible causes include:

missing Authorization header
incorrect bearer key
client sending x-api-key instead
expired static or dynamic key
disabled dynamic key
removed key
using a key hash instead of the raw key
using an administrator key on a client route
using a client key on an administrator route

Do not retry until the credential or header is corrected.

`model_forbidden`

HTTP 403

The key uses an explicit model allowlist and the requested model is not included.

{
  "error": {
    "code": "model_forbidden",
    "message": "your API key does not have access to model: restricted-model",
    "source": "router"
  }
}

Do not retry the same key and model combination. Use an allowed model or update the key’s access configuration.

A model missing from quota.per_model does not produce model_forbidden.That strict-enumeration case produces rate_limit_exceeded.

`model_not_found`

HTTP 404

For inference, this normally means no agent currently registers the exact model ID.

{
  "error": {
    "code": "model_not_found",
    "message": "No agents registered for this model",
    "source": "router"
  }
}

The model-detail endpoint also returns this code when the model exists but is not visible to the caller:

GET /v1/models/{model}

This prevents one tenant from discovering another tenant’s model catalog. Check:

curl \
  -H "Authorization: Bearer <hivenet-router-api-key>" \
  http://localhost:8080/v1/models \
  | jq -r '.data[].id'

Retry only after:

correcting the model ID
registering an agent
restoring the agent
changing the key’s access

`model_redirected`

HTTP 307
Location: https://<sibling-site>/v1/chat/completions?rt=<redirect token>

Returned only when cross-site redirect is enabled and no healthy agent on this site serves the model, while the sibling map lists another site that does. It applies to every inference endpoint, including /v1/embeddings and /v1/rerank. It is not a failure: the body explains the redirect to clients that do not follow redirects automatically.

{
  "error": {
    "code": "model_redirected",
    "message": "model llama-3-70b is served by site site-de-1: repeat this request at the Location URL (enable redirect following in your HTTP client)",
    "source": "router"
  }
}

Enable redirect following in the client, for example follow_redirects=True with httpx. An invalid X-Hivenet-Scope request header is refused with 400 request_invalid instead.

`model_waking`

HTTP 503
Retry-After: 30

Returned only for a request that hold at zero held, or refused to hold, for a model that the sibling map places on this site with scale_from_zero: true:

the model did not wake within the hold budget;
a hold limit is reached: per model, per API key and model, or in total;
the model woke, but the request still found no room in the admission gate before its hold budget ended. In this case a healthy agent already serves the model.

The message differs in each case; see model_waking for the three messages. Match on code, not on the message.

{
  "error": {
    "code": "model_waking",
    "message": "model llama-3-70b is waking up; retry in 30 seconds",
    "source": "router",
    "retry_after": 30
  }
}

retry_after repeats the Retry-After header, in seconds. Retry after that delay. To have the retry recognized as the same request, send a UUID of your own in X-Client-Request-ID (or X-Request-ID) and resend the same one; the router ignores an ID that is not a UUID. If the router already sent keepalive bytes to the held request, the status on the wire is 200 and this error is written in the body instead: as a final SSE event for a streaming request, as the JSON envelope after leading spaces otherwise.

Request-size rejection

HTTP `413`

Hivenet Router limits request bodies under /v1/* to:

10485760 bytes

by default, which is 10 MiB. A request above HIVENET_ROUTER_MAX_REQUEST_BYTES is rejected before it reaches the normal inference handler. That means the response body is not guaranteed to use Hivenet Router’s structured error envelope or a stable error.code. Handle this case using the HTTP status:

413 Request Entity Too Large

Reduce the request body, split large embedding or reranking batches, or change the configured byte limit. Set:

export HIVENET_ROUTER_MAX_REQUEST_BYTES=0

to disable the built-in limit, while accounting for any smaller reverse-proxy or ingress limit. Do not retry the unchanged request.

Quota errors

`rate_limit_exceeded`

HTTP 429

This code covers request-rate admission, serverless per-key admission, and a missing strict per-model quota declaration.

RPM bucket exhausted

{
  "error": {
    "code": "rate_limit_exceeded",
    "message": "rate limit exceeded",
    "source": "router"
  }
}

The response includes:

X-RateLimit-Remaining-Requests: 0

The RPM limiter refills continuously rather than resetting at the beginning of each wall-clock minute. Retry with bounded exponential backoff and jitter. Hivenet Router does not currently return a:

Retry-After

header, so the client must choose its own retry timing.

Serverless per-key limits

For a mode: serverless policy, this code also reports:

per-key occupancy share exceeded, please retry
input token rate exceeded, please retry
output token rate exceeded, please retry

These responses include Retry-After: 1. Occupancy and input-token limits reject the current request. Output tokens are charged after a response finishes, so an exhausted output bucket rejects a subsequent request.

Missing per-model quota declaration

A key using quota.per_model must enumerate every permitted model. An undeclared model returns:

{
  "error": {
    "code": "rate_limit_exceeded",
    "message": "no quota declared for model restricted-model on this API key",
    "source": "router"
  }
}

This is a configuration error, not temporary RPM exhaustion. Do not retry until the key has a complete entry such as:

quota:
  per_model:
    restricted-model:
      requests_per_minute_per_replica: 0
      tokens_per_day: 0

`token_limit_exceeded`

HTTP 429

Hivenet Router can reject a token budget at two stages.

Before routing

The router checks whether this worst case fits:

estimated input tokens
+ requested maximum output

A request may therefore be rejected even when it would probably generate a shorter answer. When output was reserved in the request, the message is:

{
  "error": {
    "code": "token_limit_exceeded",
    "message": "insufficient daily token budget for requested max_tokens",
    "source": "router"
  }
}

The response can include a nonzero remaining value:

X-RateLimit-Remaining-Tokens: 250

The request may succeed before the UTC reset when you reduce:

prompt size
max_tokens
max_completion_tokens

After non-streaming inference

The backend may complete the request before Hivenet Router knows the actual output-token count. When the completion no longer fits:

the response body is discarded
Hivenet Router returns 429
the prompt-token deduction remains
actual usage and the selected agent are preserved in the audit record

The response includes:

X-RateLimit-Remaining-Tokens: 0

A token_limit_exceeded response does not always mean that the backend performed no work.For a post-response rejection, inference has already completed.

Streaming responses

Once a stream has started, Hivenet Router cannot replace the HTTP 200 response with a later JSON 429. Streaming output is accounted for after delivery. When it exceeds the remaining budget, Hivenet Router records the quota event in metrics, but the client retains the already delivered response.

Admission errors

`input_too_long`

HTTP 400

This response is produced before routing when the prompt estimate exceeds 1.5× a policy’s max_input_tokens, or the image count exceeds images_max. Prompts closer to the limit are forwarded, and the backend answers context_length_exceeded if they do not fit:

{
  "error": {
    "code": "input_too_long",
    "message": "input is an estimated 200000 tokens, well over the model limit of 131072",
    "source": "router"
  }
}

Reduce the prompt or image count. Retrying an unchanged request will not help.

`concurrency_limit_exceeded`

HTTP 429
Retry-After: 1

This response is used when either:

the request would exceed the model’s token-weighted occupancy budget or max_inflight limit and capacity did not become available within HIVENET_ROUTER_ADMIT_PARK_TIMEOUT; or
aggregate healthy-pool engine pressure breached a configured shed_if threshold before queueing.

{
  "error": {
    "code": "concurrency_limit_exceeded",
    "message": "server at capacity for this model, please retry",
    "source": "router"
  }
}

Retry with a short randomized delay. See Admission control for footprint and budget calculation.

Routing and capacity errors

`no_agents_available`

HTTP 503

This is broader than “all agents are offline.” It means the full local policy chain ended without an eligible agent and no more specific model or capacity reason was available. Possible causes include:

agents offline or unhealthy
unhealthy inference backends
capability mismatch
static match filters
dynamic exclude_if gates
agents excluded after earlier failed attempts
a model-specific policy that matches no agents
all fallback steps exhausted

Messages can include:

All agents for this model are currently offline or unhealthy

or a more general policy-exhaustion explanation. Check:

curl \
  -H "Authorization: Bearer <admin-api-key>" \
  http://localhost:8080/admin/health \
  | jq .

Then inspect:

curl \
  -H "Authorization: Bearer <admin-api-key>" \
  http://localhost:8080/admin/routing-table \
  | jq .

Retry with backoff when the failure is caused by a temporary fleet or health condition. A policy or capability mismatch requires configuration changes instead.

`no_capacity`

HTTP 503

Healthy, filter-compatible agents exist, but every eligible agent has reached its declared concurrency capacity.

{
  "error": {
    "code": "no_capacity",
    "message": "All agents for this model are at maximum capacity",
    "source": "router"
  }
}

Before returning the error, Hivenet Router may:

wait in the per-model capacity queue
retry selection after a slot becomes available
advance through fallback steps

Retry after a short randomized delay. A sustained rate indicates that you should investigate:

agent --capacity
fleet size
request duration
engine queues
context and output lengths
per-model wait-queue depth

`agent_disconnected`

HTTP 503

The selected agent could not be reached over the libp2p request path. Possible causes include:

stale peer connection
unreachable announced address
firewall or NAT change
agent restart
lost network path
process termination

Hivenet Router normally handles this internally first. It grants one connection reset and redial attempt per agent without consuming the policy step’s normal try budget. Further failures can cause another agent or fallback step to be selected. The client may therefore receive a later error such as:

no_agents_available
no_capacity
request_timeout

rather than the original agent_disconnected. When the code does reach the client, retry with backoff and inspect the agent connection path.

`queue_full`

HTTP 503

The router’s global pending-request channel stayed full for five seconds.

{
  "error": {
    "code": "queue_full",
    "message": "Request queue is full, please retry",
    "source": "router"
  }
}

This is controlled by:

--queue-size

It is distinct from the per-model capacity wait queue. When the per-model queue is full, the routing session advances through fallback steps. That condition does not directly return queue_full; the final response is normally no_capacity or no_agents_available. Retry with exponential backoff and investigate:

request arrival rate
--queue-size
--max-concurrent
agent capacity
backend latency
router CPU and memory

`request_timeout`

HTTP 504

The router’s end-to-end deadline expired. The deadline can include:

waiting for processor concurrency
waiting in the per-model capacity queue
policy selection
retries
agent redial
agent forwarding
inference
provider fallback

Messages vary according to where the deadline expired:

Request expired in queue
Request timed out waiting for an available agent
Request deadline exceeded while contacting agent
Request timed out before provider fallback could be attempted
Request timeout

When the request’s own deadline expires while the router is forwarding to an agent, Hivenet Router treats it as a slow request rather than a broken agent. The router:

returns 504 with request_timeout
keeps the agent registered and does not reset its connection, so the agent’s other in-flight streams continue
does not retry the request on another agent
does not count the timeout against the agent’s success rate, failure streak, or SRTT

Instead, the router increments the agent’s consecutive_timeouts count. Any successful forward resets it to 0. The default policy excludes an agent once consecutive_timeouts reaches 3 or more, which removes an agent whose backend hangs from rotation. See Exclude hung agents to tune or disable this gate:

exclude_if:
  consecutive_timeouts:
    gte: 3

Other deadline-shaped failures, such as a timeout while opening the stream to the agent, are still handled as connection failures. Those evict the agent and fail over as described in agent_disconnected.

A timeout is an ambiguous result.The backend may have received or even completed the request before the client received the timeout. Hivenet Router does not deduplicate retries by X-Request-ID.

Retry only when the application operation is safe to repeat. For systematic timeouts, inspect the request path before increasing:

--request-timeout

Backend errors

`backend_unavailable`

HTTP 503

For native Chat Completions, this can represent:

backend connection failure
model still loading
backend HTTP 429
backend HTTP 503
temporary engine overload

Example:

{
  "error": {
    "code": "backend_unavailable",
    "message": "backend unreachable: connection refused",
    "source": "backend"
  }
}

Hivenet Router normally treats this as retryable inside the routing session. It can try:

another agent
another local fallback step
an external provider fallback

The final client-visible error may therefore be no_agents_available rather than backend_unavailable. Retry with backoff when the backend condition is temporary.

`backend_error`

HTTP 500 or 502

This is the catch-all for failures that Hivenet Router cannot classify more specifically. Possible causes include:

inference backend HTTP 500 or 502
unexpected backend response
invalid response JSON
provider fallback failure
unstructured error from a transparent proxy endpoint
storage or authentication-provider failure
another internal processing error

Example:

{
  "error": {
    "code": "backend_error",
    "message": "provider openai: upstream request failed",
    "source": "backend"
  }
}

A local agent backend error is normally retried inside the policy chain. An external provider fallback failure returns backend_error directly because no routing step remains after that fallback. Retry once only when:

the request is safe to repeat
the failure appears transient
retrying will not worsen an overloaded backend

Repeated errors with the same input require investigation.

How native Chat Completions errors are classified

The Hivenet Router agent classifies non-success responses from the native:

POST /v1/chat/completions

handler before returning them to the router.

JSON error types

The classifier recognizes both:

{
  "message": "Prompt is too long",
  "type": "context_length_exceeded"
}

and:

{
  "error": {
    "message": "Prompt is too long",
    "type": "context_length_exceeded"
  }
}

Known type mappings are:

Backend type	Hivenet Router code
`context_length_exceeded`	`context_length_exceeded`
`invalid_request_error`	`invalid_parameter`
`invalid_parameter`	`invalid_parameter`

A recognized type takes precedence over the HTTP-status fallback.

HTTP and text fallback

When the error type is not recognized:

Backend response	Hivenet Router code
`400` or `422` with context and length, window, or token wording	`context_length_exceeded`
Other `400` or `422`	`invalid_parameter`
`429` or `503`	`backend_unavailable`
Other non-success status	`backend_error`

For this native path, Hivenet Router extracts a message from the backend JSON where possible. Otherwise, it uses the raw body, truncated to 512 bytes.

A backend HTTP 429 is classified as backend_unavailable, not as Hivenet Router’s rate_limit_exceeded.rate_limit_exceeded describes Hivenet Router client-key quota admission.

Errors from transparent proxy endpoints

These paths use the agent’s generic transparent proxy:

POST /v1/messages
POST /v1/messages/count_tokens
POST /v1/embeddings
POST /v1/rerank

The agent initially forwards the backend’s status and body without applying the native Chat Completions classifier. When the router receives a non-success response:

It checks whether the body already contains a Hivenet Router structured error.
If it does, that code and source are preserved.
Otherwise, the router wraps the body as backend_error.

For example, an ordinary Anthropic-format backend error:

{
  "type": "error",
  "error": {
    "type": "invalid_request_error",
    "message": "max_tokens is required"
  }
}

does not match Hivenet Router’s own envelope:

{
  "error": {
    "code": "...",
    "message": "...",
    "source": "..."
  }
}

The final client response can therefore become:

HTTP 502
backend_error

rather than preserving the backend’s original HTTP 400.

Backend classification is currently more precise for native Chat Completions than for Messages, token counting, embeddings, and reranking.Inspect the backend log when one of the transparent endpoints returns a broad backend_error.

Internal routing retries

A backend or agent error is not necessarily returned to the client immediately. Hivenet Router distinguishes several categories.

Request-level errors

These stop the routing session immediately:

request_invalid
context_length_exceeded
invalid_parameter
token_limit_exceeded

Trying the same request on another agent would normally produce the same outcome.

Connection failures

A stale or disconnected agent connection receives one free redial attempt per request. That redial does not consume the policy step’s max_tries budget.

Retryable forwarding failures

Other agent and backend errors can:

mark the selected agent as failed in the current policy step
select another eligible agent
continue until max_tries is reached
advance through the fallback chain
use provider fallback where configured

Because of this, the client sees the final routing outcome rather than every internal attempt. For example:

backend_unavailable
  → another agent tried
  → fallback step exhausted
  → client receives no_agents_available

Use audit logs, metrics, traces, and backend logs when the final error does not reveal the first failure.

Streaming error behavior

A structured JSON error with a non-200 status can be returned only before response headers and body have started. After an SSE stream begins:

the HTTP status is already 200
some output may already have reached the client
Hivenet Router cannot replace the stream with a JSON error envelope

When the upstream agent stream is cut short mid-generation, Hivenet Router appends a terminal error frame before closing the stream. This covers an agent stream that fails partway through and a stream that closes without the backend’s terminal marker (finish_reason or [DONE] for OpenAI, message_stop for Anthropic). See stream_truncated for the frame format. If the upstream stream is cut in the middle of an SSE line, Hivenet Router drops the incomplete line rather than forwarding a broken event. The client receives only complete events, followed by the error frame. When the client itself disconnects, Hivenet Router does not send a frame and does not count the stream as truncated. A stream can still end without a terminal frame when the connection between the client and the router fails. The client should treat any stream that ends without the normal completion marker or with a stream_truncated frame as a failed or partial operation, even when the initial HTTP status was 200. Use the request ID to inspect the associated audit record, router logs, agent logs, and trace.

`stream_truncated`

HTTP 200, sent as the final SSE frame

Hivenet Router sends this frame when an upstream stream ended before completion. Everything delivered before the frame is partial output. On OpenAI-compatible streams, such as /v1/chat/completions, the frame is:

data: {"error":{"code":"stream_truncated","message":"upstream stream ended before completion; the response may be partial","type":"server_error"}}

On the Anthropic Messages endpoint, /v1/messages, the frame is an Anthropic error event with an api_error whose message starts with stream_truncated:

event: error
data: {"type":"error","error":{"type":"api_error","message":"stream_truncated: upstream stream ended before completion; the response may be partial"}}

Treat the frame as a retryable error, not as a complete answer. Discard or flag the partial output, and retry only when repeating the operation is safe. Hivenet Router counts each truncated stream in the hivenet_stream_truncated_total metric, labeled by model and reason. See Streaming responses for how the router relays SSE streams.

Error response headers

Hivenet Router can add the following headers.

Header	When present	Meaning
`X-Request-ID`	Every router response	The request ID the router generated, used for logs, audit records, and support correlation
`X-Client-Request-ID`	The client sent a valid UUID as its own ID	The client’s ID, echoed
`traceparent`	When a valid trace span exists	W3C trace context for Tempo or another tracing backend
`WWW-Authenticate`	`401` responses	Bearer authentication challenge
`X-RateLimit-Remaining-Requests`	Finite RPM quota	Remaining request-bucket capacity
`X-RateLimit-Remaining-Tokens`	Finite daily token budget	Remaining token budget known at that stage
`Retry-After`	`concurrency_limit_exceeded`	Retry delay in seconds; the current value is `1`
`Retry-After`	Serverless per-key `rate_limit_exceeded`	Retry delay in seconds; the current value is `1`

Unlimited quota values are omitted rather than represented as negative numbers. Hivenet Router does not provide a general:

Retry-After

header for every retryable error. It is set on admission-capacity rejection. Backend error headers are not guaranteed to be preserved when Hivenet Router converts a backend response into a structured error.

Correlate an error

Every router response includes:

X-Request-ID: 8b9e4a6d-dfb4-4b95-bada-319e7e5b878a

Log this value in the calling application. Search it in Loki:

{job="hivenet-router", log_type="audit"}
  | json
  | request_id =
      "8b9e4a6d-dfb4-4b95-bada-319e7e5b878a"

The audit record can show:

tenant
dynamic key ID
model
status
error code
selected agent
latency
token counts
trace ID

Use the trace ID in Tempo for the complete distributed request path.

Diagnose by status

HTTP `400`

Check:

request JSON
required model and message fields
context length
model-specific parameter support
tools and structured-output schemas
backend logs

HTTP `401`

Check:

raw bearer key
credential expiration
dynamic key state
correct client or administrator key type
reverse-proxy header forwarding

HTTP `403`

Check the explicit model allowlist.

HTTP `404`

Check:

endpoint path
base URL construction
exact model ID
client model visibility
registered agents

A plain 404 may mean an unsupported route. A structured model_not_found refers to model discovery or routing.

HTTP `413`

Reduce the request body or split a large batch. Check both HIVENET_ROUTER_MAX_REQUEST_BYTES and any reverse-proxy or ingress body limit.

HTTP `429`

Inspect:

error.code
error.message
X-RateLimit-Remaining-Requests
X-RateLimit-Remaining-Tokens

Distinguish:

RPM exhaustion
undeclared per-model quota
insufficient worst-case token budget
exhausted daily token budget

HTTP `503`

Inspect the specific code.

Code	First place to look
`no_agents_available`	Agent health, capability, policy filters, and gates
`no_capacity`	Agent load, declared capacity, and queues
`agent_disconnected`	libp2p address and network path
`backend_unavailable`	Inference backend health and startup
`queue_full`	Router concurrency and pending-request pressure

HTTP `504`

Check:

global queueing
per-model queueing
policy retries
connection resets
backend queue depth
model loading
prompt and output size
router request timeout

Client retry guidance

Before retrying, determine both:

whether the error is temporary
whether repeating the application operation is safe

Code	Default client action
`request_invalid`	Correct the request
`context_length_exceeded`	Reduce context or output
`invalid_parameter`	Remove or correct the rejected field
`input_too_long`	Reduce input text or image count
`unauthorized`	Correct or refresh credentials
`model_forbidden`	Change model or access configuration
`model_not_found`	Correct model or wait for deployment state to change
HTTP `413`	Reduce the request body or split the batch
`rate_limit_exceeded`	Back off for RPM exhaustion; change configuration for undeclared model
`token_limit_exceeded`	Reduce request size or wait for the UTC budget reset
`concurrency_limit_exceeded`	Honor `Retry-After` and retry with jitter
`no_agents_available`	Back off and inspect fleet or policy state
`no_capacity`	Retry after a short randomized delay
`agent_disconnected`	Retry with backoff after checking connectivity
`backend_unavailable`	Retry with exponential backoff
`queue_full`	Retry with exponential backoff
`backend_error`	Retry at most selectively and investigate repeated failures
`request_timeout`	Retry only when duplicate execution is acceptable
`stream_truncated`	Discard the partial output and retry when duplicate execution is acceptable

Hivenet Router does not use X-Request-ID as an idempotency key.Sending the same ID again creates another inference operation.

Official SDKs and frameworks may add their own automatic retries. Account for those retries when setting:

request quotas
application deadlines
maximum attempts
concurrency
audit expectations

Production handling

A production client should:

Parse structured errors

Parse structured errors when the body matches the Hivenet Router envelope.

Preserve the HTTP status and X-Request-ID

Preserve the HTTP status and X-Request-ID.

Handle ordinary HTTP errors when no structured envelope exists

Handle ordinary HTTP errors when no structured envelope exists.

Distinguish temporary capacity failures from request defects

Distinguish temporary capacity failures from request defects.

Apply bounded retries with exponential backoff and jitter

Apply bounded retries with exponential backoff and jitter.

Avoid retrying unsafe or ambiguous operations automatically

Avoid retrying unsafe or ambiguous operations automatically.

Log the code, status, source, request ID, and model

Log the code, status, source, request ID, and model.

Avoid logging credentials or complete sensitive prompts

Avoid logging credentials or complete sensitive prompts.

Alert on sustained error rates rather than individual transient failures

Alert on sustained error rates rather than individual transient failures.

Use audit records and traces for detailed investigation

Use audit records and traces for detailed investigation.

Next steps

Performance characteristics

Understand queueing, concurrency, latency, throughput, and benchmarking behavior.

Use from code

Handle Hivenet Router errors from Python, JavaScript, SDKs, and direct HTTP clients.

Routing concepts

See how retries, fallback steps, capacity, and policy exhaustion affect the final response.

API keys

Configure the request and token quotas behind 429 responses.

Admin endpoints

Diagnose fleet health, capacity, routing state, and dynamic key errors.

Audit logging

Correlate error codes with tenants, models, agents, request IDs, and traces.

​Response fields

​Error source

​Response-format scope

​Administration endpoints

​Unknown routes

​Error-code summary

​Request and authorization errors

​request_invalid

​context_length_exceeded

​invalid_parameter

​unauthorized

​model_forbidden

​model_not_found

​model_redirected

​model_waking

​Request-size rejection

​HTTP 413

​Quota errors

​rate_limit_exceeded

​RPM bucket exhausted

​Serverless per-key limits

​Missing per-model quota declaration

​token_limit_exceeded

​Before routing

​After non-streaming inference

​Streaming responses

​Admission errors

​input_too_long

​concurrency_limit_exceeded

​Routing and capacity errors

​no_agents_available

​no_capacity

​agent_disconnected

​queue_full

​request_timeout

​Backend errors

​backend_unavailable

​backend_error

​How native Chat Completions errors are classified

​JSON error types

​HTTP and text fallback

​Errors from transparent proxy endpoints

​Internal routing retries

​Request-level errors

​Connection failures

​Retryable forwarding failures

​Streaming error behavior

​stream_truncated

​Error response headers

​Correlate an error

​Diagnose by status

​HTTP 400

​HTTP 401

​HTTP 403

​HTTP 404

​HTTP 413

​HTTP 429

​HTTP 503

​HTTP 504

​Client retry guidance

​Production handling

​Next steps

Performance characteristics

Use from code

Routing concepts

API keys

Admin endpoints

Audit logging

Response fields

Error source

Response-format scope

Administration endpoints

Unknown routes

Error-code summary

Request and authorization errors

`request_invalid`

`context_length_exceeded`

`invalid_parameter`

`unauthorized`

`model_forbidden`

`model_not_found`

`model_redirected`

`model_waking`

Request-size rejection

HTTP `413`

Quota errors

`rate_limit_exceeded`

RPM bucket exhausted

Serverless per-key limits

Missing per-model quota declaration

`token_limit_exceeded`

Before routing

After non-streaming inference

Streaming responses

Admission errors

`input_too_long`

`concurrency_limit_exceeded`

Routing and capacity errors

`no_agents_available`

`no_capacity`

`agent_disconnected`

`queue_full`

`request_timeout`

Backend errors

`backend_unavailable`

`backend_error`

How native Chat Completions errors are classified

JSON error types

HTTP and text fallback

Errors from transparent proxy endpoints

Internal routing retries

Request-level errors

Connection failures

Retryable forwarding failures

Streaming error behavior

`stream_truncated`

Error response headers

Correlate an error

Diagnose by status

HTTP `400`

HTTP `401`

HTTP `403`

HTTP `404`

HTTP `413`

HTTP `429`

HTTP `503`

HTTP `504`

Client retry guidance

Production handling

Next steps