Chat completions and messages

Hivenet Router exposes OpenAI-compatible Chat Completions and Anthropic-compatible Messages endpoints through the same routing pipeline. The router reads the top-level model field, selects a healthy language-model agent, and forwards the original request to the same path on that agent’s backend.

OpenAI Chat Completions

POST /v1/chat/completions

Anthropic Messages

POST /v1/messages

Anthropic token counting

POST /v1/messages/count_tokens

The selected backend must support the endpoint used by the client. Hivenet Router does not translate between the OpenAI and Anthropic request formats.

How passthrough works

For each request, Hivenet Router follows the same pipeline:

Authenticate and apply quotas

Authenticate the client when API authentication is enabled, then apply configured request and token quotas.

Resolve and authorize the model

Read and validate the top-level model field, then confirm that the API key may use it.

Apply admission control

Check the model’s request caps, pressure shed, occupancy budget, and serverless per-key limits.

Select and forward

Select a healthy llm agent through the routing policy and forward the original body to the same backend path.

Return the response

Return the backend response to the client.

Apart from the fields needed for routing and quota estimation, Hivenet Router does not impose its own generation schema or defaults. Parameters such as sampling controls, tools, structured output, multimodal input, and backend-specific extensions work only when the selected backend supports them.

Allowed passthrough paths

Hivenet Router only forwards an explicit set of inference paths:

/v1/chat/completions
/v1/messages
/v1/messages/count_tokens

Other backend paths are not exposed through the router. This prevents clients from reaching backend administration, scaling, loading, metrics, or other control-plane endpoints through the public inference API.

Request requirements

Every request must contain a non-empty top-level model field:

{
  "model": "meta-llama/Llama-3.1-8B-Instruct"
}

The model name:

must match a model registered by an agent
must not exceed 256 characters
must be allowed by the caller’s API key when model restrictions are enabled

Malformed JSON, a missing model, or an overlong model name is rejected before routing.

Request-body size

Hivenet Router limits /v1/* request bodies to 10485760 bytes, or 10 MiB, by default. Requests above the configured limit are rejected with HTTP 413 before model authorization, quota accounting, queueing, or backend forwarding. Configure the limit with:

export HIVENET_ROUTER_MAX_REQUEST_BYTES=<bytes>

Set it to 0 to disable the built-in limit. A reverse proxy or ingress can still enforce a smaller limit. Do not depend on a particular JSON error body for 413 responses. Handle the HTTP status directly, because the rejection occurs before the normal inference-handler error path.

Request headers

Header	Required	Purpose
`Content-Type: application/json`	Recommended	Identifies the JSON request body
`Authorization: Bearer <api-key>`	When API auth is enabled	Authenticates the client
`X-Request-ID: <uuid>`	No	Correlates requests with logs and traces

If X-Request-ID contains a valid UUID, Hivenet Router preserves it. If it is missing or invalid, the router creates a new UUID. The value is returned in the response. Hivenet Router may also return a W3C traceparent header when tracing is enabled.

OpenAI Chat Completions

Send requests to:

POST /v1/chat/completions

Basic request

curl -X POST \
  http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "meta-llama/Llama-3.1-8B-Instruct",
    "messages": [
      {
        "role": "system",
        "content": "Answer clearly and briefly."
      },
      {
        "role": "user",
        "content": "What does an inference router do?"
      }
    ]
  }'

curl -X POST \
  http://localhost:8080/v1/chat/completions \
  -H "Authorization: Bearer <api-key>" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "meta-llama/Llama-3.1-8B-Instruct",
    "messages": [
      {
        "role": "user",
        "content": "Hello"
      }
    ]
  }'

Fields used by Hivenet Router

Hivenet Router forwards the complete request, but it reads a small number of fields for routing and quota enforcement.

Field	How Hivenet Router uses it
`model`	Selects matching agents and applies model restrictions
`messages`	Estimates input use for quotas and admission; image parts count toward `images_max`
`system`	Includes an Anthropic top-level system prompt in the input estimate
`tools`	Includes raw tool-definition JSON in the input estimate
`max_completion_tokens`	Reserves the declared output footprint for occupancy and checks the daily budget
`max_tokens`	Used as a fallback when `max_completion_tokens` is absent
`stream`	Enables progressive streaming through the router and agent

All other fields are passed to the backend without Hivenet Router assigning its own default values.

The selected backend decides which request fields, message roles, modalities, tools, and sampling values it accepts.

Multimodal message content

Hivenet Router can parse message content supplied as plain text or as content-part arrays containing text, image URLs, or input audio. For example:

{
  "model": "vision-model",
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "What is shown in this image?"
        },
        {
          "type": "image_url",
          "image_url": {
            "url": "https://example.com/image.jpg"
          }
        }
      ]
    }
  ]
}

The backend must support the requested modality. Hivenet Router routes and forwards the request but does not add vision or audio support to a text-only engine.

Non-streaming responses

For a non-streaming request, Hivenet Router returns the response produced by the selected backend. A typical OpenAI-compatible response resembles:

{
  "id": "chat-123",
  "object": "chat.completion",
  "created": 1760000000,
  "model": "meta-llama/Llama-3.1-8B-Instruct",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "An inference router directs requests to an available model server."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 20,
    "completion_tokens": 13,
    "total_tokens": 33
  }
}

The exact fields and values depend on the backend. If the backend omits token usage from a successful non-streaming response, Hivenet Router estimates prompt and completion tokens for accounting and audit records.

Streaming responses

Set:

{
  "stream": true
}

and use a client that reads server-sent events progressively.

curl -N -X POST \
  http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "meta-llama/Llama-3.1-8B-Instruct",
    "stream": true,
    "messages": [
      {
        "role": "user",
        "content": "Count from one to five."
      }
    ]
  }'

Hivenet Router forwards streaming chunks as they arrive from the backend rather than buffering the full response. A typical stream resembles:

data: {"choices":[{"delta":{"content":"One"}}]}

data: {"choices":[{"delta":{"content":", two"}}]}

data: {"choices":[{"delta":{},"finish_reason":"stop"}]}

data: [DONE]

The exact event structure comes from the backend.

Hivenet Router can relay a backend’s SSE stream, but it cannot convert a non-streaming backend response into streaming output.

Anthropic Messages

Send Anthropic-compatible requests to:

POST /v1/messages

The request is routed using its top-level model and forwarded unchanged to /v1/messages on the selected backend.

curl -X POST \
  http://localhost:8080/v1/messages \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <api-key>" \
  -d '{
    "model": "Qwen/Qwen3.6-27B",
    "max_tokens": 256,
    "messages": [
      {
        "role": "user",
        "content": "Write a haiku about routing."
      }
    ]
  }'

The backend must natively implement the Anthropic Messages endpoint. If it does not, its error is returned through Hivenet Router. Model names must match both:

the name registered by the Hivenet Router agent
the name accepted by the backend

Use Claude Code

Point Claude Code at the router:

unset ANTHROPIC_API_KEY

export ANTHROPIC_BASE_URL="https://<router-host>"
export ANTHROPIC_AUTH_TOKEN="<hivenet-router-api-key>"
export ANTHROPIC_DEFAULT_OPUS_MODEL="Qwen/Qwen3.6-27B"
export ANTHROPIC_DEFAULT_SONNET_MODEL="Qwen/Qwen3.6-35B-A3B"

claude

The router should be exposed through HTTPS before using it outside a trusted network. See Claude Code for the complete setup and compatibility guidance.

Count Anthropic input tokens

Hivenet Router also allows:

POST /v1/messages/count_tokens

The request uses the same top-level model field and is routed to a backend that supports the token-counting endpoint.

curl -X POST \
  http://localhost:8080/v1/messages/count_tokens \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <api-key>" \
  -d '{
    "model": "Qwen/Qwen3.6-27B",
    "messages": [
      {
        "role": "user",
        "content": "How many tokens are in this request?"
      }
    ]
  }'

A compatible backend returns:

{
  "input_tokens": 11
}

This endpoint counts input tokens without running model inference.

Authentication and model access

When API authentication is enabled, send:

Authorization: Bearer <api-key>

An API key can restrict access to specific models. A request for a model outside the key’s allowlist returns HTTP 403:

{
  "error": {
    "code": "model_forbidden",
    "message": "your API key does not have access to model: restricted-model",
    "source": "router"
  }
}

When authentication is disabled, requests do not need an authorization header.

Request and token quotas

API keys can enforce:

requests per minute
daily token budgets
separate quotas for individual models
on serverless policies, input and output tokens per minute and a per-key occupancy share

Before routing, Hivenet Router checks the request-rate quota. Independently of key quotas, LLM generation also passes the model’s admission gates: per-request input and image caps, live pool-pressure shedding, and replica-scaled occupancy limits. These protect the serving pool even when the key has unlimited quotas. When a daily token budget is configured, it also estimates the prompt and checks whether the remaining budget can cover:

estimated input tokens + requested maximum output tokens

max_completion_tokens is used when present. Otherwise Hivenet Router uses max_tokens. The estimated input tokens are charged at admission. Actual completion tokens from local responses are charged after the backend responds.

Hivenet Router uses one per-model learned estimate for request caps, occupancy, serverless input rates, and daily input admission. It includes message text, the Anthropic top-level system prompt, and tool-definition JSON. Exact OpenAI or Anthropic backend usage corrects the active reservation and trains the model-specific ratio. Images are bounded separately by images_max and are not used to train the text ratio.

POST /v1/messages/count_tokens is exempt from the admission and token-quota checks because it performs no generation. The request-per-minute limiter still protects it from flooding.

Rate-limit headers

When a finite quota is configured, responses may include:

X-RateLimit-Remaining-Requests
X-RateLimit-Remaining-Tokens

Header	Meaning
`X-RateLimit-Remaining-Requests`	Requests remaining in the active per-minute bucket
`X-RateLimit-Remaining-Tokens`	Tokens remaining in the active daily bucket

Unlimited quotas do not produce a remaining-value header. When a request-rate bucket is exhausted, Hivenet Router returns HTTP 429 with X-RateLimit-Remaining-Requests: 0. A token-budget rejection can still report a nonzero X-RateLimit-Remaining-Tokens value when the remaining budget is positive but too small for the estimated prompt plus requested maximum output. Post-response token rejection reports 0.

Error responses

Router errors use this envelope:

{
  "error": {
    "code": "error_code",
    "message": "Human-readable explanation",
    "source": "router"
  }
}

{
  "error": {
    "code": "error_code",
    "message": "Human-readable explanation",
    "source": "backend"
  }
}

Common responses include:

HTTP status	Example code	Meaning
`400`	`request_invalid`	Malformed request or missing model
`400`	`input_too_long`	Prompt estimate exceeds `max_input_tokens`, or the request exceeds `images_max`
`400`	`context_length_exceeded`	Backend rejected the context length
`413`	Not guaranteed	Request body exceeds `HIVENET_ROUTER_MAX_REQUEST_BYTES`
`401`	`unauthorized`	Missing or invalid credentials
`403`	`model_forbidden`	API key cannot use the requested model
`404`	`model_not_found`	No registered model matches the request
`429`	`rate_limit_exceeded`	Request rate or a serverless per-key token or occupancy limit is exhausted
`429`	`concurrency_limit_exceeded`	Pool occupancy, `max_inflight`, or front-door pressure shed rejected the request
`429`	`token_limit_exceeded`	Daily token budget exhausted
`503`	`no_agents_available`	No healthy matching agent
`503`	`no_capacity`	Matching agents are at capacity
`503`	`queue_full`	Router request queue remained full
`504`	`request_timeout`	Request exceeded the router timeout

See Error codes for the complete reference.

Next steps

Embeddings

Generate vectors through capability-specific embedding agents.

Models

Discover which models and capabilities are currently available.

API keys

Configure client authentication, model access, and quotas.

Getting started

Deploy Hivenet Router

Use the API

Routing and policies

Security and auth

Observability

Integrations

Reference

Project

Chat completions and messages

OpenAI Chat Completions

Anthropic Messages

Anthropic token counting

How passthrough works

Allowed passthrough paths

Request requirements

Request-body size

Request headers

OpenAI Chat Completions

Basic request

Fields used by Hivenet Router

Multimodal message content

Non-streaming responses

Streaming responses

Anthropic Messages

Use Claude Code

Count Anthropic input tokens

Authentication and model access

Request and token quotas

Rate-limit headers

Error responses

Next steps

Embeddings

Models

API keys

OpenAI Chat Completions

Anthropic Messages

Anthropic token counting

​How passthrough works

​Allowed passthrough paths

​Request requirements

​Request-body size

​Request headers

​OpenAI Chat Completions

​Basic request

​Fields used by Hivenet Router

​Multimodal message content

​Non-streaming responses

​Streaming responses

​Anthropic Messages

​Use Claude Code

​Count Anthropic input tokens

​Authentication and model access

​Request and token quotas

​Rate-limit headers

​Error responses

​Next steps

Embeddings

Models

API keys

How passthrough works

Allowed passthrough paths

Request requirements

Request-body size

Request headers

OpenAI Chat Completions

Basic request

Fields used by Hivenet Router

Multimodal message content

Non-streaming responses

Streaming responses

Anthropic Messages

Use Claude Code

Count Anthropic input tokens

Authentication and model access

Request and token quotas

Rate-limit headers

Error responses

Next steps