Reranking

Send a query and a set of candidate documents to a reranking model, then receive the documents ordered by relevance. Hivenet Router routes these requests to agents registered with the reranker capability.

POST /v1/rerank

Infinity is the documented reranking backend. Other engine types can register as rerankers when their backend exposes a compatible POST /v1/rerank endpoint. Ollama agents cannot use the reranker capability.

How reranking requests are routed

For each request, Hivenet Router:

authenticates the client when API authentication is enabled
applies the configured request-rate quota
reads and validates the top-level model field
checks that the API key may use that model
filters for agents registered with the reranker capability
applies routing policies and capacity checks
forwards the original request to POST /v1/rerank on the selected backend
returns the backend response to the client

The selected backend validates the query, documents, result count, and any optional reranking fields.

Request headers

Header	Required	Purpose
`Content-Type: application/json`	Yes	Identifies the JSON request
`Authorization: Bearer <api-key>`	When API auth is enabled	Authenticates the client
`X-Request-ID: <uuid>`	No	Correlates the request with logs and traces

Request body

A typical request contains:

{
  "model": "bge-reranker-large",
  "query": "What is the capital of France?",
  "documents": [
    "Paris is the capital of France.",
    "Berlin is the capital of Germany.",
    "France is located in Western Europe."
  ],
  "top_n": 2
}

Field	Type	Required	Default	Purpose
`model`	String	Yes	None	Model registered by a reranking agent
`query`	String	Yes	None	Text used to judge relevance
`documents`	Array of strings	Yes	None	Candidate documents to score
`top_n`	Integer	No	All documents	Maximum number of results to return

Hivenet Router validates the model field. The backend validates the other fields. The model name must:

match a model registered by a reranking agent
be allowed by the caller’s API key when model restrictions are enabled
contain no more than 256 characters

Request-body size

Hivenet Router limits /v1/* request bodies to 10485760 bytes, or 10 MiB, by default. The complete query and document array count toward this limit. Requests above the configured limit receive HTTP 413 before routing. Configure the limit with HIVENET_ROUTER_MAX_REQUEST_BYTES, or set it to 0 to disable the built-in check. A reverse proxy or ingress may enforce a smaller limit. Reduce the number or size of documents when a request receives 413. Do not depend on a particular JSON error body for this early rejection.

Basic request

curl -X POST \
  http://localhost:8080/v1/rerank \
  -H "Content-Type: application/json" \
  -d '{
    "model": "bge-reranker-large",
    "query": "What is the capital of France?",
    "documents": [
      "Paris is the capital of France.",
      "Berlin is the capital of Germany.",
      "France is located in Western Europe."
    ],
    "top_n": 2
  }'

With API authentication enabled:

curl -X POST \
  http://localhost:8080/v1/rerank \
  -H "Authorization: Bearer <api-key>" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "bge-reranker-large",
    "query": "What is the capital of France?",
    "documents": [
      "Paris is the capital of France.",
      "Berlin is the capital of Germany.",
      "France is located in Western Europe."
    ],
    "top_n": 2
  }'

Response

A successful response resembles:

{
  "object": "rerank",
  "results": [
    {
      "relevance_score": 0.9997,
      "index": 0,
      "document": null
    },
    {
      "relevance_score": 0.0212,
      "index": 2,
      "document": null
    }
  ],
  "model": "bge-reranker-large",
  "usage": {
    "prompt_tokens": 151,
    "total_tokens": 151
  },
  "id": "infinity-6a911ac9-d522-4fa9-8234-5b65580bc2a5",
  "created": 1776870533
}

Field	Purpose
`results`	Ranked results, normally ordered by descending relevance
`relevance_score`	Backend-generated relevance score
`index`	Position of the document in the original request array
`document`	Document content when returned by the backend; often `null` by default
`model`	Model that produced the ranking
`usage`	Token information when provided by the backend

The exact fields depend on the backend. Hivenet Router returns the successful response without changing result scores or ordering.

Use the index field to recover the original document. Do not assume that document will contain the text.

Use Python

from typing import Any

import requests

documents = [
    "Random Forest is a supervised learning algorithm.",
    "Paris is a city in France.",
    "Neural networks are used for deep learning.",
]

response = requests.post(
    "http://localhost:8080/v1/rerank",
    headers={
        "Content-Type": "application/json",
    },
    json={
        "model": "bge-reranker-large",
        "query": "machine learning algorithms",
        "documents": documents,
        "top_n": 2,
    },
    timeout=120,
)

response.raise_for_status()

payload: dict[str, Any] = response.json()

for result in payload["results"]:
    index = result["index"]
    score = result["relevance_score"]

    print(f"{score:.4f}: {documents[index]}")

With authentication:

headers = {
    "Authorization": "Bearer <api-key>",
    "Content-Type": "application/json",
}

Use reranking in a search pipeline

Reranking normally follows a broader retrieval step. A search system might first retrieve 50 approximate matches from a vector database, then ask a reranker to select the 10 most relevant documents.

from dataclasses import dataclass

import requests


@dataclass
class SearchResult:
    text: str
    source: str


candidates: list[SearchResult] = search_index.query(
    "machine learning",
    top_k=50,
)

response = requests.post(
    "http://localhost:8080/v1/rerank",
    json={
        "model": "bge-reranker-large",
        "query": "machine learning",
        "documents": [
            candidate.text
            for candidate in candidates
        ],
        "top_n": 10,
    },
    timeout=120,
)

response.raise_for_status()

ranked_candidates = [
    {
        "result": candidates[item["index"]],
        "relevance_score": item["relevance_score"],
    }
    for item in response.json()["results"]
]

This preserves the metadata held by the search system while using the reranker’s indices and scores to reorder the candidates.

Additional request fields

Hivenet Router forwards the original JSON request to the backend. You can include additional fields supported by the selected reranking service, but Hivenet Router does not interpret or normalize them. For example:

{
  "model": "bge-reranker-large",
  "query": "What is the capital of France?",
  "documents": [
    "Paris is the capital of France.",
    "Berlin is the capital of Germany."
  ],
  "top_n": 1,
  "return_documents": true
}

Whether return_documents or another optional field works depends entirely on the backend. Unsupported fields may be ignored or rejected.

No streaming support

Reranking requests are synchronous. Hivenet Router waits for the selected backend to score the documents and return the complete result set. The endpoint does not use server-sent events. For large candidate sets, keep the batch within the backend’s limits and configure an appropriate request timeout.

Authentication and model access

When API authentication is enabled, send:

Authorization: Bearer <api-key>

An API key can restrict access to specific models. A request for a model outside the key’s allowlist returns HTTP 403:

{
  "error": {
    "code": "model_forbidden",
    "message": "your API key does not have access to model: bge-reranker-large",
    "source": "router"
  }
}

When authentication is disabled, no authorization header is required.

Quotas

Reranking requests participate in request-rate quota enforcement. When a finite request quota is configured, responses may include:

X-RateLimit-Remaining-Requests

When the request-rate quota is exhausted, Hivenet Router returns HTTP 429.

The current reranking path does not charge query or document tokens against the daily token quota. Successful reranking requests are recorded with zero input and output tokens for quota accounting.Reranking requests are also exempt from the LLM admission gates, including per-request token caps, KV-occupancy budgets, and serverless token rates. These gates model KV-cache-bound generation. Request-rate, body-size, and agent-concurrency limits still protect this path.

Routing policies

Reranking agents use the same routing pipeline as other Hivenet Router workloads. For example:

routing_policy:
  match:
    engine: infinity
    region: EU-France
  exclude_if:
    success_rate:
      lt: 0.95
  strategy: least-loaded

Capability filtering happens automatically before policy ranking. A reranking request cannot be routed to an agent registered as llm or embedding. Fallback chains can route across several compatible reranking agents that expose the same model.

Discover reranking models

List the reranking models available through the router:

curl http://localhost:8080/v1/models \
  | jq '.data[]
      | select(.capability == "reranker")
      | {
          id,
          capability,
          agents
        }'

Inspect registered reranking agents:

curl http://localhost:8080/admin/routing-table \
  | jq '.agents[]
      | select(.metadata.capability == "reranker")
      | {
          peer_id,
          model: .metadata.model,
          engine: .metadata.engine,
          region: .metadata.region,
          capacity: .metadata.capacity,
          is_healthy: .status.healthy
        }'

The administration endpoint may require separate credentials depending on the router’s admin-auth configuration.

Supported agent configurations

Start the agent with:

--capability reranker

Hivenet Router accepts the reranker capability with:

Infinity
vLLM
SGLang
llama.cpp
custom engines

The selected backend must expose:

POST /v1/rerank

Ollama does not expose the required reranking endpoint. Hivenet Router rejects an agent started with both --engine ollama and --capability reranker.

For the documented Infinity setup, see Infinity agent.

Error responses

Router errors use this envelope:

{
  "error": {
    "code": "error_code",
    "message": "Human-readable explanation",
    "source": "router"
  }
}

Backend errors use:

{
  "error": {
    "code": "backend_error",
    "message": "Backend response or explanation",
    "source": "backend"
  }
}

Common responses include:

HTTP status	Code	Meaning
`400`	`request_invalid`	Missing model, malformed JSON, or model name too long
`413`	Not guaranteed	Request body exceeds `HIVENET_ROUTER_MAX_REQUEST_BYTES`
`401`	`unauthorized`	Missing or invalid API credentials
`403`	`model_forbidden`	API key cannot use the requested model
`404`	`model_not_found`	No registered model matches the request
`429`	`rate_limit_exceeded`	Request-rate quota exhausted
`503`	`no_agents_available`	No healthy matching reranking agent
`503`	`no_capacity`	Matching agents are at capacity
`503`	`queue_full`	The router queue remained full
`503`	`agent_disconnected`	The selected agent became unreachable
`502`	`backend_error`	Backend rejected or could not process the request
`504`	`request_timeout`	The request exceeded its deadline

See Error codes for the complete reference.

Troubleshooting

The model is not found

List available reranking models:

curl http://localhost:8080/v1/models \
  | jq '.data[]
      | select(.capability == "reranker")'

Check that:

the agent uses --capability reranker
the requested model matches the registered model exactly
the agent is healthy
the API key may access the model

The request reaches the wrong capability

Inspect the agent registration:

curl http://localhost:8080/admin/routing-table \
  | jq '.agents[]
      | select(.metadata.model == "bge-reranker-large")
      | {
          model: .metadata.model,
          capability: .metadata.capability,
          engine: .metadata.engine
        }'

The capability must be:

reranker

The backend rejects the request

Test it directly:

curl -X POST \
  http://localhost:7997/v1/rerank \
  -H "Content-Type: application/json" \
  -d '{
    "model": "bge-reranker-large",
    "query": "Test query",
    "documents": [
      "First document",
      "Second document"
    ],
    "top_n": 2
  }'

A backend may reject:

an empty query
an empty document list
unsupported document formats
too many documents
documents above the model’s token limit
an invalid top_n
unsupported optional fields

The response contains `document: null`

This is expected for backends that do not return document text by default. Use each result’s index to retrieve the original document from the input array.

Results do not contain every document

Check top_n. For example:

{
  "top_n": 2
}

returns at most two results, even when the request contains more documents. When top_n is omitted, the documented Infinity behavior is to return all ranked documents.

Requests time out

Reduce the number or size of candidate documents, or increase the agent’s backend timeout:

--http-timeout 10m

Also check:

backend model loading state
declared agent capacity
accelerator memory
competing embedding or reranking traffic
backend batch configuration

The request is rate-limited

Check:

X-RateLimit-Remaining-Requests

Review the API key’s request-rate quota and any per-model quota configuration.

Next steps

Models

Discover available models, capabilities, engines, and healthy capacity.

Infinity agent

Configure embedding and reranking models through Infinity.

Routing concepts

Learn how reranking agents are filtered, ranked, and used in fallback chains.

Getting started

Deploy Hivenet Router

Use the API

Routing and policies

Security and auth

Observability

Integrations

Reference

Project

How reranking requests are routed

Request headers

Request body

Request-body size

Basic request

Response

Use Python

Use reranking in a search pipeline

Additional request fields

No streaming support

Authentication and model access

Quotas

Routing policies

Discover reranking models

Supported agent configurations

Error responses

Troubleshooting

The model is not found

The request reaches the wrong capability

The backend rejects the request

The response contains `document: null`

Results do not contain every document

Requests time out

The request is rate-limited

Next steps

Models

Infinity agent

Routing concepts

​How reranking requests are routed

​Request headers

​Request body

​Request-body size

​Basic request

​Response

​Use Python

​Use reranking in a search pipeline

​Additional request fields

​No streaming support

​Authentication and model access

​Quotas

​Routing policies

​Discover reranking models

​Supported agent configurations

​Error responses

​Troubleshooting

​The model is not found

​The request reaches the wrong capability

​The backend rejects the request

​The response contains document: null

​Results do not contain every document

​Requests time out

​The request is rate-limited

​Next steps

Models

Infinity agent

Routing concepts

How reranking requests are routed

Request headers

Request body

Request-body size

Basic request

Response

Use Python

Use reranking in a search pipeline

Additional request fields

No streaming support

Authentication and model access

Quotas

Routing policies

Discover reranking models

Supported agent configurations

Error responses

Troubleshooting

The model is not found

The request reaches the wrong capability

The backend rejects the request

The response contains `document: null`

Results do not contain every document

Requests time out

The request is rate-limited

Next steps