# Hivenet Router

> Open-source routing for self-hosted and distributed AI inference.

## Docs

- [Hivenet Router](https://routerdocs.hivenet.com/index.md): Open-source routing for self-hosted and distributed AI inference.
- [Introduction to Hivenet Router](https://routerdocs.hivenet.com/getting-started/introduction.md): Learn what Hivenet Router does, when to use it, and how it fits around your inference engines.
- [Why Hivenet Router](https://routerdocs.hivenet.com/getting-started/why-hivenet-router.md): Understand when a self-hosted inference router is the right fit and what Hivenet Router adds between clients and model servers.
- [Quickstart](https://routerdocs.hivenet.com/quickstart.md): Deploy a Hivenet Router router and multiple agents across several machines, then send and observe your first routed inference request.
- [Architecture overview](https://routerdocs.hivenet.com/getting-started/architecture-overview.md): Understand how the router, agents, control plane, data plane, storage, authentication, and routing pipeline fit together.
- [Hivenet Router, Hivenet Inference API, and Compute with Hivenet](https://routerdocs.hivenet.com/getting-started/hivenet-services.md): Understand how Hivenet Router differs from Hivenet Inference API and Compute with Hivenet, and choose the right option for your workload.
- [Docker quickstart](https://routerdocs.hivenet.com/deploy/docker-quickstart.md): Deploy a Hivenet Router router and two vLLM agents across Linux hosts with Docker, then verify routing and metrics.
- [Docker Compose](https://routerdocs.hivenet.com/deploy/docker-compose.md): Use Docker Compose to run the Hivenet Router router with its full observability stack, then connect agents from separate inference hosts.
- [Bare-metal deployment](https://routerdocs.hivenet.com/deploy/bare-metal.md): Run the Hivenet Router router and agents as native Linux services managed by systemd.
- [vLLM agent](https://routerdocs.hivenet.com/deploy/agents/vllm.md): Connect a vLLM backend to Hivenet Router, configure model discovery and capacity, and expose vLLM metrics for routing and observability.
- [Ollama agent](https://routerdocs.hivenet.com/deploy/agents/ollama.md): Connect an Ollama backend to Hivenet Router, select the model an agent represents, and configure capacity for local or edge inference.
- [SGLang agent](https://routerdocs.hivenet.com/deploy/agents/sglang.md): Connect an SGLang backend to Hivenet Router and expose cache, queue, and time-to-first-token metrics for routing.
- [llama.cpp agent](https://routerdocs.hivenet.com/deploy/agents/llama-cpp.md): Connect a llama.cpp server to Hivenet Router, set a stable model name, and expose cache, latency, and throughput metrics.
- [Infinity agent](https://routerdocs.hivenet.com/deploy/agents/infinity.md): Connect Infinity embedding and reranking models to Hivenet Router by running one capability-specific agent for each model.
- [Custom engine agent](https://routerdocs.hivenet.com/deploy/agents/custom-engine.md): Connect an OpenAI-compatible chat-completion backend to Hivenet Router using an explicit model name and health endpoint.
- [Chat completions and messages](https://routerdocs.hivenet.com/use-the-api/chat-completions.md): Send OpenAI-compatible chat-completion and Anthropic Messages requests through Hivenet Router, with streaming, routing, authentication, and quota enforcement.
- [Embeddings](https://routerdocs.hivenet.com/use-the-api/embeddings.md): Generate vector embeddings through capability-specific agents, with batch requests, routing, authentication, and error handling.
- [Reranking](https://routerdocs.hivenet.com/use-the-api/reranking.md): Score and reorder candidate documents by relevance through capability-specific reranking agents.
- [Models](https://routerdocs.hivenet.com/use-the-api/models.md): List models available to the current API key and inspect their capabilities, agent distribution, health, and declared capacity.
- [Admin endpoints](https://routerdocs.hivenet.com/use-the-api/admin-endpoints.md): Inspect router health, models, routing state, storage, policies, metrics, and dynamic API keys through protected operator endpoints.
- [Routing concepts](https://routerdocs.hivenet.com/routing/routing-concepts.md): Understand how Hivenet Router filters, gates, ranks, retries, queues, and falls back across inference agents.
- [Policy YAML reference](https://routerdocs.hivenet.com/routing/policy-yaml-reference.md): Configure global and per-model routing policies, static filters, metric gates, fallback chains, and provider fallback.
- [Fallback chains](https://routerdocs.hivenet.com/routing/fallback-chains.md): Configure ordered local routing alternatives with independent filters, metric gates, retry budgets, and capacity handling.
- [Provider fallback](https://routerdocs.hivenet.com/routing/provider-fallback.md): Use OpenAI or Anthropic as a final fallback after every local routing step has been exhausted.
- [Policy gates](https://routerdocs.hivenet.com/routing/policy-gates.md): Diagnose the seven routing gates and apply practical thresholds for health, capacity, latency, engine, and hardware signals.
- [Admission control](https://routerdocs.hivenet.com/routing/admission-control.md): Configure request caps, pool pressure shedding, token-weighted occupancy, and serverless per-key limits.
- [Authentication overview](https://routerdocs.hivenet.com/security/authentication-overview.md): Understand how Hivenet Router authenticates agents, API clients, and administrators across separate trust boundaries.
- [API keys](https://routerdocs.hivenet.com/security/api-keys.md): Generate, configure, use, reload, and manage static or dynamic API keys, model access, and request and token quotas.
- [auth.yaml reference](https://routerdocs.hivenet.com/security/auth-yaml-reference.md): Configure static client authentication, administrator access, model restrictions, expiration, and flat or per-model quotas.
- [Model restrictions](https://routerdocs.hivenet.com/security/model-restrictions.md): Limit each API key to specific models and keep model discovery, inference access, and quotas aligned across tenants.
- [Key rotation](https://routerdocs.hivenet.com/security/key-rotation.md): Rotate client, administrator, agent, and provider credentials with the least possible disruption.
- [Observability overview](https://routerdocs.hivenet.com/observability/observability-overview.md): Understand Hivenet Router's health checks, operational state, metrics, dashboards, audit records, and distributed tracing.
- [Prometheus metrics](https://routerdocs.hivenet.com/observability/prometheus-metrics.md): Scrape Hivenet Router metrics for routing, agents, engines, hardware, policies, queues, tenants, quotas, and HTTP traffic.
- [Grafana dashboards](https://routerdocs.hivenet.com/observability/grafana-dashboards.md): Use the provisioned Grafana dashboards to inspect router health, agents, hardware, engine behavior, tenant quotas, audit logs, and traces.
- [Audit logging](https://routerdocs.hivenet.com/observability/audit-logging.md): Record structured request metadata, route audit files through Promtail and Loki, and correlate requests with logs and traces.
- [Hardware metrics](https://routerdocs.hivenet.com/observability/hardware-metrics.md): Understand how Hivenet Router collects, reports, and uses GPU, CPU, and system-memory metrics from agent hosts.
- [Engine metrics](https://routerdocs.hivenet.com/observability/engine-metrics.md): Understand how Hivenet Router collects cache, queue, latency, request-shape, finish-reason, and throughput metrics from supported inference engines.
- [Latency tracking](https://routerdocs.hivenet.com/observability/latency-tracking.md): Understand how Hivenet Router measures router-observed request latency and calculates per-agent SRTT and RTTVAR.
- [Hardware-aware routing](https://routerdocs.hivenet.com/observability/hardware-aware-routing.md): Route across hardware tiers and exclude agents under GPU, memory, CPU, cache, thermal, or latency pressure.
- [Admission control metrics](https://routerdocs.hivenet.com/observability/admission-control-metrics.md): Monitor admission-control occupancy, budgets, concurrency, and request rejections by gate.
- [Integrations overview](https://routerdocs.hivenet.com/integrations/integrations-overview.md): Choose the right guide for connecting coding agents, browser interfaces, SDKs, and custom applications to Hivenet Router.
- [Claude Code](https://routerdocs.hivenet.com/integrations/claude-code.md): Connect Claude Code to Hivenet Router through the Anthropic Messages API and configure authentication, model aliases, streaming, and tool use.
- [OpenCode](https://routerdocs.hivenet.com/integrations/open-code.md): Connect OpenCode to Hivenet Router through an OpenAI-compatible provider and configure models, authentication, tool use, permissions, and privacy controls.
- [Pi](https://routerdocs.hivenet.com/integrations/pi.md): Connect the Pi coding agent to Hivenet Router through OpenAI Chat Completions and configure models, authentication, tools, security, and compatibility.
- [Open WebUI](https://routerdocs.hivenet.com/integrations/open-web-ui.md): Connect Open WebUI to Hivenet Router for browser-based chat, shared access, tool use, background tasks, embeddings, and reranking.
- [Use from code](https://routerdocs.hivenet.com/integrations/use-from-code.md): Call Hivenet Router from Python, JavaScript, official SDKs, LangChain, or direct HTTP, with streaming, tools, embeddings, reranking, errors, and production controls.
- [Detailed architecture](https://routerdocs.hivenet.com/reference/detailed-architecture.md): Trace how Hivenet Router’s router, agents, authentication, policy engine, storage, request pipeline, and observability systems work together.
- [Configuration reference](https://routerdocs.hivenet.com/reference/configuration-reference.md): Review every Hivenet Router router and agent flag, supported environment variable, default value, precedence rule, and restart requirement.
- [Error codes](https://routerdocs.hivenet.com/reference/error-codes.md): Understand Hivenet Router error responses, HTTP statuses, retry behavior, backend classification, response headers, and debugging workflows.
- [Performance characteristics](https://routerdocs.hivenet.com/reference/performance-characteristics.md): Understand Hivenet Router latency boundaries, routing cost, concurrency, queueing, memory behavior, scaling limits, and benchmarking tools.
- [Project overview](https://routerdocs.hivenet.com/project/project-overview.md): Find Hivenet Router’s source code, canonical project files, contribution workflow, community standards, and Apache 2.0 license.
- [Contributing](https://routerdocs.hivenet.com/project/contributing.md): Set up a Hivenet Router development environment, run the required checks, update generated code, and submit focused pull requests.
- [Code of conduct](https://routerdocs.hivenet.com/project/code-of-conduct.md): Review the standards, scope, reporting process, and enforcement guidelines that apply across the Hivenet Router community.
- [Security policy](https://routerdocs.hivenet.com/project/security-policy.md): Report vulnerabilities privately and review the versions covered by the project security policy.
- [License](https://routerdocs.hivenet.com/project/license.md): Understand the permissions, redistribution conditions, patent grant, trademark limits, and warranty terms of the Apache License 2.0.