Hivenet Router, Hivenet Inference API, and Compute with Hivenet

Hivenet Router, Hivenet Inference API, and Compute with Hivenet all support AI inference workloads, but they solve different problems. The main difference is who operates each layer.

• With Hivenet Router, you operate the routing layer and connect your own inference infrastructure.
• With Hivenet Inference API, Hivenet operates the inference service behind a managed API.
• With Compute with Hivenet, you rent infrastructure and decide what software to run on it.

​

Hivenet Router

Hivenet Router is open-source software for routing requests across self-hosted inference backends. You deploy the router and agents yourself. You choose the inference engines, models, machines, regions, routing policies, authentication rules, fallback behavior, and monitoring setup. Hivenet Router provides:

• one OpenAI-compatible endpoint for several inference backends
• routing based on model, engine, region, tags, capacity, and live metrics
• fallback chains for unavailable or overloaded backends
• optional fallback to external providers
• API-key authentication, quotas, and model restrictions
• Prometheus metrics, Grafana dashboards, and audit logs

Hivenet Router does not provide GPUs or host models by itself. It coordinates inference servers that you already operate. Use Hivenet Router when you want to:

• run inference across several machines or locations
• combine different inference engines behind one endpoint
• keep routing policies and operational data under your control
• avoid depending on a hosted routing control plane
• build a private or distributed inference setup

​

Hivenet Inference API

Hivenet Inference API is a managed inference service. You connect your application to an API endpoint operated by Hivenet. Hivenet runs and maintains the service infrastructure behind that endpoint, so you do not need to deploy a Hivenet Router router, agents, or inference engines yourself. Use Hivenet Inference API when you want to:

• call hosted models through a managed API
• avoid operating inference infrastructure
• avoid maintaining routers, agents, and model servers
• use a commercial service with managed operations

The models, regions, service limits, and commercial terms available through Hivenet Inference API are separate from Hivenet Router and may change independently.

​

Compute with Hivenet

Compute with Hivenet provides GPU and CPU infrastructure for workloads you operate yourself. You create an instance, choose the runtime environment, and deploy the software you need. That could include an inference engine, a Hivenet Router agent, a Hivenet Router router, a notebook, or another application. Use Compute with Hivenet when you want to:

• rent GPU or CPU infrastructure
• control the operating system and runtime environment
• run your own inference engine
• deploy Hivenet Router on infrastructure you manage
• run workloads that are not covered by a managed inference API

Compute with Hivenet provides the machines. You remain responsible for the software running on them.

​

Compare the three options

​

Using Hivenet Router with Compute with Hivenet

You can run Hivenet Router on Compute with Hivenet in the same way you would run it on other infrastructure. A typical setup could include:

1. One Compute instance running the Hivenet Router router.
2. Several GPU instances running inference engines.
3. One Hivenet Router agent beside each inference engine.
4. Applications sending requests to the router’s OpenAI-compatible endpoint.

Hivenet Router can also route across a mixed environment. For example, some agents could run on your own machines while others run on Compute with Hivenet or another provider. Hivenet Router does not require Hivenet infrastructure. It can run wherever its router, agents, and inference backends can communicate.

​

Choosing the right option

Choose Hivenet Router when you want to operate your own inference setup and need routing across several backends. Choose Hivenet Inference API when you want a managed endpoint and do not want to operate the underlying inference service. Choose Compute with Hivenet when you need infrastructure for software and workloads that you control. You can also combine them where appropriate. For example, you might run Hivenet Router on your own infrastructure, use Compute with Hivenet for additional GPU capacity, and configure an external provider as a final fallback.

​

Continue with Hivenet Router