Quickstart

This guide deploys one Hivenet Router router and three agents across multiple machines. Two agents connect to vLLM backends serving the same model. A third connects to Ollama. You will send requests through the router, confirm that the agents registered, and inspect the resulting routing metrics.

The hostnames and private IP addresses in this guide are examples. Replace them with addresses from your own network.

What you will deploy

This example uses four machines:

Machine	Role	IP address	Backend
`router-server`	Hivenet Router router	`192.168.1.100`	None
`gpu-eu-1`	Agent	`192.168.1.101`	vLLM
`gpu-eu-2`	Agent	`192.168.1.102`	vLLM
`gpu-us-1`	Agent	`192.168.1.103`	Ollama

The router exposes one client-facing API to applications. Each agent connects an inference backend to the router.

Network requirements

The machines must be able to communicate over these ports:

Direction	Port	Purpose
Clients → Router	`8080`	Client HTTP API
Agents → Router	`50051`	gRPC authentication
Agents → Router	`9000`	libp2p registration and data plane
Monitoring → Router	`2112`	Prometheus metrics

Agent hosts need outbound access to the router’s gRPC and libp2p ports. They do not need to expose an inbound Hivenet Router port. Keep the Prometheus endpoint limited to your monitoring network.

Prerequisites

You need:

Go 1.25.5 or later on one build machine
Git
OpenSSL
SSH and scp
curl
jq for formatting JSON responses
vLLM installed on the two vLLM hosts
Ollama installed, or permission to install it, on the Ollama host
access to the models used in this example

The compiled Hivenet Router binaries do not require Go on the target machines.

Prepare the target machines

On each router and agent machine, create the Hivenet Router directory:

sudo mkdir -p /opt/hivenet-router
sudo chown "$USER":"$USER" /opt/hivenet-router

Run these commands on:

router-server
gpu-eu-1
gpu-eu-2
gpu-us-1

Build and distribute Hivenet Router

On the build machine:

git clone https://github.com/HivenetOSS/hivenet_router.git
cd hivenet_router

go build -o bin/hivenet-router ./cmd/router/
go build -o bin/hivenet-agent ./cmd/agent/

Copy the router binary:

scp bin/hivenet-router router-server:/opt/hivenet-router/

Copy the agent binary to each inference host:

scp bin/hivenet-agent gpu-eu-1:/opt/hivenet-router/
scp bin/hivenet-agent gpu-eu-2:/opt/hivenet-router/
scp bin/hivenet-agent gpu-us-1:/opt/hivenet-router/

Create the shared JWT secret

The router and every agent must use the same JWT secret. On the build machine:

openssl rand -hex 32 > jwt.secret
chmod 600 jwt.secret

Copy it to every machine:

scp jwt.secret router-server:/opt/hivenet-router/jwt.secret
scp jwt.secret gpu-eu-1:/opt/hivenet-router/jwt.secret
scp jwt.secret gpu-eu-2:/opt/hivenet-router/jwt.secret
scp jwt.secret gpu-us-1:/opt/hivenet-router/jwt.secret

Set restrictive permissions on each machine:

chmod 600 /opt/hivenet-router/jwt.secret

Anyone with this secret can authenticate an agent with the router. Store and distribute it through your normal secrets-management process in production.

Start the vLLM backends

Run the following command on both gpu-eu-1 and gpu-eu-2:

vllm serve meta-llama/Llama-3.1-8B-Instruct \
  --host 0.0.0.0 \
  --port 8888 \
  --tensor-parallel-size 1 \
  --max-num-seqs 32 \
  --max-model-len 8192

Wait for the model to load, then check the health endpoint:

curl http://localhost:8888/health

A ready vLLM server returns HTTP status 200.

Start the Ollama backend

On gpu-us-1, install Ollama if it is not already installed:

curl -fsSL https://ollama.com/install.sh | sh

Configure and start the server:

export OLLAMA_HOST=0.0.0.0:11434
export OLLAMA_KEEP_ALIVE=-1

nohup ollama serve > ~/ollama.log 2>&1 &

Pull the model:

ollama pull llama3.1:8b

Check that Ollama is ready:

curl http://localhost:11434/api/tags

The endpoint should return HTTP status 200 and a list containing llama3.1:8b.

Start the router

On router-server, prepare the persistent database directory:

sudo mkdir -p /var/lib/hivenet-router/badger
sudo chown "$USER":"$USER" /var/lib/hivenet-router/badger

Start the router with unauthenticated administrator endpoints enabled for this isolated quickstart:

HIVENET_ROUTER_ALLOW_INSECURE_ADMIN=true \
/opt/hivenet-router/hivenet-router \
  --jwt-secret-file /opt/hivenet-router/jwt.secret \
  --http-port :8080 \
  --grpc-port :50051 \
  --p2p-port 9000 \
  --p2p-listen-addr 0.0.0.0 \
  --metrics-port :2112 \
  --disk-db-path /var/lib/hivenet-router/badger

The --p2p-listen-addr 0.0.0.0 flag allows agents on other machines to reach the router’s libp2p endpoint.

HIVENET_ROUTER_ALLOW_INSECURE_ADMIN=true is suitable only for this isolated quickstart. It permits unauthenticated access to /admin/*. Configure administrator API-key authentication before exposing the router to shared or untrusted networks.

Check the public liveness endpoint:

curl http://localhost:8080/health

Expected response:

{
  "status": "ok"
}

Check the operational health endpoint:

curl http://localhost:8080/admin/health | jq .

Before any agents register, the response should report zero agents and a degraded operational state:

{
  "status": "degraded",
  "total_agents": 0,
  "healthy_agents": 0,
  "queue_length": 0,
  "agents": []
}

The full response also includes a Unix timestamp.

If the router runs behind NAT, inside Docker, or behind a public hostname, you may also need --p2p-announce-addr so agents receive a reachable router address.

Start the first vLLM agent

On gpu-eu-1:

/opt/hivenet-router/hivenet-agent \
  --router-grpc 192.168.1.100:50051 \
  --jwt-secret-file /opt/hivenet-router/jwt.secret \
  --engine vllm \
  --backend-url http://localhost:8888 \
  --capacity 32 \
  --region EU-Primary \
  --identity-path /opt/hivenet-router/agent_identity.key

Start the second vLLM agent

On gpu-eu-2:

/opt/hivenet-router/hivenet-agent \
  --router-grpc 192.168.1.100:50051 \
  --jwt-secret-file /opt/hivenet-router/jwt.secret \
  --engine vllm \
  --backend-url http://localhost:8888 \
  --capacity 32 \
  --region EU-Secondary \
  --identity-path /opt/hivenet-router/agent_identity.key

Start the Ollama agent

On gpu-us-1:

/opt/hivenet-router/hivenet-agent \
  --router-grpc 192.168.1.100:50051 \
  --jwt-secret-file /opt/hivenet-router/jwt.secret \
  --engine ollama \
  --backend-url http://localhost:11434 \
  --capacity 5 \
  --region US-Primary \
  --identity-path /opt/hivenet-router/agent_identity.key

This example gives the Ollama agent a capacity of 5. Set capacity according to the concurrency your backend can handle. For a multi-machine deployment, these two agent settings are particularly important:

Setting	Purpose
`--router-grpc`	Authenticates the agent and supplies the router’s libp2p connection details
`--identity-path`	Preserves the agent’s peer identity across restarts

Agents initiate the connection to the router. You do not need to expose an inbound agent port or configure an agent announce address. The agent waits until its backend is healthy and a model is available before registering. If you do not pass --model, it registers the first model returned by the backend. Each agent registers one model.

Verify agent registration

On router-server:

curl http://localhost:8080/admin/health | jq .

With all three agents healthy, the response should resemble:

{
  "status": "healthy",
  "total_agents": 3,
  "healthy_agents": 3,
  "queue_length": 0,
  "timestamp": 1760000000,
  "agents": [
    {
      "peer_id": "12D3KooW...",
      "model": "meta-llama/Llama-3.1-8B-Instruct",
      "engine": "vllm",
      "version": "dev",
      "capacity": 32,
      "region": "EU-Primary",
      "is_healthy": true,
      "last_seen": 1760000000
    },
    {
      "peer_id": "12D3KooX...",
      "model": "meta-llama/Llama-3.1-8B-Instruct",
      "engine": "vllm",
      "version": "dev",
      "capacity": 32,
      "region": "EU-Secondary",
      "is_healthy": true,
      "last_seen": 1760000000
    },
    {
      "peer_id": "12D3KooY...",
      "model": "llama3.1:8b",
      "engine": "ollama",
      "version": "dev",
      "capacity": 5,
      "region": "US-Primary",
      "is_healthy": true,
      "last_seen": 1760000000
    }
  ]
}

Peer IDs and timestamps will differ. For a fuller snapshot that includes routing, latency, hardware, and engine data:

curl http://localhost:8080/admin/routing-table | jq .

Send an inference request

From any machine that can reach the router:

curl -X POST http://192.168.1.100:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "meta-llama/Llama-3.1-8B-Instruct",
    "messages": [
      {
        "role": "user",
        "content": "Hello"
      }
    ]
  }'

The router selects one of the two vLLM agents serving the requested model and forwards the request to it.

Observe load distribution

Send ten requests in parallel:

for i in {1..10}; do
  curl -s http://192.168.1.100:8080/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
      "model": "meta-llama/Llama-3.1-8B-Instruct",
      "messages": [
        {
          "role": "user",
          "content": "Hi"
        }
      ]
    }' &
done

wait

Inspect the routing counter:

curl -s http://192.168.1.100:2112/metrics \
  | grep hivenet_router_routing_requests_routed_total

Troubleshooting

The router gRPC endpoint is unreachable

On router-server, check that the port is listening:

ss -tlnp | grep 50051

On an agent machine, test connectivity to the router:

nc -zv 192.168.1.100 50051
nc -zv 192.168.1.100 9000

An agent does not register

Check for these common causes:

--router-grpc points to the wrong or unreachable address
the router is still listening for libp2p only on 127.0.0.1
the router advertises a libp2p address the agent cannot reach
the router needs a correct --p2p-announce-addr
the inference backend is not ready
the router and agent use different JWT secrets

A backend is not ready

Check vLLM:

curl http://localhost:8888/health

Check Ollama:

curl http://localhost:11434/api/tags
ollama ps

The JWT secret does not match

Confirm that every machine has the same secret:

sha256sum /opt/hivenet-router/jwt.secret

The hash should be identical on the router and every agent host.

A firewall blocks the connection

The exact commands depend on your firewall. With UFW, a basic router configuration could look like:

sudo ufw allow 50051/tcp
sudo ufw allow 9000/tcp
sudo ufw allow 8080/tcp
sudo ufw allow from <monitoring-ip> to any port 2112 proto tcp

Agent hosts do not need an inbound Hivenet Router firewall rule. Allow their outbound connections to the router’s ports 50051 and 9000.

These are examples, not a complete network-security policy. Restrict every port to the clients, agents, or monitoring systems that require access.

Next step

Read the architecture overview to understand the router-agent control flow, data plane, storage, authentication, and routing pipeline.

Getting started

Deploy Hivenet Router

Use the API

Routing and policies

Security and auth

Observability

Integrations

Reference

Project

What you will deploy

Network requirements

Prerequisites

Prepare the target machines

Build and distribute Hivenet Router

Create the shared JWT secret

Start the vLLM backends

Start the Ollama backend

Start the router

Start the first vLLM agent

Start the second vLLM agent

Start the Ollama agent

Verify agent registration

Send an inference request

Observe load distribution

Troubleshooting

The router gRPC endpoint is unreachable

An agent does not register

A backend is not ready

The JWT secret does not match

A firewall blocks the connection

Next step

​What you will deploy

​Network requirements

​Prerequisites

​Prepare the target machines

​Build and distribute Hivenet Router

​Create the shared JWT secret

​Start the vLLM backends

​Start the Ollama backend

​Start the router

​Start the first vLLM agent

​Start the second vLLM agent

​Start the Ollama agent

​Verify agent registration

​Send an inference request

​Observe load distribution

​Troubleshooting

​The router gRPC endpoint is unreachable

​An agent does not register

​A backend is not ready

​The JWT secret does not match

​A firewall blocks the connection

​Next step

What you will deploy

Network requirements

Prerequisites

Prepare the target machines

Build and distribute Hivenet Router

Create the shared JWT secret

Start the vLLM backends

Start the Ollama backend

Start the router

Start the first vLLM agent

Start the second vLLM agent

Start the Ollama agent

Verify agent registration

Send an inference request

Observe load distribution

Troubleshooting

The router gRPC endpoint is unreachable

An agent does not register

A backend is not ready

The JWT secret does not match

A firewall blocks the connection

Next step