> ## Documentation Index
> Fetch the complete documentation index at: https://routerdocs.hivenet.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Quickstart

> Deploy a Hivenet Router router and multiple agents across several machines, then send and observe your first routed inference request.

This guide deploys one Hivenet Router router and three agents across multiple machines.

Two agents connect to vLLM backends serving the same model. A third connects to Ollama. You will send requests through the router, confirm that the agents registered, and inspect the resulting routing metrics.

<Note>
  The hostnames and private IP addresses in this guide are examples. Replace them with addresses from your own network.
</Note>

## What you will deploy

This example uses four machines:

| Machine         | Role                  | IP address      | Backend |
| --------------- | --------------------- | --------------- | ------- |
| `router-server` | Hivenet Router router | `192.168.1.100` | None    |
| `gpu-eu-1`      | Agent                 | `192.168.1.101` | vLLM    |
| `gpu-eu-2`      | Agent                 | `192.168.1.102` | vLLM    |
| `gpu-us-1`      | Agent                 | `192.168.1.103` | Ollama  |

The router exposes one client-facing API to applications. Each agent connects an inference backend to the router.

## Network requirements

The machines must be able to communicate over these ports:

| Direction           | Port    | Purpose                            |
| ------------------- | ------- | ---------------------------------- |
| Clients → Router    | `8080`  | Client HTTP API                    |
| Agents → Router     | `50051` | gRPC authentication                |
| Agents → Router     | `9000`  | libp2p registration and data plane |
| Monitoring → Router | `2112`  | Prometheus metrics                 |

Agent hosts need outbound access to the router’s gRPC and libp2p ports. They do not need to expose an inbound Hivenet Router port. Keep the Prometheus endpoint limited to your monitoring network.

## Prerequisites

You need:

* Go 1.25.5 or later on one build machine
* Git
* OpenSSL
* SSH and `scp`
* `curl`
* `jq` for formatting JSON responses
* vLLM installed on the two vLLM hosts
* Ollama installed, or permission to install it, on the Ollama host
* access to the models used in this example

The compiled Hivenet Router binaries do not require Go on the target machines.

## Prepare the target machines

On each router and agent machine, create the Hivenet Router directory:

```bash theme={null}
sudo mkdir -p /opt/hivenet-router
sudo chown "$USER":"$USER" /opt/hivenet-router
```

Run these commands on:

* `router-server`
* `gpu-eu-1`
* `gpu-eu-2`
* `gpu-us-1`

## Build and distribute Hivenet Router

On the build machine:

```bash theme={null}
git clone https://github.com/HivenetOSS/hivenet_router.git
cd hivenet_router

go build -o bin/hivenet-router ./cmd/router/
go build -o bin/hivenet-agent ./cmd/agent/
```

Copy the router binary:

```bash theme={null}
scp bin/hivenet-router router-server:/opt/hivenet-router/
```

Copy the agent binary to each inference host:

```bash theme={null}
scp bin/hivenet-agent gpu-eu-1:/opt/hivenet-router/
scp bin/hivenet-agent gpu-eu-2:/opt/hivenet-router/
scp bin/hivenet-agent gpu-us-1:/opt/hivenet-router/
```

## Create the shared JWT secret

The router and every agent must use the same JWT secret.

On the build machine:

```bash theme={null}
openssl rand -hex 32 > jwt.secret
chmod 600 jwt.secret
```

Copy it to every machine:

```bash theme={null}
scp jwt.secret router-server:/opt/hivenet-router/jwt.secret
scp jwt.secret gpu-eu-1:/opt/hivenet-router/jwt.secret
scp jwt.secret gpu-eu-2:/opt/hivenet-router/jwt.secret
scp jwt.secret gpu-us-1:/opt/hivenet-router/jwt.secret
```

Set restrictive permissions on each machine:

```bash theme={null}
chmod 600 /opt/hivenet-router/jwt.secret
```

<Warning>
  Anyone with this secret can authenticate an agent with the router. Store and distribute it through your normal secrets-management process in production.
</Warning>

## Start the vLLM backends

Run the following command on both `gpu-eu-1` and `gpu-eu-2`:

```bash theme={null}
vllm serve meta-llama/Llama-3.1-8B-Instruct \
  --host 0.0.0.0 \
  --port 8888 \
  --tensor-parallel-size 1 \
  --max-num-seqs 32 \
  --max-model-len 8192
```

Wait for the model to load, then check the health endpoint:

```bash theme={null}
curl http://localhost:8888/health
```

A ready vLLM server returns HTTP status `200`.

## Start the Ollama backend

On `gpu-us-1`, install Ollama if it is not already installed:

```bash theme={null}
curl -fsSL https://ollama.com/install.sh | sh
```

Configure and start the server:

```bash theme={null}
export OLLAMA_HOST=0.0.0.0:11434
export OLLAMA_KEEP_ALIVE=-1

nohup ollama serve > ~/ollama.log 2>&1 &
```

Pull the model:

```bash theme={null}
ollama pull llama3.1:8b
```

Check that Ollama is ready:

```bash theme={null}
curl http://localhost:11434/api/tags
```

The endpoint should return HTTP status `200` and a list containing `llama3.1:8b`.

## Start the router

On `router-server`, prepare the persistent database directory:

```bash theme={null}
sudo mkdir -p /var/lib/hivenet-router/badger
sudo chown "$USER":"$USER" /var/lib/hivenet-router/badger
```

Start the router with unauthenticated administrator endpoints enabled for this isolated quickstart:

```bash theme={null}
HIVENET_ROUTER_ALLOW_INSECURE_ADMIN=true \
/opt/hivenet-router/hivenet-router \
  --jwt-secret-file /opt/hivenet-router/jwt.secret \
  --http-port :8080 \
  --grpc-port :50051 \
  --p2p-port 9000 \
  --p2p-listen-addr 0.0.0.0 \
  --metrics-port :2112 \
  --disk-db-path /var/lib/hivenet-router/badger
```

The `--p2p-listen-addr 0.0.0.0` flag allows agents on other machines to reach the router’s libp2p endpoint.

<Warning>
  `HIVENET_ROUTER_ALLOW_INSECURE_ADMIN=true` is suitable only for this isolated quickstart. It permits unauthenticated access to `/admin/*`. Configure administrator API-key authentication before exposing the router to shared or untrusted networks.
</Warning>

Check the public liveness endpoint:

```bash theme={null}
curl http://localhost:8080/health
```

Expected response:

```json theme={null}
{
  "status": "ok"
}
```

Check the operational health endpoint:

```bash theme={null}
curl http://localhost:8080/admin/health | jq .
```

Before any agents register, the response should report zero agents and a degraded operational state:

```json theme={null}
{
  "status": "degraded",
  "total_agents": 0,
  "healthy_agents": 0,
  "queue_length": 0,
  "agents": []
}
```

The full response also includes a Unix timestamp.

<Note>
  If the router runs behind NAT, inside Docker, or behind a public hostname, you may also need `--p2p-announce-addr` so agents receive a reachable router address.
</Note>

## Start the first vLLM agent

On `gpu-eu-1`:

```bash theme={null}
/opt/hivenet-router/hivenet-agent \
  --router-grpc 192.168.1.100:50051 \
  --jwt-secret-file /opt/hivenet-router/jwt.secret \
  --engine vllm \
  --backend-url http://localhost:8888 \
  --capacity 32 \
  --region EU-Primary \
  --identity-path /opt/hivenet-router/agent_identity.key
```

## Start the second vLLM agent

On `gpu-eu-2`:

```bash theme={null}
/opt/hivenet-router/hivenet-agent \
  --router-grpc 192.168.1.100:50051 \
  --jwt-secret-file /opt/hivenet-router/jwt.secret \
  --engine vllm \
  --backend-url http://localhost:8888 \
  --capacity 32 \
  --region EU-Secondary \
  --identity-path /opt/hivenet-router/agent_identity.key
```

## Start the Ollama agent

On `gpu-us-1`:

```bash theme={null}
/opt/hivenet-router/hivenet-agent \
  --router-grpc 192.168.1.100:50051 \
  --jwt-secret-file /opt/hivenet-router/jwt.secret \
  --engine ollama \
  --backend-url http://localhost:11434 \
  --capacity 5 \
  --region US-Primary \
  --identity-path /opt/hivenet-router/agent_identity.key
```

This example gives the Ollama agent a capacity of `5`. Set capacity according to the concurrency your backend can handle.

For a multi-machine deployment, these two agent settings are particularly important:

| Setting           | Purpose                                                                     |
| ----------------- | --------------------------------------------------------------------------- |
| `--router-grpc`   | Authenticates the agent and supplies the router’s libp2p connection details |
| `--identity-path` | Preserves the agent’s peer identity across restarts                         |

Agents initiate the connection to the router. You do not need to expose an inbound agent port or configure an agent announce address.

The agent waits until its backend is healthy and a model is available before registering. If you do not pass `--model`, it registers the first model returned by the backend.

Each agent registers one model.

## Verify agent registration

On `router-server`:

```bash theme={null}
curl http://localhost:8080/admin/health | jq .
```

With all three agents healthy, the response should resemble:

```json theme={null}
{
  "status": "healthy",
  "total_agents": 3,
  "healthy_agents": 3,
  "queue_length": 0,
  "timestamp": 1760000000,
  "agents": [
    {
      "peer_id": "12D3KooW...",
      "model": "meta-llama/Llama-3.1-8B-Instruct",
      "engine": "vllm",
      "version": "dev",
      "capacity": 32,
      "region": "EU-Primary",
      "is_healthy": true,
      "last_seen": 1760000000
    },
    {
      "peer_id": "12D3KooX...",
      "model": "meta-llama/Llama-3.1-8B-Instruct",
      "engine": "vllm",
      "version": "dev",
      "capacity": 32,
      "region": "EU-Secondary",
      "is_healthy": true,
      "last_seen": 1760000000
    },
    {
      "peer_id": "12D3KooY...",
      "model": "llama3.1:8b",
      "engine": "ollama",
      "version": "dev",
      "capacity": 5,
      "region": "US-Primary",
      "is_healthy": true,
      "last_seen": 1760000000
    }
  ]
}
```

Peer IDs and timestamps will differ.

For a fuller snapshot that includes routing, latency, hardware, and engine data:

```bash theme={null}
curl http://localhost:8080/admin/routing-table | jq .
```

## Send an inference request

From any machine that can reach the router:

```bash theme={null}
curl -X POST http://192.168.1.100:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "meta-llama/Llama-3.1-8B-Instruct",
    "messages": [
      {
        "role": "user",
        "content": "Hello"
      }
    ]
  }'
```

The router selects one of the two vLLM agents serving the requested model and forwards the request to it.

## Observe load distribution

Send ten requests in parallel:

```bash theme={null}
for i in {1..10}; do
  curl -s http://192.168.1.100:8080/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
      "model": "meta-llama/Llama-3.1-8B-Instruct",
      "messages": [
        {
          "role": "user",
          "content": "Hi"
        }
      ]
    }' &
done

wait
```

Inspect the routing counter:

```bash theme={null}
curl -s http://192.168.1.100:2112/metrics \
  | grep hivenet_router_routing_requests_routed_total
```

## Troubleshooting

### The router gRPC endpoint is unreachable

On `router-server`, check that the port is listening:

```bash theme={null}
ss -tlnp | grep 50051
```

On an agent machine, test connectivity to the router:

```bash theme={null}
nc -zv 192.168.1.100 50051
nc -zv 192.168.1.100 9000
```

### An agent does not register

Check for these common causes:

* `--router-grpc` points to the wrong or unreachable address
* the router is still listening for libp2p only on `127.0.0.1`
* the router advertises a libp2p address the agent cannot reach
* the router needs a correct `--p2p-announce-addr`
* the inference backend is not ready
* the router and agent use different JWT secrets

### A backend is not ready

Check vLLM:

```bash theme={null}
curl http://localhost:8888/health
```

Check Ollama:

```bash theme={null}
curl http://localhost:11434/api/tags
ollama ps
```

### The JWT secret does not match

Confirm that every machine has the same secret:

```bash theme={null}
sha256sum /opt/hivenet-router/jwt.secret
```

The hash should be identical on the router and every agent host.

### A firewall blocks the connection

The exact commands depend on your firewall. With UFW, a basic router configuration could look like:

```bash theme={null}
sudo ufw allow 50051/tcp
sudo ufw allow 9000/tcp
sudo ufw allow 8080/tcp
sudo ufw allow from <monitoring-ip> to any port 2112 proto tcp
```

Agent hosts do not need an inbound Hivenet Router firewall rule. Allow their outbound connections to the router’s ports `50051` and `9000`.

<Warning>
  These are examples, not a complete network-security policy. Restrict every port to the clients, agents, or monitoring systems that require access.
</Warning>

## Next step

Read the [architecture overview](/getting-started/architecture-overview) to understand the router-agent control flow, data plane, storage, authentication, and routing pipeline.