---
title: "Set up the ico host"
description: "ClickHouse and the collector on one small machine, with Docker Compose."
canonical_url: "https://inference-cost-optimizer.com/install/ico-host"
---
# Set up the ico host

> ClickHouse and the collector on one small machine, with Docker Compose.

The ico host runs two containers: ClickHouse, which stores everything, and the collector, which receives telemetry from the GPU machines and writes it to ClickHouse. It needs Docker with Compose and about ten minutes.

<steps level="3">

### Get the code

```bash [ico host]
git clone --branch v0.1.1 https://github.com/Gudjon01/inference-cost-optimizer.git
cd inference-cost-optimizer/deploy
```

### Set the secrets

```bash [ico host]
cp .env.example .env
```

In `.env`, set `ICO_ADMIN_TOKEN` and `ICO_CLICKHOUSE_PASSWORD` to long random strings, such as the output of `openssl rand -hex 24`. Keep the admin token: every gateway needs the same one for `ico replay` to drain replicas. The ClickHouse password is the password of the `ico` user.

### Write the config

```bash [ico host]
cp config.example.yaml config.yaml
```

The collector reads four things from it: your replicas, and how it reaches each engine's `/metrics`; your GPU exporters; what your GPUs cost; and where ClickHouse is. For two GPU machines on a private network, with one vLLM replica each:

```yaml [deploy/config.yaml]
replicas:
  - name: gpu-1-r1             # unique; the gateway and vLLM's traces use the same name
    url: http://10.0.0.11:8000 # how the ico host reaches this engine
    host: gpu-1                # matches the exporter's host below
    gpu_model: H100-SXM        # matches a key of gpu_price_per_hour
    gpu_count: 1               # GPUs this replica uses (tensor parallel size)
  - name: gpu-2-r1
    url: http://10.0.0.12:8000
    host: gpu-2
    gpu_model: H100-SXM
    gpu_count: 1

# USD per GPU-hour, what you pay. Idle hours are billed too.
gpu_price_per_hour:
  H100-SXM: 2.99

# For the replay's memory check: what each served model weighs.
models:
  Qwen/Qwen3-8B:
    params: 8_190_735_360
    weight_bytes_per_param: 2      # BF16
    kv_bytes_per_token: 147_456    # 2 x layers(36) x kv_heads(8) x head_dim(128) x 2 bytes

collector:
  grpc_listen: 0.0.0.0:4317
  scrape_interval_s: 5
  gpu_exporters:
    - kind: dcgm                   # nvidia_smi for nvidia_gpu_exporter on GeForce cards
      url: http://10.0.0.11:9400/metrics
      host: gpu-1
    - kind: dcgm
      url: http://10.0.0.12:9400/metrics
      host: gpu-2

clickhouse:
  host: clickhouse                 # the ClickHouse container, inside Compose
  port: 8123
  username: ico
  database: ico
```

Every key is described in the [configuration reference](https://inference-cost-optimizer.com/reference/configuration).

### Let the GPU machines reach the collector

Compose publishes the collector (4317) and ClickHouse (8123) on 127.0.0.1 only. Publish them on the ico host's private address instead, in an override file Compose reads next to its own:

```yaml [deploy/docker-compose.override.yml]
services:
  collector:
    ports: !override
      - "10.0.0.5:4317:4317"   # the ico host's private address
  clickhouse:
    ports: !override
      - "10.0.0.5:8123:8123"   # for ico replay, which runs on the GPU machines
```

<caution>

The collector accepts writes without authentication. Publish it only on an address your GPU machines reach privately. ClickHouse asks for the `ico` user's password.

</caution>

### Start ClickHouse and the collector

```bash [ico host]
docker compose up -d clickhouse collector
```

Compose pulls the collector's image, `ghcr.io/gudjon01/ico-collector`, at the release you cloned; `--build` builds it from the checkout instead. The collector applies ClickHouse's schema on its first start. Until the GPU machines are set up, its log says that every scrape failed; that is expected:

```bash [ico host]
docker compose logs -f collector
```

</steps>

## Next

[Set up the GPU machines](https://inference-cost-optimizer.com/install/gpu-machines).


## Sitemap

See the full [sitemap](https://inference-cost-optimizer.com/sitemap.md) for all pages.