---
canonical_url: "https://inference-cost-optimizer.com"
---
<u-page-hero>
<template v-slot:title="">

What your self-hosted LLMs really cost

</template>

<template v-slot:description="">

ico, the inference cost optimizer, measures what each customer you serve from your own GPUs costs, shows how much of the bill pays for idle time, and tests engine configs on a replay of your own traffic to find the one that serves it for less.

Runs on your machines. No prompt or completion ever leaves them.

</template>

<template v-slot:links="">
<u-button color="neutral" size="xl" to="https://inference-cost-optimizer.com/getting-started/introduction" trailing-icon="i-lucide-arrow-right">

Get started

</u-button>

<u-button color="neutral" size="xl" to="https://inference-cost-optimizer.com/install/overview" icon="i-lucide-server" variant="outline">

Install on rented GPUs

</u-button>
</template>
</u-page-hero>

<u-page-section>
<template v-slot:title="">

Three questions, answered from your own traffic

</template>

<template v-slot:features="">
<u-page-feature icon="i-lucide-receipt-text" to="https://inference-cost-optimizer.com/report/read-the-report">
<template v-slot:title="">

What does each customer cost?

</template>

<template v-slot:description="">

The gateway tags every request with the tenant it served. The report splits each GPU's cost across tenants by the engine time their requests took, prefill and decode weighed apart.

</template>
</u-page-feature>

<u-page-feature icon="i-lucide-hourglass" to="https://inference-cost-optimizer.com/report/how-costs-are-counted">
<template v-slot:title="">

How much pays for idle GPUs?

</template>

<template v-slot:description="">

Rented GPUs bill every hour. ico counts the hours your engines had no request to serve, and prices them. Where the card has DCGM's profiling counters, it also shows what you paid for cores that sat unused while serving.

</template>
</u-page-feature>

<u-page-feature icon="i-lucide-flask-conical" to="https://inference-cost-optimizer.com/replay/overview">
<template v-slot:title="">

Which config costs less?

</template>

<template v-slot:description="">

The replay lab restarts one replica under other engine flags, replays the shape of your recorded traffic, and names the config that holds your latency limits for clearly less than the current one. Then it puts everything back.

</template>
</u-page-feature>

<u-page-feature icon="i-lucide-shield-check" to="https://inference-cost-optimizer.com/reference/security">
<template v-slot:title="">

Nothing leaves your network

</template>

<template v-slot:description="">

The gateway stores token counts and timings, never text. Replays use synthetic prompts rebuilt from lengths and salted prefix hashes.

</template>
</u-page-feature>

<u-page-feature icon="i-lucide-zap" to="https://inference-cost-optimizer.com/reference/gateway">
<template v-slot:title="">

Out of the way of your users

</template>

<template v-slot:description="">

The gateway adds about 0.2 ms per request. When the collector is down it keeps serving and drops telemetry, never requests.

</template>
</u-page-feature>

<u-page-feature icon="i-lucide-life-buoy" to="https://inference-cost-optimizer.com/install/install-help">
<template v-slot:title="">

Install it yourself, or with us

</template>

<template v-slot:description="">

On rented machines with Docker, installing takes an afternoon, and the first report is worth reading after a day of traffic. If you would rather not do it alone, we install it with you.

</template>
</u-page-feature>
</template>
</u-page-section>

## Resources for Agents

- [API catalog: every service document this site publishes](https://inference-cost-optimizer.com/.well-known/api-catalog)
- [Sitemap (XML)](https://inference-cost-optimizer.com/sitemap.xml)
- [Sitemap (Markdown): every page on the site](https://inference-cost-optimizer.com/sitemap.md)
- [MCP server card: MCP endpoint at https://inference-cost-optimizer.com/mcp](https://inference-cost-optimizer.com/.well-known/mcp/server-card.json)
- [MCP endpoint (streamable HTTP)](https://inference-cost-optimizer.com/mcp)
- [llms.txt: index of the documentation for LLMs](https://inference-cost-optimizer.com/llms.txt)
- [llms-full.txt: the full documentation as a single file](https://inference-cost-optimizer.com/llms-full.txt)
- [OpenAPI document: every route this site serves to agents](https://inference-cost-optimizer.com/openapi.json)


## Sitemap

See the full [sitemap](https://inference-cost-optimizer.com/sitemap.md) for all pages.