Install on rented GPUs
Install help
We install ico with you, on your machines, if you would rather not do it alone.
Everything in this section is meant to be done by your own team. If you would rather do it with us, we install it with you, usually in one working session, and stay until the first report is in.
What we do
- Look at your fleet with you: providers, machines, how vLLM runs, and where your traffic enters.
- Pick the gateway's place and the ico host, and write the configs with you.
- Set up the ico host, and the first GPU machine together; you do the rest of the fleet the same way, or we do it with you.
- Check that every source reports, at the telemetry level your GPUs allow.
- Walk through the first report with you once a day of traffic is in, and plan the first replay.
What we need from you
- Someone on your side who can restart your engines and change where your traffic goes.
- Access to the machines for the session, in the way your security policy allows: a shared screen is enough.
- A window in which restarting vLLM once is acceptable, to turn on its tracing.
We never need your prompts, your completions or your customers' data, and nothing we install sends any of it anywhere. See Privacy and security.
Ask for it
Open an issue with the Install help template. Tell us about your GPUs, how you serve your models and how traffic reaches them, and we reply there to plan the session. The same place takes bugs and questions.