LiteLLM Tutorial: Install, Configure, Run the Proxy

Install the proxy extras with uv tool install 'litellm[proxy]' (or pip install 'litellm[proxy]' in a virtualenv), write a config.yaml with a model_list, and start it with litellm --config config.yaml. The proxy listens on port 4000 by default. If you need the Admin UI, virtual keys, and spend tracking, use the Docker Compose quickstart, which adds Postgres.

Set the client’s base_url to the proxy (for example http://localhost:4000 or your ingress) and pass a LiteLLM virtual key as api_key. The model field is the public model_name from config.yaml, and responses come back in OpenAI format. On a config-only run without a database, the master key is the Bearer credential, but apps should get virtual keys in any shared setup.

Without shared Redis, each worker or replica keeps its own rate-limit counters, router state, and cache. A 100 RPM limit across four workers then effectively allows about 400, revoked keys can linger on other workers, and cache hits stay local. Configure Redis in both router_settings and litellm_settings.cache_params—setting REDIS_HOST in the environment alone is not enough.

Yes. The gateway is OpenTelemetry-compliant and plugs into Grafana, Datadog, Prometheus, or your preferred stack. Every request is traced from prompt to model execution, so metrics such as latency, cost, and errors land next to your existing dashboards.