OpenRouter vs LiteLLM: Which LLM Gateway Fits Your Stack?

If you’re choosing between OpenRouter and LiteLLM for a production gateway, the decision comes down to where the routing layer should run.

Both give you a single OpenAI-compatible API across many providers. OpenRouter runs that layer for you, so there’s no infrastructure to operate. LiteLLM runs inside your own infrastructure, so your data stays on your network and you pay no platform fee, in exchange for operating PostgreSQL, Redis, and Docker yourself.

Pick LiteLLM when data can’t leave your network, you need role-based access control inside your own infrastructure, or your model spend is high enough that the 5.5% platform fee costs more than running the proxy. Most other cases favor the managed option.

Decide where the routing layer lives

OpenRouter sits between your app and 70+ providers, running on Cloudflare’s edge. Your app calls https://openrouter.ai/api/v1, OpenRouter applies routing and failover, and forwards the request to an upstream provider. You never manage a server, a database, or per-provider credentials.

LiteLLM is a proxy you deploy yourself, as a Docker container or Kubernetes pod. It exposes the same OpenAI-compatible endpoint inside your network, rewrites each request into a provider’s native format, and forwards it. PostgreSQL stores spend data and keys; Redis handles caching and rate limits in production. You run all three.

That difference is what a compliance team asks about first. With LiteLLM, request data never leaves your network before it reaches the provider. With OpenRouter, requests pass through a managed layer first, so teams with strict data-residency rules should review the available routing and Zero Data Retention controls.

OpenRouter:  app -> OpenRouter (Cloudflare edge) -> provider
LiteLLM:     app -> LiteLLM proxy (your infra: Docker + PostgreSQL) -> provider

What each gateway costs

OpenRouter passes provider pricing through at 0% markup, then charges a 5.5% platform fee on pay-as-you-go credit purchases, with a $0.80 minimum per purchase. Bring Your Own Key drops the fee to 5%, and the first 1 million requests each month are waived. Failed requests aren’t billed.

LiteLLM is free to self-host. You pay for infrastructure instead: the PostgreSQL database, optional Redis, and compute, which typically runs a few hundred dollars a month for a production deployment. LiteLLM Enterprise adds SSO, SCIM, RBAC, audit logs, and Prometheus metrics, priced through their sales team.

The crossover is arithmetic. Divide your monthly infrastructure cost by the 5.5% fee. At roughly $200/month of infra, LiteLLM gets cheaper once your model spend passes about $3,600/month; at $500/month of infra, that line moves to about $9,100/month. Below it, the managed fee costs less than the engineering time to run the proxy.

Compare routing, failover, and latency

OpenRouter routes well by default. Its Auto Router, powered by NotDiamond, picks a model per prompt, and provider-level routing deprioritizes any provider that has seen outages in the last 30 seconds. You can constrain routing with a provider object that filters by price, throughput, latency, data policy, ZDR, and quantization.

LiteLLM gives you more strategies and full custom logic. It ships six routing modes: weighted pick, latency-based, rate-limit-aware, least-busy, lowest-cost, and a custom mode where you write Python. Fallback lists let the proxy try the next model when one fails. If you need per-team or per-model rules enforced at the proxy, LiteLLM gives you the hooks.

Latency depends on how much you tune. LiteLLM reports about 2ms of median overhead (8ms P95, 13ms P99) on a 4-instance deployment of 4 CPUs and 8 GB each, tested against a mock endpoint; drop to 2 instances and median overhead rises to about 12ms (LiteLLM’s self-reported benchmarks). OpenRouter adds a network hop on Cloudflare’s edge that you don’t tune or scale. One is a knob you own, the other is a constant you don’t.

Match compliance needs to the right gateway

OpenRouter brings third-party attestation. It’s SOC 2 Type 2 compliant, with the full report at trust.openrouter.ai, supports GDPR, offers Zero Data Retention per request or account-wide, and can route through EU-based providers for enterprise accounts. Workspaces add per-team organization, budgets, and cost attribution on top.

LiteLLM brings data sovereignty. Because you host it, requests never leave your infrastructure before reaching the provider, so you enforce your own controls. LiteLLM Enterprise adds RBAC, SSO/JWT auth, audit logs, and per-team budgets. It doesn’t hold independent SOC 2, ISO 27001, or HIPAA certifications, so your deployment is responsible for meeting those standards.

Use both, or switch in a few lines

The two aren’t mutually exclusive. LiteLLM can use OpenRouter as an upstream provider, so you get LiteLLM’s local RBAC and logging while OpenRouter handles multi-provider failover and model breadth.

model_list:
  - model_name: or-claude
    litellm_params:
      model: openrouter/anthropic/claude-opus-4.6
      api_key: "your-openrouter-key"
      api_base: "https://openrouter.ai/api/v1"
  - model_name: or-gpt4o
    litellm_params:
      model: openrouter/openai/gpt-4o
      api_key: "your-openrouter-key"
      api_base: "https://openrouter.ai/api/v1"

Switching direction is a base URL and key change, since both speak the OpenAI format.

from openai import OpenAI

# OpenRouter
client = OpenAI(
    base_url="https://openrouter.ai/api/v1",
    api_key="your-openrouter-key",
)

# LiteLLM
client = OpenAI(
    base_url="http://your-litellm-host:4000",
    api_key="your-litellm-master-key",
)

That keeps both gateways behind one client. If you aren’t standardizing on the OpenAI SDK, OpenRouter has its own (openrouter for Python, @openrouter/sdk for TypeScript) that calls the OpenRouter side natively, with no base URL to set.

from openrouter import OpenRouter
import os

with OpenRouter(api_key=os.environ["OPENROUTER_API_KEY"]) as client:
    response = client.chat.send(
        model="anthropic/claude-opus-4.6",
        messages=[{"role": "user", "content": "Hello"}],
    )

If you’re weighing more than these two, our LLM gateway comparison covers Portkey, Helicone, Cloudflare AI Gateway, and others.

Frequently asked questions

Is LiteLLM like OpenRouter?

They share an OpenAI-compatible API across many providers, but they’re built differently. LiteLLM is an open-source proxy you self-host; OpenRouter is a managed gateway that runs on infrastructure you don’t operate. The split shows up in data residency, operational overhead, and fees.

Can I use LiteLLM and OpenRouter together?

Yes. LiteLLM supports OpenRouter as an upstream provider, so you can route through LiteLLM locally for RBAC and audit logging while OpenRouter handles multi-provider failover and model breadth.

Is OpenRouter free?

OpenRouter has 20+ free models for evaluation. Paid usage passes through provider pricing at 0% markup, plus a 5.5% platform fee on credit purchases. Bring Your Own Key drops the fee to 5%, with the first 1 million requests each month waived.

Does OpenRouter store my prompts?

By default, no. OpenRouter doesn’t retain prompts or responses. Zero Data Retention is available per request via a header or account-wide in your settings.

What’s the latency overhead of OpenRouter vs LiteLLM?

LiteLLM reports about 2ms of median overhead on a tuned 4-instance deployment, tested against a mock endpoint; a 2-instance setup rises to about 12ms. OpenRouter adds a network hop on Cloudflare’s edge that you don’t tune or scale yourself.