# Best AI model routers and LLM gateways (OpenRouter, LiteLLM, Portkey, Cloudflare, Not Diamond)

> OpenRouter for model breadth and failover with nothing to run. LiteLLM to self-host routing logic you customise. Portkey for governance and guardrails over your own keys. Cloudflare AI Gateway for a free edge gateway if you are already on Cloudflare. Not Diamond for learned per-prompt routing on coding-agent fleets.

Human page: https://testedactually.com/ai-model-routing/
Mirror: https://testedactually.com/ai-model-routing.md
Updated: 2026-10-29

## Picks

- **OpenRouter** — best for one key and one bill across 300+ models with automatic provider failover and nothing to run. Don't use it if zero infrastructure discipline, the 5.5% credit fee and BYOK charges above 25,000 dollars a month are the cost of that convenience
- **LiteLLM** — best for platform teams that want routing logic, budgets, and RBAC they host and customise in Python. Don't use it if you own the operations, the proxy needs Postgres, compute, and someone on call
- **Portkey** — best for governance at scale, guardrails, cost attribution, and observability across many internal AI apps. Don't use it if budget discipline, log-based pricing grows with traffic and the free tier is not for production
- **Cloudflare AI Gateway** — best for teams already on Cloudflare that want caching, rate limiting, and analytics at no platform fee. Don't use it if shallow routing depth, it forwards and observes rather than deciding what serves each prompt
- **Not Diamond** — best for coding-agent fleets where per-prompt model choice cuts cost 20 to 40 percent without quality loss. Don't use it if it rides on a gateway rather than replacing one, adds 100 to 150 milliseconds a request, and access is by application

**AI model routing — compared**

| Tool | Free tier | Paid from | Deployment | Pricing model | Routing scope | AI crawlers |
| --- | --- | --- | --- | --- | --- | --- |
| [OpenRouter](/ai-model-routing/openrouter/) | Free models available (about 50 requests per day, 1,000 per day once 10 credits purchased) | $0/mo | Fully hosted at the edge, no server, database, or per-provider credentials to manage | Pass-through provider pricing at zero markup, platform fee on credit purchases and large BYOK volumes | Provider selection and failover across 70+ providers, Auto Router ranks models by 7-day community spend per task, variants for speed, price, and tool-calling quality | unknown |
| [LiteLLM](/ai-model-routing/litellm/) | Fully functional open-source edition | $0/mo | Self-hosted proxy (Docker plus Postgres) or Python SDK in-process | Free software, infrastructure cost instead of platform fees, Enterprise priced by capacity | Six routing modes including latency-based, lowest-cost, and custom Python logic, plus fallback lists and per-team virtual keys and budgets | unknown |
| [Portkey](/ai-model-routing/portkey/) | Developer plan free with 10,000 recorded logs per month | $49/mo | Managed control plane plus an open-source gateway you can self-host | Priced on recorded logs and features, not tokens, inference stays on your provider accounts | Universal API, fallbacks, load balancing, and guardrails over 250+ models you address with your own keys | unknown |
| [Cloudflare AI Gateway](/ai-model-routing/cloudflare-ai-gateway/) | Core gateway features free, one line of code to start | $0/mo | Hosted at Cloudflare edge, configured from the dashboard or Workers bindings | No gateway fee, provider inference passed through at cost, charges only for log storage and Logpush exports | Caching, retries, fallbacks, and rate limiting across providers, plus universal billing endpoint | unknown |
| [Not Diamond](/ai-model-routing/notdiamond/) | None published, access to pay-as-you-go is by application | $0.05/mo | Managed routing API that integrates with your existing gateway, not a gateway itself | Small fixed fee per million tokens routed, on top of whatever the chosen model bills | Learned per-prompt model and reasoning-effort selection from payload semantics and session outcomes, purpose-built for coding-agent harnesses including Claude Code | unknown |

## Don't use it if

- **OpenRouter**: your traffic is high-volume and predictable, credit fees and BYOK charges can exceed what self-hosting costs.
- **LiteLLM**: nobody on your team runs infrastructure, the proxy needs Postgres, compute, and on-call.
- **Portkey**: you want routing decisions, it governs and observes the path, it does not choose your model per prompt the way a learned router does.
- **Not Diamond**: you want a single vendor bill for models, it routes and decides but the inference still bills on provider accounts.

## Who holds the provider key is the real decision

Every tool here sits between your application and the model providers.
What separates them is not the feature list, it is the answer to one
question: whose API key bills the inference, and what does the layer in
the middle charge you for?

OpenRouter holds the key and the billing relationship. You buy credits,
it pays 70+ providers at their listed prices with no markup, and it
charges you a 5.5 percent fee when you buy those credits (minimum 0.80
dollars). Failover across providers is automatic, its Auto Router ranks
models by what the community actually spends on your task type, and
variants let you bias toward speed, price, or tool-calling reliability.
Bring your own keys and the first 25,000 dollars a month of list-price
inference carries no fee.

LiteLLM inverts the model entirely. It is open-source software you run:
a Python SDK that reaches 100+ models through one call, and a
self-hosted proxy with six routing strategies (latency-based,
lowest-cost, weighted, and a custom mode where you write Python), plus
virtual keys and budgets per team. You pay no platform fee. You pay an
engineer and a Postgres instance, and you get routing logic no hosted
catalog can offer.

Portkey is the governance layer. It sits in front of the provider keys
you already hold and adds guardrails, prompt management, cost
attribution per team or use case, and log-based observability. Now part
of Palo Alto Networks as PRISMA AIRS, it is the pick when the buyer is a
platform or security team, not a product engineer. Pricing is on
recorded logs, free to 10,000 a month, 49 dollars for 100,000, custom
beyond.

Cloudflare AI Gateway is the zero-fee option: caching, rate limiting,
analytics, and data-loss-prevention scanning at the edge, with inference
passed through at cost and no gateway fee at all. It forwards and
observes rather than deciding, so teams pair it with a routing layer
when model choice matters.

Not Diamond is that routing layer. It is not a gateway, it integrates
with the one you have and decides per prompt: which model, at what
reasoning effort, based on payload semantics and session outcomes, with
reinforcement learning personalising over time. It is purpose-built for
long-horizon coding-agent harnesses, charges 0.05 dollars per million
tokens routed, and claims 20 to 40 percent savings at frontier quality.
The trade is a 100 to 150 millisecond routing decision on every request
and an application gate to get in.

**Disclosure**: no vendor in this comparison paid or commissioned this
page, and none ran an affiliate programme when checked (2026-10-22).

## FAQ

### What is the best OpenRouter alternative?

Depends what you are replacing. LiteLLM replaces the routing layer you self-host, with six routing strategies and full custom logic for free plus infrastructure. Portkey replaces it with a managed control plane that adds guardrails and observability over your own provider keys. Cloudflare AI Gateway is the free option if you already run on Cloudflare. Not Diamond does something different, it picks the best model per prompt on top of a gateway.

### Litellm vs OpenRouter, which should we pick?

OpenRouter if you want no infrastructure, one bill across 300+ models, and automatic failover, paying a 5.5 percent fee on credit purchases and nothing on inference markup. LiteLLM if you want the routing in your own hands, it is free to self-host and your only costs are Postgres, compute, and the provider bills. Large teams often run LiteLLM with OpenRouter as one of its upstreams.

### How much do AI model routers cost?

OpenRouter charges no markup on inference, only a 5.5 percent fee on credit purchases (0.80 dollar minimum) and 5 percent on bring-your-own-key usage above 25,000 dollars a month. LiteLLM's core is free to self-host. Cloudflare AI Gateway's core features are free. Portkey is free to 10,000 logs then 49 dollars a month for 100,000. Not Diamond charges 0.05 dollars per million tokens routed.

### What is the difference between an LLM gateway and a model router?

A gateway is the plumbing, one API in front of many providers with failover, caching, rate limits, and observability. A router is the decision layer that picks which model serves each prompt. OpenRouter, LiteLLM, Portkey, and Cloudflare AI Gateway are gateways with different amounts of routing built in. Not Diamond is a decision layer that integrates with your existing gateway.


## References

- [1] OpenRouter FAQ — https://openrouter.ai/docs/faq (Vendor docs, 5.5 percent credit fee with 0.80 dollar minimum, zero inference markup, BYOK 5 percent above 25,000 dollars per month, verified 2026-10-22)
- [2] OpenRouter vs LiteLLM — https://openrouter.ai/blog/insights/openrouter-vs-litellm (Vendor comparison page, LiteLLM six routing modes, self-hosting costs, Enterprise via sales, verified 2026-10-22)
- [3] Portkey pricing — https://portkey.ai/pricing (Vendor pricing page, Developer free 10,000 logs, Production 49 dollars per month 100,000 logs, Enterprise custom, verified 2026-10-22)
- [4] Cloudflare AI Gateway pricing — https://developers.cloudflare.com/ai-gateway/pricing/ (Vendor docs, core features free on all plans, inference passed through with no markup, log storage limits by Workers plan, verified 2026-10-22)
- [5] Not Diamond pricing — https://notdiamond.ai/pricing (Vendor pricing page, 0.05 dollars per million tokens routed, Enterprise custom, verified 2026-10-22)

## Sources

- https://openrouter.ai/docs/faq — checked 2026-10-22, confidence: documented
- https://openrouter.ai/docs/faq — checked 2026-10-22, confidence: unconfirmed
- https://docs.litellm.ai/ — checked 2026-10-22, confidence: documented
- https://openrouter.ai/blog/insights/openrouter-vs-litellm — checked 2026-10-22, confidence: unconfirmed
- https://openrouter.ai/blog/insights/openrouter-vs-litellm — checked 2026-10-22, confidence: documented
- https://docs.litellm.ai/ — checked 2026-10-22, confidence: unconfirmed
- https://portkey.ai/pricing — checked 2026-10-22, confidence: documented
- https://portkey.ai/pricing — checked 2026-10-22, confidence: unconfirmed
- https://developers.cloudflare.com/ai-gateway/pricing/ — checked 2026-10-22, confidence: documented
- https://developers.cloudflare.com/ai-gateway/pricing/ — checked 2026-10-22, confidence: unconfirmed
- https://notdiamond.ai/pricing — checked 2026-10-22, confidence: documented
- https://notdiamond.ai/pricing — checked 2026-10-22, confidence: unconfirmed