Best AI model routers and LLM gateways (OpenRouter, LiteLLM, Portkey, Cloudflare, Not Diamond)

Who holds the provider key is the real decision

Every tool here sits between your application and the model providers. What separates them is not the feature list, it is the answer to one question: whose API key bills the inference, and what does the layer in the middle charge you for?

OpenRouter holds the key and the billing relationship. You buy credits, it pays 70+ providers at their listed prices with no markup, and it charges you a 5.5 percent fee when you buy those credits (minimum 0.80 dollars). Failover across providers is automatic, its Auto Router ranks models by what the community actually spends on your task type, and variants let you bias toward speed, price, or tool-calling reliability. Bring your own keys and the first 25,000 dollars a month of list-price inference carries no fee.

LiteLLM inverts the model entirely. It is open-source software you run: a Python SDK that reaches 100+ models through one call, and a self-hosted proxy with six routing strategies (latency-based, lowest-cost, weighted, and a custom mode where you write Python), plus virtual keys and budgets per team. You pay no platform fee. You pay an engineer and a Postgres instance, and you get routing logic no hosted catalog can offer.

Portkey is the governance layer. It sits in front of the provider keys you already hold and adds guardrails, prompt management, cost attribution per team or use case, and log-based observability. Now part of Palo Alto Networks as PRISMA AIRS, it is the pick when the buyer is a platform or security team, not a product engineer. Pricing is on recorded logs, free to 10,000 a month, 49 dollars for 100,000, custom beyond.

Cloudflare AI Gateway is the zero-fee option: caching, rate limiting, analytics, and data-loss-prevention scanning at the edge, with inference passed through at cost and no gateway fee at all. It forwards and observes rather than deciding, so teams pair it with a routing layer when model choice matters.

Not Diamond is that routing layer. It is not a gateway, it integrates with the one you have and decides per prompt: which model, at what reasoning effort, based on payload semantics and session outcomes, with reinforcement learning personalising over time. It is purpose-built for long-horizon coding-agent harnesses, charges 0.05 dollars per million tokens routed, and claims 20 to 40 percent savings at frontier quality. The trade is a 100 to 150 millisecond routing decision on every request and an application gate to get in.

Disclosure: no vendor in this comparison paid or commissioned this page, and none ran an affiliate programme when checked (2026-10-22).