LLM Gateway Showdown: LiteLLM vs OpenRouter vs Portkey for Front‑end Teams

Why a Gateway Matters for Front‑end Engineers

When your UI starts calling more than one LLM—ChatGPT for chat, Claude for summarization, and a fine‑tuned model for product recommendations—you quickly end up with scattered API keys, duplicated retry logic, and a cost dashboard that lives in three different services. A gateway consolidates those calls behind a single endpoint, giving you a consistent error model, unified rate‑limiting, and a single place to log usage.

LiteLLM: Self‑hosted Proxy

LiteLLM is an open‑source proxy you run in your own VPC or on a cheap VPS. It speaks the OpenAI‑compatible schema, then forwards the request to any of the 100+ supported providers. Because it lives inside your network, latency is minimal and you retain full control over logging and auth.

// nextjs/api/llm.ts
import { createRouter } from 'next-connect';
import fetch from 'node-fetch';

const router = createRouter();
router.post(async (req, res) => {
  const response = await fetch('http://localhost:4000/v1/chat/completions', {
    method: 'POST',
    headers: { 'Authorization': `Bearer ${process.env.LITELLM_API_KEY}` },
    body: JSON.stringify(req.body),
  });
  const data = await response.json();
  res.status(200).json(data);
});
export default router.handler();

Pros: zero vendor lock‑in, cheap at scale, can add custom middleware (e.g., request sanitization). Cons: you must monitor uptime, handle scaling, and keep the proxy up to date with provider changes.

OpenRouter: Hosted Aggregator

OpenRouter offers a managed API that already aggregates dozens of models. You get a single billing line, built‑in usage dashboards, and a community‑curated safety layer. The trade‑off is a small amount of extra latency (the request now hops through OpenRouter’s cloud) and a higher per‑token price compared to raw provider rates.

// lib/openrouter.ts
import axios from 'axios';

export async function chat(messages: {role:string;content:string}[]) {
  const {data} = await axios.post('https://openrouter.ai/api/v1/chat/completions', {
    model: 'anthropic/claude-2',
    messages,
  }, {
    headers: { Authorization: `Bearer ${process.env.OPENROUTER_KEY}` },
  });
  return data.choices[0].message.content;
}

Pros: no ops, instant access to new models, built‑in analytics. Cons: you lose the ability to tweak request routing, and you’re dependent on their SLA.

Portkey: Pay‑as‑You‑Go Gateway with Policy Engine

Portkey sits between your app and the providers, but unlike LiteLLM it’s a hosted service that lets you write policies in JavaScript. You can throttle expensive models, enforce content filters, or rewrite prompts on the fly without touching your codebase.

Example policy that caps Claude usage to 10 K tokens per day:

export default async function policy(request) {
  const usage = await getDailyUsage('claude-2');
  if (usage + request.max_tokens > 10000) {
    throw new Error('Daily token quota exceeded for Claude');
  }
  return request;
}

Pros: granular control, central audit logs, easy to swap providers. Cons: you pay a platform fee and you’re locked into their policy syntax.

Which Gateway Fits Your Stack?

• Small team, low traffic: OpenRouter wins. You get instant model variety and don’t have to manage a server.
• Mid‑size SaaS, strict compliance: LiteLLM + custom middleware lets you keep data on‑prem and audit every request.
• Enterprise product with cost caps: Portkey’s policy engine gives you the guardrails you need without building your own admin UI.

Practical Migration Tips

1. Abstract the LLM call behind a tiny wrapper (as shown in the Next.js examples). 2. Swap the endpoint URL and auth header to point at your chosen gateway. 3. Enable a thin logging layer that records model, token count, and latency—most gateways expose this via headers. 4. Run a smoke test on a staging branch before flipping production traffic.

Bottom line: you’ll end up with a gateway one way or another. Pick the one that matches the operational budget and the level of control you need, and treat the wrapper as part of your public API. That way your front‑end stays blissfully simple while the back‑end handles the messy multi‑model world.