LiteLLM vs a managed gateway: buy or build for Claude

While Claude is used by a couple of engineers, a gateway feels like overhead. Once usage grows past that, the same question comes up: do we put a proxy in front of the Anthropic API, and if we do, do we run it ourselves or pay for one? In 2026 the default open-source answer is LiteLLM, a popular Python proxy that hides 100+ model providers behind one OpenAI-compatible API. The default managed answer is codexkey. What follows is a level-headed comparison: real TCO numbers, the operational work you take on, and the security and compliance cases where each option simply wins.

What LiteLLM actually is

LiteLLM began as a thin Python library. A single litellm.completion() function smoothed over the calling conventions of OpenAI, Anthropic, Cohere and others. Today it is a full proxy server with a PostgreSQL-backed control plane: virtual API keys, per-key budgets, rate limits, team and user hierarchies, an audit log, an admin UI and webhook callbacks. You ship it as a container, give it the keys of one or more upstream providers, and your application talks to the proxy through the OpenAI SDK with a custom base_url.

It is good software. It is also software you now own, with the patches, incidents and capacity planning that come with it.

The real TCO of self-hosting LiteLLM

The price tag on LiteLLM is zero. Run it as a production gateway for, say, 20 engineers and a few internal apps, and the monthly bill looks closer to this:

• Compute. Two small instances behind a load balancer plus a staging copy. Count $80-$150 a month on any major cloud, load balancer included.
• PostgreSQL. A managed Postgres for the control plane (keys, spend, audit logs): $50-$120 a month for a small HA tier with backups.
• Redis. Without it, rate limiting stops being accurate once you run more than one proxy replica. $20-$50 a month.
• Observability. Logs, metrics and traces in Datadog, Grafana Cloud or whatever you already use. Once you really instrument the proxy, add $50-$200 a month.
• Egress and TLS. Certificates are free. Egress to Anthropic is not, and at scale you notice it.

That adds up to roughly $200-$500 a month of pure infrastructure before anyone has touched it. Infrastructure is the cheap part.

People are the expensive part. A production gateway needs:

• An on-call rotation that owns the proxy when Anthropic has a regional incident, when your Postgres fails over at 03:00, or when a runaway agent drains a virtual key's budget in ten minutes.
• Security patches for the LiteLLM container, its base image and its Python dependencies. The proxy sits on the path of every prompt and every response, so a CVE there cannot wait.
• Regular rotation of the upstream Anthropic keys, the virtual keys of internal teams and the proxy's own admin credentials.
• Upgrades. LiteLLM releases often and now and then brings schema migrations or config-breaking changes. Someone has to read the changelog and rehearse each upgrade on staging.

Price a senior engineer at $150/hour fully loaded, assume four hours a week of steady-state ownership plus the odd incident, and you get $2 000-$3 000 a month of human time on top of the infrastructure. The honest TCO of "free" LiteLLM at modest scale is closer to $2 500-$3 500 a month than to zero.

What the managed codexkey gateway covers

codexkey provides named API keys, Premium and Cheap channels, an optional Premium fallback for Cheap, key expiry and usage statistics. It serves both OpenAI-compatible and Anthropic-native endpoints. What the dashboard key API does not expose: per-key budgets, RPM/TPM, concurrency, model allowlists, IP filters and IDE restrictions.

So at modest scale the buy-or-build question is not really about infrastructure. You are comparing two sums: the margin codexkey takes on tokens against the $2 500-$3 500 a month you would otherwise put into running LiteLLM. As long as you stay within a couple of hundred million tokens a month, the managed option wins comfortably.

Feature comparison: LiteLLM vs codexkey

For Claude workloads the feature set is nearly the same. Two things differ: who runs it and how many providers it covers.

Security: who sees your prompts

This question should weigh more than TCO.

With self-hosted LiteLLM, prompts and completions travel through:

1. Your application.
2. Your LiteLLM proxy (in your VPC).
3. Anthropic's API.

Three parties, two of them yours. Logs, if you keep any, sit in your infrastructure under your own retention and access rules.

With codexkey, the path is:

1. Your application.
2. The codexkey gateway.
3. Anthropic's API.

Also three parties, but one of them is us. We keep prompt and completion content only as long as billing and debugging a specific request require, and we do not train on it. Still, you rely on our word and our SOC 2 controls instead of your own. For most teams that is enough. For some it is not.

Compliance: when self-hosting is required

In some situations self-hosting is not a preference but a rule:

• HIPAA workloads where a BAA with the gateway vendor is off the table. If your only BAA-covered route to Claude is straight to Anthropic through AWS Bedrock, the gateway has to live inside your HIPAA boundary.
• A rigid SOC 2 or ISO 27001 boundary. If your audit scope says no third-party processor touches customer data, a managed gateway becomes a new subprocessor. You would have to disclose it, assess it and defend it to the auditor.
• Data residency mandates. EU-only or RU-only routing, where traffic must not leave a specific region.
• Air-gapped or VPC-only environments. Nothing to debate here.

If one of these is yours, run LiteLLM and close the TCO discussion.

Code: what the setup looks like

Self-hosted LiteLLM, minimum viable production:

# 1. Postgres + Redis already provisioned, then:
docker run -d --name litellm \
  -e DATABASE_URL=postgres://... \
  -e REDIS_URL=redis://... \
  -e ANTHROPIC_API_KEY=sk-ant-... \
  -e LITELLM_MASTER_KEY=sk-master-... \
  -p 4000:4000 \
  ghcr.io/berriai/litellm:main-stable \
  --config /app/config.yaml

On top of that come a config.yaml with your model list, Terraform for the load balancer, alerting, backups and an upgrade runbook.

Then your application:

from openai import OpenAI

client = OpenAI(
    api_key="sk-virtual-key-for-this-team",
    base_url="https://litellm.internal.example.com/v1",
)
resp = client.chat.completions.create(
    model="claude-sonnet-5",
    messages=[{"role": "user", "content": "hello"}],
)

The same task through codexkey:

from openai import OpenAI

client = OpenAI(
    api_key="cdx-...",
    base_url="https://api.codexkey.online/v1",
)
resp = client.chat.completions.create(
    model="claude-sonnet-5",
    messages=[{"role": "user", "content": "hello"}],
)

The application code has the same shape by design. That is the point of the OpenAI-compatible standard. What differs is that the second snippet is the whole setup.

When LiteLLM is the better pick

We will not sell codexkey as the answer for everyone. LiteLLM fits better if:

• You need your own keys for many providers (OpenAI, Anthropic, Bedrock, Vertex, Azure OpenAI, Cohere, Mistral, Together, Groq, your own vLLM) behind one endpoint. codexkey is Claude-focused, while LiteLLM is the multi-provider Swiss army knife.
• You want custom Python routing logic inside the proxy: pre-call hooks that rewrite prompts, post-call hooks that redact PII, fallback chains across providers, or guardrails right in the request path.
• Your compliance boundary rules out a managed subprocessor, as covered above.
• A mature platform team already runs your infrastructure, and one more service is a rounding error to it.

Bottom line

If you are a Claude-first team at up to a couple of hundred million tokens a month, buying is the obvious call: a managed gateway like codexkey costs less than the fully loaded price of running LiteLLM, and your engineers build product instead of patching a proxy. If you work with several providers, need custom routing in the request path, or sit inside a compliance boundary that forbids a third-party gateway, run LiteLLM yourself and budget for what that really costs.

Either way, you still want a gateway. Calling Anthropic directly from twenty different services on a shared root key is the worst setup of the lot.