What Is LiteLLM? Proxy Setup and Master Key Safety

LiteLLM is an open-source tool that lets your app talk to many AI providers (OpenAI, Anthropic, Google, local models and more) through one OpenAI-style API. You can use it as a Python library, or run it as a proxy server that hands out its own keys, tracks spending and logs requests. The proxy is protected by a master key, and that key deserves the same care as a database password. LiteLLM's own Oct 21, 2026 incident started with exactly that kind of key left exposed.

By Gary, Greenlight My App. Checked Nov 1, 2026. How we checked this: we read LiteLLM's incident posts, its GitHub advisory and its virtual-keys docs on Nov 1, 2026 and quote them below. All versions and times come from LiteLLM's own sources. We also tested the proxy on Nov 1, 2026: we installed litellm 1.106.2 (the version LiteLLM's notice says to pin) in a throwaway home folder, used a mock provider so no real API calls were made, created a limited virtual key, and checked what it could and couldn't do. Drafted with AI tools and reviewed by a person before publishing.

What is LiteLLM?

LiteLLM solves a boring but real problem. Every AI provider has its own API shape, its own SDK and its own error codes. If your app wants to try Claude today and GPT tomorrow, you end up rewriting code.

LiteLLM puts one interface in front of all of them. You send a request in the OpenAI format, name the model, and LiteLLM translates it for the provider.

It comes in two forms:

1. The Python SDK. pip install litellm, then call litellm.completion(...) in your code. Good for scripts and single apps.
2. The proxy server, also called the AI gateway. A small server your apps call instead of calling providers directly. It holds the real provider keys, gives out its own keys, enforces budgets and logs usage. By default it listens on port 4000.

The proxy is where the security questions live, so that's most of this post.

Does LiteLLM cost money?

The core project is open source on GitHub (external), and you can self-host it without paying LiteLLM. Some features, such as scheduled key rotation, are marked "Enterprise feature" in the docs and need a paid license. You still pay your model providers for usage, whichever way you run it.

How does the LiteLLM proxy work?

Picture three layers:

• Your apps call the proxy with a LiteLLM key.
• The proxy checks that key, applies its limits, and forwards the request.
• The providers receive the request using provider keys the proxy holds.

Your apps never see the real OpenAI or Anthropic key. That's the main security win: if one app leaks its key, you block that one virtual key, and your provider keys stay put.

A minimal config file looks like this (keys come from env vars, never typed in):

model_list:
  - model_name: gpt-main
    litellm_params:
      model: openai/gpt-4o
      api_key: os.environ/OPENAI_API_KEY

general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY
  database_url: os.environ/DATABASE_URL

The os.environ/NAME syntax tells LiteLLM to read the value from an environment variable, which is the pattern the docs show. Virtual keys need a Postgres database, hence DATABASE_URL.

Your app then calls the proxy like any OpenAI-compatible endpoint:

curl http://localhost:4000/v1/chat/completions \
  -H "Authorization: Bearer sk-XXXX" \
  -H "Content-Type: application/json" \
  -d '{"model": "gpt-main", "messages": [{"role": "user", "content": "Hello"}]}'

How do you run the LiteLLM proxy locally?

After the October incident, pin the version when you install. LiteLLM's incident post advised pinning to 1.106.2:

pip install 'litellm[proxy]==1.106.2'
export LITELLM_MASTER_KEY="sk-XXXX"
litellm --config config.yaml

Check LiteLLM's incident post and release notes for newer safe versions before you pick one. The proxy starts on port 4000. Keep it on localhost while you test; in ours we started it with --host 127.0.0.1 --port 4000.

What is the LiteLLM master key?

The master key is the admin password for your proxy. With it, someone can create and delete keys, change budgets and use every model you've configured, on your bill.

LiteLLM's docs say the master key "must start with sk-", and you set it with general_settings: master_key in the config or the LITELLM_MASTER_KEY environment variable.

Where should you store the LiteLLM master key?

Not in the config file, and never in Git. Good options, from most to least careful:

1. A secret manager (your cloud provider's, or a tool like Vault). LiteLLM's Oct 27 advisory specifically recommends a secret manager for the master key.
2. Your host's encrypted environment variables, set in the dashboard.
3. A local .env file, for development only, listed in .gitignore. Our .env explainer covers this.

If the master key ever lands in a commit, treat it as public. Rotate it right away, then follow our guide for a committed .env file.

The advisory also recommends setting LITELLM_SALT_KEY. Set it once and keep it safe.

How do you use LiteLLM virtual keys safely?

Virtual keys are keys the proxy creates for each app, teammate or customer. You generate them with the master key using the /key/generate endpoint:

curl http://localhost:4000/key/generate \
  -H "Authorization: Bearer sk-XXXX" \
  -H "Content-Type: application/json" \
  -d '{"models": ["gpt-main"], "max_budget": 10, "duration": "30d"}'

What the options do, per LiteLLM's docs:

• models limits which models the key can call.
• max_budget caps spending for that key.
• duration sets an expiry, so forgotten keys die on their own.
• allowed_routes limits which proxy endpoints the key can reach.

If a key leaks, block it with /key/block. That's the whole point: one leaked key, one quick fix.

What a limited virtual key did in our test

We created a key allowed one model (demo-chat), a $1 budget, and only the chat routes. Then we tried it:

• Admin routes refused. /key/generate and /key/list returned HTTP 403 with "Virtual key is not allowed to call this route."
• Other models refused. Asking for demo-other returned 403 with key_model_access_denied.
• Chat allowed. /chat/completions with demo-chat returned HTTP 200 and the mock reply.

Our test key had no expiry ("expires": null); in real use, add duration.

A simple rule set:

• One key per app or person. Never share keys between projects.
• Smallest budget that works. You can raise it later.
• Always set an expiry.
• Never put a virtual key in browser code. Call the proxy from your server. An API key in a frontend bundle is readable by every visitor.

Should the LiteLLM proxy be on the public internet?

LiteLLM's Oct 27 advisory recommends keeping the proxy off the public internet and limiting its outbound network access. If your apps run in the same cloud network, let them reach the proxy privately. If outside access is unavoidable, put it behind HTTPS, a firewall that allows only your app servers, and strong keys.

The same thinking applies to any model gateway, including hosted ones like OpenRouter: the key is the lock, so guard the key and limit what it can open.

What happened in the LiteLLM supply-chain attack?

On Oct 21, 2026, a malicious version of LiteLLM was published. Everything below comes from LiteLLM's own incident post (external) and attack-chain write-up (external). Times are UTC, as LiteLLM gives them.

• PyPI: litellm==1.107.0 was "live from 11:39:51 to 13:23 UTC" on Oct 21.
• Docker: the v1.107.0, latest, main-latest and main-stable images were compromised from 11:46:56 to 14:20 UTC.
• Safe versions: "All previous versions are safe".
• Enterprise: "Enterprise Customers running the private enterprise release of LiteLLM were not impacted".
• Pin advice: "pin your version to 1.106.2".

LiteLLM's notice (published Oct 21, 2026, checked Nov 1, 2026) says, word for word: "In the early morning of October 21, LiteLLM's release pipeline was compromised and pushed v1.107.0 to the docker registry and PyPi. This version contained compiled malware which steals credentials and loads remote-code execution tools." An Oct 23 update on the same page notes that some early assumptions were revised as the investigation went on, so read the latest version.

Some news coverage gave different dates. We've used LiteLLM's own dates and times throughout.

How did attackers get in?

LiteLLM's attack-chain post says an admin key for an internal LiteLLM platform was committed on May 27 to a public repository and never revoked. On Oct 10, that key was used to obtain a GitHub CLI token belonging to the organization's owner. The attackers then pointed the release workflow at a lookalike fork, and that's how the bad release was published.

LiteLLM's own conclusion: "This incident has provided a strong illustration of the risks associated with failing to rotate a potentially exposed key."

That lesson applies to every small team. A key in a public repo, even an old one, even one you think nobody noticed, is a live risk until you revoke it.

Was I affected by the LiteLLM attack?

You may have been affected if, during the windows above, you:

• installed or upgraded to litellm==1.107.0 from PyPI, or
• pulled the v1.107.0, latest, main-latest or main-stable Docker images.

Check your lock files, CI logs and image digests. If you're not sure, read LiteLLM's incident post (external) and treat yourself as affected.

What should you do if you ran the bad version?

LiteLLM's incident post lists the steps. In short:

1. Don't start recovery with pip install on the affected machine.
2. Delete the files LiteLLM names (cook.pth and sitecustomize.py) as described in the post (see the full list in the post).
3. Rebuild environments and images from clean sources, pinned to a safe version.
4. Rotate everything: the master key, all virtual keys, and every provider key the proxy held.
5. Watch provider bills and usage for calls you don't recognize. Our stolen API key guide covers what to look for.

What is the separate LiteLLM advisory from Oct 27?

LiteLLM also published GitHub advisory GHSA-w4c7-34wj-hqm8 (external), rated Critical, on Oct 27, 2026. It's a different issue from the supply-chain attack and applies to proxies running with an Enterprise license. The advisory lists affected versions below 1.103.10, 1.104.8, 1.105.6 and 1.106.5 on their release lines. If you run the Enterprise edition, read the advisory and upgrade per its guidance.

LiteLLM security checklist

• Version pinned, and not 1.107.0 or a Docker tag from the Oct 21 window.
• Master key starts with sk-, is long and random, and lives in a secret manager or encrypted env var.
• LITELLM_SALT_KEY set and stored safely.
• Every app or person has its own virtual key with models, budget and expiry.
• No LiteLLM keys in browser code or committed files.
• Proxy on a private network, not the open internet.
• Old keys found in any public repo revoked, even if they look unused.
• Provider dashboards checked for unexpected usage.

LiteLLM vs calling providers directly

The proxy is worth it once you have more than one app or teammate. With a single small app, direct calls and a well-kept server key are simpler.

FAQ

What is LiteLLM used for? Calling many AI providers through one OpenAI-style API, with virtual keys, budgets and logging when run as a proxy.

Is LiteLLM free? The core is open source and free to self-host. Some features are Enterprise-only. You still pay your model providers.

Was I affected by the LiteLLM attack? Possibly, if you installed litellm 1.107.0 from PyPI or pulled affected Docker tags on Oct 21, 2026. Check LiteLLM's incident post for exact times.

Which LiteLLM version is safe? LiteLLM said all versions before 1.107.0 are safe and advised pinning to 1.106.2. Check its incident post for later updates.

Where should I store my LiteLLM master key? In a secret manager or your host's encrypted env vars, read via LITELLM_MASTER_KEY. Never in a committed config file.

What port does the LiteLLM proxy use? Port 4000 by default. Keep it on a private network where you can.

Do I need a database for LiteLLM virtual keys? Yes. Virtual keys need a Postgres database, set with DATABASE_URL.

Running a model gateway behind your app? Run a free scan with Greenlight My App to check your live site's public code for exposed API keys and missing security headers.