Build an LLM Gateway: One Door Between Your Apps and the Model
Hosted · ide
Beta

Build an LLM Gateway: One Door Between Your Apps and the Model

Put a gateway you wrote in front of the model and give it the four floor capabilities every LLM feature needs: API keys with a request log, a price on every call, per-app rate limits and daily budgets, an exact-match response cache, retries with backoff and idempotency keys, model routing with a fallback, and a report over a replayed morning of traffic that shows who spent what.

90 min8 steps3 domainsIntermediate
Part of the AI Engineer Course

Hands-on labs require Pro · $29.99/mo · cancel anytime

What you'll learn

  1. 1
    Pass-through: one door, and the model behind it
    Northwind's three apps call the model with one shared provider key, from
  2. 2
    Keys and the log: who is calling, and a record of every call
    The architecture lecture's floor has four capabilities, and two of them are
  3. 3
    Price every call on the model that was actually served
    The bill is the sum of every call's tokens times the model's rate. The
  4. 4
    Rate limits and daily budgets: stop the spend before it happens
    The provider has rate limits, but they protect the provider. When the
  5. 5
    An exact-match cache that cannot collide
    The support bot answers "How do I reset my password?" a few hundred times a
  6. 6
    Retries with backoff, and idempotency keys so clients can retry too
    Providers fail in three ways that deserve a retry and one that does not.
  7. 7
    Route by prompt and tier, fall back when a model is down
    Apps have been sending fast and smart since Step 1, and the gateway has
  8. 8
    Replay a morning of traffic and report who spent what
    traffic.jsonl is a recorded morning: the support bot asking its usual

Prerequisites

  • Python: classes, dicts, json, try/except, a while loop
  • HTTP basics: status codes, headers, JSON bodies
  • Read the module's caching, rate-limit and architecture lectures first

Exam domains covered

Using LLMs via APILLM OperationsCost Optimization

Skills & technologies you'll practice

This intermediate-level ai/ml lab gives you real-world reps across:

LLM GatewayRate LimitingCachingRetriesModel RoutingLLM APICost Optimization

Why put a gateway between your apps and the model

The demo architecture is a browser, an arrow and a provider. The first production architecture adds one box in the middle, and that box is where authentication, logging, rate limiting and observability live. Commercial gateways sell those four things plus caching, retries and routing. Building a small one yourself, on the plain chat completions API, shows what each of those features costs to get right and makes the commercial ones easy to evaluate. The lab is a single Python class behind a plain HTTP server. Each step adds one capability and tests it against a deterministic fake upstream: keys that are attributed and revoked, a request log, prices keyed on the model the provider actually served, a sliding rate-limit window and a daily budget, an exact-match cache that must not collide across users, exponential backoff that honours retry-after and stops for non-retryable errors, idempotency keys that make client retries safe, routing by prompt size and tier with a fallback model, and a per-app report over a replayed morning of traffic with cache hit rates, spend and p95 latency.

Frequently asked questions

Do I need FastAPI, LiteLLM or another gateway product for this lab?

No. The gateway is a Python class behind the standard library's HTTP server, and the model is reached through the OpenAI-compatible chat completions endpoint. Knowing what the class does makes any commercial gateway's feature list readable.

Which model does the lab call?

Hosted Llama 3.1 8B and Llama 3.3 70B instruct models behind the lab's proxy for the live steps. A deterministic fake upstream with a controllable clock stands in for the failure, rate-limit, cache and replay steps, so those checks are exact and cost nothing.

How is cost attributed per app?

Every request carries an API key that maps to a user and an app. The gateway prices each call from the usage the provider returns, keyed on the model name the provider actually served, writes it to the request log, and the report rolls spend up per app and per user.

What stops a client retry from charging twice?

An Idempotency-Key header. The gateway stores the response for that key and replays it on a repeat, so a client that retries after a network error gets the same answer without a second upstream call.