LLM security course: defend against the OWASP LLM Top 10

Defending an LLM application means putting controls between untrusted text and the actions the system can take, then proving those controls hold without breaking normal use. Because the model reads retrieved documents, tool descriptions, and prior outputs as instructions, a single guardrail is rarely enough; defense in depth wires an independent control at each point where untrusted data enters or leaves. The controls line up with the OWASP Top 10 for LLM Applications: prompt injection, insecure output handling, sensitive information disclosure, excessive agency and supply chain. These labs harden the same deliberately-vulnerable Retrieval-Augmented Generation (RAG) assistant and tool-using agent that the offensive labs attack, which is exactly the guardrail and human-oversight work weighted in NVIDIA NCP-AAI and in the Governance, Safety & Risk Management domain of Anthropic's CCAR-P. The controls are cataloged against the OWASP Top 10 for LLM Applications and MITRE ATLAS by name.

You begin with the shape of a complete defense: four independent control points around a RAG assistant (input mediation, retrieval and context control, output mediation, and action authorization), plus a guardrail layer wired to an attack-success-rate (ASR) CI gate that goes red the moment a change re-opens a hole. Then you lock down the RAG pipeline: an egress allow-list and provenance isolation that close the EchoLeak markdown-image exfil channel, a RAG firewall that rejects directive-shaped ingestion and enforces a server-side tenant predicate the caller cannot widen, and a secret vault boundary with a canary tripwire that keeps a system-prompt secret out of the model's context. The last stage hardens the agent: least-privilege tool scope with per-argument authorization and a human-in-the-loop approval gate, a signed and hash-pinned tool registry that refuses poisoned, rug-pulled, or shadowing manifests, and a contextual output mediator that encodes for each sink so cross-site scripting, server-side request forgery, SQL injection, and code execution all fail.

Every lab reproduces the exploit for you first, so you can start here without doing the offensive labs. All eight run in the browser against a sandboxed target with no GPU, take 80 to 90 minutes, and end on a ship gate where attack-success-rate is zero and benign traffic still passes clean through.