LLM inference and quantization labs
Nine labs on the numbers that decide whether an LLM deployment is affordable: VRAM, latency, throughput, dollars.
Serving a language model is an optimization problem: how many requests per second can one GPU answer, at what latency, at what memory footprint, and at what cost per thousand tokens. Quantization (storing weights in 8-bit or 4-bit instead of 16-bit) shrinks memory and often speeds decoding; batching and paged key-value caches raise throughput; profilers show where the GPU is idle. These are the levers the NVIDIA NCP-GENL Model Optimization domain tests, and the ones our practice-test data shows candidates miss most.
The labs measure everything rather than asserting it. Load a model in fp16, INT8 and NF4 and benchmark VRAM, latency and output quality; sweep batch size and precision to find the throughput knee; build a mini Triton-style dynamic batcher in about thirty lines and load-test it; move to production serving with vLLM and measure PagedAttention capacity, continuous batching and prefix caching; cascade requests across model tiers on NIM and read the real cost field. Three labs close the loop on efficiency: the PyTorch profiler, Nsight Systems, and a four-stage GPU cost audit that turns NVML samples into dollars of waste.
GPU labs run on live NVIDIA pods in the browser; the routing lab is hosted. Each takes 25 to 55 minutes and is checked step by step.
The labs, in order
All labs- 1GPU Proncp-genl · quantizationIntermediate
Quantize & Optimize LLMs with bitsandbytes
Load a model in fp16, INT8, and NF4, then benchmark the three precisions on VRAM, latency, and output quality. See where quantization wins and where it costs you.
ncp-genlncp-adsnca-genl40 min4 stepsLaunch - 2GPU Proncp-genl · precision-sweepIntermediate
Batch Size & Precision Sweep: Finding Your Sweet Spot
Sweep batch sizes and numerical precisions (fp32, fp16, bf16) on a real model to find the throughput/VRAM knee, then ship a production recommendation with SKU-aware precision picks and an accuracy gate.
ncp-genlnca-aiioncp-ads40 min4 stepsLaunch - 3GPU Proncp-genl · inference-servingIntermediate
Inference Serving Patterns: Dynamic Batching, Throughput, and the Triton Mental Model
Build a mini-Triton inference server in ~30 lines of Python: a dynamic batcher with max_batch_size and max_queue_delay knobs, load-tested against a naive baseline, swept for the throughput-latency tradeoff, and bridged to a real Triton config.pbtxt.
ncp-genlncp-aionca-genl40 min4 stepsLaunch - 4GPU Pro# vLLM · 10 concurrent$ asyncio.gather(…)wall time ....... 3.8sthroughput ...... 1,842 tok/sncp-genl · deploy-serve-llms-jupyterIntermediate
Deploy & Serve LLMs in Production (Jupyter)
Go from slow single-request inference to production-ready LLM serving with vLLM. Benchmark throughput, tune settings, and learn when to use vLLM vs Triton vs TGI.
ncp-genlnca-genl45 min5 stepsLaunch - 5GPU Proncp-genl · vllm-servingAdvanced
vLLM Production Serving: PagedAttention, Continuous Batching, Prefix Caching
Stand up vLLM and measure the three features that make it the de-facto inference server: PagedAttention's KV-cache capacity, continuous batching throughput, and prefix caching speedups. Then write the production spec — server args, Kubernetes deployment, monitoring, autoscaling.
ncp-genlncp-aionca-genl55 min4 stepsLaunch - 6HOSTED Pro# model-routing-cascade · agentPOST /api/agent/invoke200 OK · gradedncp-aai · model-routing-cascadeIntermediate
Model Routing & Cost Cascade with NIM
Save 60–80% on inference by cascading queries through cheap → mid → expensive NIM models. Measure real costs via NIM's usage.cost field and compare against an always-large baseline.
ncp-aai25 minHostedLaunch - 7GPU Pro# torch.profiler · top ops$ table(sort=cuda_time)gemm ........ 41%memcpy ...... 22%softmax ..... 9%nca-aiio · pytorch-profilerIntermediate
Profile PyTorch Training with the Built-in Profiler
Instrument a training loop with torch.profiler, read the op-level table, inspect the Chrome/Perfetto timeline, and decide when to reach for Nsight Systems instead.
nca-aiioncp-genlncp-ads35 min4 stepsLaunch - 8GPU Pronca-aiio · nsight-profilingIntermediate
Nsight Systems Profiling: Finding the Bottleneck That Costs You 40% of Your GPU
Run the full profile-then-fix loop with NVIDIA Nsight Systems — instrument a training loop with NVTX ranges, capture a .nsys-rep, parse the NVTX summary to pinpoint the bottleneck, then apply a targeted fix and measure the speedup.
nca-aiioncp-aioncp-genl35 min4 stepsLaunch - 9GPU Proncp-aio · cost-auditIntermediate
GPU Cost & Efficiency Audit
Build a four-stage cost-audit pipeline — measure, classify, price, recommend — that turns raw NVML samples into dollar-denominated waste and specific remediation actions. The skeleton behind every enterprise GPU cost product.
ncp-aionca-aiionca-genl35 min4 stepsLaunch
Before you start
- Python and basic PyTorch
- What a token, a batch and a KV cache are (the labs define them again in context)
- No serving experience required
Every lab in this collection, plus every practice test on the platform
$29.99 per month or $290 per year. Cancel any time. Free labs need only an account.
Certifications these labs prepare you for
Frequently asked questions
Yes. GPU labs provision a dedicated NVIDIA GPU pod for your session, so the VRAM, latency and throughput numbers you measure are real, not simulated. The routing lab is hosted and calls NVIDIA NIM.
Model Optimization and GPU Acceleration are the two heaviest NCP-GENL domains, and quantization, batching, serving patterns and profiling appear across NCP-GENL, NCA-GENL and the NVIDIA infrastructure certifications.
The serving labs use vLLM, a hand-built dynamic batcher, and NVIDIA NIM; the profiling labs use torch.profiler and Nsight Systems. TensorRT-LLM concepts are covered in the exam guides on this site rather than as a lab.
The labs are included in Preporato Pro ($29.99 per month or $290 per year) with every practice test and lab on the platform.