ML and inference engineer
Inference and ML infrastructure engineers, and machine learning engineers at AI labs and GPU clouds.
5
system design cases
3
coding tasks with tests
H100, H200
real hardware math
Hardware figures match NVIDIA's product pages, read Nov 3, 2026.
Prepare for a company
Cases
Every question is open. The worked answers are in Pro, except the free ones.
10questions
Are you ready? Readiness check
800 msfirst token, p95
Serve a 70B coding assistant at 400 requests a second
24,000authorizations a second
Score card payments inside a 40 ms budget
168files of 5 GB
Ship an 840 GB checkpoint to 1,536 hosts
Halfthe time GPUs look idle
Run RL fine-tuning for a 32B model on 256 GPUs
3 weeksto launch
Review a teammate's design doc for a streaming API
GQAwith RoPE, in NumPy
Fix the attention layer, then add a KV cache
Same seedsame text
A reproducible sampler with top-k, top-p and speculative acceptance
3 partsearlier tests keep passing
Build a continuous-batching scheduler with paged KV memory
5 daysto send it
Take-home: make a slow inference server fast
What the job posts ask for
- Building and running distributed systems in production, in Python or Rust (Anthropic, Inference).
- LLM inference: batching, caching, quantization, request routing and load balancing.
- GPU programming, the memory hierarchy, and profiling with roofline models, Nsight and the PyTorch profiler (NVIDIA, LLM Inference).
- Deep learning and transformers in PyTorch, and fine-tuning methods such as distillation and SFT (OpenAI, ML Engineer).
What the companies say about their interviews
OpenAI
For engineering roles, looks for well-designed solutions, high-quality code, optimal performance, good test coverage and clear communication.
Their pageAnthropic
Take-homes and live interviews are done without AI unless they say otherwise. AI is welcome for preparation.
Their pageSkills and labs
Each case names the skills it tests. These are the labs that train them, by skill.
How LLMs work
5 labs
How LLMs work
5 labs
Evals
6 labs
Evals
6 labs
Inference serving and optimization
6 labs
Inference serving and optimization
6 labs
GPU performance and CUDA
5 labs
GPU performance and CUDA
5 labs
Kubernetes and GPU cluster operations
14 labs
Kubernetes and GPU cluster operations
14 labs
- Welcome to NCA-AIIO Labs
- NVIDIA GPU Operator on k3s
- Inside the NVIDIA GPU Operator
- Kubernetes Resource Requests & Limits
- Multi-GPU-Type Targeting
- PriorityClass & Preemption
- Workload Controllers
- Persistent Storage for AI Workloads
- GPU Container Lifecycle
- Rolling Updates, Rollback & Blue-Green for AI Inference
- Stuck-Pending Triage Day
- GPU Observability
- GPU Health Checks + Auto-Remediation
- Incident Drill
Deep learning, training and fine-tuning
9 labs
Deep learning, training and fine-tuning
9 labs
Serving at scale: routing, capacity, caching, cost
4 labs
Serving at scale: routing, capacity, caching, cost
4 labs
The behavioral round
No labs yet
The behavioral round
No labs yet
Practised in the follow-ups: your stories, told with numbers.
Formulas the cases use
- Weights memory = parameters × bytes per parameter.
- KV cache per token = 2 (keys and values) × layers × KV heads × head dimension × bytes.
- A decode step takes at least (bytes read per GPU) / (memory bandwidth). Decode is bound by memory bandwidth.
- Prefill costs about 2 × parameters FLOPs per token. Prefill is bound by compute.
- Training costs about 6 × parameters FLOPs per token.
Certification prep in the same plan
Optional. Practice exams for certifications close to this role.
A source tag says which company and role reported a format, how many independent sources describe it and when. It doesn't mean a company uses our question. Every scenario and number is ours. Preporato is not affiliated with Anthropic, OpenAI, NVIDIA. These cases help you prepare and don't promise an interview or an offer.