Readiness check: 10 questions

Use these figures where needed (H100 SXM): 989 TFLOPS dense BF16, 80 GB of memory at 3.35 TB/s, NVLink at 900 GB/s in total (450 GB/s each way), PCIe Gen5 x16 at 128 GB/s in total (64 GB/s each way), one network port at 400 Gb/s (50 GB/s).

1. A 13-billion-parameter model runs at batch size 1 on one H100. What is the fastest possible decode speed in BF16, and in FP8? Why does compute not set the limit?
2. You must move 40 GB of model state from one GPU to another. How long does it take over NVLink, over PCIe Gen5 x16, and over one 400 Gb/s network port? What does this tell you about where tensor parallelism should run?
3. A model has 48 layers and grouped-query attention with 8 KV heads of dimension 128, with a BF16 KV cache. How much KV cache do 24 sequences of 16,000 tokens need? With 16 GB of weights, does it fit on one 80 GB GPU? If not, name two fixes.
4. 64 GPUs each hold 10 GB of gradients and run a ring all-reduce. How many bytes does each GPU send, and how long does it take at 45 GB/s per GPU?
5. 20 data scientists share 4 H100 GPUs for notebooks. Compare MIG, time-slicing and MPS, and pick one.
6. How long does it take to train a 7-billion-parameter model on 1 trillion tokens with 64 GPUs that each deliver 400 TFLOPS in practice?
7. A 2-node nccl-tests run on H100 nodes with 8 ports of 400 Gb/s each gives 9 GB/s of bus bandwidth. NCCL_DEBUG=INFO shows NET/Socket. What happened, and what do you check first on Kubernetes?
8. What does RoCE need from the network to perform well for training, and what does InfiniBand give you by design?
9. A pod that requests 2 GPUs stays Pending with "Insufficient nvidia.com/gpu", but nvidia-smi on the nodes shows idle GPUs. Give three likely reasons.
10. A customer's VP asks you to commit, in writing, that their training run will finish in 30 days. What do you say?

Answer all 10 questions on paper in 25 minutes, without notes. Show the math where there is math. Then score yourself with the answer guide.