Are you ready? Readiness check

All hardware figures are assumptions stated in each question.

1. (M7) A model has 60 layers and 8 KV heads of dimension 128. The KV cache is stored in FP8 (1 byte per value). How much cache does one 32,768-token sequence need?
2. (M7, M8) A 22B model in BF16 runs with tensor parallelism across 2 GPUs, each with 2.0 TB/s of memory bandwidth (assume). At batch 1, what is the fastest possible decode speed, in tokens a second? What would FP8 weights give?
3. (M1, M7) An 8B model reads a 10,000-token prompt on a GPU that delivers 400 TFLOPS of useful compute (assume). About how long does prefill take?
4. (M12) One GPU runs batches in t(b) = 4 ms + 0.5 ms × b. A request may wait for the batch ahead of it, then run in its own. Waiting plus running must fit in 30 ms. What is the largest batch, and the best-case throughput of one GPU?
5. (M7, M12) Time to first token is fine for most users. But whenever a few 100,000-token prompts arrive, time between tokens jumps for everyone on the same replicas. GPU utilization is high all day. Which change fixes the cause?
   • (a) Autoscale when GPU utilization passes 70%.
   • (b) Chunked prefill with a token budget per step, or a separate pool for long prompts.
   • (c) A bigger maximum batch size.
   • (d) A longer batching window.
6. (M8) Assume an H100 gives 989 TFLOPS dense BF16 and 3.35 TB/s. In decode, each weight in BF16 (2 bytes) is used for 2 FLOPs per sequence in the batch. About what batch size makes the weight matrix multiplies compute-bound?
7. (M11) You fully fine-tune a 7B model on 2 billion tokens on H100s at 40% of 989 TFLOPS (assume). About how many GPU-hours does it take?
8. (M1) The next-token probabilities are [0.5, 0.3, 0.15, 0.05]. With top-p = 0.9, which tokens remain, and what is the first token's probability after renormalizing?
9. (M5) A quantized model scores 146 of 200 on your eval; the original scored 150. On 14 questions only the original was right, and on 10 only the quantized model was right. Can you ship it and say "no regression"?
10. (M9, M12) A 200 GB checkpoint must reach 300 hosts with 50 Gb/s NICs. What is the lower bound on time? How long would one source host take, sending to each host in turn?

Answer all 10 on paper in 20 minutes, with units, before you look at the answer guide. Show the math where there is math. For the math questions, an answer within about 2 times the right one counts if your steps are right.