Take-home: an AI platform for a bank

The brief, as sent to the candidate

Aurelia Bank (fictional), a retail and commercial bank, wants an on-premises AI platform. Design it.

Workloads:

1. Continued pretraining. Adapt a 14-billion-parameter dense model on 500 billion tokens of internal and licensed text, twice a year. Each run must finish within 12 days.
2. Fine-tunes. About 30 LoRA fine-tunes a month, about 200 GPU-hours each.
3. Overnight documents. 2.4 million pages a day (loan files, identity checks) through a vision-language model, inside an 8-hour overnight window. About 1,800 input tokens per page (image and prompt) and 300 output tokens.
4. Internal assistant. Runs on the 14B model, weekdays from 8 to 18. Peak 120 requests a second, 1,500 tokens in (a 600-token system prompt shared by all requests) and 300 out. Time to first token under 1 s at p95, time between tokens under 50 ms at p95.

Constraints:

• All data and models stay on the bank's premises, in the country.
• 420 kW of IT power. Air-cooled racks up to 35 kW.
• The team knows VMware, Linux and Ethernet well, and some Kubernetes.
• Regulators require that every model version and every inference request (input, output, model version, time) be kept for 7 years, unchangeable.
• Capital ceiling: €20 million, including 3 years of support. Go-live in 9 months.

Assume:

• 8-GPU H100 SXM nodes at €300,000 each, up to 10.2 kW. Per GPU: 989 TFLOPS dense BF16, 1,979 TFLOPS dense FP8, 80 GB at 3.35 TB/s. 8 network ports at 400 Gb/s per node.
• Ethernet or InfiniBand switches with 64 ports at 400 Gb/s, €60,000 each. Storage at €800 per usable TB.
• 14B model: 48 layers, grouped-query attention with 8 KV heads of dimension 128.
• Vision-language model: 11 billion parameters, 40 layers, 8 KV heads of dimension 128.
• Any other number you need: state it as an assumption and say how you would confirm it.

Deliverables:

1. A design document of at most 10 pages: the requirements as you understood them, with open questions; the architecture (compute, fabric, storage, scheduling, security, observability); the sizing summary; the trade-offs; a phased plan to go-live.
2. A sizing workbook (spreadsheet or Python notebook) with every calculation and assumption, so a reviewer can change one input and see the result.
3. A bill of materials: nodes, switches, links, storage capacity, racks and power.
4. An acceptance test plan with pass marks.
5. A risk register: the top 8 risks, each with likelihood, impact, mitigation and owner.
6. A one-page summary for the bank's CIO.

Time box: about 6 hours of work, within 7 days. Then a 45-minute debrief with interviewers playing the CIO and the head of infrastructure.

Rules: do the work yourself. If a real employer sends you a take-home, follow their rules on AI tools. State every assumption. Where information is missing, write the question and the assumption you used.

Before you size anything, list the questions you would send the customer, and the assumption you will use for each one if no answer comes.