The compute fabric for a regional GPU cloud

Sundara Cloud (fictional), a regional GPU cloud in Southeast Asia, is fitting out a new hall: 192 nodes with 8 GPUs each, 1,536 GPUs. Tenants rent from 1 node up to 64 nodes. Next year Sundara wants 4,096 GPUs in the same building.

Assume:

• Each node has 8 compute network cards at 400 Gb/s (one per GPU), 2 ports at 200 Gb/s for storage and tenant traffic, and a 1 Gb/s management port.
• Compute switches have 128 ports at 400 Gb/s.
• Prices: $90,000 per switch, $1,000 per 400 Gb/s transceiver, two transceivers per link. A node costs $300,000.
• Sundara rents GPUs at $2.20 per GPU-hour and plans on 70% of hours rented.
• The network team runs a large EVPN-VXLAN Ethernet network today and has no InfiniBand experience.

Asks:

1. Design a non-blocking compute fabric: topology, number of leaf and spine switches, number of links. Can it grow to 4,096 GPUs without a third tier?
2. The CFO asks for 2:1 oversubscription to save money. How much does it save, and what does it cost tenants?
3. InfiniBand or Ethernet with RoCE, for a multi-tenant cloud? How do you keep tenants apart?
4. What other networks does the hall need?

Write down your leaf count, spine count and link count for a non-blocking fabric, and one sentence for the CFO on 2:1 oversubscription.