NCP-ADS Multi-GPU Scaling: Dask, Memory and Cloud Economics [2026]

GPU & Cloud Computing is 16% of the NCP-ADS exam, and it asks the questions that begin where a single comfortable GPU ends: the dataset that no longer fits, the workstation with four devices and a pipeline using one, the cloud bill that finance wants halved without touching the schedule. Underneath every stem in this domain sit two budgets, memory and money, and the graded answers are the moves that respect both. This guide covers the device memory model and the dtype arithmetic that solves most out-of-memory stems, the Dask-cuDF machinery for scaling past one GPU, the scale-up versus scale-out judgment, and the cloud economics patterns (spot instances, right-sizing, utilization) that the exam grades like engineering decisions, because they are.

The memory model: two pools, one budget

A GPU pipeline lives across two memories. Host memory (system RAM) is large and slow to reach from the device; device memory (GPU VRAM) is fast and finite, and it is where cuDF DataFrames, cuML models, and every intermediate buffer live. Transfers between the two cost real time, which is why the previous articles in this series keep repeating "stay on the device," and why this domain's questions treat an unnecessary transfer as a defect.

Out-of-memory stems are the domain's bread and butter, and most of them are arithmetic in disguise. A 21 GB float64 DataFrame is a 10.5 GB float32 DataFrame. A string column with thousands of repeats collapses into a categorical. A join's intermediate buffers scale with row width, so every dropped column shrinks the spike. The exam expects you to do this math before reaching for hardware: representation first, then chunking, then more silicon.

When the working set legitimately exceeds one device, two mechanisms extend the budget. Spilling moves idle data from device to host memory automatically (slower, but the job survives), and partitioning streams the dataset through the GPU in pieces so the resident set stays small. Both arrive properly with the Dask layer below.

Dask-cuDF: one API from laptop to cluster

Dask is the parallelism layer of the RAPIDS stack: it splits a DataFrame into partitions, builds a task graph over them, and schedules the work across workers. Dask-cuDF makes each partition a cuDF DataFrame on a GPU, which turns the pandas-like API you already write into a multi-GPU program.

The exam tests the setup and the model more than the internals. LocalCUDACluster (from the dask-cuda package) is the canonical single-machine entry point: it launches one worker per visible GPU, wires in device memory management and spill-to-host, and a Client connected to it distributes partitions across all devices. Stems that describe four GPUs and one busy device are asking for exactly this. The distractors recycle three misconceptions: more CPU threads (does not enlist GPUs), running the whole script once per GPU (no coordination, no data splitting), and imaginary APIs that assign a DataFrame to a list of devices.

Two operational facts round it out. Partition count matters: too few partitions underuse workers and make spills enormous, too many drown the scheduler in overhead, and the working heuristic is partitions that fit comfortably in a fraction of each device's memory. And lazy execution means Dask builds the graph and computes on demand, so profiling stems distinguish "defined the pipeline" from "ran it."

Scale up, scale out, or fix the representation

When a workload outgrows its hardware, the exam offers three families of response, and grades the order:

The pattern to internalize: hardware answers are graded correct only after the stem forecloses the cheap answers. If the question mentions float64 columns, unused fields, or object-dtype strings, the representation fix is the intended choice, and the bigger GPU is the trap for candidates who skipped the arithmetic.

Cloud economics: utilization is the metric

Cloud stems in this domain read like FinOps tickets: a bill, a schedule, a constraint, and four levers. The graded levers are consistent:

Spot and preemptible instances (discounted spare capacity the provider can reclaim) fit fault-tolerant batch work, and the qualifier the exam checks is checkpointing: a nightly training job that saves state every ten minutes loses at most ten minutes to a preemption, which makes the discount nearly free. Interactive serving and long uncheckpointed jobs are the wrong workloads for it.

Right-sizing and utilization: a GPU at 12% utilization is an economics bug before it is a performance one, and the fixes are batching (covered in the MLOps guide), consolidation of workloads onto fewer devices, and autoscaling that lets idle capacity go. Reserved capacity appears as the answer for steady, predictable baseline load, and as the distractor when the workload is spiky or experimental.

Data locality: moving hundreds of gigabytes between regions or in and out of object storage costs time and egress fees, so the graded architecture keeps compute next to data and reads columnar formats directly.

Worked scenario 1: four GPUs, one working

A feature pipeline on a four-GPU workstation processes 600 GB of Parquet monthly and takes nine hours, saturating a single device while three idle. The code is cuDF end to end. What is the graded change?

Stand up a LocalCUDACluster and convert the pipeline to Dask-cuDF: reads partition across the four workers, the task graph parallelizes the transforms, and the wall clock divides by roughly the device count for a partition-friendly workload. The distractors are the usual suspects: threads (wrong resource), four manual copies of the script (no coordination), and a bigger single GPU (buys memory, and the stem's problem is throughput). If the stem also mentions float64 everywhere, do the representation fix in the same breath.

Worked scenario 2: the training bill

Nightly model training on on-demand cloud GPUs takes three hours, checkpoints every ten minutes, resumes cleanly, and must finish by 6 a.m. Finance wants a materially smaller bill with the same schedule. Which lever, and what is the risk trade?

Spot instances with checkpoint resume: the workload is the textbook fit (batch, fault-tolerant, deadline with slack), the saving is the largest available without touching the job, and the quantified risk is minutes of recompute per preemption plus a rare bad night, which the 6 a.m. buffer absorbs. Reserved capacity is the runner-up distractor (helps steady loads, does nothing dramatic here), fewer epochs changes the model (the stem said schedule only), and consolidation onto a bigger on-demand card rearranges the same spend.

Next steps

This domain converts to points fastest through reps: run the CUDA fundamentals lab, then take a timed full-length practice test and note how many GPU items became arithmetic. The study plan places this domain in week three, right after the DataFrame layer it builds on.

Sources:

• Dask-CUDA documentation
• Dask-cuDF documentation
• RAPIDS Memory Manager (RMM) documentation
• NVIDIA Accelerated Data Science Professional certification