Preporato

Write the code behind machine learning

63 problems, from your first NumPy array to the attention inside a GPT. Solve each one in your browser, then practice it at full scale in our labs.

Start with the first problem See the path
one_step.py10 of 10 tests passed
import numpy as np def gradient_step(params, grads, lr): # Step against the gradient. return params - lr * grads w = np.array([-2.6, 1.35]) lr = 0.15 for step in range(30): w = gradient_step(w, grad(w), lr)
step 12 · loss 0.011lr 0.15

The path

Twelve steps in order. Each one builds on the one before.

  1. Step 1: NumPy and broadcasting

    Arrays, shapes and broadcasting: the language every ML library speaks.

    5 problems

  2. Step 2: Linear algebra for ML

    Dot products, projections and least squares, the math under every model.

    5 problems

  3. Step 3: Probability and statistics

    Expectations, uncertainty and information, computed from the definitions.

    5 problems

  4. Step 4: Derivatives and gradients

    How a change in one number moves the loss: the chain rule, in code.

    5 problems

  5. Step 5: Gradient descent

    The update rule behind every trained model, and how it goes wrong.

    5 problems

  6. Step 6: Linear and logistic regression

    The two models every data scientist ships first, built by hand.

    6 problems

  7. Step 7: Model evaluation

    Is the model any good? Metrics, thresholds and honest splits.

    6 problems

  8. Step 8: Trees and ensembles

    Splits, impurity and boosting: how tabular models really work.

    5 problems

  9. Step 9: Clustering and PCA

    Finding structure without labels: k-means and principal components.

    4 problems

  10. Step 10: Neural networks and backprop

    Layers, activations and backpropagation, written out by hand.

    6 problems

  11. Step 11: Optimizers and normalization

    Adam, weight decay, clipping and norms: what keeps training stable.

    5 problems

  12. Step 12: Attention and transformers

    From attention to a token: the parts of a GPT, one function at a time.

    6 problems

Preparing for an AI interview?

Cases at the level of the rounds candidates report at Anthropic, OpenAI and NVIDIA, with worked answers.

Interview prep

Solving is open to everyone. The worked solution to each topic's first problem is free; the others come with Pro, along with the labs.