Step 1: Tables become tensors
Brightline Books pays postage by weight, and the warehouse wants to price parcels before the books are
even picked. books.csv lists 400 books with their pages, whether they are hardback, their height and
their weight in grams. You will teach PyTorch to predict the weight, and along the way meet the four
things every deep learning model is built from: tensors, gradients, a training loop and layers.
A tensor is PyTorch's array: a table of numbers with a shape such as (400, 3) (400 rows, 3
columns) and a dtype such as float32, the number type neural networks use.
Pages run into the hundreds while hardback is 0 or 1. Training works best when every column is on the same scale, so you standardise: subtract each column's mean and divide by its standard deviation.
1. Write to_tensors(df, features, target). torch.tensor(df[features].to_numpy(), dtype=torch.float32)
gives X. Use df[[target]], with two brackets, for y, so it stays a column of shape (400, 1).
2. Write standardise(X). X.mean(dim=0, keepdim=True) averages down the rows, one value per
column. (X - mean) / std then works on every row at once (broadcasting).
3. Answer the question below, then Run.
tensors.py, the file you edit79 lines
"""PyTorch in 45 minutes: tensors, autograd, gradient descent by hand, then the same with torch.nn."""
import pandas as pd
import torch
from torch import nn
torch.set_num_threads(1) # the lab has one CPU; more threads only fight over it
# ---------- Step 1: tables become tensors ----------
def to_tensors(df, features, target):
"""(X, y): X is a float32 tensor of shape (rows, len(features)); y is float32 of shape (rows, 1)."""
# TODO (Step 1): X = torch.tensor(df[features].to_numpy(), dtype=torch.float32)
# y = the same for df[[target]] (double brackets keep it 2-D: shape (rows, 1))
raise NotImplementedError("Step 1: write to_tensors()")
def standardise(X):
"""(X_scaled, mean, std): every column shifted to mean 0 and scaled to standard deviation 1.
mean and std have shape (1, columns), so they broadcast over the rows."""
# TODO (Step 1): mean = X.mean(dim=0, keepdim=True); std = X.std(dim=0, keepdim=True)
# return (X - mean) / std, mean, std
raise NotImplementedError("Step 1: write standardise()")
# ---------- Step 2: autograd ----------
def square_error_grad(w, x, y):
"""dL/dw for L = (w * x - y) ** 2, computed by autograd, as a Python float."""
raise NotImplementedError("square_error_grad() arrives in Step 2")
def backward_twice(w, x, y):
"""w.grad after computing the same loss and calling backward() twice without zeroing, as a float."""
raise NotImplementedError("backward_twice() arrives in Step 2")
# ---------- Step 3: gradient descent by hand ----------
def fit_by_hand(X, y, lr=0.1, steps=200):
"""Fit y ≈ X @ w + b by gradient descent on the mean squared error.
Returns (w, b, losses): w of shape (columns, 1), b of shape (1,), losses a list of floats, one per step."""
torch.manual_seed(0)
w = torch.zeros(X.shape[1], 1, requires_grad=True)
b = torch.zeros(1, requires_grad=True)
losses = []
for _ in range(steps):
raise NotImplementedError("fit_by_hand() arrives in Step 3")
return w.detach(), b.detach(), losses
# ---------- Step 4: the same with torch.nn ----------
def fit_linear(X, y, lr=0.1, epochs=200):
"""The same fit with nn.Linear, nn.MSELoss and torch.optim.SGD. Returns (model, losses)."""
torch.manual_seed(0)
raise NotImplementedError("fit_linear() arrives in Step 4")
def in_real_units(model, std):
"""The weights in the original units (grams per page, per hardback, per cm): weight / std, as a list of floats."""
raise NotImplementedError("in_real_units() arrives in Step 4")
# ---------- Step 5: when a straight line is not enough ----------
def fit_mlp(X, y, hidden=64, lr=0.02, epochs=1500):
"""A small network: Linear -> ReLU -> Linear -> ReLU -> Linear, trained with Adam on the MSE. Returns (model, losses)."""
torch.manual_seed(0)
raise NotImplementedError("fit_mlp() arrives in Step 5")
def mse(model, X, y):
"""Mean squared error of a model on (X, y), as a float, without tracking gradients."""
with torch.no_grad():
return float(((model(X) - y) ** 2).mean())
def split(X, y, val_share=0.2):
"""A fixed shuffle, then the last val_share of rows for validation: (X_train, X_val, y_train, y_val)."""
g = torch.Generator().manual_seed(0)
idx = torch.randperm(len(X), generator=g)
cut = int(len(X) * (1 - val_share))
return X[idx[:cut]], X[idx[cut:]], y[idx[:cut]], y[idx[cut:]]books.csvtry_it.py