Mini-batches in a given order
Easy10 min9 hidden tests
A house price model trains on mini-batches: small groups of rows, with one gradient step per group. The data loader has already shuffled the row numbers into order. Write make_batches(X, y, order, batch_size) that walks through order and cuts the rows into batches of batch_size. When the rows do not divide evenly, the last batch is shorter, and no row is dropped.
Change the batch size or the order
batches of batch_size rows, in `order`order
batches
batch 0
X [[4, 40], [1, 10]]
y [400, 100]
batch 1
X [[5, 50], [2, 20]]
y [500, 200]
batch 2, the rest
X [[3, 30]]
y [300]
order = [3, 0, 4, 1, 2] in batches of 2: [3, 0], [4, 1], [2]. The last batch holds the 1 row left over, so no row is dropped. Each row’s features and target travel together, at the same position.
3 batches.
Examples
Five rows in batches of 2: the last batch holds one row
make_batches([[1.0, 10.0], [2.0, 20.0], [3.0, 30.0], [4.0, 40.0], [5.0, 50.0]], [100.0, 200.0, 300.0, 400.0, 500.0], [3, 0, 4, 1, 2], 2)
returns [([[4, 40], [1, 10]], [400, 100]), ([[5, 50], [2, 20]], [500, 200]), ([[3, 30]], [300])]
One full batch, in the shuffled order
make_batches([[0.5], [1.5], [2.5]], [1.0, 2.0, 3.0], [2, 1, 0], 3)
returns [([[2.5], [1.5], [0.5]], [3, 2, 1])]
Requirements
Xis an (n, d) array or list of lists,yholds n targets,orderlists every row number from 0 to n - 1 once, andbatch_sizeis 1 or more.- Return a list of tuples
(X_batch, y_batch)in the order the rows appear inorder.X_batchis a 2-D array of shape (b, d), andy_batchis a 1-D array of shape (b,). - Every batch has
batch_sizerows except possibly the last one, which holds the rest. Never return an empty batch. - Keep each row's features and its target in the same batch, at the same position.
The idea behind it
Gradient descent on the whole dataset takes one step per pass over the data. Mini-batch gradient descent splits the data into small groups, called mini-batches, and takes one step per group, so the model learns many times per pass. Each full pass is an epoch. Before an epoch, the loader shuffles the row numbers, so batches mix different examples and the gradients do not follow the file's order. Two details trip people up. First, the features and the targets must be indexed with the same row numbers, or the model learns from mismatched labels and the loss stays high for no clear reason. Second, when n is not a multiple of the batch size, the leftover rows form a short last batch. PyTorch's DataLoader keeps it by default and drops it only when you pass drop_last=True.