Project a direction out of embeddings
Medium12 min10 hidden tests
A review-search team finds that all their sentence embeddings share one strong direction, v, that mostly tracks how long a review is. They want similarity to reflect what a review says, so they remove that direction. Write remove_direction(X, v) that projects v out of every row of X: each row loses its component along v and keeps everything else. The vector v can have any nonzero length.
Drag v
x − (x · u) u, with u = v / |v|u = v / |v| = (0, 1). The row (3, 4) has x · u = 4, so it becomes (3, 4) − 4 × u = (3, 0). Every result sits on the dashed line, where its dot product with v is 0.
u is (0, 1). Row 3, 4 becomes (3, 0).
Examples
The direction (0, 2) is the second axis, so the second coordinate goes to 0
remove_direction([[3, 4], [1, 0]], [0, 2])
returns [[3, 0], [1, 0]]
A row that points along v becomes all zeros
remove_direction([[2, 0], [1, 1]], [1, 1])
returns [[1, -1], [2.22045e-16, 2.22045e-16]]
Requirements
Xis a 2-D list or array of shape (n, d), andvis a nonzero list or 1-D array of length d.- Return a float array of shape (n, d). Each row x becomes x - (x . u) u, where u = v / |v| is
vscaled to length 1. - Every output row has a dot product of 0 with
v, up to rounding. - Leave
Xandvunchanged.
The idea behind it
Any vector x splits into two parts relative to a direction u of length 1: the part along u, (x . u) u, and the part at right angles to it, x - (x . u) u. Removing a direction means keeping only the second part. This is called projecting out u, and the result has a dot product of 0 with u.
The unit length matters. For a general v the part along it is (x . v) / (v . v) * v. Scaling v to u = v / |v| first makes that (x . u) u, and forgetting the scaling makes the removed amount wrong by a factor of |v| squared.
Teams use this to clean embeddings. A shared direction tied to text length, a language or a sensitive attribute can dominate cosine similarity, and projecting it out makes the remaining directions count. The same step is the core of Gram-Schmidt orthogonalization and of removing the top principal component from a set of vectors.