Why is the dot product connected to projection?
You have already used coordinates, vector lengths, and normalization. Here we also use angles: 0° means the same direction, 90° means perpendicular, and 180° means opposite directions. We will introduce the formulas after two experiments.
Imagine a blue arrow a above a horizontal line. Drop a perpendicular from its tip to that line. The green arrow from the origin to the landing point is its “shadow.” The orange arrow b points right along the same direction.
The explorer also reports a number called the dot product. Can you work out how that number relates to the shadow?
Keep a record of your investigation
Keep an experiment record in a notebook or text file: settings, prediction, observed signed shadow length, dot product, and projection vector. Use angle 0° with lengths 10 and 1 as the first reference. Change only the angle in the first experiment; in the second, fix and the direction of , then change only its length. Before opening an explanation, compare a trial with its reference and write what the evidence says about your prediction.
First experiment: rotate without stretching
Keep a at length 10 and b at length 1. Before trying the controls, predict:
- At 0°, how long should the shadow be?
- At 90°, can the shadow disappear even though the arrow still has length?
- At 180°, which direction should the shadow point? What sign would record that direction?
Now select , , and . Then try . Compare the shadow and the two numerical readouts.
Blue: vector aGreen: projectionOrange: vector b
b points right and is drawn below the line for visibility. Both vectors use the same scale.
What changed even though the arrow’s length stayed fixed? Explain the sign before opening the comparison.
Compare your observations
The signed shadow length is 10 at 0°, 0 at 90°, and −10 at 180°. The minus sign records a direction opposite to b, not a negative geometric length. At 60°, it is 5. With b at length 1, the dot product matches this signed number in every case.
The shadow lies on the whole line through b. It can extend beyond the tip of b or point backward.
Second experiment: can the number change without the shadow moving?
In the same explorer, select . This sets b to . Record the signed shadow length, the dot product, and the projection vector.
Before changing from 1 to 2, predict which of those three quantities will change. Keep a fixed, make the change, and compare.
Now predict the results for length 3 and test again. What rule explains both experiments?
Compare your rule
The shadow vector stays and its signed length stays 3. The dot product takes values 3, 6, and 9 as the length of b changes from 1 to 2 to 3. The direction of b fixes where the shadow falls; its length scales the dot product.
Name the quantities and explain the pattern
The shadow arrow is the vector projection. Its signed length along b is the scalar projection, which we will call .
For an acute angle , the arrow, its shadow, and the perpendicular form a right triangle. Cosine is the adjacent side divided by the hypotenuse, so . The signed version extends through 90° to 180°:
The dot product has an algebraic view: multiply matching coordinates and add the results. If we treat the entries of b as weights applied to the entries of a, this is a weighted sum. In two dimensions:
For our second experiment, , while . Doubling the horizontal component of b doubled the dot product, while the shadow stayed at .
The two experiments show what the dot product responds to. In the first, both lengths stayed fixed while the angle changed. In the second, the angle stayed fixed while the length of b changed. One comparison isolated alignment; the other isolated magnitude.
That’s the geometric intuition behind the dot product: with their lengths held fixed, the more two vectors are aligned, the more strongly they contribute to the resulting scalar.
The geometric view describes that same weighted sum using lengths and alignment:
These are two interpretations of the same dot product. The weighted sum already accounts for alignment: we do not calculate a cosine separately and apply it afterward. The effect of the angle is contained in the coordinate products and how they add or cancel.
For example, : the vectors point in the same direction, so . Changing the second vector to keeps its length fixed but makes it perpendicular to the first. The weighted sum becomes . Algebraically, the contributions cancel; geometrically, the same result follows from .
There are two things happening:
- Magnitudes () determine the maximum absolute value of the dot product for those fixed lengths.
- Alignment () determines how much of that maximum is realized.
Here, stronger alignment means a larger signed value. The cosine is 1 for the same direction, 0 for perpendicular vectors, and −1 for opposite directions. Opposite vectors therefore give the most negative value, with the same absolute size as the positive maximum.
For example, with , , and , the magnitude product is . The alignment factor is , so
At this angle, the dot product is about 77.2% of its maximum positive value. Keeping both lengths fixed and rotating the vectors into the same direction would give the full .
To connect this back to the shadow, substitute into the boxed formula:
The dot product is the signed projection length multiplied by . A unit vector has length 1, so using one makes the dot product equal to , as in the first experiment. The angle formula assumes nonzero vectors; the coordinate formula also works for the zero vector.
Three related quantities
For a nonzero reference vector b:
| Quantity | Formula | What you get |
|---|---|---|
| Dot product | Signed projection length multiplied by | |
| Scalar projection | Signed length along b | |
| Vector projection | A vector on the line through b |
If b is the zero vector, the dot product is zero, but projection onto its direction is undefined: it has no direction to project onto.
Try a new case: reverse the reference arrow
Work this case on paper: keep and change from to . Predict the dot product, scalar projection, and vector projection before calculating. Does reversing the arrow change the line it lies on?
Check your reasoning
The dot product is . Dividing by gives scalar projection . The vector projection is
The line and shadow vector stay the same. The signed quantities change because the reference direction reverses.
Coordinates are projections, too
The horizontal coordinate of is 3 because its dot product with the horizontal unit vector is 3. Its vertical coordinate is 4 because its dot product with is 4.
More generally, in an orthonormal basis (perpendicular unit vectors), each coordinate is a dot product:
Choosing a different orthonormal basis means measuring the same vector along different directions. This is the geometry behind the projection steps in PCA and SVD.
Why this matters for a neuron
A neuron’s weighted sum has these same two views. Algebraically, each input is multiplied by its weight and the results are added. Geometrically, the weight vector defines a direction: the sum measures the signed projection of the input x onto that direction, scaled by . Both describe the dot product before a bias shifts the result and an activation function transforms it.
For inputs of the same length, stronger alignment with w gives a larger weighted sum. That’s the geometric sense in which a neuron can detect a direction in its input space.
Where an individual weight appears in the geometry
Consider two input features: the counts of claim and prize in a message. Choose . Algebraically, the neuron computes . Geometrically, its weight arrow has a claim component of and a prize component of .
The unit inputs and let us measure these components separately. Start with and weights . Record the score, then predict what it will become with . Switch the input, keeping both weights fixed, and compare the score with the corresponding component of the blue arrow.
Next, keep the prize-axis input selected and predict what changing only will do to the score. Move that slider and record the result. Explain which input is multiplying the changed weight before opening the explanation.
Each weight is a component of the arrow
Blue: weights w · Orange: input x · Dashed: weight components
Weighted sum: 1.5 × 1 + 1 × 0 = 1.5
Geometric view: 1.803 × 1 × 0.832 ≈ 1.5 (length × length × cosine; rounded).
Selecting an input keeps both weights fixed. A unit input on one axis reads that weight directly. The bias is zero; no activation is applied.
Connect the weight, the component, and the response
The unit inputs read the individual weights:
Changing the claim weight cannot change the response to : that weight is multiplied by zero. The arrow’s horizontal component changes, but its vertical component stays at . Its length and angle to the input both change, with their effects combining to preserve the same dot product.
Select , then . The two weighted contributions now add: . With the weights fixed, one extra claim would raise the score by ; one extra prize would raise it by .
In general, increasing input coordinate by one gives
Each weight is both an input’s multiplier in the weighted sum and the weight vector’s signed component along that input’s axis. It tells us how much the score changes when that input increases by one and all other inputs stay fixed. Bias and activation may change the final neuron output; this statement describes the weighted sum.
Choose a negative weight and a unit-axis input to test next. Write a prediction, change only your chosen weight, and record both its arrow component and the score. Explain the sign and whether it agrees with .
Takeaway: the angle controls how much of one vector falls along another direction. The dot product measures that amount, scaled by the reference vector’s length.
Design your own comparison
Choose an angle other than the worked angles, keeping both vectors nonzero. Record the settings and the three readouts. Propose a change to the length of that you think will alter the dot product while preserving the shadow vector. Write your prediction, keep and the direction of fixed, then test your change.
Repeat at 90° as a separate comparison. Does changing the length of still change the dot product? Revise your claim to include this boundary case.
Write a short conclusion: what you predicted, what you observed, and how the evidence changed or supported your explanation.
Check the reasoning in your investigation
For a fixed direction of , its positive length does not affect either projection. The dot product scales with that length, but when the signed shadow is zero, every such dot product stays zero. Cite your settings and readouts, and distinguish this observation from the formula that explains the general rule.
Check your understanding
Try five questions about angles, scaling, and what a projection measures. Select an answer, then use to see the reasoning. You can revisit a question and try again.
Keep learning
To see how a matrix uses dot products to produce each output coordinate, continue with Dot products: building the output one row at a time.