Why is the dot product connected to projection?

You have already used coordinates, vector lengths, and normalization. Here we also use angles: 0° means the same direction, 90° means perpendicular, and 180° means opposite directions. We will introduce the formulas after two experiments.

Imagine a blue arrow a above a horizontal line. Drop a perpendicular from its tip to that line. The green arrow from the origin to the landing point is its “shadow.” The orange arrow b points right along the same direction.

The explorer also reports a number called the dot product. Can you work out how that number relates to the shadow?

Keep a record of your investigation

Keep an experiment record in a notebook or text file: settings, prediction, observed signed shadow length, dot product, and projection vector. Use angle 0° with lengths 10 and 1 as the first reference. Change only the angle in the first experiment; in the second, fix a=(3,4)\mathbf a=(3,4) and the direction of b\mathbf b, then change only its length. Before opening an explanation, compare a trial with its reference and write what the evidence says about your prediction.

First experiment: rotate without stretching

Keep a at length 10 and b at length 1. Before trying the controls, predict:

  • At 0°, how long should the shadow be?
  • At 90°, can the shadow disappear even though the arrow still has length?
  • At 180°, which direction should the shadow point? What sign would record that direction?

Now select 0°, 90°, and 180°. Then try 60°. Compare the shadow and the two numerical readouts.

Rotate a. Watch its shadow.
Vector a projected onto the horizontal direction of bAt 20.00 degrees, a has length 10 and signed projection 9.40. Vector b has length 1. The dot product is 9.40. Vector b is drawn below the projection line for visibility, at the same scale.-10-50510ab

Blue: vector aGreen: projectionOrange: vector b

b points right and is drawn below the line for visibility. Both vectors use the same scale.

Signed projection9.40
Dot product9.40
Projection vector(9.40, 0)on the horizontal line

What changed even though the arrow’s length stayed fixed? Explain the sign before opening the comparison.

Compare your observations

The signed shadow length is 10 at 0°, 0 at 90°, and −10 at 180°. The minus sign records a direction opposite to b, not a negative geometric length. At 60°, it is 5. With b at length 1, the dot product matches this signed number in every case.

The shadow lies on the whole line through b. It can extend beyond the tip of b or point backward.

Second experiment: can the number change without the shadow moving?

In the same explorer, select a = (3, 4). This sets b to (1,0)(1,0). Record the signed shadow length, the dot product, and the projection vector.

Before changing Length of b from 1 to 2, predict which of those three quantities will change. Keep a fixed, make the change, and compare.

Now predict the results for length 3 and test again. What rule explains both experiments?

Compare your rule

The shadow vector stays (3,0)(3,0) and its signed length stays 3. The dot product takes values 3, 6, and 9 as the length of b changes from 1 to 2 to 3. The direction of b fixes where the shadow falls; its length scales the dot product.

Name the quantities and explain the pattern

The shadow arrow is the vector projection. Its signed length along b is the scalar projection, which we will call pp.

For an acute angle θ\theta, the arrow, its shadow, and the perpendicular form a right triangle. Cosine is the adjacent side divided by the hypotenuse, so p/∥a∥=cos⁡θp/\|\mathbf a\|=\cos\theta. The signed version extends through 90° to 180°:

p=∥a∥cos⁡θ.p=\|\mathbf a\|\cos\theta.

The dot product has an algebraic view: multiply matching coordinates and add the results. If we treat the entries of b as weights applied to the entries of a, this is a weighted sum. In two dimensions:

a⋅b=axbx+ayby.\mathbf a\cdot\mathbf b=a_xb_x+a_yb_y.

For our second experiment, (3,4)⋅(1,0)=3(3,4)\cdot(1,0)=3, while (3,4)⋅(2,0)=6(3,4)\cdot(2,0)=6. Doubling the horizontal component of b doubled the dot product, while the shadow stayed at (3,0)(3,0).

The two experiments show what the dot product responds to. In the first, both lengths stayed fixed while the angle changed. In the second, the angle stayed fixed while the length of b changed. One comparison isolated alignment; the other isolated magnitude.

That’s the geometric intuition behind the dot product: with their lengths held fixed, the more two vectors are aligned, the more strongly they contribute to the resulting scalar.

The geometric view describes that same weighted sum using lengths and alignment:

a⋅b=∥a∥∥b∥cos⁡θ\boxed{\mathbf a\cdot\mathbf b=\|\mathbf a\|\|\mathbf b\|\cos\theta}

These are two interpretations of the same dot product. The weighted sum already accounts for alignment: we do not calculate a cosine separately and apply it afterward. The effect of the angle is contained in the coordinate products and how they add or cancel.

For example, (1,1)⋅(1,1)=1+1=2(1,1)\cdot(1,1)=1+1=2: the vectors point in the same direction, so cos⁡0∘=1\cos 0^\circ=1. Changing the second vector to (1,−1)(1,-1) keeps its length fixed but makes it perpendicular to the first. The weighted sum becomes 1−1=01-1=0. Algebraically, the contributions cancel; geometrically, the same result follows from cos⁡90∘=0\cos90^\circ=0.

There are two things happening:

  1. Magnitudes (∥a∥∥b∥\|\mathbf a\|\|\mathbf b\|) determine the maximum absolute value of the dot product for those fixed lengths.
  2. Alignment (cos⁡θ\cos\theta) determines how much of that maximum is realized.

Here, stronger alignment means a larger signed value. The cosine is 1 for the same direction, 0 for perpendicular vectors, and −1 for opposite directions. Opposite vectors therefore give the most negative value, with the same absolute size as the positive maximum.

For example, with ∥a∥=7\|\mathbf a\|=7, ∥b∥=6.5\|\mathbf b\|=6.5, and θ=39.48∘\theta=39.48^\circ, the magnitude product is 7×6.5=45.57\times6.5=45.5. The alignment factor is cos⁡(39.48∘)≈0.772\cos(39.48^\circ)\approx0.772, so

a⋅b=7(6.5)cos⁡(39.48∘)≈35.1.\mathbf a\cdot\mathbf b=7(6.5)\cos(39.48^\circ)\approx35.1.

At this angle, the dot product is about 77.2% of its maximum positive value. Keeping both lengths fixed and rotating the vectors into the same direction would give the full 45.545.5.

To connect this back to the shadow, substitute p=∥a∥cos⁡θp=\|\mathbf a\|\cos\theta into the boxed formula:

a⋅b=p∥b∥.\mathbf a\cdot\mathbf b=p\|\mathbf b\|.

The dot product is the signed projection length multiplied by ∥b∥\|\mathbf b\|. A unit vector has length 1, so using one makes the dot product equal to pp, as in the first experiment. The angle formula assumes nonzero vectors; the coordinate formula also works for the zero vector.

For a nonzero reference vector b:

QuantityFormulaWhat you get
Dot producta⋅b\mathbf a\cdot\mathbf bSigned projection length multiplied by ∥b∥\|\mathbf b\|
Scalar projectiona⋅b∥b∥\dfrac{\mathbf a\cdot\mathbf b}{\|\mathbf b\|}Signed length along b
Vector projectiona⋅b∥b∥2b\dfrac{\mathbf a\cdot\mathbf b}{\|\mathbf b\|^2}\mathbf bA vector on the line through b

If b is the zero vector, the dot product is zero, but projection onto its direction is undefined: it has no direction to project onto.

Try a new case: reverse the reference arrow

Work this case on paper: keep a=(3,4)\mathbf a=(3,4) and change b\mathbf b from (2,0)(2,0) to (−2,0)(-2,0). Predict the dot product, scalar projection, and vector projection before calculating. Does reversing the arrow change the line it lies on?

Check your reasoning

The dot product is 3(−2)+4(0)=−63(-2)+4(0)=-6. Dividing by ∥b∥=2\|\mathbf b\|=2 gives scalar projection −3-3. The vector projection is

−64(−2,0)=(3,0).\frac{-6}{4}(-2,0)=(3,0).

The line and shadow vector stay the same. The signed quantities change because the reference direction reverses.

Coordinates are projections, too

The horizontal coordinate of (3,4)(3,4) is 3 because its dot product with the horizontal unit vector (1,0)(1,0) is 3. Its vertical coordinate is 4 because its dot product with (0,1)(0,1) is 4.

More generally, in an orthonormal basis (perpendicular unit vectors), each coordinate is a dot product:

ai=a⋅eia_i = \mathbf a\cdot\mathbf e_i

Choosing a different orthonormal basis means measuring the same vector along different directions. This is the geometry behind the projection steps in PCA and SVD.

Why this matters for a neuron

A neuron’s weighted sum w⋅x\mathbf w\cdot\mathbf x has these same two views. Algebraically, each input is multiplied by its weight and the results are added. Geometrically, the weight vector defines a direction: the sum measures the signed projection of the input x onto that direction, scaled by ∥w∥\|\mathbf w\|. Both describe the dot product before a bias shifts the result and an activation function transforms it.

For inputs of the same length, stronger alignment with w gives a larger weighted sum. That’s the geometric sense in which a neuron can detect a direction in its input space.

Where an individual weight appears in the geometry

Consider two input features: the counts of claim and prize in a message. Choose w=(1.5,1)\mathbf w=(1.5,1). Algebraically, the neuron computes 1.5xclaim+xprize1.5x_{\mathrm{claim}}+x_{\mathrm{prize}}. Geometrically, its weight arrow has a claim component of 1.51.5 and a prize component of 11.

The unit inputs eclaim=(1,0)\mathbf e_{\mathrm{claim}}=(1,0) and eprize=(0,1)\mathbf e_{\mathrm{prize}}=(0,1) let us measure these components separately. Start with Claim axis (1, 0) and weights (1.5,1)(1.5,1). Record the score, then predict what it will become with Prize axis (0, 1). Switch the input, keeping both weights fixed, and compare the score with the corresponding component of the blue arrow.

Next, keep the prize-axis input selected and predict what changing only Claim weight will do to the score. Move that slider and record the result. Explain which input is multiplying the changed weight before opening the explanation.

Each weight is a component of the arrow

Weight vector and selected inputThe blue weight vector is (1.5, 1). Its claim component is 1.5 and its prize component is 1. The orange input is (1, 0). Their dot product is 1.5. Their angle is 33.69 degrees.-2-2-1-101122claim coordinateprize coordinatewx

Blue: weights w · Orange: input x · Dashed: weight components

Weighted sum: 1.5 × 1 + 1 × 0 = 1.5

Geometric view: 1.803 × 1 × 0.832 ≈ 1.5 (length × length × cosine; rounded).

Selecting an input keeps both weights fixed. A unit input on one axis reads that weight directly. The bias is zero; no activation is applied.

Connect the weight, the component, and the response

The unit inputs read the individual weights:

w⋅eclaim=1.5×1+1×0=1.5,\mathbf w\cdot\mathbf e_{\mathrm{claim}}=1.5\times1+1\times0=1.5,w⋅eprize=1.5×0+1×1=1.\mathbf w\cdot\mathbf e_{\mathrm{prize}}=1.5\times0+1\times1=1.

Changing the claim weight cannot change the response to (0,1)(0,1): that weight is multiplied by zero. The arrow’s horizontal component changes, but its vertical component stays at 11. Its length and angle to the input both change, with their effects combining to preserve the same dot product.

Select Reset, then Message (2, 1). The two weighted contributions now add: 1.5×2+1×1=41.5\times2+1\times1=4. With the weights fixed, one extra claim would raise the score by 1.51.5; one extra prize would raise it by 11.

In general, increasing input coordinate ii by one gives

w⋅(x+ei)−w⋅x=w⋅ei=wi.\mathbf w\cdot(\mathbf x+\mathbf e_i)-\mathbf w\cdot\mathbf x =\mathbf w\cdot\mathbf e_i=w_i.

Each weight is both an input’s multiplier in the weighted sum and the weight vector’s signed component along that input’s axis. It tells us how much the score changes when that input increases by one and all other inputs stay fixed. Bias and activation may change the final neuron output; this statement describes the weighted sum.

Choose a negative weight and a unit-axis input to test next. Write a prediction, change only your chosen weight, and record both its arrow component and the score. Explain the sign and whether it agrees with w⋅ei=wi\mathbf w\cdot\mathbf e_i=w_i.

Takeaway: the angle controls how much of one vector falls along another direction. The dot product measures that amount, scaled by the reference vector’s length.

Design your own comparison

Choose an angle other than the worked angles, keeping both vectors nonzero. Record the settings and the three readouts. Propose a change to the length of b\mathbf b that you think will alter the dot product while preserving the shadow vector. Write your prediction, keep a\mathbf a and the direction of b\mathbf b fixed, then test your change.

Repeat at 90° as a separate comparison. Does changing the length of b\mathbf b still change the dot product? Revise your claim to include this boundary case.

Write a short conclusion: what you predicted, what you observed, and how the evidence changed or supported your explanation.

Check the reasoning in your investigation

For a fixed direction of b\mathbf b, its positive length does not affect either projection. The dot product scales with that length, but when the signed shadow is zero, every such dot product stays zero. Cite your settings and readouts, and distinguish this observation from the formula that explains the general rule.

Check your understanding

Try five questions about angles, scaling, and what a projection measures. Select an answer, then use Check answer to see the reasoning. You can revisit a question and try again.

Question 1 of 50 checked · 0 correct
Vector a has length 10 and makes a 60° angle with unit vector b. What is a · b?

Keep learning

To see how a matrix uses dot products to produce each output coordinate, continue with Dot products: building the output one row at a time.