One matrix multiplication, two views
A matrix has three rows and four columns. In the row view, each row takes a dot product with the input and produces one output coordinate. In the column view, each of the four input coordinates scales a column, and we add those contributions.
Why do both procedures produce an output in ? What happens to the four contributions in the column view?
You will need vector addition, multiplication by a scalar, and the dot product. The matrix-equations lesson introduces both views while solving systems; here we will follow one multiplication closely. Throughout, inputs and outputs are column vectors, and means vectors with real coordinates.
Predict the shape before calculating
Keep this matrix as our reference:
Start with . Before opening the calculation, write your predictions:
- How many numbers will the row calculation produce?
- How many coordinates does one column have? How many will the sum of four scaled columns have?
- What is the output? Calculate it once using rows and once using columns.
Record the three row results and all four column contributions. Compare the outputs and explain why their sizes agree before reading on.
Compare the two calculations
The row view produces three scalars, which we stack into one vector:
The column view produces four vectors, which we add coordinate by coordinate:
Each column already has three coordinates. Adding vectors adds their matching coordinates; it does not place their entries end to end. Four contributions therefore still give one three-coordinate output.
Change one input coordinate
Keep fixed. From the reference , change only from 1 to 2. Before changing the control, predict which output coordinates will change and by how much. Which column can explain the entire difference?
In the explorer, record the reference, then change to 2. Record the new row results, the four column contributions, and the output difference.
W stays fixed. Each control changes one coordinate of h.
| Output | From a | From b | From c | From d |
|---|---|---|---|---|
| y1 | 1 | 2 | 0 | 3 |
| y2 | 0 | 1 | 2 | 1 |
| y3 | 2 | 0 | 1 | 4 |
Row view: three scalar results
y1 = 1 × 1 + 2 × 1 + 0 × 1 + 3 × 1 = 6
y2 = 0 × 1 + 1 × 1 + 2 × 1 + 1 × 1 = 4
y3 = 2 × 1 + 0 × 1 + 1 × 1 + 4 × 1 = 7
Column view: four vector contributions
1 × (1, 0, 2)
= (1, 0, 2)
Three output coordinates1 × (2, 1, 0)
= (2, 1, 0)
Three output coordinates1 × (0, 2, 1)
= (0, 2, 1)
Three output coordinates1 × (3, 1, 4)
= (3, 1, 4)
Three output coordinatesAdd matching coordinates of the four contributions:
y1 = 1 + 2 + 0 + 3 = 6
y2 = 0 + 1 + 2 + 1 = 4
y3 = 2 + 0 + 1 + 4 = 7
Current output: (6, 4, 7)
Reference output: (6, 4, 7)
Change from reference: (0, 0, 0)
Tuples list vector coordinates in order. Reset restores the original W and h = (1, 1, 1, 1). Record your prediction before changing a control, then record both calculations.
Use this record in a notebook or text file:
| Case | Three row results | Four column contributions | Sum of contributions | Change from reference |
|---|---|---|---|---|
| … | … | … | … | |
| Only becomes 2 | … | … | … | … |
Compare your prediction with both calculations. Why can changing one input affect several outputs? Why does the third output stay fixed in this comparison? Write your explanation before revealing ours.
Follow the extra copy of column 2
The output changes from to . The difference is
In the column view, only the second contribution changes: becomes . The other three contributions stay fixed.
In the row view, the first row receives an extra , the second an extra , and the third an extra . Those are exactly the entries of column 2.
More generally, increasing only input coordinate by changes the output by , where is column . The change may affect several output coordinates, but a zero in that column leaves the corresponding output unchanged.
See the same arithmetic in both views
For arbitrary input coordinates, the row view gives
The column view gives
Look at just the second coordinate of this sum. The columns contribute , , , and . Adding them produces , exactly the second row’s dot product. The same reasoning applies to every output coordinate.
This is why the agreement holds for every input, including negative and zero coordinates. The experiments illustrated it; matching the terms establishes it generally. Switching views does not transpose or change the transformation.
| View of an matrix | What you calculate | How you form the output |
|---|---|---|
| Rows | dot products, each using all inputs | Stack the scalar results |
| Columns | scaled vectors, each with coordinates | Add matching coordinates to get one vector in |
For our matrix, each row has four entries so it can pair with the input. Each column has three entries so it can contribute to the output. Four input coefficients control four contributions in the same three-dimensional space.
Write the equivalence for any matrix
Let mean the entry in row , column . The row view computes
The summation symbol means to add one term for each input coordinate. In the column view, . Coordinate of column is , so coordinate of that vector sum is the same sum of numbers.
There are choices of output coordinate , regardless of how many contributions we add.
Change one row instead
Return to the reference and the original . This is a separate comparison: keep the input and rows 1 and 3 fixed, and double row 2 from to .
Predict the new output before changing from 1 to 2 in the explorer below. Which output coordinates can change? In the column view, which entries of the contributions change?
h = (1, 1, 1, 1) stays fixed. Only the second row of W changes.
| Output | From a | From b | From c | From d |
|---|---|---|---|---|
| y1 | 1 | 2 | 0 | 3 |
| y2 | 0 | 1 | 2 | 1 |
| y3 | 2 | 0 | 1 | 4 |
Row view: three scalar results
y1 = 1 × 1 + 2 × 1 + 0 × 1 + 3 × 1 = 6
y2 = 0 × 1 + 1 × 1 + 2 × 1 + 1 × 1 = 4
y3 = 2 × 1 + 0 × 1 + 1 × 1 + 4 × 1 = 7
Column view: four vector contributions
1 × (1, 0, 2)
= (1, 0, 2)
Three output coordinates1 × (2, 1, 0)
= (2, 1, 0)
Three output coordinates1 × (0, 2, 1)
= (0, 2, 1)
Three output coordinates1 × (3, 1, 4)
= (3, 1, 4)
Three output coordinatesAdd matching coordinates of the four contributions:
y1 = 1 + 2 + 0 + 3 = 6
y2 = 0 + 1 + 2 + 1 = 4
y3 = 2 + 0 + 1 + 4 = 7
Current output: (6, 4, 7)
Reference output: (6, 4, 7)
Change from reference: (0, 0, 0)
Tuples list vector coordinates in order. Reset restores the original W and h = (1, 1, 1, 1). Record your prediction before changing a control, then record both calculations.
Record the three row results, all four column contributions, and the difference from the reference. Explain how several changed column contributions can affect just one output coordinate before opening the explanation.
Follow the second coordinate through every column
The output becomes : only the second row’s dot product doubles. In the column view, the new contributions are
Only second coordinates can change. Their sum becomes ; the first and third coordinate sums remain 6 and 7. The zero in column 1 stays zero even when doubled.
For any fixed input, changing only row can change only output coordinate . It does not guarantee a change: the new row might have the same dot product with that particular input.
Design your own comparison
Choose an input coordinate other than and your own nonzero increase or decrease. In the first explorer, use and keep and the other inputs fixed. You can use paper for values outside the controls’ range.
Write the reference, your chosen change, and a prediction of the entire output difference. State what observation would contradict your prediction. Then calculate both views, record the difference, and explain the result using one column. Include a negative or zero input in your chosen case.
For a second challenge on paper, choose an input and a genuine change to one row that leaves the output unchanged. Predict why the change will be invisible, then calculate the before and after outputs. Explain what your example says about the claim “changing a row always changes its output.”
Hints and checks for your investigation
For the input experiment, multiply the selected column by the change in the input, not its new value. For example, changing from 1 to gives , so the output difference is .
For the row experiment, choose a row difference whose dot product with your input is zero. With , adding to one row changes its weights but leaves its dot product unchanged. Alternatively, changing a weight attached to a zero input makes that change invisible for this input.
Your record should show equal row and column results in both states. The row experiment proves that a changed matrix can agree with the original on a particular input; it does not mean the two matrices agree on every input.
What a column means in a neural network
Suppose contains four hidden activations and supplies three outputs. Row collects the weights used to compute receiving neuron ‘s weighted sum. Column collects the outgoing weights from source activation to all three receiving neurons.
For example, when , its contribution is
That is one source activation’s contribution to the three outputs. The full output also includes the contributions from the other source activations. Both views apply to this same layer.
Setting just and all other inputs to zero isolates column . This input is the standard unit vector , so : each column is where one input basis vector lands. The input basis vector has four coordinates; its image has three.
If a layer adds bias, add that vector after the column sum. If it applies a nonlinear activation, apply it to the resulting weighted sums; in general, applying it separately to each column contribution and adding gives a different result.
Transfer to a taller matrix
Try a new shape without the explorer:
Before calculating, predict the input and output dimensions, the number of row dot products, and the number of column contributions. Compute both ways. Then predict and check the output difference if only the second input changes from to 0.
Check the new shape and the change
The map takes to . Four row dot products give . The two column contributions are
Increasing the second input by 1 adds the second column, , giving . The third output stays fixed because the third entry of that column is zero.
Finally, explain in your own words: Why do four columns in produce a three-coordinate output, while two columns in produce a four-coordinate output? Use the difference between counting contributions and counting coordinates.
Continue with what makes a function linear to investigate the rules behind these combinations, or what a weight matrix actually does to follow them through a neural network.