Which shifts does pooling actually ignore?
Run on Molab
- Download the notebook.
- Open Molab (opens in a new tab), sign in, and import the downloaded Python notebook.
- Write a prediction, run the reference, then change one condition. These small experiments run on CPU.
Your notebook runs in your Molab workspace. Download its report to keep your evidence.
The CNN article’s pooling section explains that pooling can tolerate some shifts without guaranteeing translation invariance. Here we test the boundary of that claim. You need array coordinates and the difference between a maximum and an average.
Our input is a feature map containing one response of value 1 and zeros everywhere else. Non-overlapping windows with stride 2 produce a output. Rows and columns are numbered 0–3 from the top left.
Establish a reference
Place the response at row 0, column 0 and use max pooling. Sketch the four pooled values before running. Record the reference input and output maps; the top-left output summarizes rows 0–1 and columns 0–1.
Change only the response column
Keep row 0, response magnitude 1, and max pooling fixed. Predict the output at column 1 and record a run. Next predict and run column 2. Compare the entire output maps, rather than just their sums.
Experiment bench
One response of value 1 in a 4 × 4 feature map; every other cell is zero. Pool each non-overlapping 2 × 2 window with stride 2 and no padding. This isolates pooling, not a trained CNN.
Load the reference, write a prediction, and run it first. For each comparison, change only the condition named in the lesson. Each run is calculated here in your browser.
Loading reference settings preserves your evidence. Work is saved in this tab across navigation and reloads. Download your report to keep it after closing the tab.
Build your explanation from the evidence
Explain why both moves shift the input by one cell but need not have the same effect on the pooled output.
Compare the windows
At columns 0 and 1, the response belongs to the same pooling window, so the output remains . At column 2 it crosses a window boundary, giving .
The sum of pooled values is still 1. That unchanged scalar summary hides the changed spatial representation. Nothing in this experiment establishes that a downstream classifier would make the same prediction.
Design your own boundary test
Choose a starting row and column. Design one one-cell move that stays inside the same window and another that crosses a window boundary. Predict both maps and test them, changing only one coordinate per comparison. Use the recorded settings to show which window each response belongs to.
Then restore the reference position and switch only the pooling operation to . Predict the nonzero output value. Repeat one of your boundary comparisons with the operation now held fixed. What changed about magnitude, and what remained true about position?
For your independent comparisons, use to select a recorded run that differs in only the condition you are testing.
Report and transfer
Use the input and output maps as your evidence. Explain why “pooling ignores shifts” is too broad, and why a constant sum is insufficient to show that two representations are identical.
On paper, put two responses of value 1 in the same window. Predict how max and average pooling would respond, keeping the window size fixed.
Check the extension
One response contributes average ; two contribute . Max pooling returns 1 in both cases. Both operations summarize a window and lose information about positions inside it, but they preserve different aspects of the values. A one-cell shift can still change which window receives the responses.
Continue experimenting
Previous: What changes when we remove the hidden activation? · Next: Can a model become more confident without getting more answers right? · All machine learning experiments