Convolutional Neural Networks

A convolutional neural network uses small learned kernels across many spatial locations. Instead of learning a separate weight for every input pixel and output unit, a CNN shares each filter over the grid. This is why CNNs remain central to computer-vision architectures, often combined with normalization, residual connections, and careful initialization.

How a CNN works

A convolutional network builds features by sliding small learned filters (kernels) across the input grid:

1. Convolve. Slide each filter over the image; at every location it computes a weighted sum of the pixels beneath it, producing one output value. The same filter weights are reused at every position (weight sharing), so a filter that detects an edge detects it anywhere.
2. Activate. Apply a nonlinearity to each output value.
3. Reduce. Pooling or a strided convolution shrinks the spatial size, summarizing each region.
4. Stack. Repeat. Because each layer sees the outputs of the previous one, deeper layers respond to a larger patch of the original image — a growing receptive field.

flowchart TD
  Image[Input image] --> Conv[Convolution: shared filters over the grid]
  Conv --> Act[Nonlinearity]
  Act --> Pool[Pooling or strided reduction]
  Pool --> Stack[Repeat: deeper layers see larger receptive fields]
  Stack --> Head[Flatten or pool, then classifier]

The convolution operation

For input channel $c$, output channel $k$, and kernel offsets $(u,v)$,

$$y_{k,i,j}=b_k+\sum _c\sum _u\sum _v{W}_{k,c,u,v}{x}_{c,i+u,j+v},$$

where $x_{c,i+u,j+v}$ is the input value at channel $c$ and position $(i+u,j+v)$, $W_{k,c,u,v}$ is the shared filter weight, $b_k$ is the bias for output channel $k$, and $y_{k,i,j}$ is the output at channel $k$, position $(i,j)$. Stride controls how far the kernel moves; padding controls boundary size. With kernel size $k_{\ell }$ and stride $s_i$, the receptive field after layer $\ell$ grows as

$$r_{\ell }=r_{\ell -1}+(k_{\ell }-1)\prod _{i<\ell }{s}_i.$$

Gradients through convolution are still handled by backpropagation; the key difference is that shared weights accumulate gradient contributions from every spatial position where the filter was used.

Worked example

This snippet applies a small convolutional filter to a toy image and reports the output alongside the receptive field after two $3\times 3$ layers.

import torch
import torch.nn.functional as F
 
x = torch.arange(16, dtype=torch.float32).view(1, 1, 4, 4)
kernel = torch.tensor([[[[1., 0.], [0., -1.]]]])
y = F.conv2d(x, kernel)
print("input")
print(x.view(4, 4).int().tolist())
print("conv_output")
print(y.view(3, 3).int().tolist())
print("receptive_field_two_3x3_layers", 5)

Observed output:

input
[[0, 1, 2, 3], [4, 5, 6, 7], [8, 9, 10, 11], [12, 13, 14, 15]]
conv_output
[[-5, -5, -5], [-5, -5, -5], [-5, -5, -5]]
receptive_field_two_3x3_layers 5

The same $2\times 2$ filter is applied at nine locations, producing identical local contrast on this linear ramp. Two stride-1 $3\times 3$ layers see a $5\times 5$ patch, not just two independent local windows.

Architectural lineage

The modern CNN traces to LeNet-5, which used convolution, pooling, and a small classifier for handwritten-digit recognition in 1998. The idea scaled up with AlexNet, whose 2012 ImageNet result — using ReLU activations, dropout, and GPU training — began the deep-learning era in vision. VGG showed that stacking many small $3\times 3$ filters builds large receptive fields with few parameters per layer, and GoogLeNet added parallel multi-scale “inception” branches. ResNet then made very deep CNNs trainable with residual connections that give gradients a direct path, and remains a common backbone for computer-vision architectures. Later designs such as vision transformers revisit the same problem with attention rather than fixed local kernels.

Caveats

Translation equivariance is useful only when the label should be insensitive to location. Aggressive pooling can discard small objects, and padding can introduce boundary artifacts. Batch normalization statistics also become brittle when image batches are tiny or distribution-shifted.

References

• Goodfellow, Bengio, and Courville, Deep Learning, Chapter 9: Convolutional Networks
• LeCun et al., 1998, Gradient-Based Learning Applied to Document Recognition
• Krizhevsky, Sutskever, and Hinton, 2012, ImageNet Classification with Deep Convolutional Neural Networks
• He et al., 2015, Deep Residual Learning for Image Recognition

Nav

Section — Deep Learning

← Residual Connections Recurrent Neural Networks →