CNN Architectures

CNN architectures organize convolution, nonlinearities, normalization, pooling or strided convolution, residual paths, and task heads. They are the standard backbone for image classification, object detection, and many segmentation models, and are the vision-specific case of convolutional neural networks. ResNet and ResNeXt are useful reference architectures for residual connections, skip paths, and grouped convolutional capacity.

Convolution, stride, and receptive field

For input $X$ and kernel $W$, a 2D convolutional layer computes

$$Y_{o,u,v}=b_o+\sum _c\sum _i\sum _j{W}_{o,c,i,j}{X}_{c,u+i,v+j}.$$

The output spatial size for one dimension is

$$\lfloor \frac{L+2p-d(k-1)-1}{s}+1\rfloor ,$$

with input size $L$, padding $p$, dilation $d$, kernel size $k$, and stride $s$. Residual blocks learn $y=x+F(x)$, making deep stacks easier to optimize; see residual connections for a more detailed treatment of skip connections, ResNet, and ResNeXt.

Worked example

This snippet runs a convolutional layer on an image-shaped tensor and computes the receptive field and effective stride after two layers.

import torch
 
torch.manual_seed(8)
x = torch.arange(1*1*8*8, dtype=torch.float32).reshape(1,1,8,8)
conv = torch.nn.Conv2d(1, 2, kernel_size=3, stride=2, padding=1, bias=False)
with torch.no_grad():
    conv.weight.fill_(1/9)
y = conv(x)
rf, jump = 1, 1
for k, s in [(3, 2), (3, 2)]:
    rf += (k - 1) * jump
    jump *= s
print("output_shape", tuple(y.shape))
print("top_left_channel0", round(float(y[0,0,0,0]), 3), "center_channel0", round(float(y[0,0,2,2]), 3))
print("two_layer_receptive_field", rf, "effective_stride", jump)

Observed output:

output_shape (1, 2, 4, 4)
top_left_channel0 2.0 center_channel0 36.0
two_layer_receptive_field 7 effective_stride 4

Stride halves the spatial resolution, while the receptive field grows. That is useful for semantics but risky for small objects and fine detection and segmentation metrics.

Caveats

Architecture comparisons are not meaningful unless resolution, augmentation, training length, and compute budget are matched. CNN locality is efficient, but long-range interactions may require larger receptive fields, attention, or a vision transformer.

References

• PyTorch documentation: torch.nn.Conv2d
• He et al., 2015, Deep Residual Learning for Image Recognition
• Xie et al., 2016, Aggregated Residual Transformations for Deep Neural Networks

Nav

Section — Computer Vision

← Detection and Segmentation Metrics Vision Transformers →