What is a convolution operation in a Convolutional Neural Network (CNN)?

A convolution operation in deep learning is a spatial mathematical transformation where a learnable matrix of weights called a filter kernel (e.g., 3×3 or 5×5) slides across an input image. At each position, it computes the Frobenius inner product (sum of element-wise products) between the kernel and the local input window (receptive field), generating an output activation in a feature map that highlights visual patterns like edges, corners, and textures.

What is Max Pooling and why is it used in CNNs?

Max pooling is a spatial downsampling operation commonly placed after convolutional and activation layers. It slides a rectangular window (typically 2×2 with stride 2) across the feature map and selects the maximum scalar value in each window. This reduces spatial dimensions by 75%, cuts memory and compute costs, provides translation invariance, and allows deeper layers to capture larger contextual receptive fields.

How do you calculate output feature map dimensions for convolution and pooling?

For convolution: Output = floor((W - K + 2P) / S) + 1, where W is input size, K is kernel size, P is zero-padding, and S is stride. For max pooling without padding: Output_Height = floor((H - Filter_Height) / Vertical_Stride) + 1 and Output_Width = floor((W - Filter_Width) / Horizontal_Stride) + 1.

What is the difference between Valid (P=0) padding and Same (P=1) padding?

Valid padding (P = 0) applies no extra border pixels, causing the output feature map to shrink by (K - 1) pixels (e.g., a 28×28 input with a 3×3 kernel becomes 26×26). Same padding (P = 1 for 3×3 kernels) wraps the input with a 1-pixel border of zeros, preserving the spatial input resolution (28×28 remains 28×28 with stride 1) and allowing deep architectures to stack dozens of convolutional layers without losing edge information.

What are Sobel and Prewitt filters and how do they detect edges?

Sobel and Prewitt filters are 3×3 spatial gradient approximation kernels. Sobel kernels use weighted coefficients [-1, 0, 1; -2, 0, 2; -1, 0, 1] to calculate horizontal (X) gradients with central smoothing, and [-1, -2, -1; 0, 0, 0; 1, 2, 1] for vertical (Y) gradients. Prewitt kernels use uniform [-1, 0, 1] weights. When convolved with an image, these kernels yield high positive or negative activation responses along boundaries where pixel intensities transition sharply.

What is the Frobenius inner product in 2D convolution?

The Frobenius inner product is the sum of the element-by-element multiplication of two equally-sized matrices. In a 2D convolution with a 3×3 kernel K and receptive field patch R, the output scalar value is computed as: Output = sum_{m=0..2} sum_{n=0..2} (R[m, n] * K[m, n]) + bias.

How does backpropagation work through a Max Pooling layer?

During the forward pass of max pooling, the network records the coordinate of the maximum neuron in each pooling window (the 'max switch'). During backpropagation, the upstream error gradient (dL/dy) is routed exclusively to that winning input pixel, while all other non-max pixels in the window receive a gradient of zero (dL/dx = 0).

Why does Max Pooling provide translation invariance in vision models?

Because max pooling extracts only the maximum activation within a spatial window, minor shifts or translations of input features within that window will still produce the exact same maximum output scalar. This ensures the neural network can recognize objects and textures regardless of slight position variations.

LLM Matrix Lab AI Workbench

Home AI Tokenizer Neural Network CNN LLM Visualization About

Interactive Computer Vision Engine

CNN Operations Visualizer

Master the fundamental computational building blocks of Convolutional Neural Networks. Toggle between Convolution Operation (sliding filter kernels & Frobenius inner products) and Max Pooling (spatial dimensionality downsampling & peak activation routing) in real time.