Lecture 001

Image: a function $f(x, y)$ to some value.

Classical Image Processing

Above methods are point-processing method, no awareness of structure. What if I want to turn a white cat into a black cat?

Antialiasing methods:

• add more pixels

• get rid of high frequency (lossy compression)

  • moving average (1D)
  • cross correlation filter (2D)

Compared to Average Filter, Gaussian Filter get rid of more high frequency stuff

Convolution and Cross-Correlation are similar. Convolution flips the kernel by $180^\circ$ (or time-inverse if it is 1D) before performing the elementwise multiplication.

Convolution is commutative and associative.

Also, a gaussian kernel convolute with itself is another gaussian kernel but with sigma becomes $\sigma \sqrt{2}$

When applied pre-filtering to convolution layers in neural nets, the classification is more stable (smaller variance) against image translation. (ICML 2019 "Making Convolutional Networks Shift-Invariant Again")

For tasks like classification, you need diverse pre-processing artifact to make the classifier robust. However, for generative model, you need to avoid artifact to get the best quality of generated result.

Gaussian Pyamid: store different down-sampled version of the same image together.

• Classification: cheaper to classify in low-resolution

• Generation: cheaper to first generate coarse image

Padding:

• constant black: will result black border

• wrap around: wrong color

• copy edge: good, but can see line effects near edge

• reflect across edge: most common choice