Interactive demo · Probabilistic Machine Learning: Advanced Topics · Chapter 18

Building kernels: sums, products and random features

A Gaussian process is chosen by its kernel. Pick an elementary kernel, optionally combine it with a second one by addition (either property) or multiplication (both properties), and look at the kernel against a fixed second input x′ and at four functions drawn from the prior. In the random-features mode the squared exponential kernel is replaced by an average of D random cosines, which is a finite model with the right kernel on average.

Mode

Kernel A

Combine with kernel B

Kernel B

Length scale ℓ

Period p

Second input x′

Try these

The four functions are drawn with a Cholesky factor of the 120-point Gram matrix on [−10, 10] (a tiny multiple of the identity is added for numerical stability) and a seeded generator, so the same draws are reused while you move a slider and only the kernel changes. The linear kernel is 0.05 x x′; the cosine kernel has period p; the rational quadratic kernel has α = 1. In the random-features mode the dashed curve is the average of D cosines with frequencies drawn from N(0, 1/ℓ²) and the lower panel shows functions drawn from the finite model wTφ(x) with w standard normal.

In the lecture notes