Interactive demo · Probabilistic Machine Learning: Advanced Topics · Chapter 33

A local linear explanation: fidelity against locality

The black box is p(x) = σ(2 sin(1.4 x1) + 1.5 x2 − 1.2 x1x2), a classifier with an interaction. To explain it at the dot, fit a plane by weighted least squares with Gaussian weights of width h around the dot (the infinite-sample limit of a local surrogate such as LIME). A narrow kernel gives a faithful but tiny explanation that agrees with the gradient; a wide kernel explains more of the model and is no longer faithful anywhere. Click the picture to move the dot.

The point to explain

The explanation

Try these

Fidelity is the weighted R² of the plane against the probability; class agreement is the weighted share of the neighbourhood where the plane and the black box predict the same class (threshold 1/2). Stability is the cosine between the coefficient vectors at the point and at the point shifted along x1. The weights are placed on a 41 × 41 grid over ±4h, so there is no sampling noise.

In the lecture notes