Interactive demo · Probabilistic Machine Learning: Advanced Topics · Chapter 33

When two explanations disagree about the same model

A lemonade stand records the temperature T (°F) and the cleanliness c of the sidewalk. Model 1 is a step in T plus a little c; Model 2 is a logistic function of both. Two explanation methods rank the features: flip (does setting a feature to its extreme change the predicted class?) and derivative (mean absolute partial derivative, shown as a share). Both models are about the same function, yet the derivative method can name a different feature, and its answer depends on the units of T.

Model

Model 1: the step

Model 2: the sigmoid

Units of the temperature

Try these

The 400 training days are a fixed low-discrepancy design, T in [50, 100) and c in [0, 1). The predicted class is 1 when the probability exceeds 1/2. Derivatives are central differences; with w = 0 the step has derivative zero at every training day. In Model 2, k multiplies the argument of the sigmoid (k = 1 is the book's model).

In the lecture notes