Interactive demo · Foundations of Deep Learning · Chapter 5

Training single-layer classifiers: least squares, logistic, probit and softmax

A single layer of weights computes activations ak = wkTφ(x) and passes them through an activation function: none for least squares, the logistic sigmoid for logistic regression, the normal distribution function for probit regression, the softmax for K classes. For the logistic and softmax models with the cross-entropy error the gradient is the same simple sum Σn(yn − tn)φn as for linear regression. This page trains the weights by batch gradient descent, starting from w = 0, with forward and backward passes written out in JavaScript.

Data

Model (trained by gradient descent)

Features

Learning rate η

Weight decay α

Training

Scoreboard

Try these

In the lecture notes