Interactive demo · Foundations of Deep Learning · Chapter 13

A graph convolutional network, trained live

Zachary's karate club split into two factions, one around the instructor (node 0) and one around the administrator (node 33). Give the network the friendship graph and the faction of only a few members (thick rings; click a node to add or remove its label) and train it here: each layer is H ← tanh( H W + b) with  = D̃−1/2(A + I)D̃−1/2, the last one feeds a softmax (13.20), and the loss is the cross-entropy over the labelled nodes only (13.21). All other nodes still pass messages: this is transductive learning. Colour shows the predicted probability of the administrator's faction.

Model

Layers L (the last one is the softmax layer)

Hidden width

Node features

Learning rate (Adam)

Seed (initial weights)

Try these

Weights start as Glorot normals from the seeded generator, biases at 0; full-batch Adam (β1 = 0.9, β2 = 0.999), weight decay 5×10−4 on the weights. "no edges" replaces  by the identity, so each node sees only itself; "residual" adds H to the output of every hidden-to-hidden layer, as in (13.36). One-hot features give each node its own input weights; constant features leave only the graph to go on.

In the lecture notes