Enter non-negative weights for a joint table p(x, y) with four values of X (columns) and four of Y (rows); they are normalised to sum to one. The bar picture is the information diagram: the joint entropy H(X,Y) splits into H(X | Y) + I(X;Y) + H(Y | X), and the marginal entropies are the first two pieces and the last two pieces. The mutual information is the KL divergence from the product of the marginals to the joint.
Logarithms are base 2 (bits). A table is rescaled by its total, so only ratios matter. Rows or columns of zeros are allowed.
code/ch06_information_theory.py (joint_quantities).