Interesting post. I just wanted to mention that your first two SVD matrix illustrations (for heads 10 and 15 of layer 22) are identical, apart from the labeled axes.