Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Decius comments on Should correlation coefficients be expressed as angles? - Less Wrong Discussion

43 Post author: Sniffnoy 28 November 2012 12:05AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (24)

You are viewing a single comment's thread.

Comment author: Decius 28 November 2012 04:44:35AM 4 points [-]

All's well and good, until you get to adding angles together, unless you visualize A,B,C existing on a sphere and the angles between them being from the center of the sphere- are the possible angles of the sphere identical to the inverse cosines of the possible correlations? (If A and B are .707 correlated, and B and C are .707 correlated, are A and C necessarily somewhere between 0 and 1 correlated?)

For four variables (six angles), you would require visualizing them on a four-dimensional surface analogous to a sphere. I can do some of the math involved in calculating the possible angles there, but I can't visualize it beyond 'there are four points on that surface such that all of them are 90 degrees apart.' I know that any three of those points uniquely define a sphere, and each of those three spheres is a cross-section of the four-dimensional surface.

Comment author: Sniffnoy 28 November 2012 05:46:42AM 6 points [-]

All's well and good, until you get to adding angles together, unless you visualize A,B,C existing on a sphere and the angles between them being from the center of the sphere- are the possible angles of the sphere identical to the inverse cosines of the possible correlations? (If A and B are .707 correlated, and B and C are .707 correlated, are A and C necessarily somewhere between 0 and 1 correlated?)

Yes. If you have n random variables, you can restrict to the subspace spanned by them (or their equivalence classes). For instance, if we have 3 random variables whose equivalence classes are linearly independent, then the span of these equivalence classes will be a 3-dimensional real inner product space, which will then necessarily be isomorphic to R^3 with the dot product.

Actually, it occurs to me that I've never actually sat down and checked that angle addition obeys the triangle inequality in 3 dimensions -- I suppose if nothing else it can be done by lots of inequality grinding -- but that's not relevant. The point is, it does hold in dimension 3, and hence by the argument above, it holds regardless of dimension, finite or infinite.

Comment author: Decius 28 November 2012 05:27:42PM 4 points [-]

That's half of it- does there exist any set of angles which are mutually compatible angles on the n-dimensional surface but not inverse cosines of correlations?

Comment author: Kindly 28 November 2012 06:04:42PM 8 points [-]

No. Given any mutually compatible angles (which means we can choose unit vectors that have those angles) we can generate appropriately correlated Gaussian variables as follows: take these unit vectors, generate an n-dimensional Gaussian, and then take its dot product with each of the unit vectors.

Comment author: Decius 29 November 2012 05:24:04PM 1 point [-]

Now the hard question: Is there a finite number n such that all finite combinations of possible correlations can be described in n-dimensional space as mutually compatible angles?

My gut says no, n+1 uncorrelated variables would require n+1 right angles, which appears to require n+1 dimensions. I'm only about 40% sure that that line of thought leads directly to a proof of the question I tried to ask.

Comment author: Kindly 29 November 2012 05:38:15PM 2 points [-]

Your gut is right, both about the answer and about its proof (n+1 nonzero vectors, all at right angles to each other, always span an n+1-dimensional space). You should trust it more!

Comment author: Decius 30 November 2012 01:52:26AM 0 points [-]

I think that my 40% confidence basis for the very specific claim is proper. Typically I am wrong about three times out of five when I reach beyond my knowledge to this degree.

I was hoping that there would be some property true of 11-dimensional space (or whatever the current physics math indicates the dimensionality of meatspace is) that allows an arbitrary number of fields to fit.

Comment author: RichardKennaway 29 November 2012 02:20:32PM 2 points [-]

Actually, it occurs to me that I've never actually sat down and checked that angle addition obeys the triangle inequality in 3 dimensions -- I suppose if nothing else it can be done by lots of inequality grinding

The ordinary triangle inequality is immediate from -- is practically identical to -- the statement that a straight line is the shortest distance between two points.

The spherical triangle inequality, in any number of dimensions, is the same thing with "straight line" replaced by "great circle". A detail that doesn't arise for flat space is that there are two angles between two lines (whereas there is only one distance between two points in flat space), and you have to choose one that is no more than pi.

Comment author: faul_sname 28 November 2012 06:25:00PM *  3 points [-]

Apparently, some people can visualize more than 3 dimensions fairly easily. As for me, I use a little trick that engages the ability of my visual-spatial mind to visualize more than one object at a time.

To visualize a 6-sphere, I usually visualize a sphere of fixed size with three axes going through it. This sphere and the axes represent the higher-order dimensions. I then imagine myself somewhere in this three-dimensional space (specifically, somewhere inside the fixed sphere). I note the distance directly from the x, y, and z axes directly through me to the edge of the sphere. Each of these distances defines the radius of the 'visible surface' of the 6-sphere from that point in higher-order space looking from the x, y, or z axis respectively. By rotating the axes, the relative sizes of these surfaces change, and I'm guessing you can already visualize a rotating normal sphere in your mind. So you can rotate the 6-sphere in two sets of 3 dimensions fairly easily. Rotating between higher and lower dimensions is a bit more challenging, but still doable. For a 4- or 5- sphere, replace the 3-dimensional higher-order sphere with a circle, or just ignore the z axis.

If you can figure out how to do that, you can get a feeling for the possible orientations of the angles to one another. And that actually is fairly interesting, and the first time I really understood on a gut level why r^2 is used instead of r for correlations.

Comment author: Decius 29 November 2012 05:16:35PM 0 points [-]

Glad that works for you. I lose sight of where (0,0,1,0,0,0) is as soon as I rotate around any axis other than the z-axis, and I can never find (0,0,0,0,.707,.707) or any other point not on the reference plane for a higher-order dimension.