oh yeah, I exploited this fact to win a beer a few months ago! blogpost here:
https://jamiesimon.io/blog/how-to-win-a-beer-with-high-dimensional-statistics/
I like this widget as a demonstration of the effect Prieto et al. who showed months-of-the-year circles arising from data correlation.
I feel like I disagree with the tone of this post---it sounds to me that you're implying that these kind of circles are invalid or misleading because the correlation is relatively small. I don't think that is necessarily an issue. (Though I agree that something like the model performing modular arithmetic does not follow from this.)
I like this widget as a demonstration of the effect Prieto et al. who showed months-of-the-year circles arising from data correlation.
Sort of, maybe?
That paper shows that correlation in data can lead to PCA circles. But the widget does not have any data. It just has artificial features.
I haven't looked in to this, but I would expect that PCA circles from data will be of the the type demonstrated here. While PCA circles that are functional (e.g. mod addition) will more likely be in a true plane. Not sure of the strong version of this claim, but probably directionally true.
I feel like I disagree with the tone of this post---it sounds to me that you're implying that these kind of circles are invalid or misleading because the correlation is relatively small. I don't think that is necessarily an issue. (Though I agree that something like the model performing modular arithmetic does not follow from this.)
I don't mean to say that PCA circles are mean nothing. Circles tell us something, but I'm not sure how much. One reason there isn't much text in this post is that I'm not sure what the conclusion should be.
I think circles can be misleading if you read too much into them. I don't know if they are misleading to you, beucasue I don't know how much you are reading into them, or what is the right amount.
On noticing that almost orthogonal directions can looks like PCA circles, I updated in the direction of: PCA circles tells us there is circular correlation, but since there is no lower bound on how small the correlation is, just a PC1 vs PC2 plot tells me less that I used to think.
Looks like the correlation size (the off diagonal elements) has to be of at least similar magnitude as the magnitude difference (difference between any on diagonal elements).
Vectors that are arbitrarily close to orthogonal, but with a small circular correlation, will look circular when projected down to the first two principal component directions.
This might be relevant to you if you are doing mech-interp and think you are seeing circular features.
Clade made this widget for me. Feel free to play around.
Acknowledgment: Thanks to @philh for proofreading this post. Thanks to @Lucius Bushnaq for contributing one (out of four) sentences (not counting the acknowledgment section).