Thank you for the insightful exploration of a alarming phenomenon of linear regression that had me stumped. I would love to see a conclusion to the explainer which might give guidance on settings where the different approaches are more or less appropriate. It currently leaves you looking for a "next page" button. We now understand why, but not what to do about it.
Thank you!
I'm not an expert on this topic, but my impression is that linear regression is useful for when you are trying to a fit a function from input to output (e.g imagine you have the alleles at various loci as your inputs and you want to predict some phenotype as your output. That's the type of problem well-suited for high-dimensional linear regression.) Whereas, for principle component analysis, it's mainly used as a dimensionality reduction technique (so using PCA for the case of two dimensions as I did in this post is a bit overkill.)
Thank you for the insightful exploration of a alarming phenomenon of linear regression that had me stumped. I would love to see a conclusion to the explainer which might give guidance on settings where the different approaches are more or less appropriate. It currently leaves you looking for a "next page" button. We now understand why, but not what to do about it.