bshannon comments on Science: Do It Yourself - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (205)
Generally a good idea. Doing science yourself and focusing on data instead of intuition seems likely to yield results but there are serious limitations.
Confirmation bias and perhaps congruence bias (related: Correlation is not causation). You test your hypothesis using only the data used to form the hypothesis, thereby only using data which supports your hypothesis. In order to have a significant degree of confidence in your hypothesis, I think it's fair to say that it must have predictive power which hasn't been established by this mode of thinking. Perhaps it is more reliable than an intuitive answer and it is therefore a (minor) win but there is at least one other major problem.
It doesn't actually answer the original question: It focuses on one extreme which may not be (and I would guess probably isn't) representative of the whole. I imagine the best way to get into the Forbes 400 is to do what many (note: not most) people do and be very, very lucky but that doesn't necessarily represent the most effective way of becoming rich.
Thought experiment: Many people randomly invest all their money in one of a number of companies. Some become very rich. Most people don't. Expected utility is very low and, if we focus on the very rich and attempt to mimic their behaviour, we will almost certainly lose.
But I have no idea whether or not this can reasonably be said to represent the top 400. I have made no effort to study them. It is an hypothesis worth considering, however.
Other problems include stooping to behaviours one might consider unethical, requiring further study, and pre-existing conditions which are difficult or impossible to replicate.
Basically, be wary about taking your own conclusions too seriously because academic-grade science is there for more than one reason and expect to do a lot more work than one might expect having read this post, including reflecting on and revising the methods used to arrive at the conclusions.