PhilGoetz comments on The Contrarian Status Catch-22 - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (99)
"What do we believe?" is a distinct question; and asking it is comitting an error of rationality. The limitations of our minds often force us to use "belief" as a heuristic; but we should remember that it is fundamentally an error, particularly when the consequences are large.
You don't do the expected-cost analysis when investigating a theory; you should do it before dismissing a theory. Because, If someday you build an AI, and hardcode in the many-worlds assumption because many years before you dismissed the one-world hypothesis from your mind and have not considered it since, you will be committing a grave Bayesian error, with possibly disastrous consequences.
(My cost-of-error statements above are for you specifically. Most people aren't planning to build a singleton.)
I can't speak for Eliezer, but if I was building a singleton I probably wouldn't hard-code my own particular scientific beliefs into it, and even if I did I certainly wouldn't program any theory at 100% confidence.