drethelin comments on Problem of Optimal False Information - Less Wrong

16 Post author: Endovior 15 October 2012 09:42PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (113)

You are viewing a single comment's thread. Show more comments above.

Comment author: drethelin 18 October 2012 12:02:15PM 0 points [-]

If you're trying to find the important decision point in real situations it can often be helpful to go to extremes to admit that things are possible. Ie, if the best lie is preferred to the worst truth, that implies that some truths are worse than some lies, and you can start talking about how to figure this out. If you just start with the actual question, you get people who say "No, the truth is most important"

Comment author: RichardKennaway 18 October 2012 12:51:32PM *  0 points [-]

Considering the extreme is only useful if the extreme is a realistic one -- if it is the least convenient possible world. (The meaning of "possible" in this sentence does not include "probability 1/3^^^3".) With extreme Omega-scenarios, the argument is nothing more than an outcome pump: you nail the conclusion you want to the ceiling of p=1 and confabulate whatever scenario is produced by conditioning on that hypothesis. The underlying structure is "Suppose X was the right thing to do -- would X be the right thing to so?", and the elaborate story is just a conjurer's misdirection.

That's one problem. A second problem with hypothetical scenarios, even realistic ones, is that they're a standard dark arts tool. A would-be burner of Dawkins' oeuvre presents a hypothetical scenario where suppressing a work would be the right thing to do, and triumphantly crows after you agree to it, "So, you do believe in censorship, we're just quibbling over which books to burn!" In real life, there's a good reason to be wary of hypotheticals: if you take them at face value, you're letting your opponent write the script, and you will never be the hero in it.