Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Alexei comments on What happens when your beliefs fully propagate - Less Wrong Discussion

20 Post author: Alexei 14 February 2012 07:53AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (78)

You are viewing a single comment's thread. Show more comments above.

Comment author: Alexei 14 February 2012 05:37:14PM 10 points [-]

Thanks, I'm actually glad to see your kind of comment here. The point you make is something I am very wary off, since I've had dramatic swings like that in the past. From Christianity to Buddhism, back to Christianity, then to Agnosticism. Each one felt final, each one felt like the most right and definite step. I've learned not to trust that feeling, to be a bit more skeptical and cautious.

You are correct that my post was full of non-sequiturs. That's because I wrote it in a stream-of-thought kind of way. (I've also omitted a lot of thoughts.) It wasn't meant to be an argument for anything other than "think really hard about your goals, and then do the absolute best to fulfill them."

Comment author: fiddlemath 14 February 2012 07:41:06PM 21 points [-]

tl;dr: If you can spot non-sequiturs in your writing, and you put a lot of weight on the conclusion it's pointing at, it's a really good idea to take the time to fill in all the sequiturs.

Writing an argument in detail is a good way to improve the likelihood that your argument isn't somewhere flawed. Consider:

  • Writing allows reduction. By pinning the argument to paper, you can separate each logical step, and make sure that each step makes sense in isolation.
  • Writing gives the argument stability. For example, the argument won't secretly change when you think about it while you're in a different mood. This can help to prevent you from implicitly proving different points of your argument from contradictory claims.
  • Writing makes your argument vastly easier to share. Like in open source software, enough eyeballs makes all bugs trivial.

Further, notice that we probably underestimate the value of improving our arguments, and are overconfident in apparently-solid logical arguments. If an argument contains 20 inferences in sequence, and you're wrong about such inferences 5% without noticing the misstep, then you have about a 64% chance of being wrong somewhere in the argument. If you can reduce your chance of a misstep in logic to 1% per inference, then you only have an 18% chance of being wrong, somewhere. Improving the reliability of the steps in your arguments, then, has a high value-of-information -- even when 1% and 5% both feel like similar amounts of uncertainty. Conjunction fallacy. It is probable, then, that we underestimate the value of information attained by subjecting ourselves to processes that improve our arguments.

If being wrong about an argument is highly costly -- if you would stand to lose much by believing incorrectly -- then it is well worth writing these sorts of arguments formally, and ensuring that you're getting them right.

All that said... I suspect I know exactly what you're talking about. I haven't performed a similar, convulsive update myself, but I can practically feel the pressure for it in my own head, growing. I fight that update longer than parts of me think I should, because I'm afraid of strong mental attractors. If you can write the sound, solid argument publicly, I will be happy to double-check your steps.

Comment author: David_Gerard 14 February 2012 07:23:53PM 3 points [-]

Yes. Even if this one is right, you're still running on corrupt hardware and need to know when to consciously lower your enthusiasm.