You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

fiddlemath comments on What happens when your beliefs fully propagate - Less Wrong Discussion

20 Post author: Alexei 14 February 2012 07:53AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (78)

You are viewing a single comment's thread. Show more comments above.

Comment author: fiddlemath 14 February 2012 07:41:06PM 21 points [-]

tl;dr: If you can spot non-sequiturs in your writing, and you put a lot of weight on the conclusion it's pointing at, it's a really good idea to take the time to fill in all the sequiturs.

Writing an argument in detail is a good way to improve the likelihood that your argument isn't somewhere flawed. Consider:

  • Writing allows reduction. By pinning the argument to paper, you can separate each logical step, and make sure that each step makes sense in isolation.
  • Writing gives the argument stability. For example, the argument won't secretly change when you think about it while you're in a different mood. This can help to prevent you from implicitly proving different points of your argument from contradictory claims.
  • Writing makes your argument vastly easier to share. Like in open source software, enough eyeballs makes all bugs trivial.

Further, notice that we probably underestimate the value of improving our arguments, and are overconfident in apparently-solid logical arguments. If an argument contains 20 inferences in sequence, and you're wrong about such inferences 5% without noticing the misstep, then you have about a 64% chance of being wrong somewhere in the argument. If you can reduce your chance of a misstep in logic to 1% per inference, then you only have an 18% chance of being wrong, somewhere. Improving the reliability of the steps in your arguments, then, has a high value-of-information -- even when 1% and 5% both feel like similar amounts of uncertainty. Conjunction fallacy. It is probable, then, that we underestimate the value of information attained by subjecting ourselves to processes that improve our arguments.

If being wrong about an argument is highly costly -- if you would stand to lose much by believing incorrectly -- then it is well worth writing these sorts of arguments formally, and ensuring that you're getting them right.

All that said... I suspect I know exactly what you're talking about. I haven't performed a similar, convulsive update myself, but I can practically feel the pressure for it in my own head, growing. I fight that update longer than parts of me think I should, because I'm afraid of strong mental attractors. If you can write the sound, solid argument publicly, I will be happy to double-check your steps.