Try this: when you find yourself ready to accept a conclusion based on your step 1, notice this before mentally committing to the conclusion ("writing the bottom line"). Consider the opposite / alternative conclusions, and write them, too. Then come up with lines of reasoning which support each conclusion. I think that's the five-second skill that I've learned, and it is very handy.
Example:
In my startup recently I was trying to decide whether the best strategy going forward was to change the product drastically, or keep pushing on the same idea trying to figure out how to make it work. I recognized this argument internally, and explicitly said to my cofounder, "so the argument for keeping our idea the same is that we've been working on this thing for months and we have lots of experience in this space. But the argument for changing it is: notice sunk costs, then think whether anything else in the world would be better to work on than this idea, considering we have no actual momentum, only ideas."
In the spirit of contrarianism, I'd like to argue against The Bottom Line.
As I understand the post, its idea is that a rationalist should never "start with a bottom line and then fill out the arguments".
It sounds neat, but I think it is not psychologically feasible. I find that whenever I actually argue, I always have the conclusion already written. Without it, it is impossible to have any direction, and an argument without any direction does not go anywhere.
What actually happens is:
It is at the point 3 that the biases really struck. Motivated Stopping makes me stop checking too early, and Motivated Continuation makes me look for better arguments when defective ones are found for the conclusion I seek, but not for alternatives, resulting in Straw Men.