If we can't lie to others, we will lie to ourselves

paulfchristiano

"the most powerful tool is adopting epistemic norms which are appropriately conservative; to rely more on the scientific method, on well-formed arguments, on evidence that can be clearly articulated and reproduced, and so on."

A simple summary: Believe Less. Hold higher standards for what is sufficient reason to believe. Of course this is in fact what most people actually do. They don't bother to hold beliefs on the kind of abstract topics on which Paul wants to hold beliefs.

"1. What my decisions are optimized for. .. 2. What I consciously believe I want."

No. 2 might be better thought of as "What my talk is optimized for." Both systems are highly optimized. This way of seeing it emphasizes that if you want to make the two results more consistent, you want to move your talk closer to action. As with bets, or other more concrete actions.

Believe Less.

As in, believe fewer things and believe them less strongly? By assigning lower odds to beliefs, in order to fight overconfidence? Just making sure I'm interpreting correctly.

don't bother to hold beliefs on the kind of abstract topics

I've read this sentiment from you a couple times, and don't understand the motive. Have you written about it more in depth somewhere?

I would have argued the opposite. It seems like societal acceptance is almost irrelevant as evidence of whether that world is desirable.

1paulfchristiano10y

I care much more about the fact that "my conscious thoughts are optimized for X" than "my talk is optimized for X," though I agree that it might be easier to figure out what our talk is optimized for. [...] I'm not very interested in consistency per se. If we just changed my conscious thoughts to be in line with my type 1 preferences, that seems like it would be a terrible deal for my type 2 preferences. [...] Sometimes bets can work and I make many more bets than most people, but quantitatively speaking I am skeptical of how much they can do (how large they have to be, on what range of topics they are realistic, what are the other attendant costs). Using conservative epistemic norms seems like it can accomplish much more. If we want to tie social benefit to accuracy, it seems like it would be much more promising to use "the eventual output of conservative epistemic norms" as our gold standard rather than "what eventually happens," rather than reality, because it is available (a) much sooner, (b) with lower variance, and (c) on a much larger range of topics. (An obvious problem with that is that it gives people larger motives to manipulate the output of the epistemic process. If you think people already have such incentives then it's not clear this is so bad.)

45

If we can't lie to others, we will lie to ourselves

45

45

45

If we can't lie to others, we will lie to ourselves

45

45