I think "believe true things" is incredibly misleading and hides way too much. Quoting myself from Facebook again:
The thing that needs to be more explicitly acknowledged in the conversation about how to promote true beliefs and social processes that help people acquire true beliefs is that acquiring true beliefs is a profoundly horrifying experience, and if you are not screaming as you do it then you are not really having it.
The quest to believe true things can take away everything you love. There was a man who commented on HPMoR about living in a highly religious community, with a loving wife and family and everything, and finding that the process of reading HPMoR had unexpectedly broken his ability to believe in God; he didn't feel like he could interact with his community and his family the way he used to ever again. What was he supposed to do? I can't imagine what it's like to transition from thinking that you live in a fundamentally safe and good universe where God will take care of you to... not that. Everyone who's ever done this has my profound sympathy and respect.
"Believe true things" and "make good choices" are dramatically in tension, in a way that this slogan doesn't even begin to acknowledge. Most people, in most situations, have to believe all sorts of false things, or at least claim to, in order to fit in with their peers. (This is worth screaming about, if you haven't already.) What's the good choice here? It's very unclear.
A more honest description of rationality would make it clear that it involves making a sacrifice; when done properly, you're sacrificing your ability to comfort yourself with self-deception because there's something you care about more than comfort. (But you can't just go around saying this to people, either; it's a dick move for several reasons.)
(Related: Venkatesh Rao on "Sociopaths".)
I think this epic LARP attitude to rationality isn't very fruitful. Most discoveries are made due to simple curiosity. To me rationality is a natural result of being interested in something, you begin to wonder what's true.
There's a kind of person that can get started on rationality this way; many can't, because other emotions get in the way and actively inhibit curiosity.
Certainly seems relevant.
(Someday I assume the "@" symbol will actually properly summon people instead of be a weird internet syntax?)
Sacrifice is definitely an important part of the process, yes. But I think waxing poetic about how miserable it is to be rational to an average person will make them think we are very weird and likely not correct, while my phrase is really difficult to disagree with and so is less likely to invite criticism. Both my phrase and talking about sacrifice are honest, but the former has a better effect on my status. I think it is a better PR move, basically, to have an agreeable elevator pitch.
Why would you want to optimize for agreeableness? There are some people who don't want to make this sacrifice, and I personally want to leave those people alone. An honest pitch is a better filter; the people who aren't turned off by an honest pitch are in fact the people we want.
Oh, interesting. To want to leave alone people who aren't willing to make the right kinds of sacrifices makes a lot of sense to me. I can come up with objections off the top of my head, and objections to those and so on, so I haven't thought about it enough to agree or disagree with the idea. The reason I was aiming for agreeableness is that the phrase is intended for answering questions like, "What are your hobbies?" or "What is your friend so-and-so like?" in a short way that communicates the basic idea and keeps the conversation pleasant. I honestly hadn't considered protecting people from rationality. But now that it's on my radar, I still am thinking that the vast majority of people who will hear this phrase from me will think "That's nice," and move on without ever considering learning more. I also think that for the people who are interested in learning more, I can filter them for devotedness immediately after they express interest.
Now that I think about it, filtering for devotedness is actually pretty tricky, because to admit that you'd rather live in a comfortable lie than with an unpleasant truth is embarrassing. People will be put under pressure to agree to take the red pill even if they really would rather not. I'm not sure what to do about this.
The reason I was aiming for agreeableness is that the phrase is intended for answering questions like, "What are your hobbies?" or "What is your friend so-and-so like?" in a short way that communicates the basic idea and keeps the conversation pleasant.
This makes sense as a thing to want but it's pretty different from either a motto or an elevator pitch.
Now that I think about it, filtering for devotedness is actually pretty tricky, because to admit that you'd rather live in a comfortable lie than with an unpleasant truth is embarrassing. People will be put under pressure to agree to take the red pill even if they really would rather not. I'm not sure what to do about this.
Yeah, I don't even bring up the self-deception angle except among people who I already think care about it.
The variation I personally use is "rationality is the study of how to make good choices." I think "study of" is doing important work. I usually start with that, and then immediately go on to say "there are two important elements of making good choices - you have to understand how the world works so you know which actions will have which effects, and you have to have the capability of actually taking those actions."
The opposite statement, "believing false things and making bad choices" really doesn't have any redeeming features. So why would everyone already not be in the ingroup?
I notice I am confused, because this seems like a solid point... but don't most things seem obviously the right thing when you state them simply? Like... the field of ethics is about being a good person, and nobody is for being a bad person, so surely everyone is in the field and we don't need it as specific subset of academia? You could probably define the goal of any field such that people aren't exactly against it, but nonetheless specialists in it are useful.
Added: I think that another framing can be taken from Paul Graham's "General and Surprising". Yes, everyone is on board with 'believing true things and making good choices', but few people can make a surprising improvement in how we do that very generally. Yet we here can give high level lessons in improving S1-S2 interactions, and give broad models that improve your ability to do this in many domains.
Yes, everyone is on board with 'believing true things and making good choices'
Respectfully disagree. Part of the package of self-deception involves deceiving yourself into believing you haven't deceived yourself; everyone has to claim to be on board with believing true things regardless of how many layers of self-deception they're in fact engaging in, and so most people are incapable of giving an honest answer to the question "are you on board with believing true things?"
It was old rationalist cannon - if you can say the opposite and it doesn't mean anything, did you say anything in the first place? I thought it was applause lights or near there but I can't find it now.
Both "believing true things and making good choices" and "believing false things and making bad choices" constrain anticipation. For example, someone doing the former more often is more likely to do well on a math test than someone doing the latter more often. They are both meaningful. The thing that they aren't is controversial. One feels like an obviously good idea and the other feels like an obviously bad idea. (note: I said "feels like an obviously good idea" instead of "obviously is a good idea" because of the whole sacrifice element Qiaochu_Yuan brought up.)
I think this is one of those things that seems obvious (to rationalists), but on consideration isn’t actually true. I know a good few people who would, unironically, defend believing false things and making bad choices! “Believe true things and make good choices” is a surprisingly non-universal position.
Rationality is believing true things and making good choices.
This phrasing isn't quite as precise as it could be, but it communicates the gist of both epistemic and instrumental rationality in very simple language. I use this wording when communicating what rationality is to non-rationalists, and it seems to do its job well enough. Maybe other rationalists will find it useful too.