= 27d567bc2d48d9f40e9ccc20ea370b15)
Jiro comments on Rationality Quotes August 2014 - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (233)
If the country of rationalists is attacked by a country of barbarians who are perfectly optimized for conquest, the rationalists will get conquered.
But there's no way to get from here to there except by Omega coming down and constructing exactly the race of barbarians necessary for the hypothetical to work. And if you're going to say that, there's no point in referring to them as barbarians and describing their actions in terms like "believes in an afterlife" and "obeys orders" that bring to mind real-life human cultures; you may as well say that Omega is just manipulating each individual barbarian like a player micromanaging a video game and causing him to act in exactly the way necessary for the conquest to work best.
Except of course that if you say "rationalists could be conquered by a set of drones micromanaged by Omega", without pretending that you're discussing a real-world situation, most people (assuming they know what you're talking about) would reply "so what?"
This is not inconsistent with the claim that, if the country of rationalists is attacked by a country of barbarians who are imperfectly optimized for conquest, the rationalists might get conquered, with the risk depending on how optimized the barbarians are. And, for that matter, the rationalist nation probably isn't theoretically optimal either...
On balance, believing true things is an advantage, but there are other kinds of advantages which don't automatically favor the rationalist side. Sheer numbers, for example.
How imperfectly optimized, though? Imperfectly optimized like Omega controlling each barbarian but occasionally rolling the barbarian's morale check, which fails on a 1 on a D100? Or imperfectly optimized like real life barbarians?
What about the Bolsheviks? Or the WW2-era Japanese?
Try the following obviously-unrealistic yet not-obviously-uninteresting hypothetical: There are two approximately equal-strength warring tribes of barbarians, Tribe A and Tribe B. One day Omega sprinkles magic rationality dust on Tribe A, turning all of its members into rationalists. Tribe B is on the move towards their camp and will arrive in a few days. This is not enough time for Tribe A to achieve any useful scientific or economic advances, nor to accumulate a significant population advantage from non-stake-burning.
Can you see, in that hypothetical, how Eliezer's points in the linked posts are important?
Or another approach: the quote about the rationalists losing says "Barbarians have advantages A, B, and C over rationalists." Your response is "But rationalists have larger advantages X, Y, and Z over barbarians, so who cares?" Eliezer's response is "screw that, if barbarians have any advantages over rationalists, the "rationalists" aren't rational enough". My hypothetical's purpose is to try to control for X, Y, and Z so we have to think about A, B, and C.
Advantages are usually advantages under a specific set of circumstances. If you "control" for X, Y, and Z by postulating a set of circumstances where they have no effect, then of course A, B, and C are better. The rationalists have a set of ideals that works better in a large range of circumstances and in more realistic circumstances. They will not, of course, work better in a situation which is contrived so that they lose, and that's fairly uninteresting--it's impossible to have ideals that work under absolutely all circumstances.
Think of being rationalist like wearing a seatbelt. Asking what if the rationalists' advances over the barbarians just happen not to apply is like asking what if not having a seatbelt would let you be thrown out of the car onto something soft but wearing a seatbelt gets you killed. I would not conclude that there is something wrong with seatbelts just because there are specific unlikely situations where wearing one might get you killed and not wearing one lets you survive.
"It's impossible to have ideals that work under absolutely all circumstances."
This is the essentially the proposition Eliezer wrote those posts to refute.
A seat belt is a dumb tool which is extremely beneficial in real-world situations, but we can easily contrive unrealistic cases where it causes harm. The point of those posts is that rationality is 'smart'; it's like a seat belt that can analyze your trajectory and disengage itself iff you would come to less harm that way, so that even in the contrived case it doesn't hurt to wear one.
I don't read it that way (at least not the long quote, taken from the second article). The way I read it, he is trying to discredit what he thinks is fake rationalism, and is giving barbarians as an example of a major failure which proves that this rationalism is fake. (Pay attention to the use of scare quotes.) I believe my response--that everything fails in unrealistic situations and what he is describing is an unrealistic situation--is on point.
The intention is to discredit a 'fake rationalism' and illustrate 'real rationalism' in its place.
I think you're right that if we're talking about the USA fighting a conventional war against the visigoths, the barbarians' advantages aren't even a rounding error. But there are other types of conflicts where that may not be the case. Some possible examples include warfare in other eras or settings, economic competition, democratic politics, and social movements.
Or even if it's not a conflict--maybe we'd like to be able to have traditional rationalist virtues like empirically testing beliefs, and also be a group that cooperates on prisoners' dilemmas? Barbarians are a dramatization to elevate these problems to an existential level, but the ultimate point is that rationalists shouldn't have this kind of disadvantage even if they do have offsetting advantages that make them stronger than any group they're likely to come into direct conflict with.
Why shouldn't they? If the "should" means "I would be happier if", maybe, but it's not a law of the universe that rationalists must always have an advantage in all situations.
(If nothing else, what if the enemy just hates rationalists? I can even think of real-life enemies who do.)
Rationalists shouldn't have those disadvantages, because there are a bunch of ways to mitigate them, which the post goes on to enumerate.
Part of Eliezer's project is to enshrine a definition of 'rational' such that a decision that predictably puts you at a disadvantage is not rational.
Are you arguing that Eliezer-rationality is a poor fit for the word's historic usage and you'd rather cultivate a different kind of rationality that doesn't allow the kind of unpleasant anti-barbarian measures described in the post? One that forbids them, not conditionally on barbarians not being a major threat, but absolutely?
If the definition was actually what you describe, then whether barbarians demonstrate that that kind of rationality "predictably puts you at a disadvantage" would partly depend on how likely you were to be attacked by barbarians of those types. (Because low probability events contribute less to your expected utility.)
In other words, if that's what he meant, then optimized barbarians don't count as an example. He'd have to either use realistic barbarians, or argue that his optimized barbarians are realistic enough that they make substantial contributions to the expected utility.
(Does "I assume he meant something different because if he meant that, his example is useless" count as steelmanning?)
He's writing about what the rationalists should do conditional on facing that kind of barbarian. The point of the post is not that rationalist communities should all implement military drafts, but rather that they should be capable of such measures if circumstances require them.
If you accept the local definition of rationality as winning (and not, say, as "thinking logically and deeply") then, well, losing means you weren't sufficiently rational :-/