Azathoth123 comments on Rationality Quotes August 2014 - Less Wrong

8 Post author: RolfAndreassen 04 August 2014 03:12AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (233)

You are viewing a single comment's thread. Show more comments above.

Comment author: Azathoth123 05 August 2014 03:13:44AM *  9 points [-]

I always interpreted "our kind" as the whole of humanity,

Did you read the linked article? In it Eliezer is contrasting rationalist and religious institutions. You may also want to read this to get an idea for the problem James Miller is trying to address. Here is a relevant quote:

Suppose that a country of rationalists is attacked by a country of Evil Barbarians who know nothing of probability theory or decision theory.

Now there's a certain viewpoint on "rationality" or "rationalism" which would say something like this:

"Obviously, the rationalists will lose. The Barbarians believe in an afterlife where they'll be rewarded for courage; so they'll throw themselves into battle without hesitation or remorse. Thanks to their affective death spirals around their Cause and Great Leader Bob, their warriors will obey orders, and their citizens at home will produce enthusiastically and at full capacity for the war; anyone caught skimming or holding back will be burned at the stake in accordance with Barbarian tradition. They'll believe in each other's goodness and hate the enemy more strongly than any sane person would, binding themselves into a tight group. Meanwhile, the rationalists will realize that there's no conceivable reward to be had from dying in battle; they'll wish that others would fight, but not want to fight themselves. Even if they can find soldiers, their civilians won't be as cooperative: So long as any one sausage almost certainly doesn't lead to the collapse of the war effort, they'll want to keep that sausage for themselves, and so not contribute as much as they could. No matter how refined, elegant, civilized, productive, and nonviolent their culture was to start with, they won't be able to resist the Barbarian invasion; sane discussion is no match for a frothing lunatic armed with a gun. In the end, the Barbarians will win because they want to fight, they want to hurt the rationalists, they want to conquer and their whole society is united around conquest; they care about that more than any sane person would."

And that's assuming the rationalists don't simply surrender without a fight on the grounds that "war is a zero sum game".

Comment author: Torello 05 August 2014 02:04:11PM *  3 points [-]

I didn't read the linked article--it certainly seems to frame the issue as rationalists vs. barbarians, not humanity vs. the environment (and the flaws of humanity), so thanks for pointing that out.

I do think fundamentalists/extremists/terrorists have an asymmetrical advantage in the short term in that it's always easier to cause damage/disorder than improvement/order. This quote above seems to be a particular example of this phenomenon.

However, I have to agree with Jiro's comment. Extremists may be able to destroy things and kill people, but I wouldn't say they've been able to conquer anything. To me, "conquer" implies taking control of a country, making its economy work for you, dominating the native population, building a palace, etc. Modern extremists commit suicide and then their mastermind hides silently for a decade until helicopters fly in and soldiers kill him.

Comment author: Jiro 05 August 2014 04:38:40AM *  3 points [-]

When referring to actual barbarians, the description of the barbarians seems to lie by omission--even if all the things described above are mostly true, the barbarians have wrecked their economy because central planning doesn't work no matter how many orders they give, burning people at the stake is bad for investment, their belief in an afterlife is associated with other beliefs that prevent them from making or even efficiently using scientific advances, and their inability to have sane discussion means they can't make tactical decisions or really plan anything well at all. (Etc.) That sort of thing is pretty much the reason that the West hasn't been conquered by Muslim fundamentalists yet:

Also, barbarism doesn't arise at random. Some social structures are more conducive to barbarism than others and they may have inherent flaws which reduce the efficiency of conquest even as their encouragement of barbarism increases it.

Comment author: Azathoth123 06 August 2014 02:43:07AM *  9 points [-]

the barbarians have wrecked their economy because central planning doesn't work no matter how many orders they give

Not all barbarians do that. The communists did that, but they also considered themselves rationalists and were considered such by many people at the time. Muslim fundamentalists generally don't.

burning people at the stake is bad for investment

Depends on whose being burned and why. Having the highest per capita rate of capital punishment doesn't seem to have hurt Singapore's ability to get investment.

their belief in an afterlife is associated with other beliefs that prevent them from making or even efficiently using scientific advances

I don't think so. It might hurt their ability to make scientific advancements, but they're perfectly capable of using them once someone else makes them.

Also, barbarism doesn't arise at random. Some social structures are more conducive to barbarism than others and they may have inherent flaws which reduce the efficiency of conquest even as their encouragement of barbarism increases it.

'Rationalist' societies can also have inherent flaws, like say problems solving the collective action problems associated with wars.

Comment author: Lumifer 05 August 2014 04:31:31PM *  3 points [-]

That sort of thing is pretty much the reason that the West hasn't been conquered by Muslim fundamentalists yet

For a counterpoint, look at the speed and magnitude of the original spread of Islam in the VII-VIII centuries.

Also there is Iran.

Comment author: Jiro 05 August 2014 04:54:26PM 2 points [-]

I don't think the spread of Islam many centuries ago counts. Fanaticism isn't as much of a disadvantage when fighting medieval socieities as it is when fighting modern ones.

Comment author: Lumifer 05 August 2014 05:06:52PM 2 points [-]

Fanaticism isn't as much of a disadvantage when fighting medieval socieities as it is when fighting modern ones.

Why is that so?

Comment author: Jiro 05 August 2014 07:00:43PM 2 points [-]

To summarize: Fanaticism keeps the culture from escaping the dark ages. If everyone is in the dark ages anyway, not being able to escape the dark ages isn't much of a disadvantage.

Comment author: Lumifer 05 August 2014 07:43:45PM *  4 points [-]

That looks to me like one of those sentences which sound pretty but don't actually mean much.

In your comment upthread you listed things which make a barbarian society "uncompetitive". They apply to medieval societies as well. Essentially, you would expect the non-fanatic society to be richer, have better technology, and be governed more effectively. That holds in any epoch (as long as we don't get too far into stone age :-/).

When Islam erupted out of the Arabian Peninsula, the "fanatics" easily took over huge -- amazingly huge -- territories. And it wasn't just pillage-and-burn, they conquered the lands and established their own rule.

Comment author: Jiro 05 August 2014 10:07:17PM 1 point [-]

Essentially, you would expect the non-fanatic society to be richer, have better technology, and be governed more effectively.

Why would I expect this when the society exists hundreds of years ago? The point is that back then, everyone lacked many of the things that fanaticism would cause a society to lack. The fanatics are not at such a disadvantage under such circumstances. The loss in efficiency from it taking weeks to communicate between distant parts of your empire is going to make the loss in efficiency from having a theocracy look like noise. The disadvantage of not getting investors in your country won't matter when there's no international investment anyway. The disadvantage of having little in the way of science and engineering won't matter if there's hardly any science yet and engineering is at the state of building bridges instead of launching satellites.

Comment author: Lumifer 06 August 2014 01:22:52AM 1 point [-]

The point is that back then, everyone lacked many of the things that fanaticism would cause a society to lack.

Really? Consider trade -- a major factor in the society's wealth and survival for the last several thousands of years. The fanatic barbarians wouldn't trade, would they?

You don't think technology mattered before the Industrial Revolution? Oh, but it did. From bronze weapons to early firearms, an army with a technological edge had a big advantage.

Governance didn't matter in ancient and medieval societies? Do you actually believe that?

Comment author: Jiro 06 August 2014 03:29:51AM -1 points [-]

Technology mattered before the Industrial Revolution. The kind of technology that fanatics are bad at did not matter before the Industrial Revolution, however, because nobody had it, fanatic or not.

Comment author: Viliam_Bur 09 August 2014 06:50:13PM *  5 points [-]

This feels to me like a just world fallacy, or perhaps choosing the most convenient world. Yes, if the barbarians are completely stupid, they are probably not so much of a danger these days. If they are completely anti-science, we probably have better guns.

Now imagine somewhat smarter barbarians, who by themselves are unable to do sophisticated science, but have no problem kidnapping a few scientists and telling them to produce a lot of weapons for them, otherwise their families will be burned at stake. (Even if their religion prevents them from doing science, they may compartmentalize and believe it is okay to use the devil's tools against the devil himself.) Suddenly, the barbarians have good guns, too.

Maybe the reason why the West hasn't been conquered by Muslim fundamentalists yet is that Muslims don't have an equivalent of Genghis Khan. Someone who would have the courage to conquer the nearest territory, kill horribly everyone who opposed them, let live those who didn't (and make this fact publicly known), take some men and weapons from the conquered territory and use them to attack the next territory immediately, et cetera, spreading like a wildfire. First attacking some smaller but civilized countries to get better weapons for attacking the next ones. With multiple leaders, so that dropping a bomb won't stop the war. (Maybe one Osama hiding in secret, giving commands to dozen wannabe Genghis Khans who don't mind getting to paradise too soon.)

Comment author: SilentCal 11 August 2014 06:08:45PM 3 points [-]

Jiro's fallacy is not in saying that the world is or has been just in this respect, but rather in implicitly saying it must be. I don't think it's a coincidence that liberal/secular/enlightenment nations are the most powerful today, but that fact doesn't negate the point of the barbarian hypothetical.

I seriously doubt the viability of your Genghis Khan plan for modern fundamentalist Islam, seeing as that same M.O. was tried recently except starting with one of the world's top industrial and scientific powers. But that's a fact about our world, and the point of the barbarian example is more universal than that.

Comment author: Azathoth123 12 August 2014 05:42:39AM 2 points [-]

I seriously doubt the viability of your Genghis Khan plan for modern fundamentalist Islam, seeing as that same M.O. was tried recently except starting with one of the world's top industrial and scientific powers.

I'm not sure. The liberal world seems to have gotten "softer" since then. Compare the general reaction in the US to the death toll in Iraq (maybe one or two US soldiers a day) with the death toll in WWII.

Comment author: Jiro 11 August 2014 07:27:53PM 1 point [-]

If the country of rationalists is attacked by a country of barbarians who are perfectly optimized for conquest, the rationalists will get conquered.

But there's no way to get from here to there except by Omega coming down and constructing exactly the race of barbarians necessary for the hypothetical to work. And if you're going to say that, there's no point in referring to them as barbarians and describing their actions in terms like "believes in an afterlife" and "obeys orders" that bring to mind real-life human cultures; you may as well say that Omega is just manipulating each individual barbarian like a player micromanaging a video game and causing him to act in exactly the way necessary for the conquest to work best.

Except of course that if you say "rationalists could be conquered by a set of drones micromanaged by Omega", without pretending that you're discussing a real-world situation, most people (assuming they know what you're talking about) would reply "so what?"

Comment author: Wes_W 11 August 2014 07:57:09PM 4 points [-]

If the country of rationalists is attacked by a country of barbarians who are perfectly optimized for conquest, the rationalists will get conquered.

This is not inconsistent with the claim that, if the country of rationalists is attacked by a country of barbarians who are imperfectly optimized for conquest, the rationalists might get conquered, with the risk depending on how optimized the barbarians are. And, for that matter, the rationalist nation probably isn't theoretically optimal either...

On balance, believing true things is an advantage, but there are other kinds of advantages which don't automatically favor the rationalist side. Sheer numbers, for example.

Comment author: Jiro 11 August 2014 08:06:24PM 1 point [-]

This is not inconsistent with the claim that, if the country of rationalists is attacked by a country of barbarians who are imperfectly optimized for conquest, the rationalists might get conquered, with the risk depending on how optimized the barbarians are.

How imperfectly optimized, though? Imperfectly optimized like Omega controlling each barbarian but occasionally rolling the barbarian's morale check, which fails on a 1 on a D100? Or imperfectly optimized like real life barbarians?

Comment author: V_V 11 August 2014 08:44:40PM 2 points [-]

What about the Bolsheviks? Or the WW2-era Japanese?

Comment author: SilentCal 11 August 2014 08:39:58PM 2 points [-]

Try the following obviously-unrealistic yet not-obviously-uninteresting hypothetical: There are two approximately equal-strength warring tribes of barbarians, Tribe A and Tribe B. One day Omega sprinkles magic rationality dust on Tribe A, turning all of its members into rationalists. Tribe B is on the move towards their camp and will arrive in a few days. This is not enough time for Tribe A to achieve any useful scientific or economic advances, nor to accumulate a significant population advantage from non-stake-burning.

Can you see, in that hypothetical, how Eliezer's points in the linked posts are important?

Or another approach: the quote about the rationalists losing says "Barbarians have advantages A, B, and C over rationalists." Your response is "But rationalists have larger advantages X, Y, and Z over barbarians, so who cares?" Eliezer's response is "screw that, if barbarians have any advantages over rationalists, the "rationalists" aren't rational enough". My hypothetical's purpose is to try to control for X, Y, and Z so we have to think about A, B, and C.

Comment author: Jiro 11 August 2014 10:17:05PM 1 point [-]

My hypothetical's purpose is to try to control for X, Y, and Z so we have to think about A, B, and C.

Advantages are usually advantages under a specific set of circumstances. If you "control" for X, Y, and Z by postulating a set of circumstances where they have no effect, then of course A, B, and C are better. The rationalists have a set of ideals that works better in a large range of circumstances and in more realistic circumstances. They will not, of course, work better in a situation which is contrived so that they lose, and that's fairly uninteresting--it's impossible to have ideals that work under absolutely all circumstances.

Think of being rationalist like wearing a seatbelt. Asking what if the rationalists' advances over the barbarians just happen not to apply is like asking what if not having a seatbelt would let you be thrown out of the car onto something soft but wearing a seatbelt gets you killed. I would not conclude that there is something wrong with seatbelts just because there are specific unlikely situations where wearing one might get you killed and not wearing one lets you survive.

Comment author: SilentCal 11 August 2014 10:37:57PM *  3 points [-]

"It's impossible to have ideals that work under absolutely all circumstances."

This is the essentially the proposition Eliezer wrote those posts to refute.

A seat belt is a dumb tool which is extremely beneficial in real-world situations, but we can easily contrive unrealistic cases where it causes harm. The point of those posts is that rationality is 'smart'; it's like a seat belt that can analyze your trajectory and disengage itself iff you would come to less harm that way, so that even in the contrived case it doesn't hurt to wear one.

Comment author: Jiro 12 August 2014 12:01:36AM 0 points [-]

I don't read it that way (at least not the long quote, taken from the second article). The way I read it, he is trying to discredit what he thinks is fake rationalism, and is giving barbarians as an example of a major failure which proves that this rationalism is fake. (Pay attention to the use of scare quotes.) I believe my response--that everything fails in unrealistic situations and what he is describing is an unrealistic situation--is on point.

Comment author: SilentCal 12 August 2014 04:28:13PM 1 point [-]

The intention is to discredit a 'fake rationalism' and illustrate 'real rationalism' in its place.

I think you're right that if we're talking about the USA fighting a conventional war against the visigoths, the barbarians' advantages aren't even a rounding error. But there are other types of conflicts where that may not be the case. Some possible examples include warfare in other eras or settings, economic competition, democratic politics, and social movements.

Or even if it's not a conflict--maybe we'd like to be able to have traditional rationalist virtues like empirically testing beliefs, and also be a group that cooperates on prisoners' dilemmas? Barbarians are a dramatization to elevate these problems to an existential level, but the ultimate point is that rationalists shouldn't have this kind of disadvantage even if they do have offsetting advantages that make them stronger than any group they're likely to come into direct conflict with.

Comment author: Jiro 10 August 2014 03:03:04AM -1 points [-]

By the same reasoning which says that fundamentalists could do better with more efficient methods of conquest, they could do better with more efficient methods of making peace, too. They won't do as well as with conquest, but they'll do better than they are doing now. Yet they don't.

Barbarism is not optimized for conquest. It's optimized for supporting a set of social structures. Those social structures make them more dangerous as conquerers than the average society, but they're still not optimized for conquest; there are things which would make conquest more efficient which they would not do.

(To use just one example, for a country to embark on conquest and use the men from the conquered country to continue conquering more countries, they'd have to grant equal rights to conquered people who agreed to work with them. Rome did that except in a few rare but famous cases. So did the Mongols. But Muslim fundamentalists can't give non-Muslims or rival Muslims equal rights without no longer being Muslim fundamentalists.)

Comment author: Azathoth123 10 August 2014 07:07:32PM *  5 points [-]

To use just one example, for a country to embark on conquest and use the men from the conquered country to continue conquering more countries, they'd have to grant equal rights to conquered people who agreed to work with them.

Not necessarily. Muslims, in particular, have a history of using slave soldiers to good effect.

But Muslim fundamentalists can't give non-Muslims or rival Muslims equal rights without no longer being Muslim fundamentalists.

You do realize it's possible to convert to fundamentalist Islam?

Comment author: Nornagest 11 August 2014 04:59:34PM *  2 points [-]

Muslims, in particular, have a history of using slave soldiers to good effect.

I seem to recall, and a glance over the Wikipedia articles suggests, that the Mamluk and Janissary systems involved raising (enslaved) boys into a military environment from a fairly young age. These boys might come from subjugated territories, but they'd in effect have been part of the dominant culture for much of their lives: it's not a system that could be used to quickly convert conquered territories into additional manpower.

That said, it hasn't been unusual for empires, modern and otherwise, to make substantial use of auxiliary forces drawn from client states. The Roman military probably relied on them as much as they did on the legions, or more in the late empire.

Comment author: Azathoth123 12 August 2014 05:39:06AM 3 points [-]

The Roman military probably relied on them as much as they did on the legions, or more in the late empire.

The late Roman Empire wasn't exactly successful at conquering anything, or even at keeping the Empire from falling apart.

Comment author: Jiro 10 August 2014 07:53:34PM *  1 point [-]

You do realize it's possible to convert to fundamentalist Islam?

Yes, but requiring that soldiers do so makes the process of conquest less optimized, since it's easier for obvious reasons to get soldiers without this requirement than with it. (The same goes for using slaves.)

Comment author: Vaniver 10 August 2014 10:22:58PM 2 points [-]

Yes, but requiring that soldiers do so makes the process of conquest less optimized, since it's easier for obvious reasons to get soldiers without this requirement than with it.

You seem to be focusing solely on cost; the difference between benefit and cost is what matters, and the benefits of a fighting force with shared values (particularly shared religious ones) are many and obvious.

Comment author: Jiro 11 August 2014 04:08:45PM 1 point [-]

By that reasoning, it's the Romans and the Mongols who are un-optimized for conquest.

Comment author: Azathoth123 12 August 2014 05:37:28AM 3 points [-]

The Mongols had the advantage of recruiting from a pool of steppe nomads with similar values.

The Roman Republic conquered the Mediterranean basin with an army consisting of Italians that were required to adopt Roman values before joining. Later the Roman legion adopted the looser system you described. Subsequently Roman legions would spend nearly as much effort fighting other Roman legions in civil wars as fighting Rome's enemies.

Comment author: [deleted] 31 August 2014 03:03:32PM 0 points [-]

Maybe the reason why the West hasn't been conquered by Muslim fundamentalists yet is that Muslims don't have an equivalent of Genghis Khan. Someone who would have the courage to conquer the nearest territory, kill horribly everyone who opposed them, let live those who didn't (and make this fact publicly known), take some men and weapons from the conquered territory and use them to attack the next territory immediately, et cetera, spreading like a wildfire.

"Caliph" Abu Bakr al-Baghdadi and his group ISIS have been behaving exactly like this. They are quite young, but don't appear quite able to take on a Western military yet.

This feels to me like a just world fallacy, or perhaps choosing the most convenient world.

And yet, by definition, a group who are better at rationality win more often. We ought to expect that rational civilizations can beat irrational ones, because rationality is systematized cross-domain winning.

Comment author: Viliam_Bur 31 August 2014 04:07:30PM 2 points [-]

by definition, a group who are better at rationality win more often

Well, there is this "valley of bad rationality" where being more rational about part of the problem but not yet more rational about other part can make people less winning.

Sometimes I feel are we are there at a society level. We have smart individuals, we have science, we fly to the moon, etc. However, superstition and blind hate can be an efficient tool for coordinating a group to fight against another group. We don't use this tool much (because it doesn't fit well with rationality and science), but we don't have an equally strong replacement. Also, only a few people in our civilization do the rationality and science. So even if there is a rationality-based defense, most of our society is too stupid to use it efficiently. On the scale from "barbarians" to "bayesians", most of our society is somewhere in the middle: not barbaric enough, but still far from rational.

Comment author: Jiro 31 August 2014 03:30:57PM 1 point [-]

A group that is better at rationality will win more often, but winning more often is not the same thing as "winning in a superset of the situations in which the irrational win".

Comment author: James_Miller 05 August 2014 04:55:49PM *  4 points [-]

That sort of thing is pretty much the reason that the West hasn't been conquered by Muslim fundamentalists yet.

Another reason: many members of our military do have the courage of the Spartans. U.S. soldiers don't put on suicide vests to kill children, but they do fall on grenades and hold hopeless positions under fire so their friends can escape death.