Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Dmytry comments on Three Worlds Collide (0/8) - Less Wrong

48 Post author: Eliezer_Yudkowsky 30 January 2009 12:07PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (96)

Sort By: Old

You are viewing a single comment's thread.

Comment author: Dmytry 30 May 2011 01:55:25PM *  0 points [-]

@Eliezer well that's to me looks very dystopian future... the principle of protection of human autonomy was tossed away already when they legalize 'non consensual' anything where it is legal for human being A to impose his notions of fun on human being B against human being B's will. So the mankind got raped very gently by superhappy - so what? They legalized this already, there's a legal precedent for a far worse case that in your universe everyone agreed on.

Plus you give zero thought to concept of human beings as autonomous, each an agent upon himself. What I have immediately thought of is that I would have told the superhappy that due to communication bottleneck and biodiversity humans do not share identical values, have massively different neural wirings and as such they would have to integrate the values of each human personally etc. Not three worlds colliding but hundreds billions. I'd tell that the human neural system is so wired in many of the people that absence of pain would lead to diminished pleasure or failure to achieve sentience. I'd have told them the truth that facing such choice or having alteration forced on you results in extreme psychological pain that can disable and/or destroy some of the individuals. That we see gross modification of existing individuals as death. Etc.

Plus, of course, the fair 'superhappy' aliens are extremely vulnerable to mechanisms that homo sapiens has evolved to take advantage of naivity. You take some clinical psychopath, the smooth talking one (a corporate executive will do fine), get him to talk with the superhappy, and they're toast. The 30x thought speed advantage won't save them. The tech won't save them. They'll get owned by first nigerian prince. We have evolved for cognitive predator-prey interactions; 99% of the bias [as in , vast majority of, not as in exact number] that you're overcoming here is the heuristics for those interactions, for surviving a smarter enemy who would convince you into doing something bad with arguments that are only subtly flawed, akin to proof that 1=2. Where do you think the bias and reluctance to accept reason comes from? Well guess what 99% of reason out there is predator's blinky lights trying to subvert neural system of the prey, as in some squid versus some shrimp. The shrimp learnt to close the eyes. Reader, once you take into your mind what I just said there, even to parse what I said, you have to effectively execute truing-complete code in your head. In wetware with no process boundaries. The rationalism is inherently vulnerable.

They're entirely toothless however, with no concept of biting and no carapace. Ditto for the babyeaters, really, sending off all their data, sending human data to the superhappy, and otherwise acting like total naive idiots instead of waging the cognitive warfare that humanity has perfected in the course of probably more than a million years. I say, in such an interaction, both of those alien races would get massively, massively exploited by humanity.

Comment author: MixedNuts 30 May 2011 03:11:39PM 4 points [-]

very dystopian future... the principle of protection of human autonomy was tossed away already when they legalize 'non consensual' anything

Non-consensual conversation is legal and socially approved. A society where non-consensual anything is illegal would look very interesting - explicit mentions of kind of interaction you're open to, long escalation of extremely subtle signals, people mostly ignoring each other all the time, ubiquitous go-betweens - but hardly the only non-dystopian one.

have massively different neural wirings

Meh. There are some differences, but not nearly as big as between two random minds.

That we see gross modification of existing individuals as death

Yes, but it's not that huge. It's a rather isolated preference change.

The 30x thought speed advantage won't save them.

Bwahaha.

They're entirely toothless however, with no concept of biting and no carapace.

Can't see why. They understand treating language as a vending machine - vibrations go in, behaviors come out. The sounds "r-i-ch-b-a-n-k-er" need not be evidence of a person's finances. They didn't evolve for the same kinds of competition, but they have a concept of non-truthtelling. So I don't understand where you're coming from here.

Comment author: Dmytry 22 August 2011 06:16:16PM *  0 points [-]

Non-consensual conversation is legal and socially approved.

The story is speaking of "non-consensual sex", the illegal kind (rape), that was legalized. Great many actions are deemed illegal without consent as to protect autonomy of humans from other humans; when you start legalizing those actions, you drop off the autonomy. Especially major things.

Conversation is not illegal and thus can't be "legalized". Also, try having conversation with someone against their will, or when they are obviously busy. It is deemed impolite, and is not illegal simply because it doesn't hurt too much - if you distract someone causing an injury you might very well get in trouble.

Meh. There are some differences, but not nearly as big as between two random minds.

What the hell is a random mind, a Boltzmann brain? See http://lesswrong.com/lw/dr/generalizing_from_one_example/

In context of the story - clearly some people would embrace superhappy and some would commit suicide at the thought. Sounds significant enough to me. Hell, the humans are in reality more diverse in their views than the babyeaters and superhappy are in the story.

"Yes, but it's not that huge. It's a rather isolated preference change." well, in the story humans consider this preference change as sufficient to fear it nearly as much as death. It's the self preservation instinct that kicks in.

"Can't see why. They understand treating language as a vending machine - vibrations go in, behaviors come out."

The story itself - the aliens act far too naive and in ways that are too exploitable and imply lack of understanding of untruth. The humans as well, though. That's because this whole rationalism thing gets really messy and complicated when you start being rational about what you tell. In particular, superhappy gone into nearly shock state (lost part of crew!) over something that the babyeaters told them, without slightest thought as to the possibility that the babyeaters could perhaps have engineered an input to the superhappy which would damage the superhappy. (which is precisely what happened, except the creator of the story had engineered what the babyeaters tell as to be shocking)

Even more than this, the superhappy, despite being in position of power, are going for some supposedly fair 1/3 1/3 1/3 thing where everyone adjusts. Frankly it makes absolutely no sense and is not in the slightest rational, plus is clearly based on some failed logic and as such prone to manipulation (like every single human treated separately and they all dissolve).