= 27d567bc2d48d9f40e9ccc20ea370b15)
shminux comments on Decision Theory FAQ - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (467)
Maybe I can chime in...
"understand" does not mean "empathize". Psychopaths understand very well when people experience these states but they do not empathize with them.
Again, understanding is insufficient for revision. The paperclip maximizer, like a psychopath, maybe better at parsing human affect than a regular human, but it is not capable of empathy, so it will manipulate this affect for its own purposes, be it luring a victim or building paperclips.
So, if one day humans discover the ultimate bliss that only creating paperclips can give, should they "create a novel conceptual scheme" of giving their all to building more paperclips, including converting themselves into metal wires? Or do we not qualify as a "superior intelligence"?
Shminux, a counter-argument: psychopaths do suffer from a profound cognitive deficit. Like the rest of us, a psychopath experiences the egocentric illusion. Each of us seems to the be the centre of the universe. Indeed I've noticed the centre of the universe tends to follow my body-image around. But whereas the rest of us, fitfully and imperfectly, realise the egocentric illusion is a mere trick of perspective born of selfish DNA, the psychopath demonstrates no such understanding. So in this sense, he is deluded.
[We're treating psychopathy as categorical rather than dimensional here. This is probably a mistake - and in any case, I suspect that by posthuman criteria, all humans are quasi-psychopaths and quasi-psychotic to boot. The egocentric illusion cuts deep.)
"the ultimate bliss that only creating paperclips can give". But surely the molecular signature of pure bliss is not in any way tried to the creation of paperclips?
They would probably disagree. They might even call it a cognitive advantage, not being hampered by empathy while retaining all the intelligence.
I am the center of my personal universe, and I'm not a psychopath, as far as I know.
Or else, they do but don't care. They have their priorities straight: they come first.
Not if they act in a way that maximizes their goals.
Anyway, David, you seem to be shifting goalposts in your unwillingness to update. I gave an explicit human counterexample to your statement that the paperclip maximizer would have to adjust its goals once it fully understands humans. You refused to acknowledge it and tried to explain it away by reducing the reference class of intelligences in a way that excludes this counterexample. This also seem to be one of the patterns apparent in your other exchanges. Which leads me to believe that you are only interested in convincing others, not in learning anything new from them. Thus my interest in continuing this discussion is waning quickly.
Shminux, by a cognitive deficit, I mean a fundamental misunderstanding of the nature of the world. Evolution has endowed us with such fitness-enhancing biases. In the psychopath, egocentric bias is more pronounced. Recall that the American Psychiatric Association's Diagnostic and Statistical Manual, DSM-IV, classes psychopasthy / Antisocial personality disorder as a condition characterised by "...a pervasive pattern of disregard for, and violation of, the rights of others that begins in childhood or early adolescence and continues into adulthood." Unless we add a rider that this violation excludes sentient beings from other species, then most of us fall under the label.
"Fully understands"? But unless one is capable of empathy, then one will never understand what it is like to be another human being, just as unless one has the relevant sensioneural apparatus, one will never know what it is like to be a bat.
And you'll never understand why we should all only make paperclips. (Where's Clippy when you need him?)
Clippy has an off-the-scale AQ - he's a rule-following hypersystemetiser with a monomania for paperclips. But hypersocial sentients can have a runaway intelligence explosion too. And hypersocial sentients understand the mind of Mr Clippy better than Clippy understands the minds of sentients.
I'm confused by this claim.
Consider the following hypothetical scenario:
=======
I walk into a small village somewhere and find several dozen villagers fashioning paper clips by hand out of a spool of wire. Eventually I run into Clippy and have the following dialog.
"Why are those people making paper clips?" I ask.
"Because paper-clips are the most important thing ever!"
"No, I mean, what motivates them to make paper clips?"
"Oh! I talked them into it."
"Really? How did you do that?"
"Different strategies for different people. Mostly, I barter with them for advice on how to solve their personal problems. I'm pretty good at that; I'm the village's resident psychotherapist and life coach."
"Why not just build a paperclip-making machine?"
"I haven't a clue how to do that; I'm useless with machinery. Much easier to get humans to do what I want."
"Then how did you make the wire?"
"I didn't; I found a convenient stash of wire, and realized it could be used to manufacture paperclips! Oh joy!"
==========
It seems to me that Clippy in this example understands the minds of sentients pretty damned well, although it isn't capable of a runaway intelligence explosion. Are you suggesting that something like Clippy in this example is somehow not possible? Or that it is for some reason not relevant to the discussion? Or something else?
I think DP is saying that Clippy could not both understand suffering and cause suffering in the pursuit of clipping. The subsidiary arguments are:-
I'm trying to figure out how you get from "hypersocial sentients understand the mind of Mr Clippy better than Clippy understands the minds of sentients" to "Mr Clippy could not both understand suffering and cause suffering in the pursuit of clipping" and I'm just at a loss for where to even start. They seem like utterly unrelated claims to me.
I also find the argument you quote here uncompelling, but that's largely beside the point; even if I found it compelling, I still wouldn't understand how it relates to what DP said or to the question I asked.
I (whowhowho) was not defending that claim.
To empathically understand suffering is to suffer along with someone who is suffering. Suffering has --or rather is -- negative value. An empath would not therefore cause suffering, all else being equal.
Maybe don't restrict "understand" to "be able to model and predict".
Posthuman superintelligence may be incomprehensibly alien. But if we encountered an agent who wanted to maximise paperclips today, we wouldn't think, ""wow, how incomprehensibly alien", but, "aha, autism spectrum disorder". Of course, in the context of Clippy above, we're assuming a hypothetical axis of (un)clippiness whose (dis)valuable nature is supposedly orthogonal to the pleasure-pain axis. But what grounds have we for believing such a qualia-space could exist? Yes, we have strong reason to believe incomprehensibly alien qualia-spaces await discovery (cf. bats on psychedelics). But I haven't yet seen any convincing evidence there could be an alien qualia-space whose inherently (dis)valuable textures map on to the (dis)valuable textures of the pain-pleasure axis. Without hedonic tone, how can anything _matter _ at all?
I'm not sure we should take a DSM diagnosis to be particularly strong evidence of a "fundamental misunderstanding of the world". For instance, while people with delusions may clearly have poor models of the world, some research indicates that clinically depressed people may have lower levels of particular cognitive biases.
In order for "disregard for [...] the rights of others" to imply "a fundamental misunderstanding of the nature of the world", it seems to me that we would have to assume that rights are part of the nature of the world — as opposed to, e.g., a construct of a particular political regime in society. Or are you suggesting that psychopathy amounts to an inability to think about sociopolitical facts?
fubarobfusco, I share your reservations about DSM. Nonetheless, the egocentric illusion, i.e. I am the centre of the universe other people / sentient beings have only walk-on parts, is an illusion. Insofar as my behaviour reflects my pre-scientific sense that I am in some way special or ontologically privileged, I am deluded. This is true regardless of whether one's ontology allows for the existence of rights or treats them as a useful fiction. The people we commonly label "psychopaths" or "sociopaths" - and DSM now categorises as victims of "antisocial personality disorder" - manifest this syndrome of egocentricity in high degree. So does burger-eating Jane.
Huh, I hadn't heard that.
Clearly, reality is so Lovecraftian that any unbiased agent will immediately realize self-destruction is optimal. Evolution equipped us with our suite of biases to defend against this. The Great Filter is caused by bootstrapping superintelligences being compassionate enough to take their compatriots with them. And so on.
Now that's a Cosmic Horror story I'd read ;)