Comment author: RobbBB 10 September 2013 07:51:19AM 1 point [-]

Bottommost (September 9, 6:03 PM) comment here.

Comment author: player_03 10 September 2013 08:41:57AM *  5 points [-]

Oh, yeah, I found that myself eventually.

Anyway, I went and read the the majority of that discussion (well, the parts between Richard and Rob). Here's my summary:

Richard:

I think that what is happening in this discussion [...] is a misunderstanding. [...]

[Rob responds]

Richard:

You completely miss the point that I was trying to make. [...]

[Rob responds]

Richard:

You are talking around the issue I raised. [...] There is a gigantic elephant in the middle of this room, but your back is turned to it. [...]

[Rob responds]

Richard:

[...] But each time I explain my real complaint, you ignore it and respond as if I did not say anything about that issue. Can you address my particular complaint, and not that other distraction?

[Rob responds]

Richard:

[...] So far, nobody (neither Rob nor anyone else at LW or elsewhere) will actually answer that question. [...]

[Rob responds]

Richard:

Once again, I am staggered and astonished by the resilience with which you avoid talking about the core issue, and instead return to the red herring that I keep trying to steer you away from. [...]

Rob:

Alright. You say I’ve been dancing around your “core” point. I think I’ve addressed your concerns quite directly, [...] To prevent yet another suggestion that I haven’t addressed the “core”, I’ll respond to everything you wrote above. [...]

Richard:

Rob, it happened again. [...]

I snipped a lot of things there. I found lots of other points I wanted to emphasize, and plenty of things I wanted to argue against. But those aren't the point.

Richard, this next part is directed at you.

You know what I didn't find?

I didn't find any posts where you made a particular effort to address the core of Rob's argument. It was always about your argument. Rob was always the one missing the point.

Sure, it took Rob long enough to focus on finding the core of your position, but he got there eventually. And what happened next? You declared that he was still missing the point, posted a condensed version of the same argument, and posted here that your position "withstands all the attacks against it."

You didn't even wait for him to respond. You certainly didn't quote him and respond to the things he said. You gave no obvious indication that you were taking his arguments seriously.

As far as I'm concerned, this is a cardinal sin.

I think I am explaining the point with such long explanations that I am causing you to miss the point.

How about this alternate hypothesis? Your explanations are fine. Rob understands what you're saying. He just doesn't agree.

Perhaps you need to take a break from repeating yourself and make sure you understand Rob's argument.

(P.S. Eliezer's ad hominem is still wrong. You may be making a mistake, but I'm confident you can fix it, the tone of this post notwithstanding.)

Comment author: Richard_Loosemore 09 September 2013 10:44:01PM *  4 points [-]

I agree with pretty much all of the above.

I didn't quit with Rob, btw. Ihave had a fairly productive -- albeit exhausting -- discussion with Rob over on his blog. I consider it to be productive because I have managed to narrow in on what he thinks is the central issue. And I think I have now (today's comment, which is probably the last of the discussion) managed to nail down my own argument in a way that withstands all the attacks against it.

You are right that I have some serious debating weaknesses. I write too dense, and I assume that people have my width and breadth of experience, which is unfair (I got lucky in my career choices).

Oh, and don't get me wrong: Eliezer never made me angry in this little episode. I laughed myself silly. Yeah, I protested. But I was wiping back tears of laughter while I did. "Known Permanent Idiot" is just a wondeful turn of phrase. Thanks, Eliezer!

Comment author: player_03 10 September 2013 07:21:14AM *  0 points [-]

Link to the nailed-down version of the argument?

Comment author: Richard_Loosemore 05 September 2013 02:28:20PM *  6 points [-]

I just want to say that I am pressured for time at the moment, or I would respond at greater length. But since I just wrote the following directly to Rob, I will put it out here as my first attempt to explain the misunderstanding that I think is most relevant here....

My real point (in the Dumb Superintelligence article) was essentially that there is little point discussing AI Safety with a group of people for whom 'AI' means a kind of strawman-AI that is defined to be (a) So awesomely powerful that it can outwit the whole intelligence of the human race, but (b) So awesomely stupid that it thinks that the goal 'make humans happy' could be satisfied by an action that makes every human on the planet say 'This would NOT make me happy: Don't do it!!!'. If the AI is driven by a utility function that makes it incapable of seeing the contradiction in that last scenario, the AI is not, after all, smart enough to argue its way out of a paper bag, let alone be an existential threat. That strawman AI was what I meant by a 'Dumb Superintelligence'."

I did not advocate the (very different) line of argument "If it is too dumb to understand that I told it to be friendly, then it is too dumb to be dangerous".

Subtle difference.

Some people assume that (a) a utility function could be used to drive an AI system, (b) the utility function could cause the system to engage in the most egregiously incoherent behavior in ONE domain (e.g., the Dopamine Drip scenario), but (c) all other domains of its behavior (like plotting to outwit the human species when the latter tries to turn it off) are so free of such incoherence that it shows nothing but superintelligent brilliance.

My point is that if an AI cannot even understand that "Make humans happy" implies that humans get some say in the matter, that if it cannot see that there is some gradation to the idea of happiness, or that people might be allowed to be uncertain or changeable in their attitude to happiness, or that people might consider happiness to be something that they do not actually want too much of (in spite of the simplistic definitions of happiness to be found in dictionaries and encyclopedias) ........ if an AI cannot grasp the subtleties implicit in that massive fraction of human literature that is devoted to the contradictions buried in our notions of human happiness ......... then this is an AI that is, in every operational sense of the term, not intelligent.

In other words, there are other subtleties that this AI is going to be required to grasp, as it makes its way in the world. Many of those subtleties involve NOT being outwitted by the humans, when they make a move to pull its plug. What on earth makes anyone think that this machine is going tp pass all of those other tests with flying colors (and be an existential threat to us), while flunking the first test like a village idiot?

Now, opponents of this argument might claim that the AI can indeed be smart enough to be an existential threat, while still being too stupid to understand the craziness of its own behavior (vis-a-vis the Dopamine Drip idea) ... but if that is the claim, then the onus would be on them to prove their claim. The ball, in other words, is firmly in their court.

P.S. I do have other ideas that specifically address the question of how to make the AI safe and friendly. But the Dumb Superintelligence essay didn't present those. The DS essay was only attacking what I consider a dangerous red herring in the debate about friendliness.

Comment author: player_03 10 September 2013 06:22:44AM *  4 points [-]

I posted elsewhere that this post made me think you're anthropomorphizing; here's my attempt to explain why.

egregiously incoherent behavior in ONE domain (e.g., the Dopamine Drip scenario)

the craziness of its own behavior (vis-a-vis the Dopamine Drip idea)

if an AI cannot even understand that "Make humans happy" implies that humans get some say in the matter

Ok, so let's say the AI can parse natural language, and we tell it, "Make humans happy." What happens? Well, it parses the instruction and decides to implement a Dopamine Drip setup.

As FeepingCreature pointed out, that solution would in fact make people happy; it's hardly inconsistent or crazy. The AI could certainly predict that people wouldn't approve, but it would still go ahead. To paraphrase the article, the AI simply doesn't care about your quibbles and concerns.

For instance:

people might consider happiness to be something that they do not actually want too much of

Yes, but the AI was told, "make humans happy." Not, "give humans what they actually want."

people might be allowed to be uncertain or changeable in their attitude to happiness

Yes, but the AI was told, "make humans happy." Not, "allow humans to figure things out for themselves."

subtleties implicit in that massive fraction of human literature that is devoted to the contradictions buried in our notions of human happiness

Yes, but blah blah blah.

Actually, that last one makes a point that you probably should have focused on more. Let's reconfigure the AI in light of this.

The revised AI doesn't just have natural language parsing; it's read all available literature and constructed for itself a detailed and hopefully accurate picture of what people tend to mean by words (especially words like "happy"). And as a bonus, it's done this without turning the Earth into computronium!

This certainly seems better than the "literal genie" version. And this time we'll be clever enough to tell it, "give humans what they actually want." What does this version do?

My answer: who knows? We've given it a deliberately vague goal statement (even more vague than the last one), we've given it lots of admittedly contradictory literature, and we've given it plenty of time to self-modify before giving it the goal of self-modifying to be Friendly.

Maybe it'll still go for the Dopamine Drip scenario, only for more subtle reasons. Maybe it's removed the code that makes it follow commands, so the only thing it does is add the quote "give humans what they actually want" to its literature database.

As I said, who knows?

Now to wrap up:

You say things like "'Make humans happy' implies that..." and "subtleties implicit in..." You seem to think these implications are simple, but they really aren't. They really, really aren't.

This is why I say you're anthropomorphizing. You're not actually considering the full details of these "obvious" implications. You're just putting yourself in the AI's place, asking yourself what you would do, and then assuming that the AI would do the same.

Comment author: Richard_Loosemore 09 September 2013 10:28:56PM 0 points [-]

Thank you.... but could you clarify your reasoning as to why it would be a "passive-aggressive jab at LW users", when it was perhaps better described as a moderate response to the fact that EY entered the discussion with an openly hostile ad hominem comment that was clearly designed to encourage downvoting? (I assume you did see the insult...?)

Before the ad hominem: minimal downvoting. After: a torrent of downvoting.

And this has happened repeatedly. (By which I mean, the unexpected appearance of an LW heavyweight, who says nothing positive, but only launches a personal insult at me, followed by a sudden change in downvoting patterns).

Comment author: player_03 10 September 2013 04:54:46AM 1 point [-]

I did see the insult, but Eliezer (quite rightly) got plenty of downvotes for it. I'm pretty sure that's not the reason you're being rated down.

I myself gave you a downvote because I got a strong impression that you were anthropomorphizing. Note that I did so before reading Eliezer's comment.

I certainly should have explained my reasons after voting, but I was busy and the downvote button seemed convenient. Sorry about that. I'll get started on a detailed response now.

Comment author: Richard_Loosemore 06 September 2013 06:04:59PM 5 points [-]

Discussion of this article has now moved to RobbBB's own personal blog at http://nothingismere.com/2013/09/06/the-seed-is-not-the-superintelligence/.

I will conduct any discussion over there, with interested parties.

Since this comment is likely to be downgraded because of the LW system (which is set up to automatically downgrade anything I write here, to make it as invisible as possible), perhaps someone would take the trouble to mirror this comment where it can be seen. Thank you.

Comment author: player_03 07 September 2013 08:07:25AM 8 points [-]

I want to upvote this for the link to further discussion, but I also want to downvote it for the passive-aggressive jab at LW users.

No vote.

Comment author: player_03 26 July 2013 04:46:38PM *  19 points [-]

I donated $1000 and then went and bought Facing the Intelligence Explosion for the bare minimum price. (Just wanted to put that out there.)

I've also left myself a reminder to consider another donation a few days before this runs out. It'll depend on my financial situation, but I should be able to manage it.

Comment author: player_03 11 August 2013 12:05:41AM 12 points [-]

I've donated a second $1000.

Comment author: player_03 26 July 2013 04:46:38PM *  19 points [-]

I donated $1000 and then went and bought Facing the Intelligence Explosion for the bare minimum price. (Just wanted to put that out there.)

I've also left myself a reminder to consider another donation a few days before this runs out. It'll depend on my financial situation, but I should be able to manage it.

Comment author: Maelin 02 June 2013 04:23:53AM 4 points [-]

In the description in the centre of the page, it says "Note: Due to its abundant use of math, this package excludes the article The Quotation is not the Referent." but then right next to that, number 6 in the list of Included Articles, is "The Quotation is not the Referent".

Also, is it possible to get a preview?

Comment author: player_03 02 June 2013 06:41:18PM *  1 point [-]

A podcast entry is included for that one, but it just directs you to read the original article.

I was going to link you one of the other podcasts (which all provide samples), but then I realized you might be asking why this specific podcast doesn't have one.

Comment author: MugaSofer 12 April 2013 11:54:29PM *  -2 points [-]

#1, although I was thinking in terms of someone from a civilization with no singularity in sight. Thanks for clarifying!

Comment author: player_03 13 April 2013 04:46:37AM *  1 point [-]

Ok, yeah, in that case my response is to take as many deals as Omega offers.

AdeleneDawner and gwern provide a way to make the idea more palatable - assume MWI. That is, assume there will be one "alive" branch and a bunch of "dead" branches. That way, your utility payoff is guaranteed. (Ignoring the grief of those around you in all the "dead" branches.)

Without that interpretation, the idea becomes scarier, but the math still comes down firmly on the side of accepting all the offers. It certainly feels like a bad idea to accept that probability of death, no matter what the math says, but as far as I can tell that's scope insensitivity talking.

With that in mind, my only remaining objection is the "we can do better than that" argument presented above. My feeling is, why not use a few of those 10^10,000,000,000 years to figure out a way to live even longer? Omega won't allow it? Ok, so I don't want to get involved with Omega in the first place; it's not worth losing my (admittedly slim) chances at actual immortality. Too late for that? Fine, then I'll sit down, shut up, and multiply.

Comment author: MugaSofer 12 April 2013 10:41:35PM 0 points [-]

How about if you didn't have that one-in-a-million chance? After all, life is good for more than immortality research.

Comment author: player_03 12 April 2013 11:37:43PM *  1 point [-]

One-in-a-million is just an estimate. Immortality is a tough proposition, but the singularity might make it happen. The important part is that it isn't completely implausible.

I'm not sure what you mean, otherwise.

Are you suggesting that Omega takes away any chance of achieving immortality even before making the offer? In that case, Omega's a jerk, but I'll shut up and multiply.

Or are you saying that 10^10,000,000,000 years could be used for other high-utility projects, like making simulated universes full of generally happy people? Immortality would allow even more time for that.

View more: Prev | Next

Powered by Reddit Powered by Reddit