nhamann comments on Short versions of the basic premise about FAI - Less Wrong

3 Post author: NancyLebovitz 31 October 2010 11:14PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Sort By: Best

You are viewing a single comment's thread. Show more comments above.

Comment author: nhamann 31 October 2010 11:45:28PM * 2 points [-]

Potentially yes, but I think the problem can be profitably restated without any reference to the Singularity or FOOMing AI. (I've often wondered whether the Friendliness problem would be better recognized and accepted if it was presented without reference to the Singularity).

Edit: See also Vladimir Nesov's summary, which is quite good, but not quite as short as you're looking for here.

Comment author: NancyLebovitz 01 November 2010 12:14:01AM 1 point [-]

Friendliness would certainly be worth pursuing-- it applies to a lot of human issues in addition to what we want from computer programs.

Still, concerns about FOOM is the source of urgency here.

Comment author: NihilCredo 01 November 2010 03:04:52AM * 7 points [-]

Concerns about FOOM are also what makes SIAI look like (and some posters talk like) a loony doom cult.

Skip the "instant godlike superintelligence with nanotech arms" shenanigans, and AI ethics still remains an interesting and important problem, as you observed.

But it's much easier to get people to look at an interesting problem so you can then persuade them that it's serious, than it is to convince them that they are about to die in order to make them look at your problem. Especially since modern society has so inured people to apocalyptic warnings that the wiser half of the population takes them with a few kilograms of salt to begin with.

Comment author: JGWeissman comment score below threshold [+] (6 children)

Comment author: Kingreaper 01 November 2010 05:20:45PM * 0 points [-]

Just because a rational person would believe something, doesn't mean a rational person would say that thing.

If telling people the fate of the world depends on you is going to make them less likely to listen, you probably shouldn't tell them that. Especially if it's true (because that just makes it more important that they listen)

Comment author: JGWeissman 01 November 2010 05:53:02PM 0 points [-]

FOOM is central to the argument that we need to solve Friendliness up front, rather than build it incrementally as patches to a slowly growing AGI. If you leave it out to get past weirdness censors, you can no longer support the same conclusions.

Comment author: Kingreaper 01 November 2010 06:03:11PM * 3 points [-]

NihilCredo said:

But it's much easier to get people to look at an interesting problem so you can then persuade them that it's serious, than it is to convince them that they are about to die in order to make them look at your problem.

Notice that Nihil didn't propose never mentioning the urgency you believe exists, just not using it as your rallying cry.

I got fascinated by friendliness theorem despite never believing in the singularity (and in fact, not knowing much about the idea other than that it was being argued on the basis of extrapolating moore's law, which explains why I didn't buy it.)

Other people could be drawn in by the interesting philosophical and practical challenges of Friendliness Theorem without the FOOM threat.

Comment author: JGWeissman 01 November 2010 06:13:22PM 0 points [-]

It is more important to convince the AGI researchers who see themselves as practical people trying to achieve good results in the real world than people who like an interesting theoretical problem.

Comment author: [deleted] 04 November 2010 04:25:00PM 0 points [-]

Because people who like theoretical problems are less effective than people trying for good results? I don't buy it.

Comment author: JGWeissman 04 November 2010 04:30:59PM 0 points [-]

No. Because it is better for the people who would otherwise be working on dangerous AGI to realize they should not do that, than to have people who would not have been working on AI at all to comment that the dangerous AGI researchers shouldn't do that.

Comment author: simpleton 31 October 2010 11:53:24PM 0 points [-]

The Hidden Complexity of Wishes

Comment author: nhamann 01 November 2010 12:41:14AM * 0 points [-]

I do not understand your point. Would you care to explain?

Comment author: simpleton 01 November 2010 03:13:27AM 1 point [-]

Sorry, I thought that post was a pretty good statement of the Friendliness problem, sans reference to the Singularity (or even any kind of self-optimization), but perhaps I misunderstood what you were looking for.

Comment author: nhamann 01 November 2010 04:22:47AM 0 points [-]

Oh, I misunderstood your link. I agree, that's a good summary of the idea behind the "complexity of value" hypothesis.

nhamann comments on Short versions of the basic premise about FAI - Less Wrong

Article Navigation

Comments (23)

Nearest Meetups

Virtual Study Room

Recent Comments

Recent Posts

Recent Rationality Quotes

Recent Wiki Edits

Top Contributors, 30 Days

Recent Karma Awards

Comments (23)

Top Contributors, 30 Days

You'll need to login or register to do that

(Don't worry, it only takes a few seconds)

Create

Login