Richard_Loosemore comments on Debunking Fallacies in the Theory of AI Motivation - LessWrong

8 Post author: Richard_Loosemore 05 May 2015 02:46AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (343)

You are viewing a single comment's thread. Show more comments above.

Comment author: Richard_Loosemore 18 May 2015 09:55:30PM 0 points [-]

I said:

How exactly do you propose that the AI "weighs contextual constraints incorrectly" when the process of weighing constraints requires most of the constraints involved (probably thousands of them) to all suffer a simultaneous, INDEPENDENT 'failure' for this to occur?

And your reply was:

I'd hazard a guess that, for any given position, less than 70% of humans will agree without reservation. The issue isn't that thousands of failures occur. The issue is that thousands of failures -always- occur.

This reveals that you are really not understanding what a weak constraint system is, and where the system is located.

When the human mind looks at a scene and uses a thousand clues in the scene to constrain the interpretation of it, those thousand clues all, when the network settles, relax into a state in which most or all of them agree about what is being seen. You don't get "less than 70%" agreement on the interpretation of the scene! If even one element of the scene violates a constraint in a strong way, the mind orients toward the violation extremely rapidly.

The same story applies to countless other examples of weak constraint relaxation systems dropping down into energy minima.

Let me know when you do understand what you are talking about, and we can resume.

Comment author: OrphanWilde 18 May 2015 10:30:57PM 4 points [-]

There is no energy minimum, if your goal is Friendliness. There is no "correct" answer. No matter what your AI does, no matter what architecture it uses, with respect to human goals and concerns, there is going to be a sizable percentage to whom it is unequivocally Unfriendly.

This isn't an image problem. The first problem you have to solve in order to train the system is - what are you training it to do?

You're skipping the actual difficult issue in favor of an imaginary, and easy to solve, issue.

Comment author: Lumifer 19 May 2015 02:25:43PM *  0 points [-]

There is no "correct" answer.

However there are a lot of "wrong" answers.

Comment author: TheAncientGeek 19 May 2015 07:41:04AM *  0 points [-]

there is going to be a sizable percentage to whom it is unequivocally Unfriendly

Unfriendly is an equivocal term.

"Friendliness" is ambiguous. It can mean safety, ie not making things worse, or it can mean making things better, creating paradise on Earth.

Friendliness in the second sense is a superset of morality. A friendly AI will be moral, a moral AI will not necessarily be friendly.

"Unfriendliness" is similarly ambiguous: an unfriendly AI may be downright dangerous; or it might have enough grasp of ethics to be safe, but not enough to be able to make the world a much more fun place for humans. Unfriendliness in the second sense is not, strictly speaking a safety issue.

A lot of people are able to survive the fact that some institutions, movements and ideologies are unfriendly to them, for some value of unfriendly. Unfriendliness doesn't have to be terminal.

Comment author: OrphanWilde 19 May 2015 01:08:21PM 0 points [-]

Unfriendly is an equivocal term.

Everything is equivocal to someone. Do you disagree with my fundamental assertion?

Comment author: TheAncientGeek 21 May 2015 09:03:12AM 0 points [-]

I can't answer unequivocally for the reasons given.

There won't be a sizeable percentage to whom the AI is unfriendly in the sense of obliterating them.

There might well be a percentage to whom the AI is unfriendly in some business as usual sense.

Comment author: OrphanWilde 21 May 2015 02:37:58PM 0 points [-]

Obliterating them is only bad by your ethical system. Other ethical systems may hold other things to be even worse.

Comment author: TheAncientGeek 22 May 2015 02:22:39PM 0 points [-]

Irrelevant.

Comment author: OrphanWilde 22 May 2015 02:32:28PM 0 points [-]

You responded to me in this case. It's wholly relevant to my point that You-Friendly AI isn't a sufficient condition for Human-Friendly AI.