Wei_Dai comments on Less Wrong Q&A with Eliezer Yudkowsky: Ask Your Questions - Less Wrong

16 Post author: MichaelGR 11 November 2009 03:00AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (682)

Comments (682)

Sort By: Best

Show:

You are viewing a single comment's thread. Show more comments above.

Comment author: StefanPernar 16 November 2009 02:21:23PM -1 points [-]

"This isn't a logical fallacy but it is cause to dismiss the argument if the readers do not, in fact, have every reason to have said belief."

But the reasons to change ones view are provided on the site, yet rejected without consideration. How about you read the paper linked under B and should that convince you, maybe you have gained enough provisional trust that reading my writings will not waste your time to suspend your disbelief and follow some of the links in the about page of my blog. Deal?

Comment author: wedrifid 16 November 2009 03:07:13PM * 5 points [-]

How about you read the paper linked under B and should that convince you

I have read B. It isn't bad. The main problem I have with it is that the language used blurs the line between "AIs will inevitably tend to" and "it is important that the AI you create will". This leaves plenty of scope for confusion.

I've read through some of your blog and have found that I consistently disagree with a lot of what you say. The most significant disagreement can be traced back to the assumption of a universal absolute 'Rational' morality. This passage was a good illustration:

Moral relativists need to understand that they can not eat the cake and keep it too. If you claim that values are relative, yet at the same time argue for any particular set of values to be implemented in a super rational AI you would have to concede that this set of values – just as any other set of values according to your own relativism – is utterly whimsical, and that being the case, what reason (you being the great rationalist, remember?) do you have to want them to be implemented in the first place?

You see, I plan to eat my cake but don't expect to be able to keep it. My set of values are utterly whimsical (in the sense that they are arbitrary and not in the sense of incomprehension that the Ayn Rand quotes you link to describe). The reasons for my desires can be described biologically, evolutionarily or with physics of a suitable resolution. But now that I have them they are mine and I need no further reason.

Comment author: StefanPernar comment score below threshold [+] (4 children)

Comment author: wedrifid 17 November 2009 12:48:07AM 5 points [-]

If that is your stated position then in what way can you claim to create FAI with this whimsical set of goals?

Were it within my power to do so I would create a machine that was really, really good at doing things I like. It is that simple. This machine is (by definition) 'Friendly' to me.

you will end up being a deluded self serving optimizer.

I don't know where the 'deluded' bit comes from but yes, I would end up being a self serving optimizer. Fortunately for everyone else my utility function places quite a lot of value on the whims of other people. My self serving interests are beneficial to others too because I am actually quite a compassionate and altruistic guy.

PS: Instead of using quotation marks you can put a '>' at the start of a quoted line. This convention makes quotations far easier to follow. And looks prettier.

Comment author: timtyler 16 November 2009 07:31:11PM * 3 points [-]

There is no such thing as an "unobjectionable set of values".

Imagine the values of an agent that wants all the atoms in the universe for its own ends. It will object to any other agent's values - since it objects to the very existence of other agents - since those agents use up its precious atoms - and put them into "wrong" configurations.

Whatever values you have, they seem bound to piss off somebody.

Comment author: StefanPernar 18 November 2009 02:56:44AM -1 points [-]

There is no such thing as an "unobjectionable set of values".

And here I disagree. Firstly see my comment about utility function interpretation on another post of yours. Secondly, as soon as one assumes existence as being preferable over non-existence you can formulate a set of unobjectionable values (http://www.jame5.com/?p=45 and http://rationalmorality.info/?p=124). But granted, if you do not want to exist nor have a desire to be rational then rational morality has in fact little to offer you. Non existence and irrational behavior being so trivial goals to achieve after all that it would hardly require – nor value and thus seek for that mater – well thought out advice.

Comment author: timtyler 18 November 2009 08:23:05AM 2 points [-]

Alas, the first link seems almost too silly to bother with to me, but briefly:

Unobjectionable - to whom? An agent objecting to another agent's values is a simple and trivial occurrence. All an agent has to do is to state that - according to its values - it wants to use the atoms of the agent with the supposedly unobjectionable utility function for something else.

"Ensure continued co-existence" is vague and wishy-washy. Perhaps publicly work through some "trolley problems" using it - so people have some idea of what you think it means.

You claim there can be no rational objection to your preferred utility function.

In fact, an agent with a different utility function can (obviously) object to its existence - on grounds of instrumental rationality. I am not clear on why you don't seem to recognise this.

Wei_Dai comments on Less Wrong Q&A with Eliezer Yudkowsky: Ask Your Questions - Less Wrong

Article Navigation

Comments (682)

Nearest Meetups

Virtual Study Room

Recent Comments

Recent Posts

Recent Rationality Quotes

Recent Wiki Edits

Top Contributors, 30 Days

Recent Karma Awards

Comments (682)

Top Contributors, 30 Days

You'll need to login or register to do that

(Don't worry, it only takes a few seconds)

Create

Login