mwaser comments on Open Thread June 2010, Part 2 - Less Wrong

7 Post author: komponisto 07 June 2010 08:37AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (534)

Tags: open_thread

Comments (534)

Sort By: Best

Show:

You are viewing a single comment's thread.

Comment author: mwaser 11 June 2010 03:05:43PM * -2 points [-]

SIAI, Yudkowsky, Friendly AI, CEV, and Morality

This post entitled A Dangerous "Friend" Indeed (http://becominggaia.wordpress.com/2010/06/10/a-dangerous-friend-indeed/) has it all.

Comment author: JoshuaZ 11 June 2010 03:48:47PM 0 points [-]

Huh. That's very interesting. I'm a bit confused by the claim that evolution bridges the is/ought divide which seems more like conflating different meanings of words more than anything else. But the general point seems strong.

Comment author: cupholder 11 June 2010 11:10:15PM * 5 points [-]

Yeah, I really disagree with this:

Evolution then is the bridge across the Is/Ought divide. An eye has the purpose or goal of seeing. Once you have a goal or purpose, what you “ought” to do IS make those choices which have the highest probability of fulfilling that goal/purpose. If we can tease apart the exact function/purpose/goal of morality from exactly how it enhances evolutionary fitness, we will have an exact scientific description of morality — and the best method of determining that is the scientific method.

My understanding is that those of us who refer to the is/ought divide aren't saying that a science of how humans feel about what humans call morality is impossible. It is possible, but it's not the same thing as a science of objective good and bad. The is/ought divide is about whether one can derive moral 'truths' (oughts) from facts (ises), not about whether you can develop a good model of what people feel are moral truths. We'll be able to do the latter with advances in technology, but no one can do the former without begging the question by slipping in an implicit moral basis through the back door. In this case I think the author of that blog post did that by assuming that fitness-enhancing moral intuitions are The Good And True ones.

Comment author: mwaser comment score below threshold [+] (8 children)

Comment author: cupholder 13 June 2010 06:48:00AM 1 point [-]

I'm not sure that I understand your comment. I can understand the individual paragraphs taken one by one, but I don't think I understand whatever its overall message is.

(On a side note, you needn't worry about your karma for the time being; it can't go any lower than 0, and you can still post comments with 0 karma.)

Comment author: Alicorn 13 June 2010 07:51:37AM 1 point [-]

It can't go any lower than 0, and you can still post comments with 0 karma.

It can go lower than 0; it just won't display lower than 0.

Comment author: Kevin 13 June 2010 08:17:58AM 1 point [-]

Yup, I've been way down in the negative karma.

Comment author: cupholder 13 June 2010 05:05:20PM 0 points [-]

My bad. I was going by past experience with seeing other people's karma drop to zero and made a flaky inference because I never saw it go below that myself.

Comment author: mwaser 14 June 2010 02:42:19AM -2 points [-]

Do me a favor and check out my blog at http://becominggaia.wordpress.com. I've clearly annoyed someone (and it's quite clear whom) enough that all my posts quickly pick up enough of a negative score to be below the threshold. It's a very effective censoring mechanism and, at this point, I really don't see any reason why I should ever attempt to post here again. Nice "community".

Comment author: JoshuaZ 14 June 2010 02:52:36AM * 6 points [-]

I don't think you are getting voted down out of censorship. You are getting voted down for as far as I can tell four reasons: 1) You don't explain yourself very well. 2) You repeatedly link to your blog in a borderline spammish fashion. Examples are here and here. In replies to the second one you were explicitly asked not to blogspam and yet continued to do so. 3) You've insulted people repeatedly (second link above) and personalized discussions. You've had posts which had no content other than to insult and complain about the community. At least one of those posts was in response to an actually reasoned statement. See this example- http://lesswrong.com/lw/2bi/open_thread_june_2010_part_2/251o 4) You've put non-existent quotes in quotation marks (second link in the spamming example has an example of this).

Comment author: timtyler 31 October 2010 09:46:21AM 1 point [-]

Brief feedback:

Your views are quite a bit like those of Stefan Pernar. http://rationalmorality.info/

However, they are not very much like those of the people here.

I expect that most of the people here just think you are confused and wrong.

Comment author: Mass_Driver 13 June 2010 01:34:33AM 1 point [-]

You're not making any sense to me.

Comment author: Vladimir_Nesov 11 June 2010 05:40:27PM * 2 points [-]

Dig a bit deeper, and you'll find too much confusion to hold any argument alive, no matter what the conclusion is supposed to be, correct or not. For that matter, what do you think is the "general point", and can you reach the point of agreement with Mark on what that is, being reasonably sure you both mean the same thing?

Comment deleted 11 June 2010 09:30:47PM [+] (22 children)

Comment author: Vladimir_Nesov 11 June 2010 09:55:22PM 3 points [-]

Vladimir, all you've presented here is slanderous dart-throwing with absolutely no factual backing whatsoever.

I state my conclusion and hypothesis, for how much evidence that's worth. I understand that it's impolite on my part to do that, but I suspect that JoshuaZ's agreement falls under some kind of illusion of transparency, hence request for greater clarity in judgment.

Comment author: JoshuaZ 11 June 2010 11:48:23PM 2 points [-]

Yeah ok. After rereading it, I'm inclined to agree. I think I was projecting my own doubts about CEV-type approaches onto the article (namely that I'm not convinced that a CEV is actually meaningful or well-defined). And looking again, they don't seem to be what the person here is talking about. It seems like at least part of this is about the need for punishment to exist in order for a society to function and the worry that an AI will prevent that. And rereading that and putting it in my own words, that sounds pretty silly if I'm understanding it, which suggests I'm not. So yeah, this article needs clarification.

Comment author: Vladimir_Nesov 12 June 2010 12:00:54AM 0 points [-]

namely that I'm not convinced that a CEV is actually meaningful or well-defined

Yes, CEV needs work, it's not technical, and it's far from clear that it describes what we should do, although the essay does introduce a number of robust ideas and warnings about seductive failure modes.

Among more obvious problems with Mark's position: "slavery" and "true morality without human bias". Seems to reflect confusion about free will and metaethics.

Comment author: JoshuaZ 12 June 2010 02:06:58AM 0 points [-]

I think the analogy is something like imagine if you were able to make a creature identical to a human except that the greatest desire they had was to serve actual humans. Would that morally be akin to slavery? I think many of us would say yes. So is there a similar issue if one programs a sentient non-human entity under similar restrictions?

Comment author: khafra 12 June 2010 02:06:44PM 3 points [-]

Not analogous, but related and possibly relevant: Many humans in the BDSM lifestyle desire to be the submissive partner in 24/7 power exchange relationships. Are these humans sane; are they "ok"? Is it ethical to allow this kind of relationship? To encourage it?

Comment author: ciphergoth 12 June 2010 02:23:16PM 5 points [-]

TBH I think this may muddy the waters more than it clears them. When we're talking about human relations, even those as unusual as 24/7, we're still operating in a field where our intuitions have much better grip than they will trying to reason about the moral status of an AI.

Comment author: Blueberry 12 June 2010 09:00:31AM 5 points [-]

Taboo "slavery" here; it's a label that masks clear thinking. If making such a creature is slavery, it's a kind of slavery that seems perfectly fine to me.

Comment author: DanielVarga 12 June 2010 12:41:03PM 3 points [-]

Voted up for the suggestion to taboo slavery. Not an endorsement of the opinion that it is a perfectly fine kind of slavery.

Comment author: JoshuaZ 12 June 2010 12:11:09PM 2 points [-]

Ok. So is it ethical to engineer a creature that is identical to human but desires primarily to just serve humans?

continue this thread »

Comment deleted 14 June 2010 02:54:44AM [-]

Comment author: JoshuaZ 14 June 2010 03:02:44AM * 5 points [-]

By any chance are you trying to troll? I just told you that you were being downvoted for blogspamming, insulting people, and unnecessary personalization. Your focus on Vladimir manages to also hit two out of three of these and comes across as combative and irrational. Even if this weren't LW where people are more annoyed by irrational argumentation styles, people would be annoyed by a non-regular going out of their way to personally attack a regular. This would be true in any internet forum and all the more so when those attacks are completely one-sided.

And having now read what you just linked to, I have to say that it fits well with another point I said in my earlier remark to you: you are being downvoted in a large part for not explaining yourself well at all. If I may make a suggestion: Maybe try reading your comments outloud to yourself before you post them? I've found that helps me a lot in detecting whether I am explaining something well. This may not work for you, but it may be worth trying.

Comment author: Vladimir_Nesov 14 June 2010 10:00:49AM 2 points [-]

Vladimir and his cronies (assuming that they aren't just him under another name)

Yay world domination! I have a personal conspiracy theory now!

Comment author: Vladimir_Nesov 12 June 2010 07:09:04PM * 0 points [-]

FAI (assuming we managed to set its preference correctly) admits a general counterargument against any implementation decisions in its design being seriously incorrect: FAI's domain is the whole world, and FAI is part of that world. If it's morally bad to have FAI in the form it was initially constructed, then, barring some penalty the FAI will change its own nature so as to make the world better.

In this particular case, the suggested conflict is between what we prefer to be done with things other than the FAI (the "serving humanity" part), and what we prefer to be done with FAI itself (the "slavery is bad" part). But FAI operates on the world as whole, and things other than FAI are not different from FAI itself in this regard. Thus, with the criterion of human preference, FAI will decide what is the best thing to do, taking into account both what happens to the world outside of itself, and what happens to itself. Problem solved.

mwaser comments on Open Thread June 2010, Part 2 - Less Wrong

Article Navigation

Comments (534)

Nearest Meetups

Virtual Study Room

Recent Comments

Recent Posts

Recent Rationality Quotes

Recent Wiki Edits

Top Contributors, 30 Days

Recent Karma Awards

Comments (534)

Top Contributors, 30 Days

You'll need to login or register to do that

(Don't worry, it only takes a few seconds)

Create

Login