shminux comments on Ritual 2012: A Moment of Darkness - Less Wrong

35 Post author: Raemon 28 December 2012 09:09AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (136)

Comments (136)

Sort By: Best

You are viewing a single comment's thread. Show more comments above.

Comment author: shminux 25 December 2012 06:14:47PM 1 point [-]

Looks like you and I have different terminal meta-values.

Comment author: MugaSofer 25 December 2012 10:32:42PM -1 points [-]

Unless you own a time machine and come from a future where salmon-people rule the earth, I seriously doubt that. If you're a neurotypical human, then you terminally value not killing people. Mindraping them into doing it themselves continues to violate this preference, unless all you actually care about is people's distress when you kill them, in which case remind me never to drink anything you give me.

Comment author: shminux 26 December 2012 12:00:34AM * 2 points [-]

Typical mind fallacy?

Comment author: MugaSofer 26 December 2012 01:21:31AM * -1 points [-]

... are you saying I'm foolish to assume that you value human life? Would you, in fact, object to killing someone if they wouldn't realize? Yes? Congratulations, you're not a psychopath.

Comment author: Kaj_Sotala 31 December 2012 10:09:06AM 0 points [-]

Everyone who voluntarily joins the military is a psychopath?

Comment author: MugaSofer comment score below threshold [+] (7 children)

Comment author: Kaj_Sotala 31 December 2012 09:17:57PM * 1 point [-]

I think they have many different justifications depending on the person, ranging from "it's a necessary evil" to "I need to pay for college and can hopefully avoid getting into battle" to "only the lives of my own countrymen matter", just like people can have many different justifications for why they'd approve modifying the terminal values of others.

Comment author: MugaSofer 01 January 2013 01:24:11PM -2 points [-]

So, despite the downvotes that bought me ...

I said "non-psychopaths consider killing a Bad Thing."
You said "But what about people who join the army?"
I said "What do you think?"
You said "I think they justify it as saving more lives than it kills, or come up with reasons it's not really killing people"

I think this conversation is over, don't you?

Comment author: Kaj_Sotala 01 January 2013 01:55:27PM 1 point [-]

Do you see my point that there are plenty of ways by which somebody can consider killing as not-so-bad, without needing to be a psychopath?

Comment author: MugaSofer 01 January 2013 02:30:18PM 2 points [-]

No. Something can be bad without being worse than the other options, and people can be mistaken about whether something an action will kill people. This is quite separate from actually having no term for human life in their utility function.

Comment author: Peterdjones 01 January 2013 02:14:36PM * 1 point [-]

There's an important difference between "not bad" and "bad but justifiable under some circumstances". I don't think believers in abortion, execution or war believe that killing per se is morally neutral. Each of those three has its justification.

continue this thread »

Comment author: wedrifid 26 December 2012 07:14:45AM 0 points [-]

Unless you own a time machine and come from a future where salmon-people rule the earth, I seriously doubt that. If you're a neurotypical human, then you terminally value not killing people.

"Neurotypical"... almost as powerful as True!

Comment author: MugaSofer 26 December 2012 03:36:49PM * -1 points [-]

Seems like a perfectly functional Schelling point to me. Besides, I needed a disclaimer for the possibility that he's actually a psychopath or, indeed, an actual salmon-person (those are still technically "human", I assume.)

Comment author: Kawoomba comment score below threshold [+] (12 children)

Comment author: MugaSofer 26 December 2012 03:44:37PM 1 point [-]

I may be confused by the sarcasm here. Could you state your objection more clearly? Are you arguing "neurotypical" is not a useful concept? Are you accusing me of somehow discriminating against agents that implement other utility functions? Are you objecting to my assertion that creating an agent with a different utility function is usually instrumentally bad, because it is likely to attempt to implement that utility function to the exclusion of yours?

Comment author: Kawoomba 26 December 2012 06:37:24PM 0 points [-]

Are you accusing me of somehow discriminating against agents that implement other utility functions?

Yes, here's your last reply to me on just that topic:

Except that humans share a utility function, which doesn't change. (...) Cached thoughts can result in actions that, objectively, are wrong. They are not wrong because this is some essential property of these actions, morality is in our minds, but we can still meaningfully say "this is wrong" just was we can say "this is a chair" or "there are five apples".

Also:

The fact that morality is acted upon in different ways (due to your "layers" or simply mistaken beliefs about the world) doesn't change the fact that it is there, underneath [emphasis mine], and that this is the standard we work by to declare something "good" or "bad". We aren't perfect at it, but we can make a reasonable attempt.

It is bizarre to me how you believe there is some shared objective morality - "underneath" - that is correct because it is "typical" (hello fallacious appeal to majority), and that outliers that have a different utility function have false values.

Even if there are shared elements (even across e.g. large vague categories such as Chinese values and Western values), such as surmised by CEV_humankind (probably an almost empty set), that does not make anyone's own morality/value function wrong, it merely makes it incongruent with the current cultural majority views. Hence the "tyranny of some supposedly existing elusive majority".

Comment author: MugaSofer 27 December 2012 02:40:44AM * 0 points [-]

Bloody hell, it's you again. I hadn't noticed I was talking to the same person I had that argument with. I guess that information does add some context to your comment.

I'm not saying they're wrong, except when "wrong" is defined with reference to standard human values (which is how I, and many others on LW, commonly use the term.) I am saying their values are not my values, or (probably) your values. That's not to say they don't have moral worth or anything, just that giving them (where "them" means salmon people, clippies or garden-variety psychopaths) enough power will result in them optimizing the universe for their own goals, not ours.

Of course, I'm not sure how you judge moral arguments, so maybe I'm assuming some common prior or something I shouldn't be.

Comment author: Kawoomba 27 December 2012 07:05:29AM 1 point [-]

Your comment of just saying "well, this is the norm" does not fit with your previously stated views, see this exchange:

I would value the suffering of my child as more important than the suffering of your child.

That seems ... kind of evil, to be honest.

Are most all parents "evil" in that regard?

I believe the technical term is "biased".

My assertion is that all humans share utility - which is the standard assumption in ethics, and seems obviously true

So if the majority of humans values the lives of their close family circle higher than random other human lives - those are the standard human values, the norm - then you still call those evil or biased, because they don't agree with your notion of what standard human values should be, based on "obviously true" ethical assumptions. *

Do you see the cognitive dissonance? (Also, you're among the first - if not the only - commenters on LW who I've seen using even just "standard human values" as an ought, outside the context of CEV - a different concept - for FAI.)

* It fits well with some divine objective morality, however it does not fit well with some supposed and only descriptive, not prescriptive "standard human values" (not an immutable set in itself, you probably read Harry's monologue on shifting human values through the ages in the recent HPMOR chapter).

Comment author: MugaSofer 27 December 2012 03:57:52PM -1 points [-]

So if the majority of humans values the lives of their close family circle higher than random other human lives - those are the standard human values, the norm - then you still call those evil or biased, because they don't agree with your notion of what standard human values should be, based on "obviously true" ethical assumptions. *

I'm asserting the values you describe are not, in fact, the standard human values. If it turned out that parents genuinely have different values to other people, then they wouldn't be biased (down to definitions on "evil".)

(Also, you're among the first - if not the only - commenters on LW who I've seen using even just "standard human values" as an ought, outside the context of CEV - a different concept - for FAI.)

We are both agents with human ethics. When I say we "ought" to do something, I mean by the utility function we both share. If I were a paperclipper, I would need separate terms for my ethics and yours. But then, why would I help you implement values that oppose my own?

Comment author: Kawoomba 27 December 2012 07:17:31PM 1 point [-]

It comes down to "I value this human over that other human" being a part of your utility function, f(this.human) > f(that.human). [Syntactical overloading for comedic relief] A bias is something affecting your cognition - how you process information, not what actions you choose based upon that processing. While you can say "your values are biased towards X", that is using the term in a different than the usual LW context.

In particular, I doubt you'll find more than 1 in a million humans who would not value some close relative's / friend's / known person's life over a randomly picked human life ("It could be anything! It could even be another boat!").

You have here a major, major part of the utility function of a majority of humans (throughout history! in-group > out-group), yet you persist on calling that an evil bias. Why, because it does not fit with what the "standard human values" should be? What god intended? Or is there no religious element to your position at all? If so, please clarify.

You realize that most humans value eating meat, right? Best pick up that habit, no? ;)

continue this thread »

Comment author: BerryPick6 25 December 2012 06:35:47PM 1 point [-]

I'm really curious to know what you mean by 'terminal meta-values'. Would you mind expanding a bit, or pointing me in the direction of a post which deals with these things?

Comment author: shminux 26 December 2012 01:59:34AM 1 point [-]

Say, whether it is ever acceptable to adjust someone's terminal values.

Comment author: MugaSofer 26 December 2012 02:09:00AM * 4 points [-]

No, I'm perfectly OK with adjusting terminal values in certain circumstances. For example, turning a Paperclipper into an FAI is obviously a good thing.

EDIT: Of course, turning an FAI into a Paperclipper is obviously a bad thing, because instead of having another agent working towards the greater good, we have an agent working towards paperclips, which is likely to get in the way at some point. Also, it's likely to feel sad when we have to stop it turning people into paperclips, which is a shame.

shminux comments on Ritual 2012: A Moment of Darkness - Less Wrong

Article Navigation

Comments (136)

Nearest Meetups

Virtual Study Room

Recent Comments

Recent Posts

Recent Rationality Quotes

Recent Wiki Edits

Top Contributors, 30 Days

Recent Karma Awards

Comments (136)

Top Contributors, 30 Days

You'll need to login or register to do that

(Don't worry, it only takes a few seconds)

Create

Login