Furcas comments on A Less Wrong singularity article? - Less Wrong

28 Post author: Kaj_Sotala 17 November 2009 02:15PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (210)

You are viewing a single comment's thread. Show more comments above.

Comment author: Furcas 18 November 2009 10:14:27PM 5 points [-]

Look, it's not very complicated: When you see Eliezer write "morality" or "oughts", read it as "human morality" and "human oughts".

Comment author: timtyler 18 November 2009 10:51:16PM *  3 points [-]

Um, that's what I just said: "presumably you are talking about ought<human>".

We were then talking about the meaning of ought<alien>.

There's also the issue of whether to discuss ought<human(2000BC)> and ought<human(2000AD)> - which are evidently quite different - due to the shifting moral zeitgeist.

Comment author: Furcas 18 November 2009 11:33:23PM 0 points [-]

Well then, I don't understand why you would find statements like "There are no alien [human oughts]" and "They don't see [human morality] differently from us" bizarre-sounding.

Comment author: timtyler 19 November 2009 06:30:32PM *  0 points [-]

Having established EY meant ought<human>, I was asking about ought<alien>.

Maybe you are right - and EY misinterpreted me - and genuinely thought I was asking about ought<human><alien>.

If so, that seems like a rather ridiculous question for me to be asking - and I'm surprised it made it through his sanity checker.

Comment author: wedrifid 18 November 2009 10:34:01PM *  3 points [-]

Look, it's not very complicated: When you see Eliezer write "morality" or "oughts", read it as "human morality" and "human oughts".

It isn't that simple either. Human morality contains a significant component of trying to coerce other humans into doing things that benefit you. Even on a genetic level humans come with significantly different ways of processing moral thoughts. What is often called 'personality', particularly in the context of 'personality type'.

The translation I find useful is to read it as "Eliezer-would-want". By the definitions Eliezer has given us the two must be identical. (Except, perhaps if Eliezer has for some reason decided to make himself immoral a priori.)