Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Decius comments on What I Think, If Not Why - Less Wrong

25 Post author: Eliezer_Yudkowsky 11 December 2008 05:41PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (100)

Sort By: Old

You are viewing a single comment's thread. Show more comments above.

Comment author: Decius 30 December 2012 11:24:16PM -1 points [-]

I think that a programmer who cared enough about CEV to be a secret-keeper would also care enough about getting CEV right to kill in order to prevent it from being done wrong. The public need not be involved at all.

Comment author: Baughn 01 January 2013 04:29:48PM 0 points [-]

Agreed, in principle, but I'm not sure that such people would make very good teammates.

(Implying that AGI is more likely to be developed by people who don't care that much.)

Comment author: Decius 01 January 2013 09:03:15PM 1 point [-]

Is a good teammate one who has the social skills to make everybody happy when they are doing something they don't want to, or someone who thinks that the team's task is so important that they will do anything to get it done?

Are major breakthroughs which require a lot of work more likely to be done by people who don't care, or by people that do?

Comment author: Baughn 04 January 2013 12:16:26PM 0 points [-]

That's not my point, which is simply this:

A good teammate is probably not one who's willing to kill you if you make the wrong move, and who -- being human -- may misinterpret your actions.

Comment author: Decius 04 January 2013 09:22:45PM 0 points [-]

If there is no move you could make which would result in your teammate trying to kill you, then you have a different problem.