ciphergoth comments on Only humans can have human values - Less Wrong

34 Post author: PhilGoetz 26 April 2010 06:57PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (159)

You are viewing a single comment's thread. Show more comments above.

Comment author: ciphergoth 01 May 2010 08:15:17AM 7 points [-]

This is an excellent question. I think it's curiosity about where reflective equilibrium would take you.

Comment author: PhilGoetz 04 May 2010 02:02:45AM *  1 point [-]

I suspect that, at an evolutionary equilibrium, we wouldn't have the concept of "morality". There would be things we would naturally want to do, and things we would naturally not want to do; but not things that we thought we ought to want to do but didn't.

I don't know if that would apply to reflective equilibrium.

Comment author: Nick_Tarleton 04 May 2010 02:08:24AM *  0 points [-]

I think agents in reflective equilibrium would (almost, but not quite, by definition) not have "morality" in that sense (unsatisfied higher-order desires, though that's definitely not the local common usage of "morality") except in some very rare equilibria with higher-order desires to remain inconsistent. However, they might value humans having to work to satisfy their own higher-order desires.