ciphergoth comments on Only humans can have human values - Less Wrong

34 Post author: PhilGoetz 26 April 2010 06:57PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (159)

You are viewing a single comment's thread. Show more comments above.

Comment author: [deleted] 01 May 2010 07:53:33AM *  10 points [-]

I may be a little slow and missing something, but here are my jumbled thoughts.

I found moral nihilism convincing for a brief time. The argument seems convincing: just about any moral statement you can think of, some people on earth have rejected it. You can't appeal to universal human values... we've tried, and I don't think there's a single one that has stood up to scrutiny as actually being literally universal. You always end up having to say, "Well, those humans are aberrant and evil."

Then I realized that there must be something more complicated going on. Else how explain the fact that I am curious about what is moral? I've changed my mind on moral questions -- pretty damn foundational ones. I've experienced moral ignorance ("I don't know what is right here.") I don't interact with morality as a preference. Or, when I do, sometimes I remember not to, and pull myself back.

I know people who claim to interact with morality as a preference -- only "I want to do this," never "I must do this." I'm skeptical. If you could really have chosen any set of principles ... why did you happen to choose principles that match pretty well with being a person of integrity? Quite a coincidence, that.

It's the curiosity and ignorance that really stumps me. I can be as curious about moral matters, or feel ignorant about moral matters, as about anything else. Why would I be curious, if not to learn how things really are? Is curiosity just another thing I have a preference for?

But It's weird to talk about a preference for curiosity, because I'm not sure that if you say "I want to be curious" that you're actually being curious. Curiosity is "I want to know why the sky is blue." It refers to something. I doubt it's coherent to make a principle of curiosity. (Curiosity is one of the Virtues of Rationality, but it's understood that you aren't curious by force of will, or by deciding to value curiosity. You're curious only if you want to know the answer.)

Comment author: ciphergoth 01 May 2010 08:15:17AM 7 points [-]

This is an excellent question. I think it's curiosity about where reflective equilibrium would take you.

Comment author: PhilGoetz 04 May 2010 02:02:45AM *  1 point [-]

I suspect that, at an evolutionary equilibrium, we wouldn't have the concept of "morality". There would be things we would naturally want to do, and things we would naturally not want to do; but not things that we thought we ought to want to do but didn't.

I don't know if that would apply to reflective equilibrium.

Comment author: Nick_Tarleton 04 May 2010 02:08:24AM *  0 points [-]

I think agents in reflective equilibrium would (almost, but not quite, by definition) not have "morality" in that sense (unsatisfied higher-order desires, though that's definitely not the local common usage of "morality") except in some very rare equilibria with higher-order desires to remain inconsistent. However, they might value humans having to work to satisfy their own higher-order desires.