You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

PhilGoetz comments on Superintelligence 23: Coherent extrapolated volition - Less Wrong Discussion

5 Post author: KatjaGrace 17 February 2015 02:00AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (97)

You are viewing a single comment's thread.

Comment author: PhilGoetz 17 February 2015 05:24:43AM 11 points [-]

One point worth making is that any society would believe they had made moral progress over time, regardless of their history. If you had two societies, and one started at point A and moved to point B, and the other moved from B to A, both would feel they had made moral progress.

Comment author: Luke_A_Somers 17 February 2015 04:20:00PM 8 points [-]

Not necessarily. If A was a Nash equilibrium while B was a Pareto improvement from that but the second society couldn't coordinate to achieve it, then they could gaze wistfully into the past, say they had fallen, and be right to do so.

Comment author: PhilGoetz 20 February 2015 07:22:42PM 2 points [-]

Yes, necessarily, if A and B are sets of moral values, not the degree to which they are attained. You're interpreting A and B as, say, wealth or power distributions.

Comment author: Luke_A_Somers 21 February 2015 03:35:35AM 1 point [-]

Hmmm. Yes. But I don't know that you would actually be able to find examples of A and B in real life.

Comment author: TheAncientGeek 18 February 2015 03:51:18PM *  0 points [-]

Presumably you are assuming that societies judge their values by their values, always coming to the answer "we're good". But societies can do better and worse at realising their values. Movoer, socieites can judge moral values by non moral values, for instance by consistency. (Yudkowsky's habit, aparently copied by Bostrom, of refusing to distingusih moral value from non-moral value, causes problems, inasmuch as making the distinction solves problems).

I am not sure putting your values into practice counts as a moral value.

Comment author: KatjaGrace 20 February 2015 07:43:35PM 1 point [-]

When people talk about moral progress, I think they are rarely talking about better achieving their fixed values.

Comment author: TheAncientGeek 20 February 2015 10:16:30PM 1 point [-]

I think almost everybody makes a distinction between moral and nonmoral value. Stamp collectors value stamps, but don't think societies with a greater supply of stamps are morally better.