Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

# Nornagest comments on Convergence Theories of Meta-Ethics - Less Wrong

6 07 February 2011 09:53PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Sort By: Best

You are viewing a single comment's thread.

Comment author: 07 February 2011 11:00:40PM *  2 points [-]

As an exercise, prove transitivity. The trick is that the definition of "better than" keeps changing at each step. You can assume that any one rational agent has a transitive "better than' relation, and that there is local agreement between the two agents involved that the new agent's moral code is better than that of his predecessor. But can you prove from this that every agent would agree that the final moral code is better than the original one?

Let's take a half-bounded sequence of moral encodings I = {m(-infinity) .. m(b)}. For each encoding m(x), there's defined a comparative morality function Mx(X, Y) that takes in encodings X and Y, outputting true if Y is judged to be superior to X.

Per your conditions, we know that Mx(m(x-1), m(x+1)) is true at every step (except the final one, which has no m(x+1)). We also know that if Mx(m(x-1), m(x+1)) is true, then so is Mx+1(m(x), m(x+2)). Now, for an arbitrary x, is Mx(m(a), m(b)) true for all a < b?

I might be missing something, but it seems to me that this falls down in the case where I describes a half-bounded slice of a periodic function's output. It's easy to think of Mx that encapsulate notions of local progress but don't deal well with values outside of their own neighborhood.