Eliezer_Yudkowsky comments on Complexity of Value ≠ Complexity of Outcome - Less Wrong

32 Post author: Wei_Dai 30 January 2010 02:50AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (198)

You are viewing a single comment's thread. Show more comments above.

Comment author: Eliezer_Yudkowsky 31 January 2010 05:33:57PM 3 points [-]

I think this is about metamorals not metaethics - yes, I'm merely defining terms here, but I consider "What is moral?" and "What is morality made of?" to be problems that invoke noticeably different issues. We already know, at this point, what morality is made of; it's a computation. Which computation? That's a different sort of question and I don't see a difficulty in having my answer depend on contingent facts I haven't learned.

In response to your question: yes, if I had given a definition of moral progress where it turned out empirically that there was no coherence in the direction in which I was trying to point and the past had been a random walk, then I should reconsider my attempt to describe those changes as "progress".

Comment author: Nick_Tarleton 31 January 2010 09:59:13PM 0 points [-]

Which computation? That's a different sort of question and I don't see a difficulty in having my answer depend on contingent facts I haven't learned.

How do you cash "which computation?" out to logical+physical uncertainty? Do you have in mind some well-defined metamoral computation that would output the answer?

Comment author: Eliezer_Yudkowsky 31 January 2010 10:44:45PM 0 points [-]

I think you just asked me how to write an FAI. So long as I know that it's made out of logical+physical uncertainty, though, I'm not confused in the same way that I was confused in say 1998.

Comment author: Nick_Tarleton 31 January 2010 10:57:54PM *  2 points [-]

"Well-specified" may have been too strong a term, then; I meant to include something like CEV as described in 2004.

Is there an infinite regress of not knowing how to compute morality, or how to compute (how to compute morality), or how to compute (how to compute (...)), that you need to resolve; do you currently think you have some idea of how it bottoms out; or is there a third alternative that I should be seeing?