gimpf comments on Open Thread June 2010, Part 2 - Less Wrong

7 Post author: komponisto 07 June 2010 08:37AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (534)

You are viewing a single comment's thread.

Comment deleted 07 June 2010 12:37:39PM *  [-]
Comment author: ata 07 June 2010 09:28:53PM 1 point [-]

I think the expectation is that, if all humans had the same knowledge and were better at thinking (and were more the people we'd like to be, etc.), then there would be a much higher degree of coherence than we might expect, but not necessarily that everyone would ultimately have the same utility function.

Comment author: Vladimir_Nesov 11 June 2010 09:45:55PM 0 points [-]

Or is it equally plausible to assume that some important values cannot be extrapolated coherently, and that a Seed-AI would therefore provide several results clustered around some groups of people?

There is only one world to build something from. "Several results" is never a solution to the problem of what to actually do.

Comment author: Gavin 07 June 2010 09:26:14PM 0 points [-]

I think the idea is that CEV lets us "grow up more together" and figure that out later.

I have only recently started looking into CEV so I'm not sure whether I a) think it's a workable theory and b)think it's a good solution, but I like the way it puts off important questions.

It's impossible to predict what we will want if age, disease, violence, and poverty become irrelevant (or at least optional).