itaibn0 comments on Siren worlds and the perils of over-optimised search - Less Wrong

27 Post author: Stuart_Armstrong 07 April 2014 11:00AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (411)

You are viewing a single comment's thread. Show more comments above.

Comment author: itaibn0 23 April 2014 02:31:03PM 0 points [-]

Remember that my original point is that I believe appearing to be good correlates with goodness, even in extreme circumstances. Therefore, I expect restructuring humans to make the world appear tempting will be to the benefit of their happiness/meaningfulness/utility. Now, I'm willing to consider that are aspects of goodness which are usually not apparent to an inspecting human (although this moves to the borderline of where I think 'goodness' is well-defined). However, I don't think these aspects are more likely to be satisfied in a satisficing search than in an optimizing search.