You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

paulfchristiano comments on Why Do We Engage in Moral Simplification? - Less Wrong Discussion

24 Post author: Wei_Dai 14 February 2011 01:16AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (35)

You are viewing a single comment's thread. Show more comments above.

Comment author: paulfchristiano 14 February 2011 04:35:01PM 1 point [-]

I believe that abstract moral intuitions do have motive power.

For example, I have never been a very good utilitarian because I am selfish, lazy,etc. However, if one year ago you had offered me the option of becoming a very good preference utilitarian, in an abstract context where my reflexes didn't kick in, I would have accepted it. If you had given me the option to implement an AI which was a preference utilitarian (in some as-yet-never-made-sufficiently-concrete sense which seemed reasonable to me) I would have taken it.

I also am not sure what particular extreme behavior preference utilitarianism endorses when followed to its logical conclusion. I'm not aware of any extreme consequences which I hadn't accepted (of course, I was protected from unwanted extreme consequences by the wiggle room of a grossly under-determined theory).