Vladimir_M comments on What is moral foundation theory good for? - Less Wrong

9 Post author: novalis 12 August 2012 05:03AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (296)

You are viewing a single comment's thread. Show more comments above.

Comment author: Lightwave 12 August 2012 08:41:49AM 7 points [-]

I would actually go further and say that they are necessary for any sort of organized human society.

While they are likely necessary for organized human society, I think the argument is that their purpose is purely instrumental. It's sort of like how in the prisoner's dilemma, the concept of 'trust' ('tit for tat with forgiveness' variants) is an instrumentally useful strategy for winning points in a group of a certain kind of agents. Even if humans have loyalty, authority and sanctity built-in, they can still recognize their instrumental role and can only instrumentally optimize for those.

Comment author: Vladimir_M 12 August 2012 07:18:50PM 5 points [-]

Even if humans have loyalty, authority and sanctity built-in, they can still recognize their instrumental role and can only instrumentally optimize for those.

The trouble is, absent certain unusually favorable circumstances, attempts at such optimization run into insurmountable practical problems. For start, such analysis would be tremendously difficult even for a superhumanly unbiased intellect. And then there is the even worse problem that realistic humans will be under an almost irresistible temptation to bias their analysis in favor of their own particular authority, sanctity, and in-group norms.