(Tangent: In some cases it's possible to find a third party that understands a participant's intuitions and is willing to explain them to participants with opposing intuitions in language that everyone, including bystanders, can understand. E.g. I think that if I was involved as a translator in the Muehlhauser--Goertzel debate then at least some of Ben's intuitions would have been made clearer to Luke. Because Luke didn't quite understand Ben's position this also led some LW bystanders to get a mistaken impression of what Ben's position was, e.g. a highly upvoted comment suggested that Ben thought that arbitrary AGIs would end up human-friendly, which is not Ben's position. A prerequisite for figuring out what to do in the case of disagreeing intuitions is to figure out what the participants' intuitions actually are. I don't think people are able to do this reliably. (I also think it's a weakness of Eliezer's style of rationality in particular, and so LessWrong contributors especially might want to be wary of fighting straw men. I don't mean to attack LessWrong or anyone with this comment, just to share my impression about one of LessWrong's (IMO more glaring and important) weaknesses.))