So, in a community where a majority of the people believe in a Christian idea of morality and only co-operate with other people with that Christian ideal of morality (strict sexual rules, go to Church every Sunday, ect), then wouldn't your system say that the majority is behaving morally?

And, conversely, that the minority of people in that society with a more humanist value system, who the majority will not co-operate with because they disagree with their value system, will therefore be defined as less moral under that system?

For that matter, if nobody is willing to co-operate with someone because their face looks ugly by the standards of most of the community, then your system would declare the ugly person to also be immoral.