So ardent communists should have the right to live in a communist society, libertarians should have the right to live in a lightly governed society, devout Muslims should have the right to live in a society governed by Sharia law, devout Christians should have the right to live in a society where adultery is severely punished, etc etc.

That not exactly the same thing. Saying Muslims have the right is not the same thing as saying every Muslim has the right. There a difference between saying a group has a right and saying that every individual who's a member of the group has the right.

It's commonly accepted that every people has a right to self determination.

If I'm the only person who thinks that values having a law that requires that every person in my society can only wear red clothes, nobody else has a responsibility to help me to live in such a society.