I think you're failing to distinguish between authority one voluntarily submits to (potentially even reserving the right to reverse the decision), e.g., meetup organizer, and authority backed by a monopoly on violence, i.e., the modern conception of government.