There's really no downside to letting people use the bathroom more often. It doesn't harm me at all if my neighbor decides to violate the stricture.

If the punishment for murder is removed, or the belief that murder will be punished ceases to be generally retained, then it is entirely likely that my neighbor may wish to murder *me*, and that decision has lots of consequences that concern me greatly.

People who believe that societal indoctrination is necessary to get people to accept certain principles, and that religion is an essential part of that indoctrination, will object to the removal of the threat of god-punishment. Without that threat, they believe societal habit alone won't be enough to keep them safe.

They may be right - when societal controls are relaxed, people act pretty nastily towards each other. Primates are nasty beings.