Even when most peoples beliefs are junk you won't know before you considered the belief in detail. You probably just increase the effect of confirmation bias when you reject beliefs without examining them.

Of course, there is an opportunity cost associated with exploring any given belief. The prior probability of the belief and the potential benefits and costs associated with the topic determine whether or not it is worth investigating further. It is not confirmation bias to ignore ideas that have a low expected value of investigation. You simply leave your level of confidence unchanged.

Thinking outside your own set of beliefs is also good training.

That is one factor consider. Another is "thinking outside your own set of beliefs can be fun".