But I was more thinking about one bias appearing stronger when a bias in another direction is eliminated. For example bias X makes people think A, bias Y makes people think B, if a person is under influence of both biases, the answer is randomly A or B. In such case, eliminating bias X leads to increase of answer B.

Biases don't have clear 'directions' often. If you are overconfident on a claim P, that's just as accurate as saying you were underconfident on claim ~P. Similarly for anchoring or priming - if you anchor on the random number generator while estimating number of African nations, whether you look "over" or "under" is going to depend on whether the RNG was spitting out 1-50 or 100-200, perhaps.

I would like to be at least 95% certain

And what does that mean? If you just want to know 'what do smart people in general believe versus normal people', you don't need large samples if you can get a random selection and your questions are each independent. For example, in my recent Wikipedia experiment I removed only 100 links and 3 were reverted; when I put that into a calculator for a Bernouilli distribution, I get 99% certainty that the true reversion rate is 0-7%. So to simplify considerably, if you sampled 100 smart people and 100 dumb people and they differ by 14%, is that enough certainty for you?