Your ability to judge both yourself, and another person, and how your personalities will interact, is limited. It's sufficient to identify people with whom you absolutely will not get along, with reasonable accuracy; this is low-hanging fruit. So let's say you've eliminated 95% of the candidate pool by this point.

The remaining 5%? You're now considering a pool of candidate partners who you can't immediately eliminate (assuming you have more than one person remaining, after all probably-unsuitable people are eliminated). At this point your list of candidates are people about whom you are uncertain. -Remember- that you're uncertain.

Or, from a different angle: If you are absolutely certain that a relationship with somebody will work out, that sense of certainty should, due to the Dunning-Kruger effect, be taken as evidence that you should be less certain.