As a non-mathematician, I enjoyed an old blog post from Dick Lipton, Guessing the Truth, which lists a bunch mathematical conjectures incorrectly expected to be true, along with their resolutions. There's also a great comment by Terry Tao on kinds of evidence for mathematical conjectures. For example:

Attempts at disproof run into interesting obstacles. This one is a bit hard to formalise, but sometimes you can get a sense that attempts to disprove a conjecture are failing not due to one’s own lack of ability, or due to accidental contingencies, but rather due to “enemy activity”; some lurking hidden structure to the problem, corners of which emerge every time one tries to build a counterexample. The question is then whether this “enemy” is stupid enough to be outwitted by a sufficiently clever counterexample, or is powerful enough to block all such attempts. Identifying this enemy precisely is usually the key to resolving the conjecture (or transforming the conjecture into a stronger and better conjecture).