I suspect that there are some interesting cases in which a proof that P is provable does not constitute a proof, but this is mainly because I've seen mathematicians break similarly intuitive propositions before.

It wouldn't surprise me either. However such cases would have to rely on a precise definition of 'proof' differently to what I use. The result would then be filed under 'fascinating technical example' but not under 'startling revelation' and I would take note of the jargon for use when talking to other mathematicians.