From the "actually" in the second, it seems as if you're endorsing that one, in which case presumably "a very good mathematical explanation" is intended as a criticism. Which doesn't make any sense to me.
I was implying that it is a rationalization. Perhaps a fair one - I have no ready counterargument available - but not the real reason for the behavior.
It's not necessarily that concrete better explanations occur to us. It's that we have a heuristic that tells us there should be one.
Yes! Exactly. And moreover - that heuristic is, as you say, useful. What is the heuristic measuring, and why?
Skipping ahead a bit: The ability to notice which improbable things require explanations is, perhaps, the heart of scientific progress (think of data mining - why can't we just run a data mining rig and discover the fundamental equations of the universe? I'd bet all the necessary data already exists to improve our understanding of reality by as much again as the difference between Newtonian and Relativistic understandings of reality). Why does it work, and how can we make it work better?
Alice: "I just flipped a coin [large number] times. Here's the sequence I got:
(Alice presents her sequence.)
Bob: No, you didn't. The probability of having gotten that particular sequence is 1/2^[large number]. Which is basically impossible. I don't believe you.
Alice: But I had to get some sequence or other. You'd make the same claim regardless of what sequence I showed you.
Bob: True. But am I really supposed to believe you that a 1/2^[large number] event happened, just because you tell me it did, or because you showed me a video of it happening, or even if I watched it happen with my own eyes? My observations are always fallible, and if you make an event improbable enough, why shouldn't I be skeptical even if I think I observed it?
Alice: Someone usually wins the lottery. Should the person who finds out that their ticket had the winning numbers believe the opposite, because winning is so improbable?
Bob: What's the difference between finding out you've won the lottery and finding out that your neighbor is a 500 year old vampire, or that your house is haunted by real ghosts? All of these events are extremely improbable given what we know of the world.
Alice: There's improbable, and then there's impossible. 500 year old vampires and ghosts don't exist.
Bob: As far as you know. And I bet more people claim to have seen ghosts than have won more than 100 million dollars in the lottery.
Alice: I still think there's something wrong with your reasoning here.