The best justification I've heard for believing simple hypotheses is an argument from probability.
Consider some event caused by a certain block. We know the block's color must be either red, yellow, blue, or green; its shape must be either square, round, or triangular; its material must be either wood or metal.
We come up with two theories about the event. Both theories explain the event adequately:
The event was caused by the block being made of wood. The event was caused by the block being blue, and triangular, and and made of metal. Before the event happens, there are twenty four different possibile configurations of the block. "Made of wood" is true of twelve configurations, "blue, triangular, and made of metal" is true of one configuration.
After the event, we dismiss all configurations except these thirteen under which we believe the event was possible. We assume all of these thirteen are equally likely. Therefore, there's a 12/13 chance that the block is made of wood and a 1/13 chance the block is blue, triangular, and made of metal.
Therefore, Theory 1 is twelve times more likely than Theory 2.
The same principle is at work any time you have a simple theory competing with a more complex theory. Because the complicated theory has more preconditions that have to be just right, it has a lower prior probability relative to the simple theory, and since the occurence of the event adjusts the probabilities of both theories equally, it has a lower posterior probability.
I know I read this explanation first on a discussion of Kolmogorov complexity on someone's rationality blog, but I can't remember who's or what the link was. If I stole your explanation, please step up and take credit.
It also helps to keep in mind that the state space with associated probability distribution is something you dress the actual state of reality in. The model helps to keep track of the structure of the data you have about the actual state of reality, that hides in one tiny point of state space. Probabilities of areas of state space (events/hypotheses) quantitatively express the relation between those aspects of the model and reality it's about.
I recently spoke with a person who... it's difficult to describe. Nominally, she was an Orthodox Jew. She was also highly intelligent, conversant with some of the archaeological evidence against her religion, and the shallow standard arguments against religion that religious people know about. For example, she knew that Mordecai, Esther, Haman, and Vashti were not in the Persian historical records, but that there was a corresponding old Persian legend about the Babylonian gods Marduk and Ishtar, and the rival Elamite gods Humman and Vashti. She knows this, and she still celebrates Purim. One of those highly intelligent religious people who stew in their own contradictions for years, elaborating and tweaking, until their minds look like the inside of an M. C. Escher painting.
Most people like this will pretend that they are much too wise to talk to atheists, but she was willing to talk with me for a few hours.
As a result, I now understand at least one more thing about self-deception that I didn't explicitly understand before—namely, that you don't have to really deceive yourself so long as you believe you've deceived yourself. Call it "belief in self-deception".
When this woman was in high school, she thought she was an atheist. But she decided, at that time, that she should act as if she believed in God. And then—she told me earnestly—over time, she came to really believe in God.
So far as I can tell, she is completely wrong about that. Always throughout our conversation, she said, over and over, "I believe in God", never once, "There is a God." When I asked her why she was religious, she never once talked about the consequences of God existing, only about the consequences of believing in God. Never, "God will help me", always, "my belief in God helps me". When I put to her, "Someone who just wanted the truth and looked at our universe would not even invent God as a hypothesis," she agreed outright.
She hasn't actually deceived herself into believing that God exists or that the Jewish religion is true. Not even close, so far as I can tell.
On the other hand, I think she really does believe she has deceived herself.
So although she does not receive any benefit of believing in God—because she doesn't—she honestly believes she has deceived herself into believing in God, and so she honestly expects to receive the benefits that she associates with deceiving oneself into believing in God; and that, I suppose, ought to produce much the same placebo effect as actually believing in God.
And this may explain why she was motivated to earnestly defend the statement that she believed in God from my skeptical questioning, while never saying "Oh, and by the way, God actually does exist" or even seeming the slightest bit interested in the proposition.