Perhaps a satisfactory answer can be found in "Jewish Philosophy as a Guide to Life: Rosenzweig, Buber, Levinas, Wittgenstein" by Hilary Putnam (who seemed to me to be a reasonable philosopher, but converted to Judaism). I've just started listening to its audiobook version, prompted by this post.
Just came across this interview with Robert Aumann. On pgs. 20-27 he describes why and how he believes in Orthodox Judaism. I don't really understand what he's saying. Key quote (I think):
Anybody have a clue what he means by all this? Do you think this is a valid way of looking at the world and/or religion? If not, how confident are you in your assertion? If you are very confident, on what basis do you think you have greatly out-thought Robert Aumann?
Please read the source (all 7 pages I referenced, rather than just the above quote), and think about it carefully before you answer. Robert Aumann is an absolutely brilliant man, a confirmed Bayesian, author of Aumann's Agreement Theorem, Nobel Prize winner, and founder / head of Hebrew University's Center for the Study of Rationality. Please don't strawman his arguments or simply dismiss them!