AI is thinking more straight than you. It doesn't need to believe in God. It can assign some uncertainty. The only good argument for non-existence of God is Occam's razor, and Occam's razor doesn't say the complex explanation is impossible, merely unlikely (and shouldn't be privileged).
Imagine the AI that knows precisely how much more complex are the rules that lead to universe full of Gods who create sims of our universe, than rules for our universe. This AI will tell you exactly how likely it is that it is sitting in a simulation, instead of making some hacks for the pascal's wager. This AI will have atheist predictor with weight p and theist predictor with weight 1-p . The theist predictor likes status quo. The theist predictor, however, won't necessarily respond to claims "you must give me all your money or else God will punish you", because it has no evidence towards this statement about God over the converse "you must not give me any of your money or else God will punish you".
What's the point of speculating about what something literally defined as having more knowledge than us would believe? All I know is that in our current state of knowledge, there's no more evidence for the existence of a God prepared to punish you with an afterlife in hell if you murder somebody than one who would do the same if you play disc golf.
I certainly don't think I have an argument for the non-existence of God, nor am I looking for one. Try disproving the idea that having a lifetime average of chewing on your left side between 60-70% will leave you...
Are there any essays anywhere that go in depth about scenarios where AIs become somewhat recursive/general in that they can write functioning code to solve diverse problems, but the AI reflection problem remains unsolved and thus limits the depth of recursion attainable by the AIs? Let's provisionally call such general but reflection-limited AIs semi-general AIs, or SGAIs. SGAIs might be of roughly smart-animal-level intelligence, e.g. have rudimentary communication/negotiation abilities and some level of ability to formulate narrowish plans of the sort that don't leave them susceptible to Pascalian self-destruction or wireheading or the like.
At first blush, this scenario strikes me as Bad; AIs could take over all computers connected to the internet, totally messing stuff up as their goals/subgoals mutate and adapt to circumvent wireheading selection pressures, without being able to reach general intelligence. AIs might or might not cooperate with humans in such a scenario. I imagine any detailed existing literature on this subject would focus on computer security and intelligent computer "viruses"; does such literature exist, anywhere?
I have various questions about this scenario, including: