I agree with you that "my desire is not do do X, therefore I wouldn't do X even if I knew it was the right thing to do" isn't a valid argument. It's also not what I said. What I said was "my desire is not do do X, therefore I wouldn't choose to desire to do X even if I could choose that." Whether it's right or wrong doesn't enter into it.

As for your scenario... yes, I agree with you that IF "eating babies is wrong" is the sort of thing that can be discovered about the world, THEN an AI could discover it, and THEREFORE is not guaranteed to continue eating babies just because it initially values baby-eating.

It is not clear to me that "eating babies is wrong" is the sort of thing that can be discovered about the world. Can you clarify what sort of information I might find that might cause me to "find out" that eating babies is wrong, if I didn't already believe that?

I don't want to eat babies.
If you gave me a pill that would make me want to eat babies, I would refuse to take that pill, because if I took that pill I'd be more likely to eat babies, and I don't want to eat babies.
That's a special case of a general principle: even if an AI can modify itself and act independently, if it doesn't want to do X, then it won't intentionally change its goals so as to come to want to do X.
So it's not pointless to design an AI with a particular goal, as long as you've built that AI such that it won't accidentally experience goal changes.

Incidentally, if you're really interested in this subject, reading the Sequences may interest you.

Wouldn't it be pointless to try to instill into an AI a friendly goal, as a self-aware improving AI should be able to act independently regardless of however we might write them in the beginning?

If this universe is completely reductionistic, which a simulation probably would be, then your "actual thoughts" (and the existence of trees, etc.) are logical implications of the configuration. Does an entity with logical uncertainty still count as omniscient? But then we've gotten into definitions again.

I still don't know whether you, personally, think a deistic god implies that one or more religions is true. It doesn't particularly matter, though. Your original point that the answer to the god question depends on the answer to the simulation question is a good one.

Took the survey and was quite unsure how to answer the god questions... If we took it, for example, that there's 30% chance of universe being simulated then the same probability should be assigned to P(God) too and to P(one of the religions is correct) as well.