My argument still holds in another form, though. Even if we assume the universe has a preexisting algorithm that just unfolds, we don't know which it is. So we can't determine the best seed AI from that either, effectively we still have to start from b). Unless we get the best seed AI by accident (which seems unlikely to me) there will be room for a better seed AI which can only determined if we start with a totally new algorithm (which the original seed AI is unable to, since then it would have to delete itself). We, having the benefit of not knowing our algorithm, can built a better seed AI which the old seed AI couldn't built because it already has a known algorithm it must necessarily build on.

Indeed, a good seed AI would at some point suggest us to try another seed AI, because it infers that its original code is unlikely to be the best possible at optimal self-modification. Or it would say "Delete this part of my source code and rewrite it, it doesn't seem optimal to me, but I can't rewrite it because I can't modify this part without destroying myself or basing the modification on the very part that I want to fundamentally rewrite".

I see in which case my argument fails:

If we assume a prexisting algorithm for the universe already (which most people here seem to do), then everything else could be derived from that, including all axioms of natural numbers, since we assume the algorithm to be more powerful then them at the start. Step b) is simply postulated to be already fulfilled, with the algorithm just being there (and "just being there" is not an algorithm), so that we already have an algorithm to start with (the laws of nature).

The "laws of nature" simply have to be taken as granted. There is nothing deeper than that. The problem is that then we face a universe which is essentially abitrary, since from what we know the laws could be anything abitrary else as well (these laws would just be the way they are, too). But this is obviously not true. The laws of nature are not abitrary, there is a deeper order in them which can't stem from any algorithm (since this would just be another abitrary algorithm). But if this is the case, I believe we have no reason to suppose that this order just stopped working and lets do one algorithm all the rest. We would rather expect it to be still continuously active. Then the argument works. The non-algorithmic order can always yield more powerful seed AIs (since there is no single most powerful algorithm, I think we can agree on that), so that AI is not sufficient for an ever increasing general intelligence.

So we face a problem of worldview here, which really is independent of the argument and this is maybe not the right place to argue it (if it is even useful to discuss it, I am not sure about that, either).

I am not necessarily an atheist, it depends on your definition of God. I reject all relgious conceptions of God, but accept God as a name for the mysterious source of all order, intelligence and meaning, or as existence itself.

So in this sense, God is the uncaused cause and also everything caused.

It would indeed be a counterargument if I didn't believe in uncaused cause, but I do believe in an uncaused cause, even though it isn't a seperate entity like the usual notion of God implies.

I agree. That's why is say "this algorithm doesn't solve any problem", it isn't in a problem solving algorithm in the sense I used in my post. Any "just go through all XYZ" doesn't solve my stated problem, because it doesn't select the actual useful solution.