And if I played Kasparov I might win. It's not a 100% guaranteed prediction.

The fallacy of gray? Between zero chance of winning a lottery, and epsilon chance, there is an order-of-epsilon difference. If you doubt this, let epsilon equal one over googolplex.

Considering that Deutsch was talking about new knowledge, and I use the same terminology as him, it is relevant.

Then the define the term.

I know that? And if I played Kasparov I might win. It's not a 100% guaranteed prediction.

So what? How is that at all relevant. It isn't 100% guaranteed that if I jump off a tall building that I will then die. That doesn't mean I'm going to try. You can't use the fact that something isn't definite as an argument to ignore the issue wholesale.

Deutsch doesn't think AGI's will do fast recursive self-improvement. They can't because the first ones will already be universal and there's nothing much left to improve, besides their knowledge (not their design, besides making it faster).

Ok. So I'm someone who finds extreme recursive self-improvement to be unlikely and I find this to be a really unhelpful argument. Improvements in speed matter. A lot. Imagine for example, that our AI finds a proofs that P=NP and that this proof gives a O(n^2) algorithm for solving your favorite NP-complete problem, and that the constant in the O is really small. That means that the AI will do pretty much everything faster, and the more computing power it gets the more disparity there will be between it and the entities that don't have access to this algorithm. It wants to engineer a new virus? Oh what luck, protein folding is under many models NP-compete. The AI decides to improve its memory design? Well, that involves graph coloring and the traveling salesman, also NP-complete problems. The AI decides that it really wants access to all the world's servers and add them to its computational power? Well most of those have remote access capability that is based on cryptographic problems which are much weaker than NP-complete. So, um, yeah. It got those too.

Now, this scenario seems potentially far-fetched. After all, most experts consider it to be unlikely that P=NP, and consider it to be extremely unlikely that there's any sort of fast algorithm for NP complete problems. So let's just assume instead that the AI tries to make itself a lot faster. Well, let's see, what can our AI do. It could give itself some nice quantum computing hardware and then use Shor's algorithm to break factoring in polynomial time and then all AI can just take over lots of computers and have fun that way.

Improving knowledge with intelligence is the same process for AGI and humans. It won't magically get super fast

This is not at all obvious. Humans can't easily self-modify our hardware. We have no conscious access to most of our computational capability, and our computational capability is very weak. We're pathetic sacks of meat that can't even multiply four or five digits numbers in our heads. We also can't save states and swap out cognitive modules. An AGI can potentially do all of that.

Don't underestimate the dangers of a recursively self-improving entity or the value of speed.