Exactly. It makes no difference how powerful quantum computers are for Deutsch's argument. If we had waited exponential time for a classical computer to do it, we would not wonder "where the number was factored." Waiting only polynomial time for it to be factored then begs the question.
I am well aware of the extremely interesting complexity limitations of quantum computing. It definitely only extends computational capability a little bit -- and we still can't even prove that P does not equal NP. But none of this is relevant to Deutsch's "where was the number factored" argument. He is saying that if quantum states are physically real and not just a calculational tool, then you have to give a physical account of how Shor's algorithm works and the orthodox views of wavefunction collapse could not do that.
From a recent paper that is getting non-trivial attention...
From my understanding, the result works by showing how, if a quantum state is determined only statistically by some true physical state of the universe, then it is possible for us to construct clever quantum measurements that put statistical probability on outcomes for which there is literally zero quantum amplitude, which is a contradiction of Born's rule. The assumptions required are very mild, and if this is confirmed in experiment it would give a lot of justification for a phyicalist / realist interpretation of the Many Worlds point of view.
More from the paper:
On a related note, in one of David Deutsch's original arguments for why Many Worlds was straightforwardly obvious from quantum theory, he mentions Shor's quantum factoring algorithm. Essentially he asks any opponent of Many Worlds to give a real account, not just a parochial calculational account, of why the algorithm works when it is using exponentially more resources than could possibly be classically available to it. The way he put it was: "where was the number factored?"
I was never convinced that regular quantum computation could really be used to convince someone of Many Worlds who did not already believe it, except possibly for bounded-error quantum computation where one must accept the fact that there are different worlds to find one's self in after the computation, namely some of the worlds where the computation had an error due to the algorithm itself (or else one must explain the measurement problem in some different way as per usual). But I think that in light of the paper mentioned above, Deutsch's "where was the number factored" argument may deserve more credence.
Added: Scott Aaronson discusses the paper here (the comments are also interesting).