Dumbledore was a credible threat to him even back when he had his full power. In his weakened state, there may be too much danger that he would end up in a direct confrontation with Dumbledore or some union of strong opponents and lose. McGonagall's reminiscences of having encountered Voldemort before, three times at Dumbledore's side, implies that they've faced each other head on before.

But even if he could manage it as a matter of convenience, he might not be able to as a matter of pride. Dumbledore said that he doesn't think Voldemort would settle for any less than the strongest instantiation of the spell that would return him to power. Even if he has some avenue to victory which would probably work, he may simply be dissatisfied with any plan which does not result in him completely regaining his powers.