Assuming worst-case input is like assuming that the hacker has to do no work to solve any of these problems, and automatically knows the inputs that will screw with your solution most.

And what do you suggest to assume, instead?

Anyway, most proofs of asymptotic security in cryptography are conditional on conjectures such as "P != NP" or "f is a one-way function".