I'm thinking that the paperclipper counts as a life not worth living - an AI that wants to obsess about paperclips is about as repugnant to me as a cow that wants to be eaten. Which is to say, better than doing either of those without wanting it, but still pretty bad. Yes, I'm likely to have problems with a lot of genuinely friendly AIs.
I was assuming that both scenarios were for keeps. Certainly the paperclipper should be smart enough to ensure that; for the other, I guess I'll assume you're actually destroying the universe somehow.
It is a fair point but do you mean that the paperclipper is wrong in its judgement that its life is worth living, or is it merely your judgement that if you were the paperclipper your life would not be worth living by your current standards? Remember that we assume that there is no other life possible in the universe anyway -- this assumption makes things more interesting.
Thought experiment:
Through whatever accident of history underlies these philosophical dilemmas, you are faced with a choice between two, and only two, mutually exclusive options:
* Choose A, and all life and sapience in the solar system (and presumably the universe), save for a sapient paperclipping AI, dies.
* Choose B, and all life and sapience in the solar system, including the paperclipping AI, dies.
Phrased another way: does the existence of any intelligence at all, even a paperclipper, have even the smallest amount of utility above no intelligence at all?
If anyone responds positively, subsequent questions would be which would be preferred, a paperclipper or a single bacteria; a paperclipper or a self-sustaining population of trilobites and their supporting ecology; a paperclipper or a self-sustaining population of australopithecines; and so forth, until the equivalent value is determined.