I'm tempted to choose B just because if I choose A someone will try to use the Axiom of Transitivity to "prove" that I value some very large amount of paperclippers more than some small amount of humans. And I don't.

I might also choose B because the paperclipper might destroy various beautiful nonliving parts of the universe. I'm not sure if I really value beautiful rock formations and such, even if there is no one to view them. I tend to agree that something requires both an objective and subjective component to be truly valuable.

On the other hand, maybe the value for beautiful things I will never see is some sort of "between the margins" value, something that I value, but that my values regarding eudaemonic life are lexically prior to. All other things being equal, I'd prefer a universe with even a tiny amount of eudaemonic life (that isn't suffering or anything like that) to a totally lifeless universe chock-full of unobserved beautiful stuff. But maybe a lifeless pretty universe is more valuable to me than a lifeless ugly universe, all other things being equal.