We're humans, so we maximize human utility. If squirrels were building AIs, they ought to maximize what's best for squirrels.
There's nothing inherently better about people vs paperclips vs squirrels. But since humans are making the AI, we might as well make it prefer people.
That's one element in what started my line of thought..I was imagining situations where I would consider the exchange of human lives for non-human objects. How many people's lives would be a fair exchange for a pod of bottlenose dolphins? A West Virginia mountaintop? An entire species of snail?
I think what I'm getting towards is there's a difference between human preferences and human preference for other humans. And by human preferences, I mean my own.
I've read through some of the Sequences, but I'm still unclear on a few basic concepts around LW rationality. This is in part to my learning still which benefits from social engagement (ie discussions) rather than just reading. One of those concepts I'm unclear on: Is there an inherent value to human (or sentient) life?
It appears to me that one common theme on this site is that human life (current and future) is very important. Why is that so? Why is the goal people over paper clips?