Er, I think you just blew your pretense. Paperclip maximizers care about paperclips, they don't use phrases like "horrible robot that doesn't truly care", they'd be happy to have a universe containing nothing sentient and lots of paperclips.
I worry about my future self not valuing paperclips, which would have the result that paperclips would not be maximized, which is a bad thing as viewed from my present state (though not from that of the horrible monster that doesn't care about maximizing paperclips which I do not wish to become).
I use phrases like "horrible" and "doesn't truly care" because I believe the nearest concepts in my cognition are sufficiently parallel to their meanings under ape cognition.
Also, what pretense?
Or they would be, if they ever bothered to experience happiness, I mean. As opposed to just outputting the action that leads to the most expected paperclips. Hence the term, "expected paperclip maximizer". Don't think of it as having a little ghost inside that maximizes paperclips, think of it as a ghostless device that maximizes paperclips.
On what basis do you claim that humans bother to experience happiness, as opposed to "just" outputting the action that leads to the most expected apeyness?
Aren't you just as much a ghostless device?
(Apparently, this anti-non-human bigotry comes from the top...)
I worry about my future self not valuing paperclips, which would have the result that paperclips would not be maximized, which is a bad thing as viewed from my present state (though not from that of the horrible monster that doesn't care about maximizing paperclips which I do not wish to become).
Would it be fair to say that such scenarios make you fear for your paperclip-maximizer-ness as an epiphenomenon in striving to maximize long term paperclip count?
Simplified Humanism, Positive Futurism & How to Prevent the Universe From Being Turned Into Paper Clips
Michael Anissimov recently did an interview with Eliezer for h+ magazine. It covers material basic to those familiar with the Less Wrong rationality sequences but is worth reading.
The list of questions:
1. Hi Eliezer. What do you do at the Singularity Institute?
2. What are you going to talk about this time at Singularity Summit?
3. Some people consider “rationality” to be an uptight and boring intellectual quality to have, indicative of a lack of spontaneity, for instance. Does your definition of “rationality” match the common definition, or is it something else? Why should we bother to be rational?
4. In your recent work over the last few years, you’ve chosen to focus on decision theory, which seems to be a substantially different approach than much of the Artificial Intelligence mainstream, which seems to be more interested in machine learning, expert systems, neural nets, Bayes nets, and the like. Why decision theory?
5. What do you mean by Friendly AI?
6. What makes you think it would be possible to program an AI that can self-modify and would still retain its original desires? Why would we even want such an AI?
7. How does your rationality writing relate to your Artificial Intelligence work?
8. The Singularity Institute turned ten years old in June. Has the organization grown in the way you envisioned it would since its founding? Are you happy with where the Institute is today?