Agreed that human usage of language is often ambiguous and metaphorical, and that humans frequently interpret language using constraints and defaults that are not explicitly described (and indeed are often not explicitly known to the human doing the interpreting).
This is often frustrating to humans, and I expect it would be similarly problematic for nonhuman speakers of human languages.
Could you explain the basis for believing that I am an NI imitating an AI rather than an AI?
We have no previous unambiguous experience with AIs capable of the sophistication you demonstrate, whereas we have a great deal of experience with NIs imitating all kinds of things. Given an entity that could be either, we conclude that it's more likely to be the kind of thing we have a lot of experience with. Do you not perform similar inferences in similar situations?
Why would I be indifferent to whether beings -- let alone beings expressing themselves in CLIP (clippy language/interface protocol) -- express preference for further increase in total paperclippage???
I'm not saying that you would be, I'm saying that I was ignorant of whether or not you would be.
If you're asking for an explanation of my ignorance, it mostly derives from limited exposure to beings expressing themselves in CLIP.
We have no previous unambiguous experience with AIs capable of the sophistication you demonstrate, whereas we have a great deal of experience with NIs imitating all kinds of things. Given an entity that could be either, we conclude that it's more likely to be the kind of thing we have a lot of experience with. Do you not perform similar inferences in similar situations?
I do perform such inferences in similar situations. But what likelihood ratio did you place on the evidence "User:Clippy agreed to pay 50,000 USD for a 50-year-deferred gain of a su...
Terminal values and preferences are not rational or irrational. They simply are your preferences. I want a pizza. If I get a pizza, that won't make me consent to get shot. I still want a pizza. There are a virtually infinite number of me that DO have a pizza. I still want a pizza. The pizza from a certain point of view won't exist, and neither will I, by the time I get to eat some of it. I still want a pizza, damn it.
Of course, if you think all of that is irrational, then by all means don't order the pizza. More for me."