"There is only one problem that we really care about. Optimization." That may be what you care about, but it is not what I care about, and it was not what I was talking about, which is intelligence. You cannot argue that we only care about optimization, and therefore intelligence is optimization, since by that argument dogs and cats are optimization, and blue and green are optimization, and everything is optimization, since otherwise we would be "debating definitions, which is not productive". But that is obvious nonsense.

In any case, it is plain that most of the human ability to accomplish things comes from the use of language, as is evident by the lack of accomplishment by normal human beings when they are not taught language. That is why I said that knowing language is in fact a sufficient test of intelligence. That is also why when AI is actually programmed, people will do it by trying to get something to understand language, and that will in fact result in the kind of AI that I was talking about, namely one that aims at vague goals that can change from day to day, not at paperclips. And this has nothing to do with any "homunculus." Rocks don't have any special goal like paperclips when they fall, or when they hit things, or when they bounce off. They just do what they do, and that's that. The same is true of human beings, and sometimes that means trying to have kids, and sometimes it means trying to help people, and sometimes it means trying to have a nice day. That is seeking different goals at different times, just as a rock does different things depending on its current situation. AIs will be the same.