Why do you have a distinction between true intelligence and not true intelligence?
I have a distinction between what I'd be willing to call intelligence and what I'd say may look like intelligence but really isn't.
For example, IBM's Watson playing Jeopardy or any of the contemporary chess-playing programs do look like intelligence. But I'm not willing to call them intelligent.
My use of intelligence is defined (roughly) as cross domain optimization.
Ah. No, in this context I'm talking about intelligence as a threshold phenomenon, notably as something that we generally agree humans have (well, some humans have :-D) and the rest of things around us do not. I realize it's a very species-ist approach.
I don't think I can concisely formulate the characteristics of it (that will probably take a book or two), but the notion of adaptability, specifically, the ability to deal with new information and new environment, is very important to it.
Hm. If this idea of intelligence seems valuable to you and worth pursuing, I absolutely implore that you wade through the reductionism sequence while or before you develop it more fully. I think it'd be an excellent resource for figuring out exactly what you mean to mean. (and the very similar Human's guide to words)
A stub on a point that's come up recently.
If I owned a paperclip factory, and casually told my foreman to improve efficiency while I'm away, and he planned a takeover of the country, aiming to devote its entire economy to paperclip manufacturing (apart from the armament factories he needed to invade neighbouring countries and steal their iron mines)... then I'd conclude that my foreman was an idiot (or being wilfully idiotic). He obviously had no idea what I meant. And if he misunderstood me so egregiously, he's certainly not a threat: he's unlikely to reason his way out of a paper bag, let alone to any position of power.
If I owned a paperclip factory, and casually programmed my superintelligent AI to improve efficiency while I'm away, and it planned a takeover of the country... then I can't conclude that the AI is an idiot. It is following its programming. Unlike a human that behaved the same way, it probably knows exactly what I meant to program in. It just doesn't care: it follows its programming, not its knowledge about what its programming is "meant" to be (unless we've successfully programmed in "do what I mean", which is basically the whole of the challenge). We can't therefore conclude that it's incompetent, unable to understand human reasoning, or likely to fail.
We can't reason by analogy with humans. When AIs behave like idiot savants with respect to their motivations, we can't deduce that they're idiots.