Some points:
We tried to model a complex phenomenon using a single scalar, and this resulted in confusion and clouded intuition.
It's sort of useful for humans because of restriction of range, along with a lot of correlation that comes from looking only at human brain operations when talking about 'g' or IQ or whatever.
Trying to think in terms of a scalar 'intelligence' measure when dealing with non-human intelligences is not going to be very productive.
It's conceivable that current level of belief in homeopathy is net positive in impact. The idea here would be that the vast majority of people who use it will follow up with actual medical treatment if homeopathy doesn't solve their problem.
Assume also that medical treatment has non-trivial risks compared to taking sugar pills and infinitely dilute solutions (stats on deaths due to medical error support this thesis). And further that some conditions just get better by themselves. Now you have a situation where, just maybe, doing an initial 'treatment' with...
It's not clear to me how you get to deceptive alignment 'that completely supersedes the explicit alignment'. That an AI would develop epiphenomenal goals and alignments, not understood by its creators, that it perceived as useful or necessary to pursue whatever primary goal it had been set, seems very likely. But while they might be in conflict with what we want it to do, I don't see how this emergent behavior could be such that it would be contradict the pursuit of satisfying whatever evaluation function the AI had been trained for in the beginning. Unles...
I am not so sure about that. I am thinking back to the Minnesota Twin Study here, and the related fact that heritability of IQ increases with age (up until age 20, at least). Now, it might be that we're just not great at measuring childhood IQ, or that childhood IQ and adult IQ are two subtly different things.
But it certainly looks as if there's factors related to adult brain plasticity, motivation (curiosity, love of reading, something) that continue to affect IQ development at least until the age of 18.