I think g is sort of a mathematical artifact, not a real thing (but don't really feel like getting into a big thing about this). Factor analysis doesn't tell people what they think it does.
The first principal component of scores on various tests is, of course, a mathematical artifact. But it's not the real thing, it's just an estimate, a finger pointing at the real thing.
I agree that people can be both stupid and smart in very different ways, but at a certain -- and useful! -- level of aggregation, there are generally smart people and generally stupid people. There is a lot of variation around that axis, but I think the axis exists. I'm not arguing that everything should be projected into that one-dimensional space and reduced to a scalar.