In my model of human behavior, there is an unobservable parameter I associate with intelligence. I observe people's behavior when playing a game, when solving a problem, when defending their beliefs, or when learning something new, and I infer something about their intelligence. This in turn informs my predictions about their success in a wide variety of pursuits. In practice, I often make strong predictions about someone's intelligence after observing their behavior on a single occasion.
An accurate conception of intelligence seems to be generally important. Understanding what easily leveraged factors affect someone's intelligence---during childhood, later education, and after formal education is complete---is important if your goal is improving intelligence generally. If you are considering relatively expensive personal engagement to develop rationality, you may want to direct efforts at individuals who have the potential to have a significant impact as researchers or entrepreneurs. And so on.
Before thinking about how to understand determiners of intelligence, how to measure intelligence effectively, or the effects of intelligence on behavior, I would first like to get a feel for what my intuitive understanding of intelligence really corresponds to, if anything. It is possible that my intuitive assessments of intelligence are largely unrelated to reality, and that my beliefs about the world could be improved by discarding them. It is also possible that some of my intuitions about intelligence are quite accurate, and I could make better decisions by giving them more credence or by changing the way I use those intuitive judgments.
Intuitively, I expect the results of many types of otherwise apparently unrelated tests to be very tightly correlated with intelligence. To understand the extent to which this intuition is correct, I am considering conducting a slightly systematic study of the relationship between different metrics. I would appreciate pointers to reliable scholarship surrounding this question, but a brief search turned up mostly very muddled thinking and a general lack of people doing good experiments.
Here is a range of metrics which I suspect correlate well with my conception of intelligence, at least in certain regimes (some of these metrics may only correlate meaningfully when applied to very bright subjects, or may not correlate meaningfully when applied to very bright subjects):
1. General intelligence factor as estimated by standardized cognitive tests, e.g. Raven's Progressive Matrices.
2. Ability to quickly learn an unfamiliar formalism. For example, to quickly learn a new game and to understand simple strategic consequences of its rules.
3. Ability to infer an underlying model. For example, to learn how to achieve a goal when allowed some constrained interaction with / observation of an unknown environment.
4. My assessment of intelligence during collaboration or discussion of a complex but rigorously defined topic; or, the assessment of anyone who I consider to be intelligent.
5. Ability to solve hard problems in a well-understood environment, potentially given hours or days. For example, performance in high school olympiads.
My hope is that by gaining a better understanding of the relationship between these metrics I may learn to what extent my current rather monolithic conception of intelligence is valid and, to whatever extent it is, how to effectively measure it. Ultimately I would like to understand what easy measurements are the best indicators of success at various particular pursuits, but is even more extraordinarily difficult to acquire data about how good someone is at, say, choosing good research problems.
What do readers expect the results of inquiry to look like? Is my choice of metrics influenced unduly by my own experience? What are other metrics I should be considering but am not? Is improving a student's ability to perform any of these tasks likely to have a positive influence on other tasks?
Oh yes, one can certainly train oneself to think more efficiently/effectively/creatively/etc. But this is not the same as improving intelligence. Think of it as using better software, instead of improving the hardware. But you can certainly think of this as improving intelligence, if you will, but then do realize that what you are doing is training a few key cognitive processes that happen to be useful in many domains. Which is to say, you won't automatically be better at other mental tasks that don't happen to require such cognitive processes.
Theories other than intelligence-as-synaptic-plasticity also don't seem to allow improvement via training. This is because most of them hypothesize intelligence has something to do with the hardware of the brain. This is because the more diverse tests one aggregates, the more correlated the aggregated measure is with g. This together with the fact that tasks with high environmental variance have higher correlation with g, suggests that what aggregation does is cancel out environmental factors. This in turn strongly suggests that our notion of intelligence, or impression of someone's intelligence, depends on a person's overall mental ability over a wide range of tasks. We wouldn't be impressed with a person who could multiply ten digit numbers if she does not also excel at a wide range of other mental tasks.
This is not to say that to be more intelligent, one has be better at everything. Because then why care for intelligence? One shouldn't be too impressed with intelligence, because the whole point is to accomplish specific intellectual tasks no? Hence my suggestion in the first paragraph to identify cognitive processes influential in the performance of the intellectual tasks you care about.
Note that intelligence is a fitness indicator. We know this from psychological studies of sexual attraction and intelligence, from the fact that g has high genetic variance, from the fact that we haven't found any genes which influences intelligence significantly. It is too easy to be impressed by intelligence and think that it can solve just about anything, without the training in the relevant intellectual tasks to go with it.