Are said "non ideological people" a majority?
Most people accept that being smart helps you perform in school, but that does not mean that they accept that IQ is an appropriate representation of intelligence, or that they support using it to sort people for tasks where intelligence is a significant qualification. Asking people whether they think schools should be able to pick the most intelligent candidates, or sort by IQ, will not get you the same answers.
As mentioned upthread, most people don't mind sorting on SAT but oppose sorting on IQ. Hypothesis: Any measure that is perceived (correctly or not) to measure native talent accurately will be opposed, because people are afraid that their kids may not be talented "enough", and if the measure is accurate, they won't be able to gimmick it. People want a measure they can manipulate to their child's advantage.
I don't have any evidence for or against, but it seems plausible to me. I would expect people to want a system that benefits their own kids over...
Related: Heuristics for Evaluating the Soundness of the Academic Mainstream, Admitting to Bias, The Ideological Turing Test