I just looked at this test and had a problem with the very first question. It asks if you are strongly conservative or strongly liberal. What? I'd expect a high percentage of libertarians taking the test.
Other problems: Policies that are pro-immigration: If I believe in some policies that are pro-immigration and some that are not (like if I believe that certain groups of immigrants are bad and others aren't), what do I answer? Am I supposed to answer the question as if it means "are you in favor of unrestricted immigration?" (which it often means in political discourse) or actually answer what it literally says?
Actually, that's a general problem with the whole survey. There are questions which might have one meaning literally, but which mean something other than their literal words in political discourse. The global warming question doesn't say "... to such a degree that in order to fix it, we should put drastic policies into place", but that's what the question usually means. You're going to end up misdetecting bias when you've actually detected ability to read subtext and context.
What am I supposed to answer for questions about subjects on which I have no opinion? There isn't a "no opinion" answer on the question about whether I support tough on crime policies. And I have no idea how many percentage points renewable energy grew by.
There's also a problem whcih I've seen in lots of other surveys. The survey question consists of a statement followed by the actual question, phrased in a way which indicates that you're supposed to assume the statement is accurate. What do you do if you don't consider the statement to be accurate? ("Advocates of American capitalism argue that global technological and economic progress is largely due to American ingenuity." Literally true since it fails to state how many such advocates, but I don't believe what it is probably intended to say. "Taxes are much lower in the US than in the average EU country. If lower taxes meant more wealth, you would therefore expect the US to be richer, in the sense of having a higher GDP per capita, than most EU countries." I would not "therefore" expect any such thing, because I would only expect it if other factors are equal or favorable for the US and the statement hasn't stipulated that.)
Cross-posted from the EA forum. I asked for questions for this test here on LW about a year ago. Thanks to those who contributed.
Rationally, your political values shouldn't affect your factual beliefs. Nevertheless, that often happens. Many factual issues are politically controversial - typically because the true answer makes a certain political course of action more plausible - and on those issues, many partisans tend to disregard politically uncomfortable evidence.
This sort of political bias has been demonstrated in a large number of psychological studies. For instance, Yale professor Dan Kahan and his collaborators showed in a fascinating experiment that on politically controversial questions, people are quite likely to commit mathematical mistakes that help them retain their beliefs, but much less likely to commit mistakes that would force them to give up those belies. Examples like this abound in the literature.
Political bias is likely to be a major cause of misguided policies in democracies (even the main one according to economist Bryan Caplan). If they don’t have any special reason not to, people without special knowledge defer to the scientific consensus on technical issues. Thus, they do not interfere with the experts, who normally get things right. On politically controversial issues, however, they often let their political bias win over science and evidence, which means they’ll end up with false beliefs. And, in a democracy voters having systematically false beliefs obviously more often than not translates into misguided policy.
Can we reduce this kind of political bias? I’m fairly hopeful. One reason for optimism is that debiasing generally seems to be possible to at least some extent. This optimism of mine was strengthened by participating in a CFAR workshop last year. Political bias seems not to be fundamentally different from other kinds of biases and should thus be reducible too. But obviously one could argue against this view of mine. I’m happy to discuss this issue further.
Another reason for optimism is that it seems that the level of political bias is actually lower today than it was historically. People are better at judging politically controversial issues in a detached, scientific way today than they were in, say, the 14th century. This shows that progress is possible. There seems to be no reason to believe it couldn’t continue.
A third reason for optimism is that there seems to be a strong norm against political bias. Few people are consciously and intentionally politically biased. Instead most people seem to believe themselves to be politically rational, and hold that as a very important value (or so I believe). They fail to see their own biases due to the bias blind spot (which disables us from seeing our own biases).
Thus if you could somehow make it salient to people that they are biased, they would actually want to change. And if others saw how biased they are, the incentives to debias would be even stronger.
There are many ways in which you could make political bias salient. For instance, you could meticulously go through political debaters’ arguments and point out fallacies, like I have done on my blog. I will post more about that later. Here I want to focus on another method, however, namely a political bias test which I have constructed with ClearerThinking, run by EA-member Spencer Greenberg. Since learning how the test works might make you answer a bit differently, I will not explain how the test works here, but instead refer either to the explanatory sections of the test, or to Jess Whittlestone’s (also an EA member) Vox.com-article.
Our hope is of course that people taking the test might start thinking more both about their own biases, and about the problem of political bias in general. We want this important topic to be discussed more. Our test is produced for the American market, but hopefully, it could work as a generic template for bias tests in other countries (akin to the Political Compass or Voting Advice Applications).
Here is a guide for making new bias tests (where the main criticisms of our test are also discussed). Also, we hope that the test could inspire academic psychologists and political scientists to construct full-blown scientific political bias tests.
This does not mean, however, that we think that such bias tests in themselves will get rid of the problem of political bias. We need to attack the problem of political bias from many other angles as well.