Harvard Prof. Richard Moran touches on this in a humorous manner:
"As to ‘experimental philosophy, I can’t claim to be very well versed in it, but it seems to be a research program in its early days. I think that by now, even its practitioners are beginning to realise that simply asking people, outside of any particular context, about their “intuitions” about some concept of philosophical interest is not really going to be informative since without any philosophical background to the question, the respondents themselves can’t really know just what question they are being asked to answer, what their responses are responses to. There are just too many different things that can be meant by a question like, “‘Was such-and-such an action intentional or not?”, for example. And without further discussion or further analysis, the experimenters themselves can’t know what answers they are being given by the respondents. It’s not good data. So I can imagine experimental philosophy evolving in a way to account for this, and starting to include some philosophical background to the investigation, perhaps even some philosophical history, to provide the needed context to the particular intuitions that they are trying to expose and test for. At that point, the experimental situation might also become less one-sided, with a researcher examining a respondent, and could allow for the experimental subjects themselves to ask questions of the experimenters, including questions of clarification and disambiguation, and perhaps even challenges to the way the experimenter has framed the questions.
Later it might be found useful to conduct such experiments in small groups rather than individually, with one experimenter and one subject, and instead the respondents could be encouraged to discuss the questions among themselves as well as with the experimenter. People could meet in these groups two or three times a week and perhaps some relevant reading could be assigned, to clarify and expand upon the question, and the respondents would be given time to do the reading, and asked to write something later on about the question in connection with the reading and the discussions they have had. Then the experimenter could provide “comments” on this writing for the experimental subjects themselves. I think grading the results would be optional on such an arrangement, and probably of no experimental interest, but other than that I think something like this could be the future of experimental philosophy. It’s worth trying anyway."
http://www.3ammagazine.com/3am/keeping-sartre-and-other-passions/2/
This seems to be implying that moral philosophy has little or nothing to do with how untrained people make moral decisions; epistemology has little or nothing to do with how untrained people gain confidence in their beliefs as knowledge, etc.
Consider these two versions of the famous trolley problem:
Here it is: a standard-form philosophical thought experiment. In standard analytic philosophy, the next step is to engage in conceptual analysis — a process in which we use our intuitions as evidence for one theory over another. For example, if your intuitions say that it is "morally right" to throw the switch in both cases above, then these intuitions may be counted as evidence for consequentialism, for moral realism, for agent neutrality, and so on.
Alexander (2012) explains:
In particular, notice that philosophers do not appeal to their intuitions as merely an exercise in autobiography. Philosophers are not merely trying to map the contours of their own idiosyncratic concepts. That could be interesting, but it wouldn't be worth decades of publicly-funded philosophical research. Instead, philosophers appeal to their intuitions as evidence for what is true in general about a concept, or true about the world.
In this sense,
But anyone with more than a passing familiarity with cognitive science might have bet in advance that this basic underlying assumption of a core philosophical method is... incorrect.
For one thing, philosophical intuitions show gender diversity. Consider again the Stranger and Child versions of the Trolley problem. It turns out that men are less likely than women to think it is morally acceptable to throw the switch in the Stranger case, while women are less likely than men to think it is morally acceptable to throw the switch in the Child case (Zamzow & Nichols 2009).
Or, consider a thought experiment meant to illuminate the much-discussed concept of knowledge:
When presented with this vignette, only 41% of men say that Peter "knows" there is a watch on the table, while 71% of women say that Peter "knows" there is a watch on the table (Starman & Friedman 2012). According to Buckwalter & Stich (2010), Starmans & Friedman ran another study using a slightly different vignette with a female protagonist, and that time only 36% of men said the protagonist "knows," while 75% of women said she "knows."
The story remains the same for intuitions about free will. In another study reported in Buckwalter & Stich (2010), Geoffrey Holtman presented subjects with this vignette:
In this study, only 35% of men, but 63% of women, said a person in this world could be free to choose whether or not to murder someone.
Intuitions show not only gender diversity but also cultural diversity. Consider another thought experiment about knowledge (you can punch me in the face, later):
Only 26% of Westerners say that Bob "knows" that Jill drives an American car, while 56% of East Asian subjects, and 61% of South Asian subjects, say that Bob "knows."
Now, consider a thought experiment meant to elicit semantic intuitions:
When presented with this vignette, East Asians are more likely to take the "descriptivist" view of reference, believing that John "is referring to" Schmidt — while Westerners are more likely to take the "causal-historical" view, believing that John "is referring to" Gödel (Machery et al. 2004).
Previously, I asked:
For one thing, we would never assume that people of all kinds would share our intuitions.
Next post: Philosophy Needs to Trust Your Rationality Even Though It Shouldn't
Previous post: Living Metaphorically