I think he is trying to say that it's more important to understand how humans currently perform cognitive tasks than to try to formulate a robust way to perform cognitive tasks that has mathematical guarantees about accuracy. This same friend has some vague, new-agey type beliefs about "emotions" and I think he takes the evidence about emotions being major players in decision making a little too far. He sees attempts to quantify what he calls "human decision making" with the same tools that we use for "computational decision making" as fruitless. For him, aesthetics, for example, cannot be analyzed down to the level of preference orderings, utility functions, and Bayesian decision rules. Aesthetics simply are what they are, directly encoded by emotions and not "understandable" in any symbolic sense.
I don't think he would object to using Bayesian reasoning to make the best choices for, say, financial planning. But if you claimed that future neuroscientists will ever be able to quantitatively understand what "love" is (or even consciousness), he would reply that since human cognition is sub-conscious and non-Bayesian, you'll never explain it in a way such that knowledge of Bayesian decision theory can help a human make "human decisions" any better than their emotions would by default.
I think the problem is that he makes a dichotomy between "human computation" and "machine computation" (owing maybe to the fact that his background is in philosophy and psychology, with a lot less emphasis on the math and physics relevant to cognition). Not only this, but he then further claims that neuroscience evidence in favor of emotions as major players in human cognition is also in favor of treating the two sides of his dichotomy with fundamentally different tools, and that Bayesian reasoning is not a successful normative model for human computation. To him, comparing the goodness of a Bayesian decision in place of a human-emotion decision is a "silly" thing to do.
I predict that the lesson behind this exchange will turn out to be "Don't argue with people who think consciousness is fundamental".
I have recently been corresponding with a friend who studies psychology regarding human cognition and the best underlying models for understanding it. His argument, summarized very briefly, is given by this quote:
I am having trouble synthesizing a response that captures the Bayesian point of view (and is sufficiently backed up by sources so that it will be useful for my friend rather than just gainsaying of the argument) because I am mostly a decision theory / probability person. Are these works of psychology and neuroscience really illustrating that human emotion governs decision making? What are some good neuroscience papers to read that deal with this, and how do Bayesians respond? It may be that everything he mentions above is a correct assessment (I don't know and don't have enough time to read the books right now), but that it is irrelevant if you want to make good decisions rather than just accept the types of decisions we already make.