Comment author: John_Maxwell_IV 20 March 2014 03:20:44AM 1 point [-]

My impression was that Kaj's essay was not original to him but rather inspired by the paper he linked to at the bottom.

Comment author: velisar 20 March 2014 01:50:38PM 0 points [-]

I edited for clarity, thanks.

Comment author: RichardKennaway 19 March 2014 05:01:56PM *  1 point [-]

But in our case, we need God to say that the nerves in the skin are thermometers, the eyes, height measuring tools and so on.

Historically, it has been the other way round. We can recognise, without the hypothesis of God, that legs are good for walking, eyes for seeing, and so on, and these observable facts were taken as proof of the existence of a Designer.

Having dispensed with the Designer, we are left with the problem of explaining why living organisms appear to be made of distinct parts serving clear functions, and how we are able to say that these functions are sometimes performed well and sometimes badly, how we can describe some processes as pathological and some as healthy.

ETA: The answer isn't "evolution!", because when we use evolutionary techniques to solve computational problems, the result is typically something that works but we can't see how. When we look at living organisms we see things that work that we largely can see how. (The brain is a notable exception. Also, protein folding. But a heart is clearly a pump.)

Comment author: velisar 19 March 2014 07:54:45PM 0 points [-]

True.

But is jealousy pathological? Or anger? Or fear?

I was arguing that the nerves in the skin are only an approximation of thermometers, likewise the eyes only a poor measure tool. By the way, there are 'evolutionary' biases: we perceive a ravine as deeper when we look down onto it and, conversely, from the bottom looking up it doesn't seem as tall (see also auditory looming). Their function is quite transparent once you think about organisms and not measure tools.

Comment author: RichardKennaway 19 March 2014 03:58:38PM 0 points [-]

Can you comment on how the concept of "ecological rationality" relates to this imaginary conversation?

Comment author: velisar 19 March 2014 04:28:05PM 0 points [-]

It seems to me that it is the discussion about optimizing versus satisficing.

If Intel builds the computer to do some division, but they found a way to approximate the results because that way the CPU can simulate, I don't know, a nuclear explosion, it should say so. But in our case, we need God to say that the nerves in the skin are thermometers, the eyes, height measuring tools and so on. The only utility function of organisms that we now for sure is that the code that build them has to make it in the next generation; we can argue about different strategies, but they depend on - sometimes - too many other things.

Comment author: Gunnar_Zarncke 19 March 2014 12:46:14PM 1 point [-]

Agreed. But it kind of means that some evolution of fallacies trending toward more complex argumentation patterns is taking place. Or? I'm not versed in the classics but I take it that they didn't have this large an (anti-)tool-set.

Comment author: velisar 19 March 2014 03:09:06PM 0 points [-]

I think any preoccupation, if it exists long enough, results in great refinements. The are people good a African rare languages, mineral water, all sorts of (noble!) sports, torture - why should't people get better at something as common as argumentation.

But we're advocating a look the other way around, to the more basic processes, they may say something about how humans work. And indeed, it would be easier with less sophisticated arguers.

Comment author: Viliam_Bur 19 March 2014 01:34:14PM *  5 points [-]

It is true that there are reasons for our biases; that human behavior was shaped by evolution and optimized for the natural environment. Many mistakes that we do are a result of behavior that contributes to survival in nature.

But I think that "contributes to survival" does not always lead to "solid inference rules". For example, imagine that a majority of the tribe is wrong about some factual question. (A question where being right or wrong is not immediately relevant for surviving.) It contributes to survival if an individual joins this majority, because it gets them allies. -- This could be excused by saying that in an ancient tribe without much specialization, a majority is more likely to be correct than an individual, therefore "follow the majority opinion" actualy is a good truth-finding heuristics. But that ignores the fact that people sometimes lie for a purpose, e.g. calumniate their opponents, or fabricate religious experience. So there is more in joining the majority than merely a decent truth-finding heuristics.

(EDIT: It's not like in the past humans lived in harmony with nature using their heuristics, and only today we have exploitable biases. People had exploitable biases even in the ancient environment -- their heuristics were correct often, but not always -- and people have exploited each other's biases even in the ancient environment. Not only we had adaptations to make mostly correct decisions, but also adaptations to exploit other people's flaws in the former adaptations.)

Also, no species is perfectly tuned to their environment. Some useful mutations simply didn't happen yet. Also, there are various trade-offs, so even if a species as a whole is optimized for given environment, some of their individual features may be suboptimal, as a price to improve other conflicting features. Therefore, assuming that every human bias is a result of a perfect behavior in the natural environment, would be assuming too much.

But otherwise, I like this.

Comment author: velisar 19 March 2014 02:55:19PM 0 points [-]

I have to admit that the text is a bit long! We sorta did say all of that you are saying, which means that the way I resumed the text here was a bit misleading.

There must be conditions when a heuristic like "follow the majority opinion" must be triggered in our heads: something is recognized maybe. There is selection pressure to find social exchange violation, but also to be ingenious in persuasion. Some of this already has experimental support. Anyway, we think that what we today call fallacies are not accidents - like the blind spot. They are good inference rules for a relatively stable environment, but cannot predict far into the future and cannot judge new complex problems. That may be why we don't spot the fallacies of small talk, of experts in domains with expertise, or in domains for which we already have intuitions.

That would imply that a bad decision today is not necessarily the product of a cognitive illusion, but that we build a bad interface for the actual human mind in the modern world (a car will be lighter and faster if it shouldn't accommodate humans). Reference class forecasting or presenting probabilities as frequencies are just technologies, interfaces. The science is about the function and the fallacies are interesting precisely because, presumably, they are a repetitive behavior. They may help in our effort to reverse engineer ourselves.

Comment author: Gunnar_Zarncke 19 March 2014 12:03:51PM *  1 point [-]

A quote from that paper:

If a style of argumentation has survived critics for millennia, we can ask several questions: Could it be that there are evolutionary programs running in our heads that systematically push us to do the same things? Are those based on inferences that correlate with good fitness? Where does epistemic value differ from ecologic utility? Do the fallacists have some observation bias; do we suffer from the Focusing illusion (Schkade & Kahneman, 1998) when observing a bad argument?

I have this heard called the fallacy fallacy (though rational wiki sees that differently).

Comment author: velisar 19 March 2014 12:14:02PM 0 points [-]

You are correct; but the Argument from fallacy is still pretty uninformative.

The ecological rationality of the bad old fallacies

7 velisar 19 March 2014 11:39AM

I think that the community here may have some of the most qualified people to judge a new frame of studying the fallacies of argumentation with some instruments that psychologists use. I and my friend Dan Ungureanu, a linguist at Charles University in Prague could use some help!

I’ll write a brief introduction on the state of argumentation theory first, for context:

There is such thing as a modern argumentation theory. It can be traced back to the fifties when Perelman and Olbrechts-Tyteca published their New Rhetoric and Toulmin published his The uses of argument. The fallacies of argumentation, now somewhat popular in the folk argumentation culture, have had their turning point when the book Fallacies (Hamblin, 1970) argued that most fallacies are not fallacies at all, they are most of the time the reasonable option. Since then some argumentation schools have taken Hamblin’s challenge and tried to come up with a theory of fallacies. Of them, the Informal logic school and the pragma-dialectics are the most well-known. They even have made empirical experiments to verify their philosophies.

Another normative approach, resumed here by Kaj Sotala in Fallacies as weak Bayesian evidence, is comparing fallacious arguments with the Bayesian norm (Hahn & Oaksford, 2007; also eg. Harris, Hsu & Madsen, 2012; Oaksford & Hahn, 2013).

We cherry-pick a discourse to spot the fallacies. We realized that a couple of years ago when we had to teach the informal fallacies to journalism masters students: we would pick a text that we disagree with, and then search for fallacies. Me and Dan, we often come up with different ones for the same paragraph. They are vague. Than we switched to cognitive biases, as possible explanations for some fallacies, but we were still in the ‘privileging the hypothesis territory’, I would say now, with the benefit of hindsight.

Maybe the world heuristic has already sprung to some of you. I’ve seen this here and somewhere else on the net: fallacies as heuristics. Argumentation theorists only stumbled on this idea recently (Walton, 2010).

Now here’s what this whole intro was for: lesswrong and before overcoming bias are sites build on the idea that we can improve our rationality by doing some things in relation to the now famous Heuristics&Biases program. The heuristics as defined by Twersky and Kahneman are only marginally useful for assessing the heuristic value of a type of argument that we use to call a fallacy. The heuristic elicitation design is maybe a first step: we can see if we have some form of attribute substitution (we always have, if we think that a Bayesian daemon is the benchmark).

We started with the observation that if people generally fall back to some particular activity when they are “lazy”, that activity could be a precious hint about human nature. We believe that it is far easier to spot the fallacy a) when you are looking for it and b) that you are looking for it usually if the topic is interesting, complex, grey: theology, law, politics, health and the like. If indeed the fallacies of argumentation are stable and universal behaviors across (at least some) historical time and across cultures, we can see those “fallacies” as rules of thumb that use other, lower-level fast and frugal heuristics as solid inference rules in the right ecology. Ecological rationality is a match between the environment and the – bounded rational – agent’s decision mechanisms (G. Gigerenzer 1999, V. Smith, 2003).

You can’t just invent a norm and then compare behaviors of organisms or artifacts with it. Not even Bayes rule: the decision of some organisms will have to be Bayesian only in their natural environment (E.T. Jaynes observed this). That is why we need a computational theory of people even when we study arguments: there is no psychology which isn’t evolutionary psychology. We need to know the function, but saying fallacy is about valence, so people traditionally ask why we are so narrow or stupid or, recently, when are the fallacies irrational and when they are not. (no, we don’t want to start again the 1996 polemic between Gigerenzer and Tversky&Kahneman!).

Well, that is what we think, anyway. And if you spot a big flaw, please point it to us before we send our paper to a journal.

Here’s the draft of our paper:

https://www.academia.edu/6271737/The_Ecological_Rationality_of_Argumentation_Fallacies

 

Thanks

Comment author: fortyeridania 01 February 2014 07:13:43AM 0 points [-]

The one from Carnap ("Anything you can do, I can do meta") might not really be from Carnap. Can anyone find a source besides this one, which only gets it back to 1991?

Comment author: velisar 08 February 2014 09:03:59PM 0 points [-]

I think it's Daniel Dennet (said to Hofstadter).

Comment author: mwengler 23 September 2011 01:59:25PM 3 points [-]

I grew up in Long Island 20 miles from JFK airport. We could see the Concorde once in a while at JFK airport and if we were very lucky we would see it landing or taking off. The amount of mindspace in the world occupied by that beautiful plane was gigantic compared to that occupied by most other planes. Whether the Concorde was still a net deficit to the UK and France would require, I think, a calculation similar to figuring the deficit or surplus to the U.S. of putting people on the moon.

Comment author: velisar 28 January 2013 09:51:20AM *  0 points [-]

You might be right - as I never saw one - but the project didn't start with a plan to built a spectacular flying sculpture. So they fell first to the planning fallacy (which may not be so much a psychological cognitive bias but the very structure of possible outcomes of everything - the top of the frequency distribution is to the right of the "arrival" time), then to sunk costs which later were half acknowledged, thus making them highly suspicious of trying to resolve a cognitive dissonance (rationalization).

One has to take into account the original prediction to make a probabilistic interpretation...

Comment author: Virge2 13 May 2008 02:14:32PM 4 points [-]

Eliezer, I guess the answer you want is that "science" as we know it has at least one bias: a bias to cling to pragmatic pre-existing explanations, even when they embody confused thinking and unnecessary complications. This bias appears to produce major inefficiencies in the process.

Viewing science as a search algorithm, it follows multiple alternate paths but it only prunes branches when the sheer bulk of experimental evidence clearly favours another branch, not when an alternate path provides a lower cost explanation for the same evidence. For efficiency, science should instead prune (or at least allocate resources) based on a fair comparison of current competing explanations.

Science has a nostalgic bias.

Comment author: velisar 22 May 2012 08:27:00PM 1 point [-]

The science world, as much as the rest of the "worlds" comprised by people who share something which everybody cherishes, has to have the status quo bias. (the enigmatic add-on: One cannot escape the feeling that there is such thing as time)

View more: Next