Epiphany comments on 2012 Survey Results - Less Wrong

80 Post author: Yvain 07 December 2012 09:04PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (640)

Sort By: Popular

You are viewing a single comment's thread.

Comment author: Epiphany 30 November 2012 07:09:06AM *  2 points [-]

Alternate Explanations for LW's Calibration Atrociousness:

Maybe a lot of the untrained people simply looked up the answer to the question. If you did not rule that out with your study methods, then consider seeing whether a suspiciously large number of them entered the exact right year?

Maybe LWers were suffering from something slightly different from the overconfidence bias you're hoping to detect: difficulty admitting that they have no idea when Thomas Bayes was born because they feel they should really know that.

Comment author: Kindly 30 November 2012 10:35:25PM *  7 points [-]

The mean was 1768, the median 1780, and the mode 1800. Only 169 of 1006 people who answered the question got an answer within 20 years of 1701. Moreover, the three people that admitted to looking it up (and therefore didn't give a calibration) all gave incorrect answers: 1750, 1759, and 1850. So it seems like your first explanation can't be right.

After trying a bunch of modifications to the data, it seems like the best explanation is that the poor calibration happened because people didn't think about the error margin carefully enough. If we change the error margin to 80 years instead of 20, then the responses seem to look roughly like the untrained example from the graph in Yvain's analysis.

Another observation is that after we drop the 45 people who gave confidence levels >85% (and in fact, 89% of them were right), the remaining data is absolutely abysmal: the remaining answers are essentially uncorrelated with the confidence levels.

This suggests that there were a few pretty knowledgeable people who got the answer right and that was that. Everyone else just guessed and didn't know how to calibrate; this may correspond to your second explanation.

Comment author: [deleted] 01 December 2012 03:15:49PM 6 points [-]

Another thing I have noticed is that I tend to pigeonhole stuff into centuries; for example, once in a TV quiz there was a question “which of these pairs of people could have met” (i.e. their lives overlapped), I immediately thought “It can't be Picasso and van Gogh: Picasso lived in the 20th century, whereas van Gogh lived in the 19th century.” I was wrong. Picasso was born in 1881 and van Gogh died in 1890. If other people also have this bias, this can help explain why so many more people answered 17xx than 16xx, thereby causing the median answer to be much later than the correct answer.

Comment author: NancyLebovitz 01 December 2012 06:04:50PM 1 point [-]

I hate the nth century convention because it doesn't match up with the numbers used for the dates, so I always refer to the dates.... but that actually tends to confuse people.

Comment author: [deleted] 02 December 2012 11:39:23AM 1 point [-]

I was going to say “the 1700s”, but that's ambiguous as in principle it could refer either to a century or to its first decade. (OTOH, it would be more accurate, as my mental pigeonholes lump the year 1700 together with the following 99 years, not with the previous.)

Comment author: Epiphany 01 December 2012 03:05:08AM *  2 points [-]

Good points, Kindly, thank you. New alternate explanation idea:

When these people encounter this question, they're slogging through this huge survey. They're not doing an IQ test. This is more casual. They're being asked stuff like "How many partners do you have?" By the time they get down to that question, they're probably in a casual answering mode, and they're probably a little tired and looking for an efficient way to finish. When they see the Bayes question, they're probably not thinking "This question is so important! They're going to be gauging LessWrong's rationality progress with it! I had better really think about this!" They're probably like "Output answer, next question."

If we really want to test them, we need to make it clear that we're testing them. And if we want them to be serious about it, we have to make it clear that it's important. I hypothesize that if we were to do a test (not a survey) and explain that it's serious because we're gauging LessWrong's progress, and also make it short so that the person can focus a lot of attention onto each question, we'd see less atrocious results.

In hindsight, I wonder why I didn't think about the effects of context before. Yvain didn't seem to either; he thought something might be wrong with the question. This seems like one of those things that is right in front of our faces but is hard to see.

I think that people may be rationing their mental stamina, and may not be going through all the steps it takes to answer this type of question.

Comment author: Swimmer963 01 December 2012 09:35:35PM 5 points [-]

When these people encounter this question, they're slogging through this huge survey. They're not doing an IQ test. This is more casual. They're being asked stuff like "How many partners do you have?" By the time they get down to that question, they're probably in a casual answering mode, and they're probably a little tired and looking for an efficient way to finish... If we really want to test them, we need to make it clear that we're testing them.

My first thought about this is that people's rationality 'in real life' totally is determined by how likely they are to notice a Bayes question in an informal setting, where they may be tired and feeling mentally lazy. In Keith Stanovich's terms, rationality is mostly about the reflective mind: it's someone's capacity and habits to re-compute a problem's answer, using the algorithmic mind, rather than accept the intuitive default answer that their autonomous mind spits out.

IQ tests tend to be formal; it's very obvious that you're being tested. They don't measure rationality in the sense that most LWers mean it; the ability to apply thinking techniques to real life in order to do better.

It might still be valuable to know how LWers do on a more formal test of probability-related knowledge; after all, most people in the general public don't know Bayes' theorem, so it'd be neat to see how good LW is at increasing "rationality literacy". But that's not the ultimate goal. There are reasons why you might want to measure a group's ability to pick out unexpected rationality-related problems and activate the correct mindware. If your Bayesian superpowers only activate when you're being formally tested, they're not all that useful as superpowers.

Comment author: Epiphany 01 December 2012 10:55:48PM *  2 points [-]

I can see why you'd criticize someone for saying "the problem is that the setting wasn't formal enough" but that's not exactly what I was getting at. What I was getting at is that there's a limit to how much thinking that one can do in a day, everyone's limit is different, and a lot of people do things to ration their brainpower so they avoid running out of it. This comment on mental stamina explains more.

My point was, more clearly worded: It would be a very rare person who possesses enough mental stamina to be rational in literally every single situation. That's a wonderful ideal, but the reality is that most people are going to ration brainpower. If your expectation is that rationalists should never ration brainpower and should be rational constantly, this is an unrealistic expectation. A more realistic expectation is that people should identify the things they need to think extra hard about, and correctly use rational thinking skills at those times. Therefore, testing for the skills when they're trying is probably the only way to detect a difference. There are inevitably going to be times when they're not trying very hard, and if you catch them at one of those times, well, you're not going to see rational thinking skills. It may be that some of these things can be ingrained in ways that don't use up a person's mental stamina, but to expect that rationality can be learned in such a way that it is applied constantly strikes me as an unreasoned assumption.

Now I wonder if the entire difference between the control groups results and LessWrong's results was that Yvain asked the control group only one question, whereas LessWrong had answered 14 pages of questions prior to that.

Comment author: Swimmer963 01 December 2012 11:12:52PM 3 points [-]

Agreed that rationality is mentally tiring...I went back and read your comment, too. However:

A more realistic expectation is that people should identify the things they need to think extra hard about, and correctly use rational thinking skills at those times.

To me, rationality is mostly the ability to notice that "whew, this is a problem that wasn't in the problem-set of the ancestral environment, therefore my intuitions probably won't be useful and I need to think". The only way a rationalist would have to be analytical all the time is if they were very BAD at doing this, and had to assume that every situation and problem required intense thought. Most situations don't. In order to be an efficient rationalist, you have to be able to notice which situations do.

Any question on a written test isn't a great measure of real-life rationality performance, but there are plenty of situations in everyday life when people have to make decisions based on some unknown quantities, and would benefit from being able to calibrate exactly how much they do know. Some people might answer better on the written test than if faced with a similar problem in real life, but I think it's unlikely that anyone would do worse on the test than in real life.

Comment author: V_V 03 December 2012 10:00:22PM -1 points [-]

someone's capacity and habits to re-compute a problem's answer, using the algorithmic mind, rather than accept the intuitive default answer that their autonomous mind spits out.

I don't think you could really apply any 'algorithmic' method to that question (other than looking it up, but that would be cheating). It was a test on how much confidence you put in your heuristics. (BTW, It seems that I've underestimated mine, or I've been lucky, since I've got the date off by one year but estimated my confidence at 50% IIRC). Still, it was a valuable test, since most of human reasoning is necessarily heuristic.

most people in the general public don't know Bayes' theorem

Really? What probability do you assign to that statement being true? :D

I'm under the impression that Bayes' theorem is included in the high school math programs of most developed countries, and I'm certain it is included in any science and engineering college program.

Comment author: [deleted] 09 January 2013 01:50:40PM 1 point [-]

I'm under the impression that Bayes' theorem is included in the high school math programs of most developed countries, and I'm certain it is included in any science and engineering college program.

It was in my high school curriculum (in Italy, in the mid-2000s), but the teacher spent probably only 5 minutes on it, so I would be surprised if a nontrivial number of my classmates who haven't also heard of it somewhere else remember it from there. IIRC it was also briefly mentioned in the part about probability and statistics of my "introduction to physics" course in my first year of university, but that's it. I wouldn't be surprised if more than 50% of physics graduates remember hardly anything about it other than its name.

Comment author: CarlShulman 31 December 2012 02:27:12AM 2 points [-]

most people in the general public don't know Bayes' theorem

Really? What probability do you assign to that statement being true? :D

There are national and international surveys of quantitative literacy in adults. The U.S. does reasonably well in these, but in general the level of knowledge is appalling to math teachers. See this pdf (page 118 of the pdf, the in-text page number is "Section III, 93") for the quantitative literacy questions, and the percentage of the general population attaining each level of skill. less than a fifth of the population can handle basic arithmetic operations to perform tasks like this:

One task in this level, with a difficulty value of 332, asks the reader to estimate, based on information in a news article, how many miles per day a driver covered in a sled-dog race. The respondent must know that to calculate a “ per day” rate requires the use of division. A more difficult task (355) requires the reader to select from two unit price labels to estimate the cost per ounce of creamy peanut butter. To perform this task successfully, readers may have to draw some information from prior knowledge.

People who haven't learned and retained basic arithmetic are not going to have a grasp of Bayes' theorem.

Comment author: Swimmer963 04 December 2012 04:01:53AM 5 points [-]

most people in the general public don't know Bayes' theorem

Really? What probability do you assign to that statement being true? :D

I assign about 80% probability to less than 25% of adults knowing Bayes theorem and how to use it. I took physics and calculus and other such advanced courses in high school, and graduated never having heard of Bayes' Theorem. I didn't learn about it in university, either–granted, I was in 'Statistics for Nursing', it's possible that the 'Statistics for Engineering' syllabus included it.

Comment author: dbaupp 04 December 2012 11:53:07AM 2 points [-]

Only 80%?

In the USA, about 30% of adults have a bachelor's degree or higher, and about 44% of those have done a degree where I can slightly conceive that they might possibly meet Bayes' theorem (those in the science & engineering and science- & engineering-related categories (includes economics), p. 3), i.e. as a very loose bound 13% of US adults may have met Bayes' theorem.

Even bumping the 30% up to the 56% who have "some college" and using the 44% for a estimate of the true ratio of possible-Bayes'-knowledge, that's only just 25% of the US adult population.

(I've no idea how this extends to the rest of the world, the US data was easiest to find.)

Comment author: Swimmer963 04 December 2012 04:25:38PM 0 points [-]

You did your research and earned your confidence level. I didn't look anything up, just based an estimate on anecdotal evidence (the fact that I didn't learn it in school despite taking lots of sciences). Knowing what you just told me, I would update my confidence level a little–I'm probably 90% sure that less than 25% of adults know Bayes Theorem. (I should clarify that=adults living in the US, Canada, Britain, and other countries with similar school systems. The percentage for the whole world is likely significantly lower.)

Comment author: Eugine_Nier 05 December 2012 05:42:20AM 1 point [-]

adults living in the US, Canada, Britain, and other countries with similar school systems.

I hear Britain's school system is much better than the US's.

Comment author: Jayson_Virissimo 05 December 2012 12:59:57PM 4 points [-]

Once you control for demographics, the US public school system actually performs relatively well.

Comment author: Larks 09 January 2013 02:08:17PM 0 points [-]

The UK high school system does not cover Bayes Theorem.

Comment author: Peterdjones 05 December 2012 12:53:39PM 0 points [-]

It's not great by international standards, but I have heard that the US system is particularly bad for an advanced country.

Comment author: V_V 18 December 2012 04:05:20PM 0 points [-]

I took physics and calculus and other such advanced courses in high school, and graduated never having heard of Bayes' Theorem.

Must be a problem of the American school system, I suppose.

I didn't learn about it in university, either–granted, I was in 'Statistics for Nursing', it's possible that the 'Statistics for Engineering' syllabus included it.

Did they teach you about conditional probability? Usually Bayes' theorem is introduced right after the definition of conditional probability.

Comment author: Eugine_Nier 04 December 2012 01:35:17AM 3 points [-]

I'm under the impression that Bayes' theorem is included in the high school math programs of most developed countries,

Well, it's certainly not included in the US high school curriculum.

Comment author: MugaSofer 09 January 2013 12:56:55PM 0 points [-]

most people in the general public don't know Bayes' theorem

Really? What probability do you assign to that statement being true? :D

I'm under the impression that Bayes' theorem is included in the high school math programs of most developed countries, and I'm certain it is included in any science and engineering college program.

I'm pretty sure Ireland doesn't have it on our curriculum, not sure how typical we are.

Comment author: katydee 01 December 2012 03:59:28AM 7 points [-]

If we really want to test them, we need to make it clear that we're testing them. And if we want them to be serious about it, we have to make it clear that it's important.

Uh, what? The point of LessWrong is to make people better all the time, not just better when they think "ah, now it's time to turn on my rationality skills." If people aren't applying those skills when they don't know they're being tested, that's a very serious problem, because it means the skills aren't actually ingrained on the deep and fundamental level that we want.

Comment author: Epiphany 01 December 2012 10:29:01AM *  8 points [-]

You know that, Katydee, but do all the people who are taking the survey think that way? The majority of them haven't even finished the sequences. I agree with you that it's ideal for us to be good rationalists all the time, but mental stamina is a big factor.

Being rational takes more energy than being irrational. You have to put thought into it. Some people have a lot of mental energy. To refer to something less vague and more scientific: there are different levels of intelligence and different levels of intellectual supersensitivity (A term from Dabrowski that refers to how excitable certain aspects of your nervous system are.) Long story short: Some people cannot analyze constantly because it's too difficult for them to do so. They run out of juice. Perhaps you are one of those rare people who has such high stamina for analysis that you rarely run into your limit. If that's the case, it probably seems strange to you that anybody wouldn't attempt to maintain a state of constant analysis. Most people with unusual intellectual stamina seem to view others as lazy when they observe that those other people aren't doing intellectual things all the time. It frequently does not occur to them to consider that there may be an intellectual difference. The sad truth is that most people have much lower limits on how much intellectual activity they can do in a day than "constant". If you want to see evidence of this, you can look at Ford's studies where he shows that 40 hours a week is the optimum number of hours for his employees to work. Presumably, they were just doing factory work assembling car parts, which (if it fits the stereotype of factory work being repetitive) was probably pretty low on the scale for what's intellectually demanding, but he found that if they tried to work 60 hours for two weeks in a row, their output would dip below the amount he'd normally get from 40 hours. This is because of mistakes. You'd think that the average human brain could do repetitive tasks constantly but evidently, even that tires the brain.

So in reality, the vast majority of people are not capable of the kind of constant meta-cognitive analysis that is required to be rational all the time. You use the word "ingrained" and I have seen Eliezer talk about how patterns of behavior can become habits (I assume he means that the thoughts are cached) and I think this kind of habit / ingrained response works beautifully when no decision-making is required and you can simply do the same thing that you usually do. But whenever one is trying to figure something out (like for instance working out the answers to questions on a survey) they're going to need to put additional brainpower into that.

I had an experience where, due to unexpected circumstances, I developed some vitamin deficiencies. I would run out of mental energy very quickly if I tried to think much. I had, perhaps, a half an hour of analysis available to me in a day. This is very unusual for me because I'm used to having a brain that loves analysis and seems to want to do it constantly (I hadn't tested the actual number of hours for which I was able to analyze, but I would feel bored if I wasn't doing something like psychoanalysis or problem-solving for the majority of the day). When I was deficient, I began to ration my brainpower. That sounds terrible, but that is what I did. I needed to protect my ability to analyze to make sure I had enough left over to be able to do all the tasks I needed to do each day. I could feel that slipping away while I was working on problems and I could observe what happened to me after I fatigued my brain. (Vegetable like state.)

As I used my brainpower rationing strategies, it dawned on me that others ration brainpower, too. I see it all the time. Suddenly, I understood what they were doing. I understood why they kept telling me things like "You think too much!" They needed to change the subject so they wouldn't become mentally fatigued. :/

Even if the average IQ at LessWrong is in the gifted range, that doesn't give everyone the exact same abilities, and doesn't mean that everyone has the stamina to analyze constantly. Human abilities vary wildly from person to person. Everyone has a limit when it comes to how much thinking they can do in a day. I have no way of knowing exactly what LessWrong's average limit is, but I would not be surprised if most of them use strategies for rationing brainpower and have to do things like prioritize answering survey questions lower on their list of things to "give it their all" on, especially when there are a lot of them, and they're getting tired.

Comment author: Swimmer963 01 December 2012 11:24:44PM 7 points [-]

Long story short: Some people cannot analyze constantly because it's too difficult for them to do so. They run out of juice. Perhaps you are one of those rare people who has such high stamina for analysis that you rarely run into your limit.

Fascinating!

It's making me realize why my summer project, which was to read Eat That Frog by Brian Tracey, was such a failure. The book is meant to be applied to work, preferably in an office environment–i.e. during your 40 productive work-hours. I was already working 40 hours a week at my extremely stimulating job as a nurse's aid at the hospital, where I had barely any time to sit down and think about anything, and I certainly didn't have procrastination problems. Then I would get home, exhausted with my brain about to explode from all the new interesting stuff I'd been seeing and doing all day, and try to apply Brian Tracey's productivity methods to the personal interest projects I was doing in my spare time.

This was a very efficient way to make these things not fun, make me feel guilty about being a procrastinator, etc. It gave me an aversion to starting projects, because the part of my brain that likes and needs to do something easy and fun after work knew it would be roped into doing something mentally tiring, and that it would be made to feel guilty over not wanting to do it.

I'm hoping that once I'm graduated and work as a nurse for a year or two, so that I have a chance to get accustomed to a given unit and don't have to spend so much mental effort, I'll have more left over for outside interests and can start reading about physics and programming for fun again. (Used to be able to do this in first and second year, definitely can't now.)

Comment author: Epiphany 02 December 2012 12:00:42AM *  2 points [-]

I'm glad you seem to have benefited from my explanation. If you want to do mentally draining reading, maybe weekends or later on in the evenings after you've rested would be a good time for that? If you've rested first, you might be able to scrape up a little extra juice.

Of course everyone has their own mental stamina limit, so nobody can tell you whether you do or don't have enough stamina to do additional intellectual activities after work. And it may vary day to day, as work is not likely to demand the exact same amount of brainpower every day.

An interesting experiment would be to see if there's anything that restores your stamina like a bath, a 20 minute nap after work, meditation, watching TV, or playing a fun game. Simply laying down in a dark quiet place does wonders for me if I am stressed out or fatigued. I would love to see someone log their mental stamina over time and correlate that to different activities that might restore stamina.

There are also stress reduction techniques that may help prevent you from losing stamina in the first place that could be interesting to experiment with.

And if you're not taking 15 minute breaks every 90 minutes during work, you might be "over-training" your brain. Over-training might result in an amplification of fatigue. "The Power of Full Engagement: Manage Energy Not Time" is likely to be of interest.

If you decide to do mental stamina experiments, definitely let me know!

Comment author: [deleted] 02 December 2012 12:11:50PM 1 point [-]

An interesting experiment would be to see if there's anything that restores your stamina like a bath, a 20 minute nap after work, meditation, watching TV, or playing a fun game.

I've also found that pouring lots of cold water on my face helps me squeeze out the last drops of stamina I have left, and allow me to work twenty more minutes or so. (It doesn't actually restore stamina, so it doesn't work if I do that more than a couple times in a row.)

Comment author: Epiphany 02 December 2012 08:16:46PM 0 points [-]

Hmmm. That might be one or a combination of the following:

  1. Taking a five minute break.
  2. Enjoying physical sensation. (Enjoyment seems to restore stamina for me, perhaps that's because the brain uses neurotransmitters for processing, and triggering pleasure involves increasing the amount of certain neurotransmitters.)

    Fifteen minute breaks are supposed to be optima, and if you maximized pleasure during your break, I wonder what amount of stamina that would restore?

Comment author: [deleted] 03 December 2012 11:22:43AM 0 points [-]

Probably 2. -- the break actually lasts about one minute.

Comment author: Swimmer963 02 December 2012 01:27:11PM 0 points [-]

If you decide to do mental stamina experiments, definitely let me know!

I hadn't actually thought of that before...but it's an awesome idea! I will let you know if I get around to it.

Comment author: Epiphany 02 December 2012 08:17:35PM 0 points [-]

Woo-hoo! (:

Comment author: Vladimir_Nesov 01 December 2012 11:30:30PM *  5 points [-]

Re the problem of having to think all the time: a good start is to develop a habit of rejecting certainty about judgments and beliefs that you haven't examined sufficiently (that is, if your intuition shouts at you that something is quite clear, but you haven't thought about that for a few minutes, ignore (and mark as a potential bug) that intuition unless you understand a reliable reason to not ignore it in that case). If you don't have the stamina or incentives to examine such beliefs/judgments in more detail, that's all right, as long as you remain correspondingly uncertain, and realize that the decisions you make might be suboptimal for that reason (which should suitably adjust your incentives for thinking harder, depending on the importance of the decisions).

Comment author: Epiphany 02 December 2012 12:52:48AM *  2 points [-]

The process of choosing a probability is not quite that simple. You're not just making a boolean decision about whether you know enough to know, you're actually taking the time to distinguish between 10 different amounts of confidence (10%, 20%, 30%, etc), and then making ten more tiny distinctions (30%, 31%, 32% for instance)... at least that's the way that I do it. (More efficient than making enough distinctions to choose between 100 different options.) When you are wondering exactly how likely you are to know something in order to choose a percentage, that's when you have to start analyzing things. In order to answer the question, my thought process looked like this:

  • Bayes. I have to remember who that is. Okay, that's the guy that came up with Bayesian probability. (This was instant, but that doesn't mean it took zero mental work.)

  • Do I have his birthday in here? Nothing comes to mind.

  • Digs further: Do I have any reason to have read about his birthday at any point? No. Do I remember seeing a page about him? I can't remember anything I read about his birthday.

  • Considers whether I should just go "I don't know" and put a random year with a 0% probability. Decides that this would be copping out and I should try to actually figure this out.

  • When was Bayesian probability invented? Let's see... at what point in history would that have occurred?

  • Try to brainstorm events that may have required Bayesian probability, or that would have suggested it didn't exist yet.

  • Try to remember the time periods for when these events happened.

  • Defines a vague section of time in history.

  • Considers whether there might be some method of double-checking it.

  • Considers the meaning of "within 20 years either way" and what that means for the probability that I'm right.

  • Figures out where in my vague section of time the 40 year range should be fit.

  • Figures out which year is in the middle of the 40 year range and types it in.

  • Consider how many years Bayes would likely have to have lived for before giving his theorems to the world and adjust the year to that.

  • Considers whether it was at all possible for Bayesian probability to have existed before or after each event.

  • If possible, consider how likely it was that Baye's probability existed before/after each event.

  • Calculate how many 40-year ranges there are in the vague section of time between the events where Bayes could not have been born.

  • Calculate the chance that I chose the correct 40-year section out of all the possible sections, if odds are equal.

  • Compare this to my probabilities regarding how likely it was for Bayes theorem to have existed before and after certain events.

  • Adjust my probability figure to take all that into account.

My answer to this question took at least twenty steps, and that doesn't even count all the steps I went through for each event, nor does it count all the sub steps I went through for things that I sort of hand-waved like "Adjust my probability figure to take all that into account".

If you think figuring out stuff is instant, you underestimate the number of steps your brain does in order to figure things out. I highly recommend doing meditation to improve your meta-cognition. Meta-cognition is awesome.

Comment author: Vladimir_Nesov 02 December 2012 01:57:44AM *  7 points [-]

you're actually taking the time to distinguish between 10 different amounts of confidence (10%, 20%, 30%, etc), and then making ten more tiny distinctions (30%, 31%, 32% for instance)... at least that's the way that I do it

The straightforward interpretation of your words evaluates as a falsity, as you can't estimate informal beliefs to within 1%.

Comment author: [deleted] 02 December 2012 12:06:57PM 4 points [-]

I'd put it more in terms of decibels of log-odds than percentages of probability. Telling 98% from 99% (i.e. +17 dB from +20 dB) sounds easier to me than telling 50% from 56% (i.e. 0 dB from +1 dB).

Comment author: Epiphany 02 December 2012 02:10:27AM 0 points [-]

Well, you can, but it would be a waste of time.

Comment author: Vladimir_Nesov 02 December 2012 02:13:01AM *  3 points [-]

No, I'm pretty certain you can't. You can't even formulate truth conditions for correctness of such an evaluation. Only in very special circumstances getting to that point would be plausible (when a conclusion is mostly determined by data that is received in an explicit form or if you work with a formalizable specification of a situation, as in probability theory problems; this is not what I meant by "informal beliefs").

Comment author: Vladimir_Nesov 02 December 2012 02:05:54AM *  3 points [-]

If you think figuring out stuff is instant, you underestimate the number of steps your brain does in order to figure things out.

(I was commenting on a skill/habit that might be useful in the situations where you don't/can't make the effort of explicitly reasoning about things. Don't fight the hypothetical.)

Comment author: Epiphany 02 December 2012 02:09:37AM 1 point [-]

(I was commenting on a skill that might be useful in the situations where you don't/can't make the effort of explicitly reasoning about things. Don't fight the hypothetical.)

Is it your position that there is a thinking skill that is actually accurate for figuring stuff out without thinking about it?

Comment author: Vladimir_Nesov 02 December 2012 02:29:41AM *  2 points [-]

I expect you can improve accuracy in the sense of improving calibration, by reducing estimated precision, avoiding unwarranted overconfidence, even when you are not considering questions in detail, if your intuitive estimation has an overconfidence problem, which seems to be common (more annoying in the form of an "The solution is S!" for some promptly confabulated arbitrary S, when quantifying uncertainty isn't even on the agenda).

(I feel the language of there being "positions" has epistemically unhealthy connotations of encouraging status quo bias with respect to beliefs, although it's clear what you mean.)

Comment author: katydee 01 December 2012 11:27:31PM 3 points [-]

Being rational takes more energy than being irrational. You have to put thought into it. Some people have a lot of mental energy. To refer to something less vague and more scientific: there are different levels of intelligence and different levels of intellectual supersensitivity (A term from Dabrowski that refers to how excitable certain aspects of your nervous system are.) Long story short: Some people cannot analyze constantly because it's too difficult for them to do so. They run out of juice. Perhaps you are one of those rare people who has such high stamina for analysis that you rarely run into your limit. If that's the case, it probably seems strange to you that anybody wouldn't attempt to maintain a state of constant analysis.

The point is to make these things automatic so that one doesn't have to analyze all the time. I definitely don't feel like I "maintain a state of constant analysis," even when applying purportedly advanced rationality techniques. It basically feels the same as thinking about things normally, except that I am right more often.

So in reality, the vast majority of people are not capable of the kind of constant meta-cognitive analysis that is required to be rational all the time. You use the word "ingrained" and I have seen Eliezer talk about how patterns of behavior can become habits (I assume he means that the thoughts are cached) and I think this kind of habit / ingrained response works beautifully when no decision-making is required and you can simply do the same thing that you usually do. But whenever one is trying to figure something out (like for instance working out the answers to questions on a survey) they're going to need to put additional brainpower into that.

I don't believe that your claim is true, but if it is I think LessWrong is doomed as a concept. I frankly do not think people will be able to accurately evaluate when they need to apply thinking skills to their decisions, so if we cannot teach skills on this level-- teach habits, as you say-- I do not think LessWrong will ever accomplish anything of real worth.

One example of a skill that I have taken on on this level is reference class forecasting. If I need to estimate how long something will take, my go-to method is to take the outside view. I am so used to this that it is now the automatic response to questions of estimating times.

As I used my brainpower rationing strategies, it dawned on me that others ration brainpower, too. I see it all the time. Suddenly, I understood what they were doing. I understood why they kept telling me things like "You think too much!" They needed to change the subject so they wouldn't become mentally fatigued. :/

I don't use "brainpower rationing" because I frankly have never felt the need to do so. I have told people that they "think too much" under certain circumstances (most notably when thinking is impeding action), and the thought of "brainpower rationing" has never come to mind until I saw this post.

Comment author: Epiphany 02 December 2012 01:20:45AM *  3 points [-]

What do you make of this?

I don't believe that your claim is true

Maybe I misinterpreted here but it sounds like you're saying you don't believe in mental stamina limits? Maybe you mean that you don't think rationality requires much brainpower?

but if it is I think LessWrong is doomed as a concept.

I don't think we'd be doomed, and there are a few reasons for that:

  1. There are people in existence who really can analyze pretty much constantly. THOSE people would theoretically have a pretty good chance of being rational all the time.

  2. People who cannot analyze anywhere near constantly can simply choose their battles. If they're aware of their mental stamina limits, they can work with them. Realizing you don't know stuff and that you don't have enough mental stamina to figure it out right now is kind of sad but it is still perfectly rational, so perhaps rationalists with low mental stamina can still be good rationalists that way.

  3. There are things that decrease mental fatigue. For instance, taking 15 minute breaks every 90 minutes (The book "The power of full engagement: manage energy not time" talks about this). We could do experiments on ourselves to find out what other things reduce or prevent mental fatigue. There may be low-hanging apples we're totally unaware of.

One example of a skill that I have taken on on this level is reference class forecasting. If I need to estimate how long something will take, my go-to method is to take the outside view. I am so used to this that it is now the automatic response to questions of estimating times.

Okay, so you've learned to instantly go to a certain method. I can believe that this does not take much brainpower. However, how much brainpower does it take to execute the outside view method, on average, for the types of things you use it for? How many times can you execute the outside view in a day? Have you ever tried to reach your mental stamina limit?

I don't use "brainpower rationing" because I frankly have never felt the need to do so.

Do you ever get home from work and feel relieved that you can relax now, and then do something that's not mentally taxing? Do you ever find that you're starting to hate an activity, and notice you're making more and more mistakes? Do you ever feel lazy and can't be bothered to do anything useful? I bet you do experience mental fatigue but don't recognize it as such. A lot of people just berate themselves for being unproductive, and don't consciously recognize that they've hit a real limit.

Comment author: katydee 02 December 2012 03:21:52AM *  3 points [-]

What do you think of this?

My method of doing the same calculation was:

  • see name
  • remember "mid-1700s"
  • therefore he must have been born in the early 1700s.
  • 20 year margin means I don't have to be precise.
  • answer: 1700 or 1705 (can't remember which I put)
  • get question right

The more difficult part was the probability estimate. But using the heuristics taught to me by this book, this took only a few calculations. And the more I do these types of calculations, the faster and more calibrated I become. Eventually I hope to make them automatic at the 8 + 4 = 12 level.

If I were doing the calculation "for real" and not on a survey my algorithm would be much easier:

  • see name
  • copy/paste name into Google
  • look at Wikipedia
  • look at other sources to confirm (if important)

Maybe I misinterpreted here but it sounds like you're saying you don't believe in mental stamina limits?

I know they exist on some level thanks to my experience with dual n-back, but I've yet to encounter any practical situation that imposes them (aside from "getting tired, which is different), and if I did I'm sure I could train my way out, just as I trained my way out of certain physical stamina limits. For example, it was once hard for me to maintain my endurance throughout a full fencing bout, but following some training I can do several in a row without becoming seriously fatigued. I'm sure better fencers than me can do even more.

LessWrong and CFAR, in my view, should provide the mental equivalent of that training if it is indeed necessary for the practice of rationality. I'm not, however, convinced that it is.

However, how much brainpower does it take to execute the outside view method, on average, for the types of things you use it for? How many times can you execute the outside view in a day? Have you ever tried to reach your mental stamina limit?

Immeasurably small (no perceived effort and takes less time than the alternative)/indeterminate/not in this respect. Most of the effort was involved in correctly identifying situations in which the method was useful, not in actually executing the method, but once the method became sufficiently ingrained that too went away.

Do you ever get home from work and feel relieved that you can relax now, and then do something that's not mentally taxing?

No. My work is generally fun.

Do you ever find that you're starting to hate an activity, and notice you're making more and more mistakes?

Not really. Sometimes I get bored, does that count?

Do you ever feel lazy and can't be bothered to do anything useful?

Negative.

Comment author: NancyLebovitz 02 December 2012 11:18:45AM 0 points [-]

I think mental stamina is an important concept.

I'll add mental exuberance (not an ideally clear word, but I don't have a better one)-- how much people feel an impulse to think.

Comment author: Epiphany 02 December 2012 08:05:00PM *  1 point [-]

Nancy, there is already a term for this. It's "intellectual overexcitability" or "intellectual supersensitivity". These are terms from Dabrowski. Look up the "Theory of Positive Disintegration" to learn more.

Comment author: NancyLebovitz 02 December 2012 11:21:14PM 1 point [-]

Those terms seem like pathologizing-- which is not surprising, considering that Dabrowski puts emphasis on the difficulties of the path. I was thinking more of the idea that some people like thinking more than others, just as some people like moving around more than others, which is something much less intense.

I was wondering whether Dabrowski was influenced by Gurdjieff, and it turns out that he was.

Comment author: NancyLebovitz 01 December 2012 01:11:06PM 4 points [-]

Thanks for the details. If I remember correctly, I was running out of the ability to care by the time I got to the Bayes question.

What were the vitamin deficiencies?

Comment author: Qiaochu_Yuan 01 December 2012 11:36:01PM 0 points [-]

I'm not sure I can reliably recognize what mental fatigue feels like. I'd like to be able to diagnose it in myself (because I suspect that I have less mental energy than I used to), so do you know of any reasonably quick way to induce something that feels like mental fatigue, e.g. alcohol?

Comment author: [deleted] 02 December 2012 12:15:19AM *  2 points [-]

Alcohol doesn't induce mental fatigue in me; high temperatures and dehydration do. YMMV.

EDIT: So does not eating enough sugars.

Comment author: Epiphany 02 December 2012 12:14:36AM *  2 points [-]

Whatever your worst subject is, do a whole bunch of exercises in it until you start making so many mistakes it is not worth continuing. No need for alcohol, might as well wear out your brain.

It would be interesting to see if you'd get different types of fatigue from doing different kinds of activities. For instance, if I do three hours of math problems, I have trouble speaking after that - it's like my symbol manipulation circuitry is fried. (I have dyslexia, so that's probably related.) If I wear out my verbal processor (something that I think only started happening to me after I developed some unexpected vitamin deficiencies) this results in irritation. I can't explain myself very well, so people jump on me for mistakes, and it's really hard to tell them what I meant instead, so I get frustrated.

So, exercising each area of mental abilities might yeild different fatigue symptoms.

If you decide to experiment on yourself I'm definitely curious about your results!

Comment author: [deleted] 02 December 2012 11:59:26AM 2 points [-]

It would be interesting to see if you'd get different types of fatigue from doing different kinds of activities.

That happens to me, too.

Comment author: Epiphany 02 December 2012 08:06:29PM 2 points [-]

What are your fatigue symptoms? How much can you do of each activity before becoming fatigued?

Comment author: [deleted] 03 December 2012 11:28:14AM 1 point [-]

What are your fatigue symptoms?

If I've been reading/studying too long, I find much harder to concentrate and am more easily distracted by stray thoughts.

If I've been writing computer code/doing maths too long, I make the kind of trivial mistakes that screw up the results but are hard to locate way more often.

How much can you do of each activity before becoming fatigued?

It depends -- usually between 20 minutes and 3 hours.

Comment author: Kindly 01 December 2012 03:56:31AM 2 points [-]

If you'll excuse the expression, I'm suspicious of your sudden epiphany. That is, I accept your suggestion as a possible explanation (although I'm not convinced, mainly because this doesn't describe the way I answered the question; I don't know about anyone else). But I think saying "Oh gosh! The true answer has been staring us in the face all along!" is premature.

Comment author: Epiphany 01 December 2012 09:53:52AM *  2 points [-]

I am not sure why you took "a new explanation" so seriously. I guess I have to be really careful on LessWrong to distinguish ideas from actual beliefs. I do not think it's "The True Answer". I just think it's a rather obvious alternate explanation that should have occurred to me immediately, and didn't, and I'm surprised about that, and about the fact that it didn't seem to occur to Yvain either. I reworded some things to make it more obvious that I am not trying to present this as "The True Answer" but just as an idea.

Comment author: Kindly 01 December 2012 01:57:57PM 1 point [-]

Thank you, I appreciate that.

Comment author: Epiphany 01 December 2012 09:06:55PM *  0 points [-]

Would you mind trying to avoid jumping to the conclusion that I'm acting stupid in the future, Kindly? I definitely don't mind being told "Your statement could be interpreted as such-and-such stupid behavior, so you may want to change it." but it's a little frustrating when people speak to me as if they really believe I am as confused as your "The True Answer" interpretation would imply.

Comment author: Kindly 01 December 2012 09:57:19PM 0 points [-]

I'm not sure why you're accusing me of this. I often disagree with people, but I usually don't assume the people I disagree with are stupid. This is especially true when we disagree due to a misunderstanding.

(I don't intend to continue this line of conversation.)