Lightwave comments on Open Thread, October 16-31, 2012 - Less Wrong

5 Post author: OpenThreadGuy 16 October 2012 10:43PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (271)

You are viewing a single comment's thread.

Comment author: Lightwave 17 October 2012 12:51:37PM *  1 point [-]

I'm planning on doing a presentation on cognitive biases and/or behavioral economics (Kahneman et al) in front of a group of university students (20-30 people). I want to start with a short experiment / demonstration (or two) that will demonstrate to the students that they are, in fact, subject to some bias or failure in decision making. I'm looking for suggestion on what experiment I can perform within 30 minutes (can be longer if it's an interesting and engaging task, e.g. a game), the important thing is that the thing being demonstrated has to be relevant to most people's everyday lives. Any ideas?

I also want to mention that I can get assistants for the experiment if needed.

Edit: Has anyone at CFAR or at rationality minicamps done something similar? Who can I contact to inquire about this?

Comment author: Larks 17 October 2012 09:57:14PM 4 points [-]

Get people to give 90% confidence intervals on 10 questions, and then at the end ask

"Ok, so who got all 10 within their intervals. 9? That's what you should have got... ok, 8? Still no-one? Ok, how about 7?"

Comment author: Kindly 18 October 2012 12:00:37AM 0 points [-]

90% might not be the best number for demonstrating the idea of a confidence interval. It's too close to 100%. There's not much room to be underconfident. What about 50% confidence intervals?

Comment author: Morendil 18 October 2012 06:13:59AM *  1 point [-]

Have you tried it? I have, and I can tell you most people I tried it on are over-confident when asked for 90% intervals.

Comment author: Morendil 18 October 2012 06:24:02AM *  3 points [-]

I've done this in a few small groups, using:

  • the "this is an attention test" video (aka "how many passes does the team in white make") - always a hit
  • a calibration exercise just as Larks suggests below, which works pretty well
  • the Wason card task - some people find it unconvincing "because it's logic" (!)
  • a "learned helplessness" experiment using impossible anagrams - works so-so
  • a "priming" experiment where two groups have a phrase-building task with different priming words - my version of that sucks, need to rework it
  • I also show the Spinning Dancer often
Comment author: maia 18 October 2012 01:27:08AM 1 point [-]

For something very brief, anchoring bias is easy to demonstrate and fairly dramatic. I tried this on a friend a couple weeks ago, anchoring her on 1 million people as the population of Ghana; she guessed 900,000. Turned out to be 25 million.

Comment author: sixes_and_sevens 17 October 2012 01:17:17PM 1 point [-]

The Wason selection task is a good go-to example of confirmation bias.

Comment author: Lightwave 17 October 2012 01:48:23PM 2 points [-]

Well the thing is that people actually get this right in real life (e.g. with the rule 'to drink you must be over 18'). I need something that occurs in real life and people fail at it.

Comment author: sixes_and_sevens 17 October 2012 01:59:29PM 1 point [-]

They get it correct when it's in an appropriate social context, not simply because it's happening in real life. If it didn't happen in real life, confirmation bias wouldn't be a real thing.

Comment author: Lightwave 17 October 2012 02:27:23PM 2 points [-]

Right, but I want to use a closer to real life situation or example that reduces to the wason selection task (and people fail at it) and use that as the demonstration, so that people can see themselves fail in a real life situation, rather than in a logical puzzle. People already realize they might not be very good at generalized logic/math, I'm trying to demonstrate that the general logic applies to real life as well.

Comment author: Vaniver 17 October 2012 05:42:23PM 0 points [-]

Well the thing is that people actually get this right in real life (e.g. with the rule 'to drink you must be over 18'). I need something that occurs in real life and people fail at it.

No, people are more likely to get it right in real life. Some fraction of your audience will get it wrong, even with ages and drinks.

Comment author: faul_sname 22 October 2012 06:17:45PM 1 point [-]

To a first approximation, people get it right in real life.

Comment author: Barry_Cotter 17 October 2012 01:13:08PM *  1 point [-]

Confirmation bias, the triplet number test where the rule is “Any triplet where the second number is greater than the first and the third greatet than the second”. Original credit (edit:for my exposure)to Eliezer in HPmoR but I thought of it because that was what Yvain did at a meetup I was at.

Comment author: wedrifid 17 October 2012 01:34:29PM 3 points [-]

Confirmation bias, the triplet number test where the rule is “Any triplet where the second number is greater than the first and the third greatet than the second”. Original credit to Eliezer in HPmoR but I thought of it because that was what Yvain did at a meetup I was at.

To be clear, since reading this made me double-take, I think by "original credit" you mean "original credit for your personal exposure to the concept".