AShepard comments on Minicamps on Rationality and Awesomeness: May 11-13, June 22-24, and July 21-28 - Less Wrong

24 Post author: AnnaSalamon 29 March 2012 08:48PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (239)
Tags: practical training minicamp

Comments (239)

You are viewing a single comment's thread. Show more comments above.

Comment author: Eliezer_Yudkowsky 29 March 2012 10:39:13PM 13 points [-]

We've actually noticed in our weekly sessions that our nice official-looking yes-we're-gathering-data rate-from-1-to-5 feedback forms don't seem to correlate with how much people seem to visibly enjoy the session - mostly the ratings seem pretty constant. (We're still collecting useful data off the verbal comments.) If anyone knows a standard fix for this then PLEASE LET US KNOW.

Comment author: AShepard 30 March 2012 02:00:05AM *  14 points [-]

I'd suggest measuring the Net Promoter Score (NPS) (link). It's used in business as a better measure of customer satisfaction than more traditional measures. See here for evidence, sorry for the not-free link.

  1. "On a scale of 0-10, how likely would you be to recommend the minicamp to a friend or colleague?"
  2. "What is the most important reason for your recommendation?

To interpret, split the responses into 3 groups:

  • 9-10: Promoter - people who will be active advocates.
  • 7-8: Passive - people who are generally positive, but aren't going to do anything about it.
  • 0-6: Detractor - people who are lukewarm (which will turn others off) or will actively advocate against you

NPS = [% who are Promoters] - [% who are Detractors]. Good vs. bad NPS varies by context, but +20-30% is generally very good. The followup question is a good way to identify key strengths and high priority areas to improve.

Comment author: Vaniver 30 March 2012 03:21:25AM 10 points [-]

NPS is a really valuable concept. Means and medians are pretty worthless compared to identifying the percentage in each class, and it's sobering to realize that a 6 is a detractor score.

(Personal anecdote: I went to a movie theater, watched a movie, and near the end, during an intense confrontation between the hero and villain, the film broke. I was patient, but when they sent me an email later asking me the NPS question, I gave it a 6. I mean, it wasn't that bad. Then two free movie tickets came in the mail, with a plea to try them out again.

I hadn't realized it, but I had already put that theater in my "never go again" file, since why give them another chance? I then read The Ultimate Question for unrelated reasons, and had that experience in my mind the whole time.)

Comment author: tgb 30 March 2012 03:41:15AM 2 points [-]

Good anecdote. It made me realize that I had just 20 minutes ago made a damning non-recommendation to a friend based off of a single bad experience after a handful of good ones.

Comment author: jsalvatier 30 March 2012 06:25:00PM 5 points [-]

Here is the evidence paper.

Comment author: thomblake 30 March 2012 02:02:38AM 3 points [-]

Right, I'd forgotten about that. I concur that it is used, and I work in market research sort of.

Powered by Reddit Powered by Reddit