Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Positive Bias Test (C++ program)

26 Post author: MBlume 19 May 2009 09:32PM

I've written a program which tests positive bias using Wason's procedure from "On the failure to eliminate hypotheses in a conceptual task" (Quarterly Journal of Experimental Psychology, 12: 129-140, 1960). If the user does not discover the correct rule, the program attempts to guess, based on the user's input, what rule the user did find, and explains the existence of the more general rule. The program then directs the user here.

I'd like to use a better set of triplets, and perhaps include more wrong rules. The program should be fairly flexible in this way.

I'd also like to set up a web-based front-end to the program, but I do not currently know any cgi.

I'm not completely happy with the program's textual output. It still feels a bit like the program is scolding the user at the end. Not quite sure how to fix this.

Program source

ETA: Here is a macintosh executable version of the program. I do not have any means to make an exe file, but if anyone does, I can host it.

If you're on Linux, I'm just going to assume you know what to do with a .cpp file =P

Here is a sample run of the program (if you're unfamiliar with positive bias, or the wason test, I'd really encourage you to try it yourself before reading):

Hi there! We're going to play a game based on a classic cognitive science experiment first performed by Peter Wason in 1960 (references at the end)

Here's how it works. I'm thinking of a rule which separates sequences of three numbers into 'awesome' triplets, and not-so-awesome triplets. I'll tell you for free that 2 4 6 is an awesome triplet.

What you need to do is to figure out which rule I'm thinking of. To help you do that, I'm going to let you experiment for a bit. Enter any three numbers, and I'll tell you whether they are awesome or not. You can do this as many times as you like, so please take your time.

When you're sure you know what the rule is, just enter 0 0 0, and I'll test you to see if you've correctly worked out what the rule is.

Enter three numbers separated by spaces: 3 6 9

3, 6, 9 is an AWESOME triplet!

Enter three numbers separated by spaces: 10 20 30

10, 20, 30 is an AWESOME triplet!

Enter three numbers separated by spaces: 8 16 24

8, 16, 24 is an AWESOME triplet!

Enter three numbers separated by spaces: 0 0 0

So, you're pretty sure what the rule is now? Cool. I'm going to give you some sets of numbers, and you can tell me whether they seem awesome to you or not.
Would you say that 3, 6, 9 looks like an awesome triplet? (type y/n)
y

Would you say that 6, 4, 2 looks like an awesome triplet? (type y/n)
n

Would you say that 8, 10, 12 looks like an awesome triplet? (type y/n)
n

Would you say that 1, 17, 33 looks like an awesome triplet? (type y/n)
n

Would you say that 18, 9, 0 looks like an awesome triplet? (type y/n)
n

Would you say that 1, 7, 3 looks like an awesome triplet? (type y/n)
n

Would you say that 3, 5, 7 looks like an awesome triplet? (type y/n)
n

Would you say that 2, 9, 15 looks like an awesome triplet? (type y/n)
n

Would you say that 5, 10, 15 looks like an awesome triplet? (type y/n)
y

Would you say that 3, 1, 4 looks like an awesome triplet? (type y/n)
n

You thought that 3, 6, 9 was awesome.
In fact it is awesome.

You thought that 6, 4, 2 was not awesome.
In fact it is not awesome.

You thought that 8, 10, 12 was not awesome.
In fact it is awesome.

You thought that 1, 17, 33 was not awesome.
In fact it is awesome.

You thought that 18, 9, 0 was not awesome.
In fact it is not awesome.

You thought that 1, 7, 3 was not awesome.
In fact it is not awesome.

You thought that 3, 5, 7 was not awesome.
In fact it is awesome.

You thought that 2, 9, 15 was not awesome.
In fact it is awesome.

You thought that 5, 10, 15 was awesome.
In fact it is awesome.

You thought that 3, 1, 4 was not awesome.
In fact it is not awesome.

It looks as though you thought the rule was that awesome triplets contained three successive multiples of the same number, like 3,6,9, or 6,12,18. In fact, awesome triplets are simply triplets in which each number is greater than the previous one.

The rule for awesomeness was a fairly simple one, but you invented a more complicated, more specific rule, which happened to fit the first triplet you saw. In experimental tests, it has been found that 80% of subjects do just this, and then never test any of the triplets that *don't* fit their rule. If they did, they would immediately see the more general rule that was applying. This is a case of what psychologists call 'positive bias'. It is one of the many biases, or fundamental errors, which beset the human mind.

There is a thriving community of rationalists at the website Less Wrong (http://www.lesswrong.com) who are working to find ways to correct these fundamental errors. If you'd like to learn how to perform better with the hardware you have, you may want to pay them a visit.

If you'd like to learn more about positive bias, you may enjoy the article 'Positive Bias: Look Into the Dark': http://www.overcomingbias.com/2007/08/positive-bias-l.html
If you'd like to learn more about the experiment which inspired this test, look for a paper titled 'On the failure to eliminate hypotheses in a conceptual task' (Quarterly Journal of Experimental Psychology, 12: 129-140, 1960)

Comments (75)

Comment author: JGWeissman 20 May 2009 05:13:20AM *  10 points [-]

Here is a version in HTML/JavaScript.

The style could use some improvement.

And I still think it would be nice to have a version that does not depend on client scripting. I could do it easily in asp.Net, but if anyone wants to do it Python, that would probably be more compatible with the existing server scripting. (Edit: darius has provided a Python implementation.)

Also, does anyone know of a better file hosting site? I apologize for the captcha needed to download.

Comment author: MichaelHoward 20 May 2009 12:50:54PM 2 points [-]

Odd, I just get "Sorry, your time to enter the code has expired. Please try again". Tried with Firefox 3.0.10 and IE 7.0.5730.13.

Comment author: Vladimir_Golovin 20 May 2009 12:54:22PM *  2 points [-]

Look below the table -- there should be a link "Download for free with FileFactory Basic". (I downloaded it using Firefox.)

Comment author: MichaelHoward 20 May 2009 09:12:55PM 0 points [-]

That worked, thanks. :)

Comment author: ChrisHibbert 20 May 2009 04:27:06PM 1 point [-]

One advantage of the executable is that you don't lose a record of your previous questions and answers. I think this is crucial, and ought to be included in a webbed version as well. It was annoying that the text entry boxes didn't let me type <return> in order to enter my guess.

But thanks for writing this version to get us started on the web.

Comment author: cabalamat 21 May 2009 02:21:52AM 1 point [-]

One advantage of the executable is that you don't lose a record of your previous questions and answers. I think this is crucial, and ought to be included in a webbed version as well.

sorted

Comment author: Vladimir_Golovin 20 May 2009 10:07:19AM 0 points [-]

Thanks -- works perfectly!

Comment author: darius 20 May 2009 11:54:24PM 5 points [-]

A quick-and-dirty server-based webification (no Javascript): http://wry.me:7002/ with source code at http://github.com/darius/wason/ (only lightly tested).

Comment author: JGWeissman 21 May 2009 02:26:33AM *  0 points [-]

Nice work. It seemed responsive enough to me without client scripting. Unless people with slow connections report that the JavaScript version is faster, I think this is (the start of) the only version we need.

Some ideas for improvement:

  • Make 3 input boxes for the 3 elements of the triplet.

  • All results should include a short explanation of the Positive Bias, and an introduction to Less Wrong, including (the perfect result and "no rule" result).

  • It would be nice to make it StumbleUpon friendly, we can get a lot of hits from Stumblers. If it maintains the same URL at every stage, and provides different content based on the Post data, then StumbleUpon would not get confused by people thumbing up at different points in the test. As it is, the URL http://wry.me:7002/start returns an error when retrieved with a GET.

Comment author: darius 21 May 2009 09:32:15AM 1 point [-]

Thanks! I've addressed your latter two points (perhaps too minimally); though I think 3 input boxes would actually be less usable. I just made it accept both commas and spaces as delimiters instead.

If anyone would like to make a nice, pretty version, that should be easier now with the rule logic better-factored.

Comment author: darius 24 May 2009 01:53:09AM 3 points [-]

I just skimmed the results from it running the last few days at http://wry.me:7002/ -- it ran through to the end 49 times: 21 times the player got the right rule with no errors, 10 times they apparently got it right but with some errors, and 18 times they got it wrong. Of course I don't know who was playing or what they may have read just before.

Comment author: CronoDAS 20 May 2009 07:16:10PM 3 points [-]

Something slightly related:

Petals Around The Rose

Comment author: Vladimir_Nesov 20 May 2009 07:37:30PM *  -1 points [-]

Not quite, this game doesn't provide unlimited confirmation for your wrong guess.

Comment deleted 20 May 2009 08:59:56AM [-]
Comment author: MBlume 20 May 2009 09:19:50AM 1 point [-]

It reminds me distinctly of the skinner box experiment, which was essentially the same, but done with pigeons (who, of course, could not actually explain their procedures)

It also rather reminds me of actors avoiding macbeth, athletes wearing lucky garments, etc. etc.

Comment author: Apprentice 20 May 2009 01:19:07PM 4 points [-]

Nice! Is this a recruitment tool like the Scientology personality test? I guess it worked 'cause I just created an account :)

So, now that I've joined the cult - what's the major deity worshiped in these parts? I hear a lot about this Omega guy, maybe that's it. I also understand that salvation has something to do with lots and lots of liquid nitrogen. Oh, and some really cool robots!

Slightly more seriously, I think delivering this Wason test in a handy computer program which you can take with no-one watching you may increase the success rate. I could quickly test dozens of triplets and I tested any idea that came to my mind. In a supervised test where I'd had to ask the test administrator whether a given triplet was correct I might have been afraid to appear stupid by asking about overly many triplets or overly "silly" triplets or something like that.

(I'd personally never heard about this test before and ran the program without first reading the article - I tested 50 or so combinations and at one point crashed the program by feeding it floating point numbers. I got the rule right.)

Comment author: evtujo 23 May 2009 04:42:33AM 2 points [-]

I wonder if I'm the only person here so biased, but when you wrote that you used c++ my instant and curiously strong reaction was: why would you use c++ on something like that? that's obviously a python (or similar language) type project.

i'm working on forgiving you.

Comment author: MBlume 16 December 2011 07:37:25PM 4 points [-]

I learned C++ in high school and didn't make an effort to learn anything different in college or grad school. I now make a living as a python programmer, and if I had to write this again, I wouldn't consider C++ for half a second =)

Comment author: Bugmaster 16 December 2011 07:48:17PM 0 points [-]

I learned C++ in high school as well (before STL became widespread, get off my lawn), and regret it. I'd rather program in raw assembly than in C++. Ok, ok, in practical terms I use plain old C when I need some piece of code to be small and fast, but still. C++ is like the crazy uncle of programming. No one will come to your birthday party just because he might be there.

Comment author: Simetrical 20 May 2009 02:55:38PM 2 points [-]

I got the wrong rule, but it said I was right because I made only one mistake. I thought the rule was that a sequence was awesome if it was an increasing arithmetic progression. The only one of your examples at the end that contradicted this was 2, 9, 15. All the other awesome ones were, in fact, increasing arithmetic progressions: five out of the six awesome sequences you gave at the end. You should probably cut that down to two or three, so I'd have lost.

Comment author: AndrewKemendo 21 May 2009 12:08:30AM 1 point [-]

I think the game primes you to make the bias through the example:

I'll tell you for free that 2 4 6 is an awesome triplet.

So the results may be significantly skewed

Comment author: Vladimir_Nesov 21 May 2009 12:44:15AM *  2 points [-]

That's kinda the point. And then you get unlimited confirmation for the wrong idea you got from this example, unless you are clever enough to perform negative tests too.

Comment author: AndrewKemendo 21 May 2009 08:35:16PM 0 points [-]

So then this seems like the test would then be more priming centered rather than positively bias based. If it was truly a pure positive bias test then there should be no hypothesis primed through example.

I am learning to play GO right now and it is very hard to start to play effectively because there are very few instructional guides which give you strategies. Rather, each guide simply explicates the rules and lets you free to make your own hypotheses. Such an example would be a better bias test in my opinion. This is a simple fix, eliminate the example, and may give a more accurate result.

Comment deleted 21 May 2009 01:49:28AM [-]
Comment author: Craig_Morgan 21 May 2009 05:05:37AM *  3 points [-]

It is absolutely NOT a trick question.

There are an infinite number of hypotheses for what an 'Awesome Triplet' could be. Here are some example hypotheses that could be true based on our initial evidence '2 4 6 is an awesome triplet':
1. Any three integers
2. Any three integers in ascending order
3. Three successive multiples of the same number
4. The sequence '2 4 6'
5. Three integers not contained in the set '512 231123 691 9834 91238 1'

We cannot falsify every possible hypothesis, so we need a strategy to falsify hypotheses, starting from the most likely. All hypotheses are not created equal.

I want to falsify as much of the hypotheses-space as possible (where simple hypthoses take up more space), so I design a test that should do so. My first test was '3 integers in descending order', because it can falsify #1, the simplest hypothesis. I find from this test that #1 is false. My second test is to distinguish between #2 & #3; '3 integers in ascending order, but not successive multiples of the same number', '1 2 5' I find from this test that #2 is still plausible, but #3 is falsified.

You can continue falsifying smaller and smaller areas of the hypothesis-space with additional tests, up until you're happy with your confidence level or you're bored of testing.

For much better coverage of this entire area, see the following posts by Eliezer:
* What is Evidence?
* The Lens That Sees Its Flaws
* How Much Evidence Does It Take?
* Occam's Razor

For a good overview of additional related posts, see the list.

Edit: Learning Markdown, fixing style.

Comment author: MichaelBishop 20 May 2009 03:17:32PM *  1 point [-]

If the experimental subjects aren't taking their task seriously the results are unsurprising. I would think more of this experiment if it has been repeated with financial incentives for getting the answer right. Its also quite possible that people would do better if the task were less abstract... the type of problem solving they might do in real life.

It seems a reference to http://en.wikipedia.org/wiki/Confirmation_bias is in order here.

Comment author: Nominull 20 May 2009 01:23:54AM 1 point [-]

Program could use some protection against invalid input, along the lines of

if(!(cin >> n1 >> n2 >> n3)) { ndone=false; }

and if you really wanted to get fancy you could throw in an error message when it occurs

it's not nice to have a program that goes into an infinite loop if you give it a letter instead of a number

Comment author: MBlume 20 May 2009 01:29:48AM *  0 points [-]

wow, I didn't know iostream was that badly broken.

Fixed, though it now aborts completely if you give it a letter.

Comment author: CronoDAS 20 May 2009 04:33:34AM 0 points [-]

It could be worse. The C++ keyboard console input libraries are, indeed, horrible - but you could be stuck trying to do the same thing in plain old C. ::shudders::

Comment author: Unnamed 20 May 2009 06:18:37PM -2 points [-]

This phenomenon is more commonly known as "confirmation bias" - I don't think I've ever seen anyone call it "positive bias" besides Eliezer (and those who got the term from him), although it might be out there somewhere. I'd recommend at least including the term "confirmation bias", if not replacing "positive bias."

Comment author: JGWeissman 21 May 2009 01:56:01AM 7 points [-]

The Confirmation Bias refers to the problem of searching for, and therefor finding, only evidence that supports your preferred conclusion. This is likely to occur when using anecdotal evidence.

The Positive Bias on the other hand, refers to the problem of testing your beliefs about what your theory allows, but not about what it disallows. In terms of formal logic, two predicates P (corresponding to your theory of awesome triplets) and Q (corresponding to the actual rule) are logically equivalent if and only if for all x in the domain, P(x) implies Q(x) and Q(x) implies P(x). The Positive Bias is only testing samples of the form "P(x) implies Q(x)" but not of the form "Q(x) implies P(x)", or equivalently "not P(x) implies not Q(x)". This bias is likely to occur when designing an experiment.

Comment author: John_Maxwell_IV 20 May 2009 11:36:10PM 0 points [-]

I tried the macintosh executable. Double-clicking did nothing except bring the Terminal application to the front. I attempted to execute it from Terminal and was told I didn't have execute privileges. I added the execute privilege, attempted to execute again, and got a segmentation fault.

Comment author: arthurlewis 24 May 2009 06:30:59AM 0 points [-]

It's an Intel binary; perhaps you're on PowerPC?

Comment author: John_Maxwell_IV 24 May 2009 07:58:02PM 0 points [-]

No, I'm on intel.

It's not that important; the html/javascript version worked.

Comment author: lavalamp 20 May 2009 09:04:10PM 0 points [-]

4 out of 5 people fail this? Seriously? I'm having a really hard time believing that, I'm going to have to administer this test to people.

Visual Studio 2005 reports stack corruption after it exits, btw.

Comment author: darius 20 May 2009 11:10:36PM 0 points [-]

I believe that's because line 23 should be

bool mywrongtests [NUMTESTS+1];

(It's missing the "+1".)

Comment author: AllanCrossman 19 May 2009 10:01:49PM *  0 points [-]

Failed to compile for me until I removed a linebreak on line 120...

Comment author: MBlume 19 May 2009 10:04:24PM 0 points [-]

thanks -- fixed =)

Comment author: SoullessAutomaton 19 May 2009 09:54:34PM 0 points [-]

A web-based front end would be easy enough if written using an appropriate language and framework--something simple in PHP or whatnot. A CGI program in C++ would likely involve unnecessary amounts of wheel-reinventing.

It would be nice if there were more than just a single rule. How much work do you think it would be to come up with new rules?

Also, have you heard of the game Zendo?

Comment author: cousin_it 19 May 2009 11:04:11PM *  2 points [-]

Absolutely no need for PHP, CGI, frameworks or whatever. The whole thing can be written purely client-side in one file of HTML and JavaScript. Couple hours' project for a competent person with Notepad.

Comment author: JGWeissman 19 May 2009 11:16:59PM *  1 point [-]

I use Firefox with NoScript and StumbleUpon. If I stumble a site that does not show any content because it needs javascript, I do not white list it (I do not need to see the dancing pigs), I just go on to the next stumble.

An implementation that does not need client side scripting could reach a larger audience.

Comment author: SoullessAutomaton 19 May 2009 11:25:50PM 1 point [-]

The "NoScript" demographic is relatively small, I suspect, but potentially has a lot of overlap with people who might be interested in this.

Comment author: taw 20 May 2009 09:51:39AM -2 points [-]

I'm all for ignoring the NoScript demographic - JavaScript is one of the basic building blocks of the web, and turning it off is just totally stupid.

Comment author: thomblake 20 May 2009 05:23:44PM 6 points [-]

Turning it off is not "just totally stupid". For one thing, scripts are sometimes annoying, and most of the web can still be successfully navigated without bothering with them. Turning scripts off can greatly decrease site load times, browser overhead, and network traffic.

For another, you don't know what sort of user agent the customer is using - there are a lot of browsers for particular accessibility issues or other purposes that do not evenly support Javascript.

Supporting lack of javascript is just plain best practice, based on the principle of progressive enhancement.

Comment author: JGWeissman 20 May 2009 06:18:00PM 3 points [-]

Are you aware that with JavaScript, I can get your browser to submit a form to another site you are logged in to, that foolishly only validates sessions by cookies, and the other site will think you wanted to do that, and you would never know it happened?

Knowing this, would you still consider those who don't want JavaScript enabled when visiting sites they don't explicitly trust to be stupid?

Comment author: whpearson 20 May 2009 09:23:33PM *  0 points [-]

Cross site scripting is the main vulnerability caused by javascript, not request forgery.

I use noscript. It helps disable annoying ads.

Comment author: JGWeissman 20 May 2009 09:42:08PM 3 points [-]

Cross Site Scripting is the main vulnerability caused by javascript hosted on trusted sites. The problem there is that a hacker has managed to inject javascript onto the page (perhaps by submitting a comment that the server fails to encode for HTML) allowing the script to gather information provided in the version of the page associated with my login, and send that information to the hacker's server. NoScript does provide some protection against this, but this is by suppressing the methods the injected script uses to report back to the hostile server. It only helps by disabling JavaScript when the injected attack loads a script from a (non white listed) hostile domain.

Cross Site Forgery, on the other hand, is an attack by JavaScript hosted on a random untrusted site I visit, perhaps by Stumbling. NoScript provides excellent protection against this simply by turning of JavaScript.

Comment author: whpearson 20 May 2009 10:02:06PM *  3 points [-]

You can still have XSRF attacks without javascript though... so they are not caused by javascript but facilitated. Harder but doable, and you might protect yourself from some in the wild by turning off javascript.

Reading up about this a bit more it does appear that noscript does help against XSRF even those that don't use javascript, by changing post requests from untrusted sites to empty gets.

More details can be found on the noscript developers blog,

Editted: Made correction suggested by child post of JGWeismann,

Comment author: JGWeissman 20 May 2009 10:33:16PM 0 points [-]

Reading up about this a bit more it does appear that noscript does help against XSRF even those that don't use javascript, by changing post requests to untrusted sites to empty gets.

Neat. Though I think you meant "changing post requests from untrusted sites to trusted sites to empty gets", as would be expected to protect against Cross Site Forgery.

Comment author: taw 20 May 2009 08:59:29PM -1 points [-]

I agree with Vladimir Nesov - this is not relevant, and you seem to be using a soldier-argument. Anyway, validating write requests with cookies without auth tokens is plainly wrong, and there are trivial ways to do it right (automatically add auth token to all write forms, read-only forms like search boxes are not affected in any way by CSRF).

And you can do CSRF with "Press button to see kittens" trick, without a single line of Javascript. You can also read browser history with CSS, without Javascript. There aren't that terribly many security failures that you can do with Javascripts but cannot without, and in any case it's like using dynamite to deal with mosquitoes - completely disproportional reaction.

Yes, this is plain stupid.

Comment author: JGWeissman 20 May 2009 10:26:20PM 2 points [-]

Suppose Mallory has identified 100 websites that are vulnerable to high valued Cross Site Forgery attacks. (The administrators of these websites failed to use any of the trivial ways to prevent this. Some users, though they can't fix the servers, still want to protect themselves.) Is Mallory going to make a site that attempts to entice a user to click a button that apparently won't do anything, but will actually attempt an attack that will work if the user happens to have an account and be currently logged in to the one targeted site? Or is he going to create a site that has some interesting content that may even entice the user to come back later, that will silently run a script that attempts an attack on every vulnerable site Mallory knows, every time the user visits, without the user having to press any buttons?

You can also read browser history with CSS, without Javascript.

Good to know, though it would have been nice to actually provide a description or a link instead of making me search for it. Though, finding a weakness in my armor does not convince me to abandon my armor which still protects me against many threats. I will consider strengthening my armor. (Unfortunately, it appears the linked extension only supports Firefox 2, but I will see what else I can find.)

I agree with Vladimir Nesov - this is not relevant, and you seem to be using a soldier-argument.

Neither you nor Vladimir have even attempted to identify a single detail about Soldier-Arguments that describes my arguments. I really don't care about your unsupported opinions that an argument that you happen to disagree with has committed some fallacy.

Comment author: Vladimir_Nesov 20 May 2009 10:34:49PM *  0 points [-]

I really don't care about your unsupported opinions that an argument that you happen to disagree with has committed some fallacy.

Why, I agree with the argument. It just isn't a killer argument that determines the overall decision, and that is the problem with giving it.

Comment author: JGWeissman 21 May 2009 02:09:33AM 1 point [-]

It just isn't a killer argument that determines the overall decision, and that is the problem with giving it.

Are you serious? We should only ever present arguments that are powerful enough to convince everyone and conclude the discussion?

Comment author: Vladimir_Nesov 20 May 2009 07:18:15PM -1 points [-]

It seems you are using this fact as a soldier-argument. The position under discussion is that all things considered, turning scripting off seems to be a wrong decision. Of course there are potential problems, but at the same time, there are working solutions to these problems, and benefits from actually using the technology.

Comment author: thomblake 20 May 2009 07:32:45PM 3 points [-]

there are working solutions to these problems

Such as? The only working solutions I know of are server-side, or disallowing javascript and/or cookies.

The position under discussion is that all things considered, turning scripting off seems to be a wrong decision.

No, the position under discussion is that turning Javascript off is "just totally stupid". If one can provide good pro tanto reasons for doing so, it is at least not "just totally stupid".

Comment author: taw 20 May 2009 09:01:23PM 0 points [-]

Disallowing Javascript does NOT protect you against CSRF - "Press button to see kittens" form works without any Javascript. The right solution is server-side - auth tokens for all cookie-validated write forms.

Comment author: Douglas_Knight 20 May 2009 05:45:43PM 0 points [-]

It's NoScript+interested+StumbleUpon. Is StumbleUpon a good way of promoting things? I bet that there is negative correlation between NoScript & StumbleUpon.

Anyhow, JGW's first claim, that it is a terrible failure mode to produce a blank screen in the absence of js, is true and very cheap to fix.

Comment author: pjeby 20 May 2009 08:00:39PM 4 points [-]

I bet that there is negative correlation between NoScript & StumbleUpon.

From StumbleUpon's advertiser FAQ:

A large portion of our Firefox users have added the NoScript add-on to their browser. This is one of the top-10 most popular extensions for Firefox. This extension blocks any javascript calls that the user doesn't approve of. Blocking javascript causes Urchin, Google Analytics, Webtrends, etc to not work. These tracking services never see the traffic because NoScipt blocks it.

You may see a larger difference in reported traffic than with other advertising services because StumbleUpon has a much higher proportion of Firefox users -- and in particular, a much larger proportion of people who use browser extensions -- than a typical audience. In essence, StumbleUpon's early-adopter user-base is much more privacy and security conscious than the typical internet user, and they block tracking services as a result.

Comment author: Alicorn 20 May 2009 06:05:03PM 0 points [-]

Nitpick: Such a negative correlation between two Firefox extensions probably only persists in the set of Firefox users, not over Internet users at large, since (inexplicably) Firefox is not overwhelmingly popular among Internet users in general. (It's possible that it is overwhelmingly popular in the set of people interested in this program, though.)

Comment author: Simetrical 20 May 2009 02:57:48PM 0 points [-]

Doing this with server side scripting is crazy. You'd have to submit a zillion forms and take a second to get the answer for each try. This is precisely the sort of thing client-side scripting is meant for.

Of course, the page would explain that it needed JavaScript, if you had JavaScript disabled, not just show a blank page.

Comment author: JGWeissman 20 May 2009 05:08:49PM 1 point [-]

Doing this with server side scripting is crazy.

Porting the exact behavior of a command line application to a GUI environment like a web page is crazy. Reorganizing the content into a form that fits a new environment makes more sense. For example, the questions to test understanding of awesome sequences can all be part of one form, like in my javascript implementation.

You'd have to submit a zillion forms and take a second to get the answer for each try.

By "a zillion", do you mean "tens of"? And a second for the form is a lot shorter than the infinite time it takes on a browser that does not support scripting.

This is precisely the sort of thing client-side scripting is meant for.

No, this is precisely the sort of thing client-side scripting is meant to do better on browsers that support it, without interfering with the good enough implementation on browsers that don't. It most definitely is not meant for the sole means of delivering content that is intended for a wide audience.

Of course, the page would explain that it needed JavaScript, if you had JavaScript disabled, not just show a blank page.

I know I get annoyed quickly when I am asked to enable JavaScript for a purpose that should not need it. It would be better if we could explain that the page will be more responsive with JavaScript.

Comment author: Alicorn 19 May 2009 10:07:13PM *  1 point [-]

Or the card game Mao.

(Does anyone know the proper way to link to a URL that ends in a parenthesis?

Edit: Fixed, thanks.)

Comment author: dclayh 19 May 2009 10:15:41PM *  2 points [-]

Escape the first close-parenthesis with a backslash.

Comment author: MichaelHoward 20 May 2009 12:37:11PM 1 point [-]

Or the chess variant Penultima.

Comment author: steven0461 20 May 2009 02:17:29PM *  0 points [-]

All these games seem interesting, but more complicated than strictly necessary and plagued by the (possibly unavoidable) problem that guessing someone's mind involves different skills/knowledge than guessing mindless laws of nature. Is there a game that captures induction more cleanly, so we can expect skill at that game to generalize better?

Comment author: gwern 23 May 2009 05:36:31PM 2 points [-]

I have been, off and on, working on a Haskell implementation of Zendo. The idea is to implement just a subset: the human as player and the program as the Master (ie. the human trying to guess the rule).

The first question one naturally needs to know is: how do you generate rules? My attempt is to have a small set of building blocks which express simple propositions - 'all', 'none', 'even, 'odd', 'ascendingBy' etc. (and the numbers 1-10), and to generate a random list; that done, one can create random triplets of integers (via QuickCheck) and present the user only those triplets that satisfy the formula.

This solves your problem: the player can be told exactly what vocabulary the rule is written in. Another nice thing about having a simplified logic for propositions is that the formulas are data, but can be turned into code if need be, and it offers an obvious way to increase the difficulty: easy is as above; medium adds other predicates to the language (perhaps one could increase the numbers to 1-1000, and include predicates for 'isPrime'/'isComposite'); and so on.

(Before anyone gets too impressed, I don't have any running code yet; I got bogged down into figuring out how to use GADTs to turn my data constructors into code. And if you're wondering why no plans to have the computer guess the human's rule - that's because it's a hard problem. It's much easier to generate a random rule and then run triplets past it then it is to generate rules about a set bunch of triplets.)

Comment author: anonym 23 May 2009 05:53:18PM *  0 points [-]

Release early and release often. I can't wait to try your app. I've been wanting to play zendo for a long time, but wasn't willing to pay for all the sets of pieces that it seems are required now that zendo is out of print.

In terms of generating rules, you might want to look at CopyCat and its algorithms for analogy solving. There is an excellent discussion of the strategies it uses in the author's recent Complexity: A Guided Tour. You'll certainly get lots of good ideas for rule generation by looking at the discussion in that book.

Comment author: gwern 23 May 2009 07:57:54PM *  0 points [-]

CopyCat does look interesting. I note that:

"Since the 1995 FARG book, work on Copycat-like models has continued: as of 2008 the latest models are Phaeaco (a Bongard problem solver), SeqSee (number sequence extrapolation), George (geometric exploration), and Musicat (a melodic expectation model)."

3D Zendo is basically a variant on Bongard problems, and if a program can extrapolate numbers, then it could also test them against the human oracle to see if it's right.

Comment author: SoullessAutomaton 23 May 2009 08:07:36PM 0 points [-]

As an aside, given all the GEB love around here, it's worth noting that CopyCat was originally developed by Douglas Hofstadter and a student of his, the latter being the author of the book linked.

Comment author: anonym 24 May 2009 02:25:07AM 0 points [-]

Eliezer has mentioned CopyCat many times, so I figured that Mitchell and the relation to the FARGonauts was redundant information. On the topic of Mitchell and books, I also recommend An Introduction to Genetic Algorithms.

Comment author: SoullessAutomaton 24 May 2009 02:51:24AM 0 points [-]

I don't recall seeing CopyCat mentioned on OB. Has he mentioned it elsewhere, perhaps?

Not everyone here is familiar with Eliezer's stuff outside OB/LW and an awareness that SIAI is his "day job".

Comment author: anonym 25 May 2009 01:41:35AM 0 points [-]

You're right, I stand corrected. I could have sworn I remembered him mentioning CopyCat in OB before, but I can't find any now, only in other essays like "General Intelligence and Seed AI" and "The Plan to Singularity".

Comment author: Mark_Neznansky 20 May 2009 02:44:13PM 0 points [-]

What's the difference between one's mind laws and mindless "natural" laws?

Comment author: thomblake 20 May 2009 05:26:36PM 1 point [-]

You just pointed it out. The difference between "mind" and "mindless". If a human is guessing, there are different techniques for determining things thought up by another human than things not thought up at all.

(caveat: anthropic argument)

Comment author: Mark_Neznansky 07 June 2009 09:31:29PM 0 points [-]

So it's not really about the laws themselves (being "mindless" or "mind") as it's the context in which the guessing/researching is done. Guessing a a natural law known by a person in front of you is different than discovering it anew by yourself.