Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Comment author: BlueSun 07 June 2013 06:36:54PM 1 point [-]

I was writing an article and trying to refer to www.worstargumentintheworld.com but it appears to be down. Is the registration still valid and/or going to be renewed?

Comment author: joshkaufman 20 June 2013 05:09:16PM 0 points [-]

Sorry for the downtime - transferred the domain to a new registrar, and thought the forward would be automatically detected and carried over. It wasn't. Should be back up once the record updates.

Comment author: prase 27 August 2012 07:41:33PM 6 points [-]

When in the discussion under the well-written page created by a third party the first party openly admits registering the domain in order to use it as argumentum ad verecundiam, the whole thing loses much of its power.

Comment author: joshkaufman 27 August 2012 08:24:13PM 1 point [-]

Anyone who visits this page can judge the merits themselves: there's no argument from authority involved. No one is claiming this form of argument is invalid because it's on LW, or because Yvain wrote it, or because it has a catchy name that's published on a website, or because it now has an easy-to-remember URL. I made a simpler citation, nothing more.

Comment author: wedrifid 27 August 2012 06:55:14AM 32 points [-]

I just registered http://worstargumentintheworld.com - it redirects to this post, and should be available shortly. Much easier to mention in conversation when other people use this argument, and don't believe it's a "real thing."

"Real things" have their own domain. I registered this domain, therefore...

Comment author: joshkaufman 27 August 2012 02:10:05PM *  22 points [-]

Hahaha, nice.

I was imagining a situation in which someone makes an argument of this type, you say something along the lines of "that's a great example of the 'Worst Argument in the World'," and the person replies "you just made that up..." or "that's just your opinion..."

Providing a pre-existing URL that links to a well-written page created by a third-party is a form of evidence that shifts "Worst Argument in the World" from something that feels like an opinion to the title of a logical fallacy. That can be quite useful in certain circumstances.

Comment author: joshkaufman 27 August 2012 06:11:34AM 54 points [-]

I just registered http://worstargumentintheworld.com - it redirects to this post, and should be available shortly. Much easier to mention in conversation when other people use this argument, and don't believe it's a "real thing."

Great piece of work, Yvain - it's now on my list of all-time favorite LW posts.

Comment author: Unnamed 11 November 2011 01:14:43AM 4 points [-]

It's not a Bayesian method, but you could use Randall Munroe's algorithm, probably adding on the assumption of transitivity for each participant except where it is explicitly violated. The way that works is:

Step 1: you say that participant 1 has A>B if 1) they explicitly ranked A>B, or if 2) they have a transitive path giving A>B and do not have a transitive path giving B>A. Repeat for each pair of alternatives for each participant. (note that a participant may have A=B)

Step 2: you say that the group has A>B if more participants have A>B than have B>A. Repeat for each pair of alternatives. (note that the group may have A=B)

Step 3: you count up how many alternatives A is preferred to by the group, and divide by that number plus the number of objects that are preferred to A. This is A's score. Repeat for each alternative.

If you only care about which object is most likely to be #1, and not any other ranks, then another method is to look for each participant only at the objects that are undefeated. Each object that was never rejected by participant 1 would get a score of 1/(# of undefeated objects), and the other objects would get a score of 0. Then you just sum the scores across participants. Or, you might want not want to give the same score to an object that the participant never saw as you do to one that they preferred three times, so you could weight by number of times selected; perhaps give the object a score of (1 + # of objects it was preferred to)/(sum numerators), where you divide by the sum to normalize each participant's scores to add to one.

Comment author: joshkaufman 11 November 2011 01:24:43AM *  1 point [-]

Wow, that's so simple it could possibly work!

Many thanks - this is most likely what I'll go with. I appreciate your help. :-)

Comment author: [deleted] 11 November 2011 12:42:45AM 4 points [-]

Even if every participant answered all 21 questions, and if every participant answers with respect to some total ordering, then this still only reduces to the problem of preferential voting. This is a really hard problem and I'm not aware of any voting system that's provably "correct" -- in fact, there are results that for some definitions of "correct", there are no correct voting systems period. There are various voting systems that satisfy some, but not all, reasonable properties you could name, and you should pick one.

"Ranked pairs" is one natural choice (I don't know how well it performs in practice, though) because it deals in pairs to begin with. Basically, for each pair (A,B) you tally how many participants picked A over B. You go through the pairs in order of how significant a majority they represent. In this case, you wouldn't want to use the usual method for deciding which majority is better, because you have to take into account the number of people who answered the question, too. There was a post on LW once which linked to an article about this (in the context of e.g. ratings of movies on IMDB, where a movie rated 10.0 by 5 users isn't as good as a movie rated 9.0 by 100 users). Anyway, the next thing you do with the pairs is lock them in one by one, except when it would create a cycle together with pairs that have already been locked in. If you do this, the "locked in" pairs will show the most preferred object.

Comment author: joshkaufman 11 November 2011 12:48:34AM 0 points [-]

Thanks - the voting system analogy didn't occur to me. Reading up on ranked pairs: http://en.wikipedia.org/wiki/Ranked_pairs

Comment author: dlthomas 11 November 2011 12:37:00AM 1 point [-]

I don't think I'm following you.

We see a new piece of evidence - one of the people prefers C to E

C will be preferred to E in half the lists. Those lists become more probable, the other half become less probable. How much more/less probable depends on how much error you expect to see and of what type.

Repeat on all the data.

You only actually look at the first member when asking the odds that a particular object is there - at which point, yes, you sum up the probability of those 720 sets.

Comment author: joshkaufman 11 November 2011 12:40:53AM 0 points [-]

Ah, I see. Instead of updating half the lists, I was updating the 720 sets where C is the #1 preference. Thanks for the clarification.

Comment author: VincentYu 10 November 2011 11:48:34PM *  1 point [-]

If the aggregated preferences are transitive (i.e., 'not inconsistent' in your and Manfred's wording), then this preference relation defines a total order on the objects, and there is a unique object that is preferred to every other object (in aggregate). (Furthermore, this is isomorphic to the set {1,2,3,...,7} under the ≤ relation.)

Comment author: joshkaufman 11 November 2011 12:00:01AM *  0 points [-]

Very helpful - reading about this now. Starting here: http://en.wikipedia.org/wiki/Ranking

Comment author: dlthomas 10 November 2011 11:14:12PM 1 point [-]

The alternate path would be to assume that there is a preference ordering from 1 to 7, with some sort of noise applied. This feels clunky to me, though.

While clunky, this seems to be a perfectly workable approach for 7 objects. There are 5040 permutations. For each piece of evidence, add probability mass to those that correspond and remove it from those that differ (perhaps weighted by how much they correspond or differ?). The probability of your object being most preferred, then, is the sum of the probabilities of those permutations in which it occupies the highest point.

Comment author: joshkaufman 10 November 2011 11:57:38PM *  0 points [-]

Okay, if A is preferred from { A , [B-G] }, that should add probability mass to [A, [B,...,G] ], where [A, [B,...,G] ] is a ranked set of objects where the first slot is most preferred. That would represent 720 (6!) sets out of 5040.

All other sets (7! - 6! = 4,320) should either stay the same probability or have probability mass removed.

Then, the probability of A being "most preferred" = the sum of the probability mass of all 720 sets that have A as the highest ranked member. Likewise for B through G. Highest total probability mass wins.

Am I understanding that correctly?

Comment author: dlthomas 10 November 2011 11:01:44PM 0 points [-]

"There are no priors" is not correct - there are necessarily priors. I think what you mean is "we start with no information about rankings" and so would assume perfect entropy - a 1:1 chance of either member of any pair being preferred.

Comment author: joshkaufman 10 November 2011 11:12:14PM *  1 point [-]

Right - thanks for the correction. Posted a correction in the main text.

View more: Next