Jack comments on But Somebody Would Have Noticed - Less Wrong

36 Post author: Alicorn 04 May 2010 06:56PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (250)

You are viewing a single comment's thread. Show more comments above.

Comment author: SilasBarta 07 May 2010 03:58:54PM *  6 points [-]

I must confess, I'm a bit disturbed by how Alicorn's post continues to be voted up after its promotion. It is an overbroad criticism of the "Would someone have noticed?" heuristic which, as Tehom and timtyler point out, is actually very useful.

The fact that Alicorn has identified an uncommon, bizarre failure mode in the heuristic's use, where such a failure mode results from a very naive application of it, is not a reason to be suspicious of it in general and seems to reflect more of a negative affect Alicorn has developed toward those words than any serious shortcoming in asking, "Would someone have noticed?"

I don't say this to insult Alicorn -- no, really, I don't -- because I've been in the position of certain phrases becoming tainted in my mind because of their frequent misuse. I just want to distinguish between this kind of rejection and one grounded in demonstrable failure of a heuristic.

The test of a heuristic is its average performance, not its worst-case performance.

Comment author: Jack 07 May 2010 04:33:04PM 1 point [-]

I wouldn't say I'm disturbed. But I am confused.

It is an overbroad criticism of the "Would someone have noticed?" heuristic which, as Tehom and timtyler point out, is actually very useful.

I took myself to be making the same kind of point here though in a bit of a round-a-bout and indirect way. All of these criticisms were heavily voted up, as well. I wonder if front page posts have a de facto karma floor in the high twenties just because they get more traffic than posts that aren't promoted. Aside from the occasional work of brilliance and the special threads almost every promoted post has a karma total between 25 and 33. I think the promotion system probably needs more scrutiny or at least we need a way of distinguishing "Promoted for discussion purposes" and "Promoted for truth".

Comment author: komponisto 07 May 2010 04:52:59PM 1 point [-]

It seems to me that posts are pretty much automatically promoted once they reach 20 or so; some posts are promoted before then, leading one to infer that the editor thinks especially highly of them. (Others, by contrast, seem to be promoted only with considerable reluctance; although it might just mean the editor wasn't paying attention.)

Comment author: SilasBarta 07 May 2010 05:08:29PM 1 point [-]

IIRC, this post was at 9 on promotion :-[

Comment author: komponisto 07 May 2010 05:39:38PM *  4 points [-]

That's a bit surprising, but in any case it seems like a decent post to me; I don't think the current score of 25 is excessive.

(And there have been some excessive scores recently. E.g. Yvain's post on excuses -- it was a fine post, to be sure, and I'm a big Yvain fan, but... 97?? Really? I would have put it at 30-40.)

Comment author: Morendil 07 May 2010 05:58:45PM 3 points [-]

I've long settled on interpreting the meaning of upvotes as "I like this post and want to see more like this".

I vote on posts before knowing who authored them or what their current score is, using the Anti-Kibitz script. This is because I've become more aware of my own bias as a result of reading LW, which I believe was the intended result. (I liked Yvain's post and voted it up, but not because I'm a "fan", just because I thought it'd be nice to have more posts like it.)

After I vote a post up, I turn off the script to see who it was from. If I thought they deserved an upvote in the first place, my vote still means the same, and it's natural to wish that my vote aggregates with others' in giving the author feedback about their post. So, I don't as a rule go back on a vote once I've given it.

So it kind of puzzles me why you seem to think there should be some kind of "vote ceiling", or why you expect that your own evaluation of a post should be a good indicator of how others like it. What I'm saying, I guess, is that I don't get the point of your parenthetical.

What would you want us to adopt as a voting norm?

Comment author: thomblake 07 May 2010 06:43:35PM *  2 points [-]

I've long settled on interpreting the meaning of upvotes as "I like this post and want to see more like this".

I agree, though I still intuitively get "This post was worth more points" or "97 points? it was only as good as this other post, which has 30 points".

So it kind of puzzles me why you ... expect that your own evaluation of a post should be a good indicator of how others like it.

Really? That seems like a completely natural expectation to me. Like, I like strawberries dipped in chocolate, so I would assume (with no other info) that a random person would like strawberries dipped in chocolate. We are far more alike than not.

Comment author: komponisto 07 May 2010 07:23:39PM *  1 point [-]

I liked Yvain's post and voted it up, but not because I'm a "fan",

Cheap shot detected here. I said I was a fan in order to soften the effect of saying that the post was overrated; without that disclaimer, my statement might have been interpreted as a criticism of Yvain or his post. Nothing I said implies that I make a habit of upvoting posts just because of who their author is.

What I'm saying, I guess, is that I don't get the point of your parenthetical.

The point was that I don't think that that post was as as outstanding relative to other posts as its score suggests.

I've long settled on interpreting the meaning of upvotes as "I like this post and want to see more like this".

What would you want us to adopt as a voting norm?

That's fine as a voting norm. Under that norm, the proper interpretation of my remark is that my eagerness to see more posts like Yvain's "Eight short studies on excuses" is comparable to my eagerness to see more posts like those with scores in the 30-40 range; in particular, the first quantity is not 2-3 times the second.

Comment author: Morendil 07 May 2010 08:10:46PM 1 point [-]

Yes, and for that reason it may not be correct to interpret the score of a post as the "collective eagerness" to see more posts like it, and therefore not entirely appropriate to draw the kind of comparison you're drawing.

Unless people upvote Yvain's articles merely because they are Yvain's (which was what I thought you were getting at, and all I was getting at, with the term "fan"), then we want to interpret high scores as marking posts that have broad appeal, rather than posts which have intense appeal.

Not, "people liked Studies On Excuses almost as much as they liked Generalizing from One Example", but "almost as many people liked Studies as liked Generalizing". It makes a difference to me to think of it that way, not sure if it will to you...

Comment author: komponisto 07 May 2010 08:40:02PM 2 points [-]

If post X has a score strictly less than post Y, then it follows that there are either people who upvoted Y and did not upvote X, or people who downvoted X and did not downvote Y. If I think the score of X should be equal to the score of Y, then I am disagreeing with the voting behavior of the persons in those sets, at least one of which (as I said) is nonempty.

Comment deleted 09 May 2010 10:24:31AM [-]
Comment author: jimrandomh 07 May 2010 05:05:13PM 0 points [-]

The algorithm is more complicated than that. I don't recall the exact details, but I'm pretty sure it includes the rate of upvotes, not just the number of them. And while it can be overriden by moderators, I doubt that they're doing that very often.

Comment author: jimrandomh 07 May 2010 05:45:10PM 1 point [-]

I just checked, and there is in fact no such auto-promote feature in the code base. I was misremembering a post in which Eliezer talked about it being planned, but apparently it never happened.

Comment author: Alicorn 07 May 2010 05:23:51PM 1 point [-]

Eliezer promotes posts by hand. If he likes them and they have a reasonable number of upvotes, they go up faster. If he doesn't like them, they need more votes before he'll promote them. If he doesn't see them for a while, they'll take longer to be promoted.

Comment author: komponisto 07 May 2010 05:29:40PM *  0 points [-]

That's exactly what I thought. (And I assume your source for this information is Eliezer, making it very likely to be correct!)

Comment author: komponisto 07 May 2010 05:26:55PM *  0 points [-]

I didn't realize promotion was automated; I thought editors (meaning basically EY) did it manually.

Comment author: Jack 07 May 2010 05:20:21PM 0 points [-]

The algorithm really ought to be public.

Comment author: Morendil 07 May 2010 05:22:34PM 0 points [-]

If there is such an algorithm in the codebase that's published on github, it shouldn't be too hard to find.

Comment author: SilasBarta 07 May 2010 04:38:49PM *  0 points [-]

Couldn't have said it better myself.

Maybe we should do something like: require promotion to penalize the user 50 karma if the post doesn't get at least 20 net upvotes? (I'm guessing this one of mine would have gotten more than 5 additional net upvotes if it had been promoted...)