Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Vaniver comments on LW 2.0 Strategic Overview - Less Wrong

47 Post author: Habryka 15 September 2017 03:00AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (294)

You are viewing a single comment's thread. Show more comments above.

Comment author: Vaniver 16 September 2017 01:42:41AM 2 points [-]

My impression of how to do this is to give experts an "as an expert, I..." vote. So you could see that a post has 5 upvotes and a beaker downvote, and say "hmm, the scientist thinks this is bad and other people think it's good."

Multiple flavors lets you separate out different parts of the comment in a way that's meaningfully distinct from the Slashdot-style "everyone can pick a descriptor;" you don't want everyone to be able to say "that's funny," just the comedians.

This works somewhat better than simple vote weighting because it lets people say whether they're doing this as just another reader or 'in their professional capacity;' I want Ilya's votes on stats comments to be very highly weighted and I want his votes on, say, rationality quotes to be weighted roughly like anyone else's.

Of course, this sketch has many problems of its own. As written, I lumped many different forms of expertise into "scientist," and you're trusting the user to vote in the right contexts.

Comment author: SaidAchmiz 16 September 2017 04:01:04AM 3 points [-]

If you have a more-legible quality signal (in the James C. Scott sense of "legibility"), and a less-legible quality signal, you will inevitably end up using the more-legible quality signal more, and the less-legible one will be ignored—even if the less-legible one is tremendously more accurate and valuable.

Your suggestion is not implausible on its face, but the devil is in the details. No doubt you know this, as you say "this sketch has many problems of its own". But these details and problems conspire to make such a formalized version of the "expert's vote" either substantially decoupled from what it's supposed to represent, or not nearly as legible as the simple "people's vote". In the former case, what's the point? In the latter case, the result is that the "people's vote" will remain much more influential on visibility, ranking, inclusion in canon, contribution to a member's influence in various ways, and everything else you might care to use such formalized rating numbers for.

The question of reputation, and of whose opinion to trust and value, is a deep and fundamental one. I don't say it's impossible to algorithmize, but if possible, it is surely quite difficult. And simple karma (based on unweighted votes) is, I think, a step in the wrong direction.

Comment author: ingres 16 September 2017 04:17:28AM 1 point [-]

As far as an algorithm for reputation goes, academia seems to have something that sort of scales in the form of citations and co-authors:

http://www.overcomingbias.com/2017/08/the-problem-with-prestige.html

It's certainly a difficult problem however.

Comment author: IlyaShpitser 17 September 2017 02:05:36AM *  0 points [-]

Vaniver, I sympathize with the desire to automate figuring out who experts are via point systems, but consider that even in academia (with a built-in citation pagerank), people still rely on names. That's evidence about pagerank systems not being great on their own. People game the hell out of citations.


Probably should weigh my opinion of rationality stuff quite low, I am neither a practitioner nor a historian of rationality. I have gotten gradually more pessimistic about the whole project.

Comment author: Vaniver 19 September 2017 06:34:40PM 0 points [-]

Vaniver, I sympathize with the desire to automate figuring out who experts are via point systems

To be clear, in this scheme whether or not someone had access to the expert votes would be set by hand.

Comment author: Lumifer 19 September 2017 07:42:31PM 0 points [-]

What is going to be the definition of "an expert" in LW 2.0?

Comment author: gjm 19 September 2017 10:26:35PM 0 points [-]

From context, it's clearly (conditional on the feature being there at all) "someone accepted by the administrators of the site as an expert". How they make that determination would be up to them; I would hope that (again, conditional on the thing happening at all) they would err on the side of caution and accept people as experts only in cases where few reasonable people would disagree.

Comment author: Lumifer 20 September 2017 12:28:23AM 0 points [-]

"All animals are equal... " X-)

The issue is credibility.

Comment author: John_Maxwell_IV 18 September 2017 06:19:19AM 0 points [-]

People game the hell out of citations.

Is there anyone whose makes it their business to guard against this?

Comment author: IlyaShpitser 18 September 2017 11:30:13PM 0 points [-]

Academics make it their business, and they rely on name recognition and social networks.