= 27d567bc2d48d9f40e9ccc20ea370b15)
BenAlbahari comments on Bayesian Collaborative Filtering - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (23)
I'm the founder of TakeOnIt, so let me add a little here.
JGWeissman's algorithm, or an evolution of it based on the feedback here, will be replacing the crude algorithm that's already online. You can see the results of that algorithm here:
Predicting Eliezer Yudkowsky's Opinions
If anyone else needs access to the database let me know.
Part of the vision here is to help people choose their beliefs in areas where they don't have domain expertise. This concept was described here.
In addition, this technique can also be used to detect correct contrarian opinions, as described here. The algorithm will predict that person S should believe in a minority opinion I if S has similar opinions to the set of people T, where T hold the opinion I.
An extremely minor quibble with your site (I'm just trying to be helpful):
You ask for an opinion using a question: "Does god exist?", but then you allow people to provide answers as if they're agreeing or disagreeing with an affirmative statement (options: agree, neutral, disagree). The grammatical disconnect caused me some confusion when I first looked at the site. I think changing this to be either:
would make this more clear.
I haven't gotten to really read the above article yet, but do you think the proposed method would perform substantially better than a simple cluster analysis?
That's a good point. I've added it to the user voice feature suggestions:
http://takeonit.uservoice.com/forums/44305-general
Can you describe an algorithm that uses "simple cluster analysis" to predict what position a person S will take on an issue I?
There's a system (I think maintained by NASA) called AutoClass, which is fairly easy to use. As I understand it, it accepts input "points" (who here would be people) and outputs clusters of similar points (people).
In order to predict using AutoClass, I think you would model unanswered questions as "missing values", and then predict based on observed frequencies from the same cluster.
There's some ad-hoc-ish-ness about the way AutoClass decides how many clusters there should be, but it's a solid, existing technology that has been used successfully in many applications.
If a collaborative filter algorithm is accurate, that's all that really matters to the consumers of the algorithm. It's primarily the designers of the algorithm who care about the scientific basis as to why the algorithm works.
A decent overview of the various CF algorithms:
http://www.hindawi.com/journals/aai/2009/421425.html
I find it both amusing and disturbing how pioneers in this field have been trying to optimize guessing movie preferences (recall the famous Netflix $1 million prize), when we can use these techniques to actually predict stuff that IMBO "really matters". It's yet another interesting data point as to what people really care about.
Rather, it's a reminder that more effort is spent on projects that can be immediately profitable, however trivial they may be in scope. If there were a pay market for intellectual content as robust as the one for movies currently is, we'd have seen this done already. (Alas, I don't see how that could happen in the near future.)
I've been considering the question: why has using Collaborative Filtering to predict the "trivial" opinions about movies been prioritized over predicting the "important" opinions about political/social/economic/etc. issues?
On reflection, I don't actually think it's because people care more about the former than the latter. Would you rather have a prediction regarding the opinion as to whether the movie Titanic was good, or a prediction regarding the opinion as to whether there's a housing bubble?
I think the answer is that opinions about products are naturally schematized, and hence easy to collate. Products are already tracked everywhere in databases, so it's pretty easy to extend that model to add opinions about those products. In contrast, opinions about issues, although often even more passionate than opinions about products, are not as naturally schematizable, hence they're harder to collate. Even in terms of representing the identity of an issue, it's not like we have the equivalent of an ISBN number for each issue. So opinions about issues are not adequately schematized and hence we can't collate those opinions into the nice big datasets we'd want to make predictions. Obviously websites like TakeOnIt are trying to change that. Each question ID is analogous to an ISBN number for that issue, if you will.
Yes, I agree with you that opinions of products can help sell products, so predicting opinions about products has the incentive of an immediately obvious monetization strategy. But if there's money to be made depending on correctly predicting the answer to a question, then the potential for monetization of the prediction of those opinions is also there.
Well there's a subset of questions on TakeOnIt where the correct answer has a financial reward/impact. An example of such a question was Is there a housing bubble in the United States?. These type of questions overlap with the kind of questions seen on prediction markets (which is a nice model for monetizing intellectual content). I'd be curious as to the relative accuracy between prediction markets and using collaborative filtering on expert predictions.
I had vaguely what Jonicholas mentions in mind.
I've finally taken a look at this site. I'm strongly tempted to add H.P. Lovecraft quotes to many issues, but of course he's not an expert in the relevant senses, and the easily findable quotes are from fiction which should not generally be taken as his own.
Based on your idea and the discussion that followed, I've added the feature to flag a quote as fictional.
On the question pages, fictional quotes are put in their own group:
Is information-theoretic death the most real interpretation of death?
On the expert pages, fictional quotes are flagged per quote:
H.P.Lovercraft's Opinions
Fictional quotes are discounted from the prediction analysis.
Thinking out loud (this could be a terrible idea, "green hat" thinking alert!): I wonder if it would be interesting to be able to tag a quote as "fiction". There's so many insightful quotes that are spoken through the fictional characters of great authors. It seems a shame that such quotes are "illegitimate". Better to perhaps allow the quotes but tag them appropriately so they can be filtered out of prediction analysis. Thoughts?
How would they be attributed? Valentine Michael's opinions are substantially different from Lazarus Long's.
Simple model: a flag on a quote, present if it's a fictional character, with text preceding the quote explaining the source.
Complex model: Each fictional character is on par with an expert/influencer, with an extra field referencing back to the expert/influencer who's the author. E.g. you could look up all the quotes of "Sherlock Holmes" or all the fictional quotes of characters written by Arthur Conan Doyle.
It seems worth trying, if you want to code it up. While it certainly doesn't make much sense to base predictions about others based on quite possibly incoherent groupings of characters, predicting the other way could be interesting.
But it does occur to me that I could just create user account and post them there, though that wouldn't let others add quotes.
I generally include the name of the character along with the rest.
That doesn't always work -- sometimes it's from an impersonal narrator.
Point - "Narrator", perhaps?
Fun Fact: My algorithm's most confident prediction about Ben is that with 67% probability, he Disagrees on the issue Is the unconscious philosophical zombie possible?.
(Ben has registered his position on 7 issues.)