In response to Linkposts now live!
Comment author: VipulNaik 28 September 2016 10:56:22PM *  4 points [-]

I'm unable to edit past posts of mine; it seems that this broke very recently and I'm wondering if it's related to the changes you made.

Specifically, when I click the Submit or the "Save and Continue" buttons after making an edit, it goes to lesswrong.com/submit with a blank screen. When I look at the HTTP error code it says it's a 404.

I also checked the post after that to see if the edit still went through, and it didn't. In other words, my edit did not get saved.

Do you know what's going on? There were a few corrections/expansions on past posts that I need to push live soon.

Comment author: ThisSpaceAvailable 21 August 2016 01:31:02AM *  4 points [-]

I suppose this might be better place to ask than trying to resurrect a previous thread:

What kind of statistics can Signal offer on prior cohorts? E.g. percentage with jobs, percentage with jobs in data science field, percentage with incomes over $100k, median income of graduates, mean income of graduates, mean income of employed graduates, etc.? And how do the different cohorts compare? (Those are just examples; I don't necessarily expect to get those exact answers, but it would be good to have some data and have it be presented in a manner that is at least partially resistant to cherry picking/massaging, etc.) Basically, what sort of evidence E does Signal have to offer, such that I should update towards it being effective, given both E, and "E has been selected by Signal, and Signal has an interest in choosing E to be as flattering rather than as informative as possible" are true?

Also, the last I heard, there was a deposit requirement. What's the refund policy on that?

Comment author: VipulNaik 22 August 2016 11:48:33PM *  0 points [-]

One relevant consideration in such an evaluation is that Signal's policies with respect to various things (like percentage of income taken, initial deposit, length of program) may have changed since the program's inception. Of course, the program itself has changed since it started. Therefore, feedback or experiences from students in initial cohorts needs to be viewed in that light.

Disclosure: I share an apartment with Jonah Sinick, co-founder of Signal. I have also talked extensively about Signal with Andrew J. Ho, one of its key team members, and somewhat less extensively with Bob Cordwell, the other co-founder. ETA: I also conducted a session on data science and machine learning engineering in the real world (drawing on my work experience) with Signal's third cohort on Saturday, August 20, 2016.

Comment author: gwern 15 July 2016 05:15:00PM 6 points [-]

Their motivation is public education & outreach:

Vipul and I ultimately want to get a better sense of the value of a Wikipedia pageview (one way to measure the impact of content creation), and one way to do this is to understand how people are using Wikipedia. As we focus on getting more people to work on editing Wikipedia – thus causing more people to read the content we pay and help to create – it becomes more important to understand what people are doing on the site.

This is a topic I've wondered about myself, as I occasionally spend substantial amounts of time trying to improve Wikipedia articles; most recently GCTA, liability threshold model, result-blind peer review, missing heritability problem, Tominaga Nakamoto, & debunking urban legends (Rutherford, Kelvin, Lardner, bicycle face, Feynman IQ, MtGox). Even though I've been editing WP since 2004, it can be deeply frustrating (look at the barf all over the result-blind peer review right now) and I'm never sure if it's worth the time.

Results:

  • most people in LW/SSC/WP/general college-educated SurveyMonkey population/Vipul Naik's social circles read WP regularly (with a skew to reading WP a huge amount), have some preference for it in search engines & sometimes search on WP directly, every few months is surprised by a gap in WP which could be filled (sounding like a long tail of BLPs and foreign material; the latter being an area that the English WP has always been weak in)

    • reading patterns in the total sample match aggregate page-view statistics fairly well; respondents tend to have read the most popular WP articles
  • they primarily skim articles; reading usage tends to be fairly superficial, with occasional use of citations or criticism sections but not any more detailed evaluation of the page or editing process

At face value, this suggests that WP editing may not be that great a use of time. Most people do not read the articles carefully, and aggregate traffic suggests that the sort of niche topics I write on is not reaching all the people one might hope. For example, take threshold models & GCTA traffic statistcs - 74/day and 35/day respectively, or maybe 39k page views a year total. (Assuming, of course, that my contributions don't get butchered.) This is not a lot in general - I get more like 1k page views a day on gwern.net. A blogpost making it to the front page of Hacker News frequently gets 20k+ page views within the first few days, for comparison.

I interpret this as implying that a case for WP editing can't be made based on just the traffic numbers. I may get 1k page views a day, but relatively little of that is to pages using GCTA or threshold models even in passing. It may be that writing those articles is highly effective because when someone does need to know about GCTA, they'll look it up on WP and read it carefully (even though they don't read most WP pages carefully), and over the years, it'll have a positive effect on the world that way. This is harder to quantify in a survey, since people will hardly remember what changed their beliefs (indeed, it sounds like most people find it hard to remember how they use WP at all, it's almost like asking how people use Google searches - it's so engrained).

My belief is that WP editing can have long-term effects like that, based primarily on my experiences editing Neon Genesis Evangelion and tracking down references and figuring out the historical context. I noticed that increasingly discussions of NGE online took on a much better informed hue, and in particular, the misguided obsession with the Christian & Kabbalic symbolism has died down a great deal, in part due to documenting staff quotes denying that the symbolism was important. On the downside, if you look through the edit history, you can see that a lot of terrific (and impeccably sourced) material I added to the article has been deleted over the years. So YMMV. Presumably working on scientific topics will be less risky.

Comment author: VipulNaik 17 July 2016 12:16:46AM *  3 points [-]

I think Issa might write a longer reply later, and also update the post with a summary section, but I just wanted to make a quick correction: the college-educated SurveyMonkey population we sampled in fact did not use Wikipedia a lot (in S2, CEYP had fewer heavy Wikipedia users than the general population).

It's worth noting that the general SurveyMonkey population as well as the college-educated SurveyMonkey population used Wikipedia very little, and one of our key findings was the extent to which usage is skewed to a small subset of the population that uses it heavily (although almost everybody has heard of it and used it at some point). Also, the responses to S1Q2 show that the general population rarely seeks Wikipedia actively, in contrast with the small subset of heavy users (including many SSC readers, people who filled my survey through Facebook).

Your summary of the post is an interesting take on it (and consistent with your perspective and goals) but the conclusions Issa and I drew (especially regarding short-term value) were somewhat different. In particular, both in terms of the quantity of traffic (over a reasonably long time horizon) and the quality and level of engagement with pages, Wikipedia does better than a lot of online content. Notably, it does best in terms of having sustained traffic, as opposed to a lot of "news" that trends for a while and then drops sharply (in marketing lingo, Wikipedia content is "evergreen").

Comment author: JonahSinick 12 April 2016 01:26:52AM *  1 point [-]

Hi Toggle,

Thanks for your question!

Most of our students have just started looking for jobs over the past ~2 weeks, and the job search process in the tech sector typically takes ~2 months, from sending out resumes to accepting offers (see, e.g. "Managing your time" in Alexei's post Maximizing Your Donations via a Job).

The feedback loop here is correspondingly longer than we'd like. We expect to have an answer to your question by the time we advertise our third cohort.

Comment author: VipulNaik 07 July 2016 05:51:45AM 1 point [-]

Following up!

Comment author: VipulNaik 16 April 2016 04:16:59PM 2 points [-]

[Comment cross-posted to the Effective Altruism Forum]

[I will use "Effective Altruists" or "EAs" to refer to the people who self-identify as members of the community, and "effective altruists" (without capitalization) for people to whom effectiveness matters a lot in altruism, regardless of whether they self-identify as EAs.]

I think this post makes some important and valuable points. Even if not novel, the concise summary here could make for a good WikiHow article on how to be a more effective fundraiser. However, I believe that this post falls short by failing to mention, let alone wrestle with, the tradeoffs involved with these strategies.

I don't believe there is a clear and obvious answer to the many tradeoffs involved with adopting various sales tactics that compromise epistemic value. I believe, however, that not even acknowledging these tradeoffs can lead to potentially worse decisions.

My points below overlap somewhat.

First, effective altruists in general, and EAs in particular, are a niche segment in the philanthropic community. The rules for selling to this niche can differ from the rules of selling to the general public. So much so that sales tactics that are considered good for the general public are actively considered bad when selling to this niche. Putting an identifiable victim may help with, say, 30% of potential donors in the general public, but alienate 80% of potential donors among effective altruists, because they have (implicitly or explicitly) learned to overcome the identifiable victim effect. In general, using messaging targeted at the public for a niche that is often based, implicitly or explicitly, on rejecting various aspects of such messaging, is a bad thing. A politician does not benefit from taking positions held by the majority of people all the time; rather, whereas some politicians are majoritarian moderates, others seek specific niches where their support is strong, often with the alienation of a majority as a clear consequence (for instance, a politician in one subregion of a country may adopt rhetoric and policies that make the politician unpopular countrywide but guarantee re-election in that subregion). Similarly, not every social network benefits from adopting Facebook's approach to partial openness and diversity of forms of expression. Snapchat, Pinterest, and Twitter have each carved a niche based on special features they have.

Second, in addition to the effect in rhetorical terms, it's also important to consider the effect in substantive terms on how the organizations involved spend their money and resources, and make decisions. Ideally, you can imagine a wall of separation: the organization focuses on being maximally effective, and a separate sales/fundraising group optimizes the message for the general public. However, many of the strategies suggested here actually affect the organization's core functions. Pairing donors with individual recipients significantly affects the organization's operations on the ground, raising costs. Could this in the long run lead to e.g. organizations selecting to operate in areas where recipients have characteristics that make them more interesting to donors to communicate with (e.g., they are more familiar with the language of the donor's country?). I don't see a way of making overall effectiveness, in the way that many EAs care about, still the dominant evaluation criterion if fundraising success is tied heavily to other outreach strategies.

Third (building somewhat on the first), insofar as there is a tradeoff between being able to sell more to effective altruists versus appealing more to the general public, the sign of the financial effect is actually ambiguous. The number of donors in the general public is much larger, but the amount that they donate per capita tends to be smaller. One of the ingredients to EA success is that its strength lies not so much in its numbers but in the depth of convictions of many self-identified EAs, plus other effective altruists (such as GiveWell donors). People who might have previously donated a few hundred dollars a year for an identifiable victim may now be putting in tens of thousands of dollars because the large-scale statistics have touched them in a deeper way. GiveWell moved $103 million to its top charities in 2015, of which $70 million was from Good Ventures (that's giving away money from a Facebook co-founder) and another $20 million is from individual donors who are giving amounts in excess of $100,000 each. To borrow sales jargon, these deals are highly lucrative and took a long time to close. Closing them required the donor to have high confidence in the epistemic rigor from a number of donors, many of whom were probably jaded by psychologically pitch-perfect campaigns. I'm not even saying that GiveWell's reviews are actually rigorous, but rather, that the perception of rigor surrounding them was a key aspect to many people donating to GiveWell-recommended charities.

Fourth, if the goal is to spread better, more rational giving habits, then caving in to sales tactics that exploit known forms of irrationality hampers that goal.

None of these imply that the ideas you suggest are inapplicable in the context of EA or for effective altruists in general. Nor am I suggesting that EAs (or effective altruists in general) are bias-free and rational demigods: I think many EAs have their own sets of biases that are more sophisticated than those of the general public but still real. I also think that many of the biases, such as the identifiable victim, can actually be epistemically justified somewhat, and you could make a good epistemic case for using individual case studies as not just a sales strategy but something that actually helps provide yet another sanity check (this is sort of what GiveWell tried to do by sponsoring field trips to the areas of operation of its top charities). You could also argue that the cost of alienating some people is a cost worth bearing in order to achieve a somewhat greater level of popularity, or that a wall of separation is not that hard to achieve.

But acknowledging these tradeoffs openly is a first step to letting others (including the orgs and fundraisers you are targeting) make a careful, informed decision. It can also help people figure out new, creative compromises. Perhaps, for instance, showing an identifiable victim and, after people are sort-of-sold, then pivoting to the statistics, provides the advantages of mass appeal and epistemic rigor. Perhaps there are ways to use charities' own survey data to create composite profiles of typical beneficiaries that can help inform potential donors as well as appeal to their desire for an identifiable victim. Perhaps, at the end of the day, raising money matters more than spreading ideas, and getting ten million people to donate a few hundred dollars a year is better than the current EA donor profile or the current GiveWell donor profile.

Comment author: Petter 30 March 2015 08:59:46PM 1 point [-]

Mobile is a larger platform than desktop 2015. That fact and the knowledge graph seem like very plausible explanations.

Comment author: VipulNaik 31 March 2015 11:07:42PM 1 point [-]
Comment author: John_Maxwell_IV 30 March 2015 08:05:16AM 4 points [-]

Regression to the mean is a potential problem when you choose to examine the most extreme data points in a data set (highly viewed wikipedia pages in this case).

Comment author: VipulNaik 31 March 2015 11:06:04PM 2 points [-]

I didn't pick them as points that were most extreme as of earlier years, I picked them as generically popular topics. There should be no particular temporal directionality to view counts for such pages.

Comment author: tog 16 February 2015 12:15:21PM 0 points [-]

Did you ever find the answer to this?

Comment author: VipulNaik 17 February 2015 11:08:49PM 0 points [-]

No

Comment author: Shri 25 July 2014 04:01:26AM *  3 points [-]

You may be interested in this white paper by a Google enginer using a NN to predict power consumption for their data centers with 99.6% accuracy.

http://googleblog.blogspot.com/2014/05/better-data-centers-through-machine.html

Looking at the interals of the model he was able to determine how sensitive the power consumption was to various factors. 3 examples were given for how the new model let them optimize power consumption. I'm a total newbie to ML but this is one of the only examples I've seen where: predictive model -> optimization.

Here's another example you might like from Kaggle cause-effect pairs challenge. The winning model was able to accurately classify whether A->B, or B->A with and AUC of >.8 , which is better than some medical tests. A writeup and code were provided by the top three kagglers.

http://clopinet.com/isabelle/Projects/NIPS2013/

Comment author: VipulNaik 25 July 2014 06:49:15AM 0 points [-]

Thanks, both of these look interesting. I'm reading the Google paper right now.

Comment author: John_Maxwell_IV 16 July 2014 03:08:22AM *  2 points [-]

Since the complexity of many machine learning algorithms grows at least linearly (and in some cases quadratically or cubically) in the data, and the quantity of data itself will probably grow superlinearly, we do expect a robust increase in demand for computing.

Algorithms to find the parameters for a classifier/regression, or algorithms to make use of it? And if I've got a large dataset that I'm training a classifier/regression on, what's to stop me from taking a relatively small sample of the data in order to train my model on? (The one time I used machine learning in a professional capacity, this is what I did. FYI I should not be considered an expert on machine learning.)

(On the other hand, if you're training a classifier/regression for every datum, say every book on Amazon, and the number of books on Amazon is growing superlinearly, then yes I think you would get a robust increase.)

Comment author: VipulNaik 16 July 2014 05:37:31AM *  1 point [-]

Good question.

I'm not an expert in machine learning either, but here is what I meant.

If you're running an algorithm such as linear or logistic regression, then there are two dimension numbers that are relevant: the number of data points, and the number of features (i.e., the number of parameters). For the design matrix of the regression, the number of data points is the number of rows and the number of features/parameters is the number of columns.

Holding the number of parameters constant, it's true that if you increase the number of data points beyond a certain amount, you can get most of the value through subsampling. And even if not, more data points is not such a big issue.

But the main advantage of having more data is lost if you still use the same (small) number of features. Generally, when you have more data, you'd try to use that additional data to use a model with more features. The number of features would still be less than the number of data points. I'd say that in many cases it's about 1% of the number of data points.

Of course, you could still use the model with the smaller number of features. In that case, you're just not putting the new data to much good use. Which is fine, but not an effective use of the enlarged data set. (There may be cases where even with more data, adding more features is no use, because the model has already reached the limits of its predictive power).

For linear regression, the algorithm to solve it exactly (using normal equations) takes time that is cubic in the number of parameters (if you use the naive inverse). Although matrix inversion can in principle be done faster than cubic, it can't be faster than quadratic, which is a general lower bound. Other iterative algorithms aren't quite cubic, but they're still more than linear.

View more: Next