lukeprog comments on Why Academic Papers Are A Terrible Discussion Forum - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (54)
Re-replying:
For people who "are extremely busy, and they use "Did they bother to pass peer review?" as a filter for what they choose to read", which specific examples are you thinking of, and how much any of them become nontrivial members of our community, or helped us out in nontrivial ways?
I'm sure there are people who a) are very smart, b) look impressive on paper, who c) we've contacted about FAI research, and d) have said "I'm not going to pay attention, since this isn't peer reviewed" (or some equivalent). However, I think that for most of those people, that isn't their true rejection (http://lesswrong.com/lw/wj/is_that_your_true_rejection/), and they aren't going to take us seriously anyway. But I could be wrong - what evidence do you have in mind?
A lot of your points are criticisms of blog posts, like "a lot of them don't have citations", or "a lot of them are poorly organized". These are true in many cases. However, if SIAI is considering whether to publish some given idea in paper or blog post form, they could simply spend the (fairly small) effort to write a blog post which was well organized and had citations, thereby making these problems moot.
Journal editors obviously aren't perfectly analogous to mob bosses. However, I've heard many stories from academics of authors spending huge amounts of time and effort trying to get stuff published. In the most recent case, which I discussed with a grad student just a few hours ago, it took hundreds of hours, over a full year. If it's usually easy to get around that sort of thing, by just publishing in a different journal, why don't more academics do so?
Your first two questions ask about evidence that I already said I'm not in a position to share yet. I know that's unsatisfying, but... are your priors on my claims being true really very low? Famous scientists, especially, are barraged with a few purported unifications of quantum theory and relativity every month, and "Did they bother to pass peer review?" is a pretty useful heuristic for them. When you visualize a busy academic receiving CFAI from one person, and The Singularity and Machine Ethics from somebody else, which one do you think they're more likely to read and take seriously, and why? (Feel free to take this as a rhetorical question.)
The effort required may be much larger than you think. Eliezer finds it very difficult to do that kind of work, for example. (Which is why his papers still read like long blog posts, and include very few citations. CEV even contains zero citations, despite re-treading ground that has been discussed by philosophers for centuries, as "The Singularity and Machine Ethics" shows.)
And if you've done all that work, then why not also tweak it for use in a scholarly AI risk wiki, and then combine it with a couple other wiki articles into a paper?
Because their career depends on satisfying their advisors, or on getting published in particular journals. SI researchers' careers don't depend on investing hundreds of hours making revisions. If publishing in a certain journal is going to require 30 hours of revisions that don't actually improve the paper in our eyes, then we aren't going to bother publishing in that journal.
Both links go to the same place.
Fixed, thanks.
If this is the case, then a significant benefit to Eliezer of trying to get papers published would be that it would be excellent discipline for Eliezer, and would make him an even better scholar.
A benefit that would follow on is that it would establish by example that nobody is above showing their work, acknowledging their debts and being current on the relevant literature. Conceivably Eliezer is such a talented guy that it is of no benefit to him to do these things, but if everyone who thought they were that talented were excused from showing their work and keeping current then progress would slow significantly.
It also avoids reinventing the wheel. No matter how smart Eliezer is, it's always conceivable that someone else thought of something first and expressed it in rigorous detail with proper citations. A proper literature review avoids this waste of valuable research time.
Luke (and his remote research assistants) have this angle covered.