Recently, I gave some reasons for SIAI to begin publishing in mainstream journals, and outlined how it could be done.
I've recently been made aware of some pretty good reasons for SIAI to not publish in mainstream journals, so here they are:
- Articles published to websites (e.g. Yudkowsky's work, Bostrom's pre-prints) seem to have gotten more attention, and had more positive impact, than their in-journal counterparts.
- Articles in mainstream journals take a relatively large amount of time, money, and expertise to produce.
- Articles in mainstream journals must jump through lots of hoops - journals' aversion to novelty, reviewer bias, etc.
- It is easier to simply collaborate with (and greatly influence) established mainstream academics who have already jumped through mainstream academia's many hoops (as Carl Shulman has been doing, for example).
I agree with previous comments about publishing in journals being an important status issue, but I think there is other value as well which is being ignored. For all of its annoyances and flaws, one good thing about peer review is that it really makes your paper better. When you submit a pretty good paper to a journal and get back the "revise and resubmit" along with the detailed list of criticisms and suggestions, then by the time the paper actually makes it into the journal, chances are that it will have become a really good paper.
But to return to the issue of papers being taken more seriously when published in a journal, I think that this view is actually quite justified. For researchers who are not already very knowledgeable in the precise area that is the topic for a given paper, whether or not the paper has withstood peer review is a very useful heuristic cue toward how much weight you should place on it. Basically, peer review keeps the author honest. An author posting a paper on his website can say pretty much whatever he wants. One of the purposes of peer reviewers is to make sure that the author isn't making unreasonable claims, mischaracterizing theoretical positions, "reviewing" the relevant previous literature in a grossly selective way, etc. Like I said, if someone is already very familiar with the area, then they can evaluate these aspects of the paper for themselves. But if you'd like to communicate your position to a wider academic audience, peer review can help carry your paper a longer way.
-- Paraphrase of a speaker at the Northeast Conference on Science and Skepticism
If it's not published, it might be correct, but it's not science.