Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

hamnox comments on Fact Posts: How and Why - Less Wrong

74 Post author: sarahconstantin 02 December 2016 06:55PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (31)

You are viewing a single comment's thread.

Comment author: hamnox 04 December 2016 04:20:29AM 2 points [-]

I find myself very confused about how to tell which journals are reputable. Do you have a good heuristic (or list) for finding this out?

Comment author: btrettel 06 December 2016 01:09:41AM *  3 points [-]

Learning of the reputation of the journal from someone knowledgeable about its field is the most reliable way I can think of for someone outside the field of interest.

Impact factors seem inappropriate to me, as they can vary wildly between fields and even wildly among subfields. A more specialized, but still high quality, journal could have a much lower impact factor than a more general journal, even if the two are at roughly the same average quality. Also, some foreign language journals can be excellent despite having low impact factors for the field of the journal. This unfortunately is true even for cover-to-cover or partial translations of those foreign language journals.

You also could learn what signs to avoid. Some journals publish nonsense, and that's usually pretty obvious after looking at a few articles. (Though, in some fields it can be hard to separate nonsense from parody.)

Beyond the previous recommendations, it's probably better to focus on the merits of the article rather than the journal. I can immediately think of one article in particular which has a non-obvious major flaw that is published in a small journal that I consider excellent. This error should have been caught in review, but it was not, probably because catching the error requires redoing math the authors skipped over in the article. (In intend to eventually publish a paper on this error after I finish my correction to it.)

Comment author: Manfred 08 December 2016 11:30:26PM 2 points [-]

The worst part is that there's a lot the journal doesn't protect you from, no matter how reputable. Data shown in the paper can be assumed to be presented in the prettiest possible way for the specific data set they got, and interpretation of the data can be quite far off base and still get published if none of the reviewers happen to be experts on the particular methods or theories.

Comment author: waveman 06 December 2016 11:08:22AM 1 point [-]

Ask people in the field, if you know someone.

Comment author: ChristianKl 05 December 2016 02:16:25AM 0 points [-]

Looking at the impact factor of the journal can be a way to filter out junk journals.

Comment author: Douglas_Knight 05 December 2016 11:55:27PM 4 points [-]

Right, the highest impact factors are noise mining.