Comment author: Nick_Tarleton 23 August 2009 04:28:47PM *  3 points [-]

Many folks keep pointing out how published research is itself biased towards positive results, and how replication (and failed replication!) trumps mere "first!!!11" publication.

Clarity check: "trumps" = "is (normatively) more important than"?

Also,

("he can't implant engineered tissue in a rat heart and he calls himself a scientist?!")

will be really confusing if/when that entry drops off the front page.

Comment author: Vladimir_Gritsenko 24 August 2009 10:50:29AM 0 points [-]

Clarity check: "trumps" = "is (normatively) more important than"?

Yes.

will be really confusing if/when that entry drops off the front page.

Hehe :-) if you propose a less confusing quip, I'll edit it in.

Comment author: Christian_Szegedy 24 August 2009 02:49:22AM *  2 points [-]

The current system of scientific publishing is clearly outdated.

There are issues with the biases, but also with for-profit nature of the system that charges huge sums of money for accessing the work of researchers funded by public grants and reviewed by researchers for free, financed by public money.

Then add to this all those phony journals, that are theoretically peer reviewed, but have very low (or non-existent) standards and accept everything to make money or simply exist because some big-pharma company uses them to publish skewed tests to get FDA approvals, etc.

I think one or two additional special-purpose journals would not really change the landscape.

IMO, what we need is a complete modern infrastructure based on state of the art IT/social networking. One that allows the review of articles even after they officially appeared with an elaborate voting system that factors in the credibility of the reviewers, It should make it possible to add (publish) refutations and the publication of positive or negative attempts of replications, etc and organize the articles with their support/refutations/endorsements in an easily accessible database.

Ideally, such a system could work both as a rating and publication medium, but with the current scientific publishing lobby, it would not have much chance to take off. The only chance is to do this by extending an existing meta-system (e.g. citeseer) with a general discussion/rating/publishing forum, that would allow the publication of critics/refusals/extensions of existing papers maybe even in a peer-reviewed manner.

In the field that I work, I see that the scientific community discusses and generally supports such changes and given all the efforts and progress of the last decade I'd be surprised if we won't see such (or similar) one or more systems emerging in the next 10 years.

Comment author: Vladimir_Gritsenko 24 August 2009 10:17:15AM 0 points [-]

Yes, that would be better, but as yourself note, it's a big change that's unlikely to happen in one go. On the other hand, specialized journals are not a novelty, and considering that at least some folks took that specific specialization up, it appears to be more an issue of advertising than invention.

But nobody said this problem should be attacked on just one front. More (different) attempts mean more chances of success, no?

Comment author: cousin_it 23 August 2009 01:10:49PM *  0 points [-]

A novel negative result isn't the same as failing to replicate a study published by someone else.

Comment author: Vladimir_Gritsenko 23 August 2009 02:37:21PM 1 point [-]

At least in the second journal (of ecology and evolutionary biology), they do say they accept replication studies.

Comment author: AndrewKemendo 23 August 2009 12:18:57PM *  5 points [-]

If regular journals don't have good incentives to publish "mere" (failed) replication studies, why not create a journal that would be dedicated entirely to them?

There are a few Journals of negative results already out there:

http://www.jnrbm.com/

http://www.jnr-eeb.org/index.php/jnr

Comment author: Vladimir_Gritsenko 23 August 2009 02:34:35PM 1 point [-]

Cool, thanks! (Also, Google-fu fail on my part.)

One other journal I just found (although no publications there yet): http://www.arjournals.com/ojs/

If this is representative, then it's both encouraging (at least a few folks are taking the problem seriously) and discouraging (they're too few). At least now there's something concrete to evangelize :-)

The Journal of (Failed) Replication Studies

11 Vladimir_Gritsenko 23 August 2009 09:15AM

One of Seed Magazine's "Revolutionary Minds" is Moshe Pritsker, who created the Journal of Visualized Experiments, which to me looks like a very cool idea. I imagine that early on it may have looked somewhat silly ("he can't implant engineered tissue in a rat heart and he calls himself a scientist?!"), so it's nice to know JoVE is picking up pace.

Many folks keep pointing out how published research is itself biased towards positive results, and how replication (and failed replication!) trumps mere "first!!!11" publication. If regular journals don't have good incentives to publish "mere" (failed) replication studies, why not create a journal that would be dedicated entirely to them? I can't speak about the logistics, but I imagine it can be anything from a start-up (a la JoVE) to an open depository (a la arxiv.org).

I am not part of academia, but I understand that there are a few folks here who are. What do you say?

[EDIT: Andrew Kemendo notes two such journals in the comments: http://www.jnrbm.com/ and http://www.jnr-eeb.org/index.php/jnr.]

Comment author: scientism 16 April 2009 08:55:39PM 4 points [-]

It's interesting that China's leadership is full of engineers and economists; as a whole they're probably far more qualified to do their jobs than the leadership of most Western countries. I think there's some truth to the idea that you can have qualified leaders or you can have elected leaders but you can't have both. Getting elected just doesn't translate into being the best candidate for the job. If positions in the private sector were filled by popular vote, industry would grind to a halt. I think if you honestly consider that as a though experiment, it's obvious that only a truly absurd and impractical amount of voter rationality would solve the inherent problem, which is just that it isn't a sensible way to fill vacancies.

Comment author: Vladimir_Gritsenko 24 April 2009 05:24:04PM *  2 points [-]

For what it's worth, I offer this summary of a study about Chinese and American education. Even though Chinese students know a heck of a lot more science, they can't reason scientifically any better than their American counterparts.

I confess I don't know a lot about China, and so my preference to live in almost any Western country and not in China may be biased by ignorance, but... would you prefer to live in China, or another authoritarian state but whose management would be experts in various fields? Do you honestly think such a state would be better at various important parameters of societal welfare?

A final point: while congressfolk may be less competent than we might wish, actual state managers - civil servants in high positions - are often accredited veterans in their fields.

Comment author: PhilGoetz 16 April 2009 03:49:10PM 1 point [-]

Can you propose a better use of our time?

Comment author: Vladimir_Gritsenko 16 April 2009 04:08:00PM 0 points [-]

Novel information. I liked your post "on dollars, utility, and crack cocaine", for example.

Comment author: Vladimir_Gritsenko 16 April 2009 10:46:41AM 1 point [-]

Downvoted.

As a gut feeling, I agree with the sentiment. But... Most if not all of us agree that neither politicians nor voters are as educated or as rational as they should be, and we voice our agreements frequently. Is this the best use of our time? Considering that many folks have called for "thinking-based" education for a long time now, we're not even innovating. So, what are we doing? Reinforcing virtually uncontroversial beliefs? Priming our own private affective death spiral?

In response to Where are we?
Comment author: ciphergoth 02 April 2009 09:53:09PM 1 point [-]

Post in this thread if you live outside Europe, the US, or Canada.

In response to comment by ciphergoth on Where are we?
Comment author: Vladimir_Gritsenko 04 April 2009 10:09:30AM 1 point [-]

I can't be the only guy to generate the 410 hits from Israel... if I am, that is very sad.

Comment author: Vladimir_Gritsenko 30 March 2009 04:01:37PM *  7 points [-]

For what it's worth:

Ampakines are a new class of compounds known to enhance attention span and alertness, and facilitate learning and memory. ...

Unlike earlier stimulants (e.g. caffeine, methylphenidate (Ritalin), and the amphetamines), ampakines do not seem to have unpleasant, long-lasting side effects such as sleeplessness.

Apparently, only the military is interested in its mind-enhancing effects. Any chemists here interested in a start-up? ;-)

View more: Prev | Next