If MIRI doesn't publish reasonably frequently (via peer review), how do you know they aren't wasting donor money?
How did science get done for the centuries before peer review? Why do you place such weight on such a recently invented construct like peer review (you may remember Einstein being so enraged by the first and only time he tried out this new thing called 'peer review' that he vowed to never again submit anything to a 'peer reviewed' journal), a construct which routinely fails anytime it's evaluated and has been shown to be extremely unreliable where the same paper can be accepted and rejected based on chance? If peer-review is so good, why do so many terrible papers get published and great Nobel-prize-winning work rejected repeatedly? If peer review is such an effective method of divining quality, why do many communities seem to get along fine with desultory use of peer review where it's barely used or left as the final step long after the results have been disseminated and evaluated and people don't even bother to read the final peer-reviewed version (particularly in economics, I get the impression that everyone reads the preprints & working papers and the final publication comes as a non-event; which has caused me serious trouble in the past in trying to figure out what to cite and whether one cite is the same as another; and of course, I'm not always clear on where various statistics or machine learning papers get published, or if they are published in any sense beyond posting to ArXiv)? And why does all the real criticism and debate and refutations seem to take place on blogs & Twitter if peer-review is such an acid test of whether papers are gold or dross, leading to the growing need for altmetrics and other ways of dealing with the 'post-publication peer review' problem as journals increasingly fail to reflect where scientific debates actually are?
I've said it before and I'll said it again: 'peer review' is not a core element of science. It's barely even peripheral and unclear it adds anything on net. For the most part, calls for 'peer review' are cargo culting. What makes science work is replication and putting your work out there for community evaluation. Those are the real review by peers.
If you are a donor who wants to evaluate MIRI, whether some arbitrary reviewers pass or fail its papers is not very important. There are better measures of impact: is anyone building on their work? have MIRI-specific claims begun filtering out? are non-affiliated academics starting to move into the AI risk field? Heck, even citation counts would probably be better here.
How did science get done for the centuries before peer review? Why do you place such weight on such a recently invented construct like peer review
Is this an "arguments as soldiers" thing? Compare an isomorphic argument: "how did medicine get done for the centuries before antibiotics."
(you may remember Einstein being so enraged by the first and only time he tried out this new thing called 'peer review' that he vowed to never again submit anything to a 'peer reviewed' journal),
Leaving aside that this an argument from authority, ...
If it's worth saying, but not worth its own post (even in Discussion), then it goes here.
Notes for future OT posters:
1. Please add the 'open_thread' tag.
2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)
3. Open Threads should be posted in Discussion, and not Main.
4. Open Threads should start on Monday, and end on Sunday.