When I showed up at the Singularity Institute, I was surprised to find that 30-60 papers' worth of material was lying around in blog posts, mailing list discussions, and people's heads — but it had never been written up in clear, well-referenced academic articles.
Why is this so? Writing such articles has many clear benefits:
- Clearly stated and well-defended arguments can persuade smart people to take AI risk seriously, creating additional supporters and collaborators for the Singularity Institute.
- Such articles can also improve the credibility of the organization as a whole, which is especially important for attracting funds from top-level social entrepreneurs and institutions like the Gates Foundation and Givewell.
- Laying out the arguments clearly and analyzing each premise can lead to new strategic insights that will help us understand how to purchase x-risk reduction most efficiently.
- Clear explanations can provide a platform on which researchers can build to produce new strategic and technical research results.
- Communicating clearly is what lets other people find errors in your reasoning.
- Communities can use articles to cut down on communication costs. When something is written up clearly, 1000 people can read a single article instead of needing to transmit the information by having several hundred personal conversations between 2-5 people.
Of course, there are costs to writing articles, too. The single biggest cost is staff time / opportunity cost. An article like "Intelligence Explosion: Evidence and Import" can require anywhere from 150-800 person-hours. That is 150-800 paid hours during which our staff is not doing other critically important things that collectively have a bigger positive impact than a single academic article is likely to have.
So Louie Helm and Nick Beckstead and I sat down and asked, "Is there a way we can buy these articles without such an egregious cost?"
We think there might be. Basically, we suspect that most of the work involved in writing these articles can be outsourced. Here's the process we have in mind:
- An SI staff member chooses a paper idea we need written up, then writes an abstract and some notes on the desired final content.
- SI pays Gwern or another remote researcher to do a literature search-and-summary of relevant material, with pointers to other resources.
- SI posts a contest to LessWrong, inviting submissions of near-conference-level-quality articles that follow the provided abstract and notes on desired final content. Contestants benefit by starting with the results of Gwern's literature summary, and by knowing that they don't need to produce something as good as "Intelligence Explosion: Evidence and Import" to win the prize. First place wins $1200, 2nd place wins $500, and 3rd place wins $200.
- Submissions are due 1 month later. Submission are reviewed, and the authors of the best submissions are sent comments on what could be improved to maximize the chances of coming in first place.
- Revised articles are due 3 weeks after comments are received. Prizes are awarded.
- SI pays an experienced writer like Yvain or Kaj_Sotala or someone similar to build up and improve the 1st place submission, borrowing the best parts from the other submissions, too.
- An SI staff member does a final pass, adding some content, making it more clearly organized and polished, etc. One of SI's remote editors does another pass to make the sentences more perfect.
- The paper is submitted to a journal or an edited volume, and is marked as being co-authored by (1) the key SI staff member who provided the seed ideas and guided each stage of the revisions and polishing, (2) the author of the winning submission, and (3) Gwern. (With thanks to contributions from the other contest participants whose submissions were borrowed from — unless huge pieces were borrowed, in which case they may be counted as an additional co-author.)
If this method works, each paper may require only 50-150 hours of SI staff time per paper — a dramatic improvement! But this method has additional benefits:
- Members of the community who are capable of doing one piece of the process but not the other pieces get to contribute where they shine. (Many people can write okay-level articles but can't do efficient literature searches or produce polished prose, etc.)
- SI gets to learn more about the talent that exists in its community which hadn't yet been given the opportunity to flower. (We might be able to directly outsource future work to contest participants, and if one person wins three such contests, that's an indicator that we should consider hiring them.)
- Additional paid "jobs" (by way of contest money) are created for LW rationalists who have some domain expertise in singularity-related subjects.
- Many Less Wrongers are students in fields relevant to the subject matter of the papers that will be produced by this process, and this will give them an opportunity to co-author papers that can go on their CV.
- The community in general gets better at collaborating.
This is, after all, more similar to how many papers would be produced by university departments, in which a senior researcher works with a team of students to produce papers.
Feedback? Interest?
(Not exactly the same, but see also the Polymath Project.)
When I tried to write up some decision theory results as a full-length article, it felt really pointless and unpleasant. I couldn't get through even a single paragraph without thinking how much I hate it.
One problem is that even though I enjoy coming up with short and sweet proofs, I don't know in advance which parts will trip up readers and require clarification, and feel very averse to guessing. Here's a recent example. Maybe the right way is to write discussion posts first, then debug the presentation based on reader comments?
But the bigger problem is that academic articles seem to require a lot of fluff that doesn't add value. Moldbug called it "grant-related propaganda", but I'm not sure grants are the main reason why people add fluff. Contrast a typical paper today with John Nash's 1950 paper which doesn't waste even a single word on explaining why the subject matter is relevant to anything. I'd be happy to write things up in the same style, but then journals will just reject my writings, won't they?
Are you still considering it, since creating the 'writeup' thread on the list, or are you describing what preceded that?