Wei? Your response?
I'm not sure that my thoughts on this topic should be taken that seriously, since I'm quite confused, uncertain, and conflicted (a part of me just wants to see these intellectual puzzles solved ASAP), but since you ask... My last thoughts on this topic were:
...It's the social consequences that I'm most unsure about. It seems like if SIAI can keep "ownership" over the decision theory ideas and use it to preach AI risk, then that would be beneficial, but it could also be the case that the ideas take on a life of their own and we just end up having m
When I showed up at the Singularity Institute, I was surprised to find that 30-60 papers' worth of material was lying around in blog posts, mailing list discussions, and people's heads — but it had never been written up in clear, well-referenced academic articles.
Why is this so? Writing such articles has many clear benefits:
Of course, there are costs to writing articles, too. The single biggest cost is staff time / opportunity cost. An article like "Intelligence Explosion: Evidence and Import" can require anywhere from 150-800 person-hours. That is 150-800 paid hours during which our staff is not doing other critically important things that collectively have a bigger positive impact than a single academic article is likely to have.
So Louie Helm and Nick Beckstead and I sat down and asked, "Is there a way we can buy these articles without such an egregious cost?"
We think there might be. Basically, we suspect that most of the work involved in writing these articles can be outsourced. Here's the process we have in mind:
If this method works, each paper may require only 50-150 hours of SI staff time per paper — a dramatic improvement! But this method has additional benefits:
This is, after all, more similar to how many papers would be produced by university departments, in which a senior researcher works with a team of students to produce papers.
Feedback? Interest?
(Not exactly the same, but see also the Polymath Project.)