The danger is realized by what happens eventually. After decades of additional progress in theory and tools, it might become plausible for someone to build an AGI as a personal project (even if it's not a particularly probable outcome, compared to alternatives). This progress is built step by step, and even though each step may be small, eventual danger is made out of such steps, the only thing that causes the danger are the steps.
The possible harm of individual steps is in what kind of differential intellectual progress they represent. You can't stop progress, others will take the steps you avoid, but you can make some of the steps come earlier by taking them yourself before anyone else does, and you can choose what kind of steps these are: those that move you closer to the cliff, or those that teach you to fly.
I cannot imagine the circumstances under which a stray hobbyist would be able to beat a massive government or corporate effort to the punch in AI. The imbalance of resources is simply too great. Concern yourself with what the goals and methodologies of those large corporate or government efforts should look like.
The possible harm of individual steps is in what kind of differential intellectual progress they represent.
I get the concept, but I am totally unconvinced that anything MIRI is putting out could increase x-risk; in fact I think it's wildly im...
I'm pleased to announce friendly-artificial-intelligence, a google group intended for research-level discussion of problems in FAI and AGI, in particular for discussions that are highly technical and/or math intensive.
Some examples of possible discussion topics: naturalized induction, decision theory, tiling agents / Loebian obstacle, logical uncertainty...
I invite everyone who want to take part in FAI research to participate in the group. This obviously includes people affiliated with MIRI, FHI and CSER, people who attend MIRI workshops and participants of the southern california FAI workshop.
Please, come in and share your discoveries, ideas, thoughts, questions et cetera. See you there!