How would someone who has made an AGI breakthrough convince the AGI risk community with out building it?
In the usual way someone who has made a breakthrough convinces others. Reputation helps. Whitepapers help. Toy examples help. Etc., etc.
I don't understand the context, however. That someone, how does he know it's a breakthrough without testing it out? And why would he be so concerned with the opinion of the AI risk community (which isn't exactly held in high regard by most working AI researchers)?
Okay. A good metaphor might be the development of the atomic bomb. Lots of nuclear physicists thought that nuclear reactions couldn't be used for useful energy (e.g. Rutherford). Leo Szilard had the insight that you could do a chain reaction and that this might be dangerous. He did not build it the bomb (he could not, he didn't know about neutrons) and assigned the patent to the admiralty to keep it secret.
But he managed to convince other high profile physicists that it might dangerous without publicizing it too much (no whitepapers etc). He had the reputa...
If it's worth saying, but not worth its own post, then it goes here.
Notes for future OT posters:
1. Please add the 'open_thread' tag.
2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)
3. Open Threads should start on Monday, and end on Sunday.
4. Unflag the two options "Notify me of new top level comments on this article" and "