Nick that's an interesting thought, but it is possible that someone does know how to build a friendly AI, but may not know that he knows this. What I mean is that a person may be working on something unrelated to AI or is not interested in AI, but knows all the ingredients and technical things needed to make a friendly AI. Maybe he just doesn't know the implications of his own knowledge.
I think it's fair to say many scientists fall into this category, if only due to specialization. Maybe some inorganic chemist knows the cure to cancer. But simply due t...
"Destroying the world" describes things fairly accurately. Who believes a nuclear strike on the major Soviet Union and American cities wouldn't have destroyed the world? Life, I'm sure wouldn't have ended (even human), but the fall out and human loss would have been catastrophic. It's easy to question this 20-30 years after the fact, but I remember the fear the world festered in in the 70's and 80's. If a nuclear missile had been launched, dozens if not hundreds would have been launched.
The change in world view and behavior would have been la...
Eliezer,
"[L]ogical ignorance is still ignorance; if this were not so, I would know a proof of the Riemann Hypothesis." I understand what you mean by logical ignorance is still ignorance; as for the second part of your statement, I don't think that's necessarily true.
For example, if the RH were true you would need n statements in a particular order. Meaning that you have stated assumptions, then you proceed to S1->S2->...->Sn (where Si is the ith logical statement) and Sn is the conclusion of the proof. What you are saying is that you ... (read more)