wedrifid comments on Should I believe what the SIAI claims? - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (600)
You have correctly identified the area in which we do not agree.
The most relevant knowledge needed in this case is knowledge of game theory and human behaviour. They also need to know 'friendliness is a very hard problem'. They then need to ask themselves the following question:
What is likely to happen if people have the ability to create an AGI but do not have a proven mechanism for implementing friendliness? Is it:
I don't (with that phrasing). I actually suspect that the problem is too difficult to get right and far too easy to get wrong. We're probably all going to die. However, I think we're even more likely to die if some fool goes and invents a AGI before they have a proven theory of friendliness.