How can we have Friendly AI even if we humans cannot agree about our ethical values? This is an SQ because probably this was the first problem solved -it just so obivious - yet I cannot find it.
I have not finished the sequences yet, but they sound a bit optimistic to me - as if basically everybody is a modern utilitarian and the rest of the people just don't count. To give you the really dumbest question: what about religious folks? Is it just supposed to be a secular-values AI and they can go pound sand, or some sort of an agreement, compromise drawn with them and then that implemented? Is some sort of a generally agreed Human Values system a prerequisite?
My issue here is that if we want to listen to everybody, then this will be a never-ending debate. If you draw the line and e.g. include on people with reasonably utilitarian value systems, where do you draw the line etc.
As I told someone else, this pdf has preliminary discussion about how to resolve differences that persist under extrapolation.
The specific example of religious disagreements seems like a trivial problem to anyone who gets far enough to consider the question. Since there aren't any gods, the AI can ask what religious people would want if they accepted this fact. (This is roughly why I would oppose extrapolating only LW-ers rather than humanity as a whole.) But hey, maybe the question is more difficult than I think - we wouldn't specifically tell the AI to b...
This thread is for asking any questions that might seem obvious, tangential, silly or what-have-you. Don't be shy, everyone has holes in their knowledge, though the fewer and the smaller we can make them, the better.
Please be respectful of other people's admitting ignorance and don't mock them for it, as they're doing a noble thing.
To any future monthly posters of SQ threads, please remember to add the "stupid_questions" tag.