= 27d567bc2d48d9f40e9ccc20ea370b15)
Eugine_Nier comments on Evidence for the orthogonality thesis - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (289)
How's that any different from:
"What should we have the AI's beliefs be?"
"Eh, just make it self-improve, once it's smart it can figure out the true beliefs."
It's not very different. They are both different from:
"The AI will acquire accurate beliefs by using a well understood epistemology to process its observations, as it is explicitly designed to do."
"Smart" implicitly entails "knows the true beliefs", whereas it doesn't entail "has the right goals".
It doesn't exclude having the right goals, either. You could engineer something whose self-improvement was restricted from affecting its goals. But if that is dangerous, why would you?
Well, the difference is that building an AI without figuring out where goals come from gives you a dangerous AI, while building an AI without figuring out where beliefs come from gives you a highly-optimized compiler that wants to save humanity.
factual beliefs != moral beliefs
And the methods for investigating them are very different.