ChristianKl comments on Open thread, Aug. 10 - Aug. 16, 2015 - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (283)
You set two goals. One is to maximize expressed desires which likely leads to wireheading. The other is to keep constant communication with doesn't allow sleep.
Controlling the information flow isn't getting around your restriction. It's the straightforward way of matching expressed desires with results. Otherwise the human might ask for two contradictory things and the AGI can't fulfill both. The AGI has to prevent that case from arising to get a 100% fulfillment score.
You are not the first person who thinks that taming an AGI is trival but MIRI thinks that taming an AGI is a hard task. That's the result of deep engagement with the issue.
I don't object to a red button and you didn't call for one at the start. Maximizing expressed desires isn't a red button.