= 27d567bc2d48d9f40e9ccc20ea370b15)
AlexMennen comments on Proper value learning through indifference - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (50)
If the AI's utility function changes randomly for no apparent reason because the AI has litterally zero incentive to make sure that doesn't happen, then you have an AI design problem.
It didn't change for no reason. It changed because someone fed new data into the AI's utility-learning algorithm which made it change. Don't give people root access if you don't want them using it!
Being changed by an attacker is only one of the scenarios I was suggesting. And even then, presumably you would want the AI to help prevent them from hacking its utility function if they aren't supposed to have root access, but it won't.
Anyway, that problem is just a little bit stupid. But you can also get really stupid problems, like the AI wants more memory, so it replaces its utility function with something more compressible so that it can scavange from the memory where its utility function was stored.