= 27d567bc2d48d9f40e9ccc20ea370b15)
G0W51 comments on Open Thread, Apr. 06 - Apr. 12, 2015 - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (128)
Do you happen to know of any good places to read about this issue? I have yet to look into what others have thought of it.
I've seen not-especially-theoretical discussion about this problem for human organizations, though mostly from the point of view of lower status people complaining that they're being given impossible, incomprehensible, and/or destructive commands.
This points at another serious problem-- you need communication (even if mostly not forceful communication) to flow up the hierarchy as well as down the hierarchy.
You're pointing at a failure mode (I have a JOB! I wanna do my JOB!) which is quite real, but there are equal and opposite failure modes. For example, obsessive compulsive disorder would be an example of lower level functions insisting on taking charge. Eating disorders are (at least in some cases) examples of higher level functions overriding competent lower level functions.
What I'm concluding from this is that if Friendliness is to work, it has to pervade the hierarchy of agents.
Remember that making humans in organizations cooperate is a rather different from making the many parts of an AI cooperate with the other parts, because people in organizations can't (feasibly) be reprogrammed to have their values be aligned, but AI HLAs can.
True. The real issue is that if you give a lower-level action the same goals and utility functions as the higher ones, you've lost all benefit of having HLAs!.