= 27d567bc2d48d9f40e9ccc20ea370b15)
mfb comments on The Human's Hidden Utility Function (Maybe) - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (87)
As a first reaction (and without being read up on the details), I'm very skeptical. Assuming these three systems are actually in place, I don't see any convincing reason why any one of them should be trusted in isolation. Natural selection has only ever been able to work on their compound output, oblivious to the role played by each one individually and how they interact.
Maybe the "smart" system has been trained to assign some particular outcome a value of 5 utilons, whereas we would all agree that it's surely and under all circumstances worth more than 20, because as it happens throughout evolution one of the other "dumb" systems has always kicked in and provided the equivalent of at least 15 utilons. If you then extract the first system bare and naked, it might deliver some awful outputs.
As I understand it, the first system should be able to predict the result of the other two - if the brain knows a bit about how brains work.
While I don't know if the brain really has three different systems, I think that the basic idea is true: The brain has the option to rely on instincts, on "it worked before", or on "let's make a pro/contra list" - this includes any combination of the concepts.
The "lower" systems evolved before the "higher" ones, therefore I would expect that they can work as a stand-alone system as well (and they do in some animals).