= 27d567bc2d48d9f40e9ccc20ea370b15)
eli_sennesh comments on Value learning: ultra-sophisticated Cake or Death - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (15)
The problem is that there is no conservation of expected evidence for mixed statements, unless we put it there by hand.
Key reason: value loading AIs do not follow a utility function, but a dynamic construct, that doesn't have all the same properties.
At least as I read the original value-learning paper, they do follow a utility function: the maximum likelihood utility function in some distribution that is subject to Bayesian updating. The hard part was how to construct that distribution and subject it to evidence; the concept that the AI is going to want to have incorrect beliefs (since, after all, the process by which the updates are performed is epistemic, not moral) hadn't occurred to me.