= 27d567bc2d48d9f40e9ccc20ea370b15)
Stuart_Armstrong comments on An Oracle standard trick - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (33)
The "act as if it doesn't believe its messages will be read" is part of its value function, not its decision theory. So we are only requiring the value function to be stable over self improvement.
Why is that? The value function tells you what is important, but the "act" part requires decision theory.
What I mean is that I haven't wired the decision theory to something odd (which might be removed by self improvement), just chosen a particular value system (which has much higher chance of being preserved by self improvement).