= 27d567bc2d48d9f40e9ccc20ea370b15)
Zvi comments on Even with default points, systems remain exploitable - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (9)
Even with zero knowledge of the other guy's function, you'd always start with Lie #1: Always represent any outcome that leaves you worse off as having infinite negative utility (or at least more bad than your utopia point is good).
This cuts off any outcome that decreases your utility, and thus is very, very good for you - even if you need to self-modify and make it real. Note that this is how actual negotiations work.
Another easy hack is to limit your goals, and pretend that impossibly good outcomes are no better for you than the best possible outcome, in order to increase the value of utility to you via decreasing your Utopia point.