= 27d567bc2d48d9f40e9ccc20ea370b15)
IlyaShpitser comments on Building Phenomenological Bridges - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (116)
[ I am sure this is not new to anyone who's been thinking about this. ]
I worry that the world is not benign in general, but contains critters that want to mislead you to win utility. There might be such critters that may exploit the difficulty of the problem faced by Cai by trying to mislead Cai into constructing the wrong mapping from "stuff out there" to "stuff in Cai" (of course even the easy version is hard :( ).
The mapping might be relatively easy to check (i.e. hard to construct a misleadement that would look plausible).
Misleading an agent to believe that agent has a different utility function could be more interesting though.