= 27d567bc2d48d9f40e9ccc20ea370b15)
shokwave comments on Connecting Your Beliefs (a call for help) - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (73)
If we restrict the space of its terminal goals to things we can imagine (and then set about proving each thing to be friendly) then we can be sure that even thinking in ways we cannot fathom, as long as its goal structure doesn't change (this seems decoupled from intelligence ie paperclip maximiser) it won't ever do bad things X Y or Z (because it checks them against its terminal goal).
That directly contradicts the EY's CEV, where whatever we can imagine is no more than a part of the Initial Dynamics. "Thou shalt..." or "Thou shalt not..." is not going to do the trick.
Right. Downgrading my estimate of how well I understand the problem.