= 27d567bc2d48d9f40e9ccc20ea370b15)
Armok_GoB comments on The Blue-Minimizing Robot - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (159)
Good point, but the fact that humans are consequentialists (at least partly) doesn't seem to make the problem much easier. Suppose we replace Yvain's blue-minimizer robot with a simple consequentialist robot that has the same behavior (let's say it models the world as a 2D grid of cells that have intrinsic color, it always predicts that any blue cell that it shoots at will turn some other color, and its utility function assigns negative utility to the existence of blue cells). What does this robot "actually want", given that the world is not really a 2D grid of cells that have intrinsic color?
To avoid SEEING blue things. If the model is good enough for it it'd search out a mirror and laser it's own camera so that it could NEVER see a blue pixel again.
This can be modelled using human empathy by equating the sensation of seeing blue with pain. You don't care to minimize damage to your body (if it's not somehting that actually cripples you), but you care about not getting the signal about it happening, and your reaction to a pill that turned you masochist would be very different than your reaction to a murder pill.
Edit: huh? I am surprised that this is downvoted, and the most probable reason is that I'm wrong in some obvious way that I can't see, can someone please tell me how? (Or maybe my usage of empathy was interpreted way to literally. )