I'm afraid this didn't clarify anything for me. Sorry! Pretend you're explaining it to someone stupid.
Actually, I think I understand it, but only with moderate confidence, and I have no prior experience with this particular terminology. So let me take this opportunity to put my thoughts on the firing line. :-) Whether they get shot down or confirmed, either way I'll have learned something about my ability to figure out things at first glance.
So: we're agents and we live in an environment.
We value certain things in the environment, and try to make decisions so that the environment arrives at the states we like, the states that have more value. Furthermore, ...
A comment to http://singinst.org/blog/2010/10/27/presentation-by-joshua-foxcarl-shulman-at-ecap-2010-super-intelligence-does-not-imply-benevolence/: Given as in the naive reinforcement learning framework (and that can approximate some more complex notions of value) that the value is in the environment, you don't want to be too hasty with the environment lest you destroy a higher value you haven't yet discovered! So you especially wouldn't replace high complexity systems like humans with low entropy systems like computer chips, without first analyzing them.