Again, you've pulled a statement out of a discussion the context of the behavior of a self-modifying AI. So, fine. In my current condition I wouldn't build a baby mulcher. That doesn't mean that I might not build a baby mucher if I had the ability to change my values. You might as well say that I terminally value not flying when I flap my arms. The thing you're discussing just isn't physically allowed. People terminally value only what they're doing at any given moment because the laws of physics say that they have no choice.
I think you're confusing "terminal" and "immutable". Terminal values can and do change.
In my current condition I wouldn't build a baby mulcher
And why is that? Do you, perchance, have some terminal moral value which disapproves?
People terminally value only what they're doing at any given moment because the laws of physics say that they have no choice.
Huh? That makes no sense. How do you define "terminal value"?
Part 1 was previously posted and it seemed that people likd it, so I figured that I should post part 2 - http://waitbutwhy.com/2015/01/artificial-intelligence-revolution-2.html