= 27d567bc2d48d9f40e9ccc20ea370b15)
cousin_it comments on Less Wrong views on morality? - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (145)
Okay, is a paperclip maximizer a moral realist?
I see no reason to think a paperclip maximizer would need to have any particular meta-ethics. There are possible paperclip maximizers that are and one's that aren't. As rule of thumb, an agent's normative ethics, that is, what it cares about, be it human flourishing or paperclips does not logically constrain it's meta-ethical views.
That's a nice and unexpected answer, so I'll continue asking questions I have no clue about :-)
If metaethics doesn't influence paperclip maximization, then why do I need metaethics? Can we point out the precise difference between humans and paperclippers that gives humans the need for metaethics? Is it the fact that we're not logically omniscient about our own minds, or is it something deeper?
Perhaps I misunderstood. There are definitely possible scenarios in which metaethics could matter to a paperclip maximizer. It's just that answering "what meta-ethics would the best paperclip maximizer have?" isn't any easier than answering "what is the ideal metaethics?". Varying an agent's goal structure doesn't change the question.
That said, if you think humans are just like paperclip maximizers except they're trying to maximize something else than you're already 8/10ths of the way to moral anti-realism (Come! Take those last two steps the water is fine!).
Of course it's also the case that meta-ethics probably matters more to humans than paperclip maximizers: In particular metaethics matters for humans because of individual moral uncertainty, group and individual moral change, differences in between individual moralities, and the overall complexity of our values. There are probably similar possible issues for paperclip maximizers-- like how should they resolve uncertainty over what counts as a paperclip or deal with agents that are ignorant of the ultimate value of paperclips-- and thinking about them pumps my anti-realist intuitions.
Well, there's certainly that. Also, human algorithms for decision-making can feel different from simply looking up a utility - the algorithm can be something more like a "treasure map" for locating morality, looking out at the world in a way that can feel as if morality was a light shining from outside.
Consider dealings with agents that have morals that conflict with your own. Obviously, major value conflicts preclude co-existence. Let's assume it is a minor conflict - Bob believes eating cow milk and beef at the same meal is immoral.
It is possible to develop instrumental or terminal values to resolve how much you tolerate Bob's different value - without reference to any meta-ethical theory. But I think that meta-ethical considerations play a large role in how tolerance of value conflict is resolved - for some people, at least.
Not obvious. (How does this "preclusion" work? Is it the best decision available to both agents?)
Well, if I don't include that sentence, someone nitpicks by saying:
I was trying preempt by making it clear that McH gets imprisoned or killed, even by moral anti-realists (unless they are exceptionally stupid).
I would certainly say a paperclip maximizer morality falls in the former camp (objective, able to be scientifically investigated, real), although I'm not intimately familiar with the realist/anti-realist terminology.
Hank, why would Clippy believe that maximizing paperclips is based on something external to its own mind? Clippy could just as easily be programmed to desire staples, and Clippy is probably intelligent enough to know that.
That said, I think Jack's general point about the relationship between ethics and meta-ethics is probably right.
Presumably Clippy has a hard-coded utility function sitting in his source code somewhere. It's a real set of 0s and 1s sitting on a disk somewhere, and we could open the source file and investigate the code.
Clippy's value system is a specific objective, pre-programmed utility function that's inherent in his design and independent/prior to any knowledge or other cognitive content Clippy eventually gains or invents.
And yes, it could have been easily changed (until such a time as Clippy is all grown up and would prevent such change) to make him desire staples. But then that's a different design and we'd probably better call him Stapley at that point.
Ideally, one would like objective facts to be universally compelling. If Clippy shows its source code to me, or to another AGI, neither of us would update in favor of believing that paper-clip maximizing is an appropriate terminal value.
Ah, no, I don't mean "objective morality" in the sense of something like a "one true, universal, metaphysical morality for all mind-designs" like the Socratic/Platonic Form of Good or any such nonsense. I just mean what I said above, something in reality that's mind-independent and can be investigated scientifically - a definite "is" from which we can make true "ought" statements relative to that "is".
See drethelin's comment below.
Hank, I definitely don't think there's any activity that (1) can reasonably be labeled "scientific investigation" and (2) can solve the is-ought divide.
I didn't think you would :) I'm curious about the consensus on LW, though. But incidentally, what do you think of Thou Art Godshatter?
First, that essay is aimed primarily at those who think dualism is required in order to talk about morality at all - obviously that's not the discussion we are having.
Second, the issue is not whether there are (1) universal (2) morally relevant (3) human preferences that (4) have been created by evolution. The answer to that question is yes (i.e. hunger, sexual desire). But that alone does not show that there is a universal way for humans to resolve moral dilemma.
If we study and quantify the godshatter to the point that we can precisely describe "human nature," we aren't guaranteed in advance to know that appeals to human nature will resolve every moral dilemma. If reference to human nature doesn't, then evolutionary preference doesn't prove moral realism.
I'm not sure why "universality" is really that important here. Suppose we are just talking about one person, why can't they reduce their value judgments down to their own precisely described nature to resolve every moral dilemma they face? With a read-out of their actual terminal values defined by the godshatter, they can employ the usual consequentialist expected utility calculus to solve any question, in principle.
Clippy's code is his mind.
No. The physical instantiation of a utility function is not an argument for moral realism. On the complete contrary, defining moral actions as "whatever an agents utility function says" is straight-forward, definitional, no-bones-about-it moral subjectivism.
Put it this way: the paperclip maximizer is not going to approve of your behavior.