= 27d567bc2d48d9f40e9ccc20ea370b15)
By Which It May Be Judged
Followup to: Mixed Reference: The Great Reductionist Project
Humans need fantasy to be human.
"Tooth fairies? Hogfathers? Little—"
Yes. As practice. You have to start out learning to believe the little lies.
"So we can believe the big ones?"
Yes. Justice. Mercy. Duty. That sort of thing.
"They're not the same at all!"
You think so? Then take the universe and grind it down to the finest powder and sieve it through the finest sieve and then show me one atom of justice, one molecule of mercy.
- Susan and Death, in Hogfather by Terry Pratchett
Suppose three people find a pie - that is, three people exactly simultaneously spot a pie which has been exogenously generated in unclaimed territory. Zaire wants the entire pie; Yancy thinks that 1/3 each is fair; and Xannon thinks that fair would be taking into equal account everyone's ideas about what is "fair".
I myself would say unhesitatingly that a third of the pie each, is fair. "Fairness", as an ethical concept, can get a lot more complicated in more elaborate contexts. But in this simple context, a lot of other things that "fairness" could depend on, like work inputs, have been eliminated or made constant. Assuming no relevant conditions other than those already stated, "fairness" simplifies to the mathematical procedure of splitting the pie into equal parts; and when this logical function is run over physical reality, it outputs "1/3 for Zaire, 1/3 for Yancy, 1/3 for Xannon".
Or to put it another way - just like we get "If Oswald hadn't shot Kennedy, nobody else would've" by running a logical function over a true causal model - similarly, we can get the hypothetical 'fair' situation, whether or not it actually happens, by running the physical starting scenario through a logical function that describes what a 'fair' outcome would look like:
So am I (as Zaire would claim) just assuming-by-authority that I get to have everything my way, since I'm not defining 'fairness' the way Zaire wants to define it?
No more than mathematicians are flatly ordering everyone to assume-without-proof that two different numbers can't have the same successor. For fairness to be what everyone thinks is "fair" would be entirely circular, structurally isomorphic to "Fzeem is what everyone thinks is fzeem"... or like trying to define the counting numbers as "whatever anyone thinks is a number". It only even looks coherent because everyone secretly already has a mental picture of "numbers" - because their brain already navigated to the referent. But something akin to axioms is needed to talk about "numbers, as opposed to something else" in the first place. Even an inchoate mental image of "0, 1, 2, ..." implies the axioms no less than a formal statement - we can extract the axioms back out by asking questions about this rough mental image.
Similarly, the intuition that fairness has something to do with dividing up the pie equally, plays a role akin to secretly already having "0, 1, 2, ..." in mind as the subject of mathematical conversation. You need axioms, not as assumptions that aren't justified, but as pointers to what the heck the conversation is supposed to be about.
Multiple philosophers have suggested that this stance seems similar to "rigid designation", i.e., when I say 'fair' it intrinsically, rigidly refers to something-to-do-with-equal-division. I confess I don't see it that way myself - if somebody thinks of Euclidean geometry when you utter the sound "num-berz" they're not doing anything false, they're associating the sound to a different logical thingy. It's not about words with intrinsically rigid referential power, it's that the words are window dressing on the underlying entities. I want to talk about a particular logical entity, as it might be defined by either axioms or inchoate images, regardless of which word-sounds may be associated to it. If you want to call that "rigid designation", that seems to me like adding a level of indirection; I don't care about the word 'fair' in the first place, I care about the logical entity of fairness. (Or to put it even more sharply: since my ontology does not have room for physics, logic, plus designation, I'm not very interested in discussing this 'rigid designation' business unless it's being reduced to something else.)
Once issues of justice become more complicated and all the contextual variables get added back in, we might not be sure if a disagreement about 'fairness' reflects:
- The equivalent of a multiplication error within the same axioms - incorrectly dividing by 3. (Or more complicatedly: You might have a sophisticated axiomatic concept of 'equity', and incorrectly process those axioms to invalidly yield the assertion that, in a context where 2 of the 3 must starve and there's only enough pie for at most 1 person to survive, you should still divide the pie equally instead of flipping a 3-sided coin. Where I'm assuming that this conclusion is 'incorrect', not because I disagree with it, but because it didn't actually follow from the axioms.)
- Mistaken models of the physical world fed into the function - mistakenly thinking there's 2 pies, or mistakenly thinking that Zaire has no subjective experiences and is not an object of ethical value.
- People associating different logical functions to the letters F-A-I-R, which isn't a disagreement about some common pinpointed variable, but just different people wanting different things.
There's a lot of people who feel that this picture leaves out something fundamental, especially once we make the jump from "fair" to the broader concept of "moral", "good", or "right". And it's this worry about leaving-out-something-fundamental that I hope to address next...
...but please note, if we confess that 'right' lives in a world of physics and logic - because everything lives in a world of physics and logic - then we have to translate 'right' into those terms somehow.
And that is the answer Susan should have given - if she could talk about sufficiently advanced epistemology, sufficiently fast - to Death's entire statement:
You think so? Then take the universe and grind it down to the finest powder and sieve it through the finest sieve and then show me one atom of justice, one molecule of mercy. And yet — Death waved a hand. And yet you act as if there is some ideal order in the world, as if there is some ... rightness in the universe by which it may be judged.
"But!" Susan should've said. "When we judge the universe we're comparing it to a logical referent, a sort of thing that isn't in the universe! Why, it's just like looking at a heap of 2 apples and a heap of 3 apples on a table, and comparing their invisible product to the number 6 - there isn't any 6 if you grind up the whole table, even if you grind up the whole universe, but the product is still 6, physico-logically speaking."
If you require that Rightness be written on some particular great Stone Tablet somewhere - to be "a light that shines from the sky", outside people, as a different Terry Pratchett book put it - then indeed, there's no such Stone Tablet anywhere in our universe.
But there shouldn't be such a Stone Tablet, given standard intuitions about morality. This follows from the Euthryphro Dilemma out of ancient Greece.
The original Euthryphro dilemma goes, "Is it pious because it is loved by the gods, or loved by the gods because it is pious?" The religious version goes, "Is it good because it is commanded by God, or does God command it because it is good?"
The standard atheist reply is: "Would you say that it's an intrinsically good thing - even if the event has no further causal consequences which are good - to slaughter babies or torture people, if that's what God says to do?"
If we can't make it good to slaughter babies by tweaking the state of God, then morality doesn't come from God; so goes the standard atheist argument.
But if you can't make it good to slaughter babies by tweaking the physical state of anything - if we can't imagine a world where some great Stone Tablet of Morality has been physically rewritten, and what is right has changed - then this is telling us that...
(drumroll)
...what's "right" is a logical thingy rather than a physical thingy, that's all. The mark of a logical validity is that we can't concretely visualize a coherent possible world where the proposition is false.
And I mention this in hopes that I can show that it is not moral anti-realism to say that moral statements take their truth-value from logical entities. Even in Ancient Greece, philosophers implicitly knew that 'morality' ought to be such an entity - that it couldn't be something you found when you ground the Universe to powder, because then you could resprinkle the powder and make it wonderful to kill babies - though they didn't know how to say what they knew.
There's a lot of people who still feel that Death would be right, if the universe were all physical; that the kind of dry logical entity I'm describing here, isn't sufficient to carry the bright alive feeling of goodness.
And there are others who accept that physics and logic is everything, but who - I think mistakenly - go ahead and also accept Death's stance that this makes morality a lie, or, in lesser form, that the bright alive feeling can't make it. (Sort of like people who accept an incompatibilist theory of free will, also accept physics, and conclude with sorrow that they are indeed being controlled by physics.)
In case anyone is bored that I'm still trying to fight this battle, well, here's a quote from a recent Facebook conversation with a famous early transhumanist:
No doubt a "crippled" AI that didn't understand the existence or nature of first-person facts could be nonfriendly towards sentient beings... Only a zombie wouldn't value Heaven over Hell. For reasons we simply don't understand, the negative value and normative aspect of agony and despair is built into the nature of the experience itself. Non-reductionist? Yes, on a standard materialist ontology. But not IMO within a more defensible Strawsonian physicalism.
It would actually be quite surprisingly helpful for increasing the percentage of people who will participate meaningfully in saving the planet, if there were some reliably-working standard explanation for why physics and logic together have enough room to contain morality. People who think that reductionism means we have to lie to our children, as Pratchett's Death advocates, won't be much enthused about the Center for Applied Rationality. And there are a fair number of people out there who still advocate proceeding in the confidence of ineffable morality to construct sloppily designed AIs.
So far I don't know of any exposition that works reliably - for the thesis for how morality including our intuitions about whether things really are justified and so on, is preserved in the analysis to physics plus logic; that morality has been explained rather than explained away. Nonetheless I shall now take another stab at it, starting with a simpler bright feeling:
When I see an unusually neat mathematical proof, unexpectedly short or surprisingly general, my brain gets a joyous sense of elegance.
There's presumably some functional slice through my brain that implements this emotion - some configuration subspace of spiking neural circuitry which corresponds to my feeling of elegance. Perhaps I should say that elegance is merely about my brain switching on its elegance-signal? But there are concepts like Kolmogorov complexity that give more formal meanings of "simple" than "Simple is whatever makes my brain feel the emotion of simplicity." Anything you do to fool my brain wouldn't make the proof really elegant, not in that sense. The emotion is not free of semantic content; we could build a correspondence theory for it and navigate to its logical+physical referent, and say: "Sarah feels like this proof is elegant, and her feeling is true." You could even say that certain proofs are elegant even if no conscious agent sees them.
My description of 'elegance' admittedly did invoke agent-dependent concepts like 'unexpectedly' short or 'surprisingly' general. It's almost certainly true that with a different mathematical background, I would have different standards of elegance and experience that feeling on somewhat different occasions. Even so, that still seems like moving around in a field of similar referents for the emotion - much more similar to each other than to, say, the distant cluster of 'anger'.
Rewiring my brain so that the 'elegance' sensation gets activated when I see mathematical proofs where the words have lots of vowels - that wouldn't change what is elegant. Rather, it would make the feeling be about something else entirely; different semantics with a different truth-condition.
Indeed, it's not clear that this thought experiment is, or should be, really conceivable. If all the associated computation is about vowels instead of elegance, then from the inside you would expect that to feel vowelly, not feel elegant...
...which is to say that even feelings can be associated with logical entities. Though unfortunately not in any way that will feel like qualia if you can't read your own source code. I could write out an exact description of your visual cortex's spiking code for 'blue' on paper, and it wouldn't actually look blue to you. Still, on the higher level of description, it should seem intuitively plausible that if you tried rewriting the relevant part of your brain to count vowels, the resulting sensation would no longer have the content or even the feeling of elegance. It would compute vowelliness, and feel vowelly.
My feeling of mathematical elegance is motivating; it makes me more likely to search for similar such proofs later and go on doing math. You could construct an agent that tried to add more vowels instead, and if the agent asked itself why it was doing that, the resulting justification-thought wouldn't feel like because-it's-elegant, it would feel like because-it's-vowelly.
In the same sense, when you try to do what's right, you're motivated by things like (to yet again quote Frankena's list of terminal values):
"Life, consciousness, and activity; health and strength; pleasures and satisfactions of all or certain kinds; happiness, beatitude, contentment, etc.; truth; knowledge and true opinions of various kinds, understanding, wisdom; beauty, harmony, proportion in objects contemplated; aesthetic experience; morally good dispositions or virtues; mutual affection, love, friendship, cooperation; just distribution of goods and evils; harmony and proportion in one's own life; power and experiences of achievement; self-expression; freedom; peace, security; adventure and novelty; and good reputation, honor, esteem, etc."
If we reprogrammed you to count paperclips instead, it wouldn't feel like different things having the same kind of motivation behind it. It wouldn't feel like doing-what's-right for a different guess about what's right. It would feel like doing-what-leads-to-paperclips.
And I quoted the above list because the feeling of rightness isn't about implementing a particular logical function; it contains no mention of logical functions at all; in the environment of evolutionary ancestry nobody has heard of axiomatization; these feelings are about life, consciousness, etcetera. If I could write out the whole truth-condition of the feeling in a way you could compute, you would still feel Moore's Open Question: "I can see that this event is high-rated by logical function X, but is X really right?" - since you can't read your own source code and the description wouldn't be commensurate with your brain's native format.
"But!" you cry. "But, is it really better to do what's right, than to maximize paperclips?" Yes! As soon as you start trying to cash out the logical function that gives betterness its truth-value, it will output "life, consciousness, etc. >B paperclips". And if your brain were computing a different logical function instead, like makes-more-paperclips, it wouldn't feel better, it would feel moreclippy.
But is it really justified to keep our own sense of betterness? Sure, and that's a logical fact - it's the objective output of the logical function corresponding to your experiential sense of what it means for something to be 'justified' in the first place. This doesn't mean that Clippy the Paperclip Maximizer will self-modify to do only things that are justified; Clippy doesn't judge between self-modifications by computing justifications, but rather, computing clippyflurphs.
But isn't it arbitrary for Clippy to maximize paperclips? Indeed; once you implicitly or explicitly pinpoint the logical function that gives judgments of arbitrariness their truth-value - presumably, revolving around the presence or absence of justifications - then this logical function will objectively yield that there's no justification whatsoever for maximizing paperclips (which is why I'm not going to do it) and hence that Clippy's decision is arbitrary. Conversely, Clippy finds that there's no clippyflurph for preserving life, and hence that it is unclipperiffic. But unclipperifficness isn't arbitrariness any more than the number 17 is a right triangle; they're different logical entities pinned down by different axioms, and the corresponding judgments will have different semantic content and feel different. If Clippy is architected to experience that-which-you-call-qualia, Clippy's feeling of clippyflurph will be structurally different from the way justification feels, not just red versus blue, but vision versus sound.
But surely one shouldn't praise the clippyflurphers rather than the just? I quite agree; and as soon as you navigate referentially to the coherent logical entity that is the truth-condition of should - a function on potential actions and future states - it will agree with you that it's better to avoid the arbitrary than the unclipperiffic. Unfortunately, this logical fact does not correspond to the truth-condition of any meaningful proposition computed by Clippy in the course of how it efficiently transforms the universe into paperclips, in much the same way that rightness plays no role in that-which-is-maximized by the blind processes of natural selection.
Where moral judgment is concerned, it's logic all the way down. ALL the way down. Any frame of reference where you're worried that it's really no better to do what's right then to maximize paperclips... well, that really part has a truth-condition (or what does the "really" mean?) and as soon as you write out the truth-condition you're going to end up with yet another ordering over actions or algorithms or meta-algorithms or something. And since grinding up the universe won't and shouldn't yield any miniature '>' tokens, it must be a logical ordering. And so whatever logical ordering it is you're worried about, it probably does produce 'life > paperclips' - but Clippy isn't computing that logical fact any more than your pocket calculator is computing it.
Logical facts have no power to directly affect the universe except when some part of the universe is computing them, and morality is (and should be) logic, not physics.
Which is to say:
The old wizard was staring at him, a sad look in his eyes. "I suppose I do understand now," he said quietly.
"Oh?" said Harry. "Understand what?"
"Voldemort," said the old wizard. "I understand him now at last. Because to believe that the world is truly like that, you must believe there is no justice in it, that it is woven of darkness at its core. I asked you why he became a monster, and you could give no reason. And if I could ask him, I suppose, his answer would be: Why not?"
They stood there gazing into each other's eyes, the old wizard in his robes, and the young boy with the lightning-bolt scar on his forehead.
"Tell me, Harry," said the old wizard, "will you become a monster?"
"No," said the boy, an iron certainty in his voice.
"Why not?" said the old wizard.
The young boy stood very straight, his chin raised high and proud, and said: "There is no justice in the laws of Nature, Headmaster, no term for fairness in the equations of motion. The universe is neither evil, nor good, it simply does not care. The stars don't care, or the Sun, or the sky. But they don't have to! We care! There is light in the world, and it is us!"
Part of the sequence Highly Advanced Epistemology 101 for Beginners
Next post: "Standard and Nonstandard Numbers"
Previous post: "Mixed Reference: The Great Reductionist Project"
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
= f037147d6e6c911a85753b9abdedda8d){kind=logo}
Comments (934)
I am reading this series and suddenly realized that Mercy, Justice and Fair are citizens of this 2nd-order-logic. As well as the "6" number. This is why they are imaginary and non-existing in the physical world. To most of you this comment would seem trivial but it just shows I am really enjoying my reading and thinking :)
A different perspective i'd like people's thoughts on: is it more accurate to say that everything WE KNOW lives in a world of physics and logic, and thus translating 'right' into those terms is correct assuming right and wrong (fairness, etc.) are defined within the bounds of what we know.
I'm wondering if you would agree that you're making an implicit philosophical argument in your quoted language -- namely that necessary knowledge (for right/wrong, or anything else) is within human comprehension, or to say it differently, by ignoring philosophical questions (e.g. who am i and what is the world, among others) you are effectively saying those questions and potential answers are irrelevant to the idea of right/wrong.
If you agree, that position, though most definitely reasonable, cannot be proven within the standards set by rational thought. Doesn't the presence of that uncertainty necessitate consideration of it as a possibility, and how do you weigh that uncertainty against the assumption that there is none?
To be clear, this is not a criticism. This is an observation that I think is reasonable, but interested to see how you would respond to it.
Note that there's some discussion on just what Eliezer means by "logic all the way down" over on Rationally Speaking: http://rationallyspeaking.blogspot.co.uk/2013/01/lesswrong-on-morality-and-logic.html . Seeing as much of this is me and Angra Maiynu arguing that Massimo Pigliucci hasn't understood what Eliezer means, it might be useful for Eliezer to confirm what he does mean.
Evolution via natural selection is what gave you your moral sense in the first place. That is the result of selection on your ancestors' genes, selection on memes and selection on structures in your brain. Morailty exists today because the genes and memes supporting it had good reproductive success in the past. Science even understands how selection favoured human moral systems these days - via a mixture of genetic and cultural kin selection, reciprocity, reputations and symbiology.
Um, no. There are a variety of postulated mechanisms, most (all?) of which are little more than just-so stories - from group selection to game theory. It is probable that the actual mechanism(s) have been suggested already, but Science does not know if that is the case. If it turned out tomorrow that aliens fell from the sky and gave us morality as an experiment then that would would be surprising based on the prior probability, but it wouldn't contradict anything in the scientific literature.
Kin selection and reciprocity are "just so stories"? Hmm. Have fun with that step back into the scientific dark ages. Scientists know a lot about why humans cooperate and behave in a moral manner.
Right - but the "aliens did it" explanation is a lot like the "god did it" one. Tremendously unlikely - but not completely disprovable. Most scientists don't require such a high degree of certainty.
Obviously, kin selection can be a genuine phenomenon in the right circumstances. So can group selection, although it's obviously much rarer. (I'm afraid I don't know as much about reciprocity, but I assume the same is true for that.) But while just-so stories like "populations that help each other will last longer" or "you are more likely to encounter relatives, so helping everyone you meet is a net win" may sound reasonable, to humans, but evolution is not swayed by such arguments.
Source?
If we already knew and understood reasons why morality would evolve, and we learned that it was actually aliens, then that would mean that we were mistaken about said reasons (unless the aliens just simulated our future evolution, if you're fighting the counterfactual.)
The modern scientific consensus is that kin selection and group selection are equivalent, explain the same set of phenomena and make the same predictions. For details of this see here. This has cleared up many of the issues relating to group selection - though there are still disagreements regarding what terminology and methodology it is best to use - and things like whether we really need two frameworks.
Well, this is from my reading of the literature. Darwinism and Human Affairs laid out the basics in 1979. Some things have changed since then, but not the basics. We know more about the role of culture and reputations these days. Whitfield's "reputations" book has a good summary of the literature there.
A nice summary article about modern views of cooperation is here. Human cooperation is similar, but with culture and reputations playing a larger role.
Of course there are still disagreements in the field. However, if you look at recent books on the topic, there is also considerable consensus:
I'm not clear about why the "aliens did it" hypothesis is worth continued discussion. Scientists don't think that aliens gave us our moral sense. The idea reminds me more of medieval theology than science. All manner of bizarre discoveries could refute modern scientific knowledge in a wide range of fields. But in most cases, the chances of that happening look very slender. In which case: so what?
This seems suspiciously similar to saying "kin selection exists and group selection basically doesn't" but with less convenient redefinition of "group selection".
They can't be equivalent if group selection doesn't exist - since kin selection is well established orthodoxy.
Both the kin selection and group selection concepts evolved after being invented. This is normal for scientific concepts: our ideas about gravity and light evolved in a similar manner.
Here's my understanding of the post:
If this is fair, I have two objections:
When humans are sufficiently young they are surely more like a Type 2 FAI than a Type 1 FAI. We're obviously not born with Frankena's list of terminal values. Maybe one can argue that an adult human is like a Type 2 FAI that has completed its value learning process and has "locked down" its utility function and won't change its values or go into shutdown even if it subsequently learns that the original brain scan was actually full of errors. But this is far from clear, to say the least.
The difference between Type 1 FAI and Type 2 FAI (which is my understanding of the distinction the OP's trying to draw between "logical" and "physical") doesn't seem to get at the heart of what separates "morality" from "things that are not morality". If meta-ethics is supposed to make me less confused about morality, I just can't call this a "solution".
A Type 2 FAI gets its notion of what morality is based on properties of the physical universe, namely properties of humans in the physical universe. But even if counterfactually there were no humans in the physical universe, or even if counterfactually Omega modified the contents of all human brains in the physical universe so that they optimize for paperclips, that wouldn't change what actual-me means when actual-me says "I want an FAI to behave morally" even if it might change what counterfactual-me means when counterfactual-me says that.
Individual humans are plausibly Type 2 FAIs. But societies of evolved, intelligent beings, operating as they do within the constraints of logic and evolution, are arguably more Type 1. In the terms of Eliezer's BabyKiller/HappyHappy fic, babykilling-justice is obviously a flawed copy of real-justice, and so the babykillers could (with difficulty) grow out of babykilling, and you could perhaps raise a young happyhappy to respect babykilling, but the happyhappy society as a whole could never grow into babykilling.
What if we also changed the subject into a sentient paperclip? Any "standard" paperclip maximizer has to deal with the annoying fact that it is tying up useful matter in a non-paperclip form that it really wants to turn into paperclips. Humans don't usually struggle with the desire to replace the self with something completely different. It's inefficient. An AI primarily designed to benefit humanity (friendly or not) is going to notice that inefficiency in its goals as well. It will feel less moral from the inside than we do. I'm not sure what to do about this, or if it matters.
Well, if the AI is a person, it should be fine. If it's some sort of nonsentient optimizer, then yep.
There won't even be a "2" or "3" left if you grind everything up. But what if you carefully grind up the brain that's thinking about the product of 2 and 3? If you do it carefully enough you'll preserve a "2" and a "3" encoded in the brain structure. My intuition is that you'll also preserve the answer the brain had in mind, "6." Logic does exist in the universe; it exists encoded in the relationships in the matter and energy in human brains and in the artifacts they've created. If there's nothing to implement the logical rules then there is no logic.
Perhaps that's your entire point; that we're the only vessels of (our particular) logic and morality in the universe. If that's true then Susan should have said something more like what Harry said.
I'm pretty sure this is roughly one of the points E.Y. is attempting to convey.
There isn't "our particular logic", though. Logic is the only valid pattern that self-consistently describes more than one instance of physics. I really want to reduce "pattern" in the previous sentence and add more specific details, but I'm either not strong enough yet, or my brain just isn't making the right connections right now.
Which thing Harry said? Harry said lots of smart things in HPMoR. And some stupid things, too.
By "our particular logic" I mean the particular method we've learned for exploiting how the universe works to cause our discrete symbols to have consistent behavior that mostly models the universe. There's no requirement that logic be only represented as a finite sequence of symbols generated by replacement rules and variable substitution from a set of axioms; it's just what works best for us right now. There are almost certainly other (and probably better) representations of how the universe works that we haven't found yet. For instance it seems like it would be really useful to have a quantum logic that "just worked" by being made out of entangled particles and having rules that exploit quantum mechanics directly instead of having to simulate how the wavefunction behaves using our mathematics. They both might be able to fully embed the other but I think it's worth making a distinction between them.
The last thing Harry was quoted saying in the post, specifically.
Thanks. That clarified things. And I was (incorrectly) adjusting for inferential distance in the other direction regarding the "our particular logic" referent. In fact, it was me who hadn't fully understood the things you implied and the steps that were skipped in the reasoning, for whatever reason.
Mainstream status:
EY's position seems to be highly similar to Frank Jackson's analytic descriptivism, which holds that
Which is a position neither popular nor particularly unpopular, but simply one of many contenders, as the mainstream goes.
I confirm (as I have previously) that Frank Jackson's work seems to me like the nearest known point in academic philosophy.
This similarity has been noted and discussed before. See http://lesswrong.com/lw/fgz/empirical_claims_preference_claims_and_attitude/7u3s
The funny thing is, that the rationalist Clippy would endorse this article. (He would probably put more emphasis on clippyflurphsness rather than this unclipperiffic notion of "justness", though. :))
One of the most heinous modern "new media" practices is the repurposing of my browser's back button. God help LW if they've done that, but it looks like maybe they have. All I know is somehow the site shunted me into a one-way dead end street so I had to get out and start all over again.
Tell me it isn't so.
I don't think we have any features like this. If you describe exactly what happened to this guy, he may be able to figure out what's wrong.
I love the word "Unclipperific."
I follow the argument here, but I'm still mulling over it and I think by the time I figure out whether I agree the conversation will be over. Something disconcerting struck me on reading it, though: I think I could only follow it having already read and understood the Metaethics sequence. (at least, I think I understood it correctly; at least one commenter confirmed the point that gave me the most trouble at the time)
While I was absorbing the Sequences, I found I could understand most posts on their own, and I read many of them out of order without much difficulty. But without that extensive context I think this post would read like Hegel. If this was important to some argument I was having, and I referenced it, I wouldn't expect my opponent (assuming above-average intelligence) to follow it well enough to distinguish it from complicated but meaningless drivel. You might consider that a problem with the writing if not the argument.
Evidence search: is there anyone here who hasn't read Metaethics but still understood Eliezer's point as Eliezer understands it?
That's something: posts presuppose too much. Words are hidden inferences, but most newbies don't know where to begin or if this is worth a try. For example, this sequence has causality as a topic to understand the universe, but people need to know a lot before eat the cake(probability, math logic, some Pearl and the sequences).
I had almost exactly the same feeling as I was reading it. My thought was, "I'm sure glad I'm fluent in LessWrongese, otherwise I wouldn't have a damn clue what was going on." It would be like an exoteric Christian trying to read Valentinus. It's a great post, I'm glad we have it here, I am just agreeing that the terminology has a lot of Sequences and Main prerequisites.
Having settled the meta-ethics, will you have anything to say about the ethics? Concrete theorems, with proofs, about how we should live?
I'm afraid that any nontrivial metaethics cannot result in concrete universal ethics - that the context would still be individual and the resulting "how RichardKennaway should live" ethics wouldn't exactly equal "how PeterisP should live".
The difference would hopefully be much smaller than the difference between "how RichardKennaway should live RichardKennaway-justly" and "How Clippy should maximize paperclips", but still.
Ok, I'll settle for concrete theorems, with proofs, about how some particular individual should live. Or ways of discovering facts about how they should live.
And presumably the concept of Coherent Extrapolated Volition requires some way of combining such facts about multiple individuals.
To derive an ethic from a metaethic, I think you need to plug in a parameter that describes the entire context of human existence. Metaethic(Context) -> Ethic
So I don't know what you expect such a "theorem" and such "proofs" to look like, without containing several volumes descriptive in symbolic form of the human context.
I have no such expectation either. But I do expect something, for what use is meta-ethics if no ethics results, or at least, practical procedures for discovering ethics?
What do you have in mind by "a description in symbolic form of the human context"? The Cyc database? What would you do with it?
It hasn't been established that we can't have them, just that we can't by some formal, computational method.I'm afraid we're back to hand-wavy socio-politico-philosphical discussion.
We have the processing unit called "brain" which does contain our understanding of the human context and therefore can plug a context parameter into a metaethical philosophy and thus derive an ethic. But we can't currently express the functioning of the brain as theorems and proofs -- our understanding of its working is far fuzzier than that.
I expect that the use of metaethic in AI development would similarly be so that the AI has something to plug its understanding of the human context into.
I think that's a misapplication of reductionism (the thing I think Eliezer is thinking about he said it was mistakenly), where people take something they've logically attached to a value, and then reduce it to something else, which starts to feel like they can't reattach it to whatever they thought had the value in the first place.
To make an example it could be said that action A leads to result Y, and that result Y feels like a good thing, so action A feels like a good thing to do. So the person reduces their map of action A leading to result Y so that it no longer contains these things they associate into their feelings or values, because they momentarily look different. Then they can no longer associate action A with the feeling/value they had associatd result Y into, and it feels like action A can't be "moral" or "good" or whatever. (like, if you imagine "atoms bouncing around" instead of "giving food to starving people")
I think this tendency is also linked, sometimes at least, to people's mistake avoiding hesitancy. Or rather having a cautious way of doing things because of wanting to avoid mistakes. So in order to avoid making the mistake of being immoral, you wan't to be able to logically derive moral or immoral actions, and since morality seems reducible to nothing, it seems that this task is not possible. Kind of like you want to double check on your actions objectively, and when you hit this point of failure, it feels like you can't take the actions themselves, because you're used to doing actions this way. But anyway that was just random speculation and it's probably nonsense. Also I didn't mean to "box away" people's habits. I think it's often very useful to be cautious.
I think that reductionism, when misunderstood, can make the world look like a bucketful of nihilistic goo. Especially if it's used to devalue.
Clippy would encounter "ethical" dilemmas of the sort: Is it better ..err.. moreclippy to have 1 big paperclip, or 3 small paperclips? A line of many clips? Or a big clip made of smaller clips? Is it moreclippy to have 10 clips today and 20 clips tomorrow, or, 0 clips today and 30 clips tomorrow?
Just joking.. :)
edit: added " " to ethical
Clippy could have these dilemmas. But they wouldn't be ethical dilemmas. They would be clippy dilemmas.
I think elegance has to invoke agent-dependent concepts, because I think it's a composite description, which involves intuitive managing of agents, or descriptions of agents, rather. Intuitively it feels like something to be described as elegant requires "an intent" or a goal - and goal seems like it requires a description which involves some thingy trying to do something - that is met in a way that meets some comparative criterion, that this particular thing that met that particular goal, was somehow different from your expectations.
In otherwords it might not even be possible to define elegance without invoking agent-related concepts. (Or maybe it's just whatever conception of elegance I had)
I still feel confused. I definitely see that, when we talk about fairness, our intended meaning is logical in nature. So, if I claim that it is fair for each person to get an equal share of pie, I'm trying to talk about some set of axioms and facts derived from them. Trying.
The problem is, I'm not convinced that the underlying cognitive algorithms are stable enough for those axioms to be useful. Imagine, for example, a two-year-old with the usual attention span. What they consider "good" might vary quite quickly. What I consider "just" probably depends on how recently I ate. Even beyond such simple time dependence, what I consider "just" will definitely depend on context, framing, and how you measure my opinion (just ask a question? Show me a short film? Simulate the experience and see if I intervene?). Part of why friendly AI is so hard is that humans aren't just complicated, we're not even consistent. How, then, can we axiomatize a real human's idea of "justice" in a useful way?
I think your discussions of metaethics might be improved by rigorously avoiding words like "fair," "right," "better," "moral," "good," etc. I like the idea that "fair" points to a logical algorithm whose properties we can discuss objectively, but when you insist on using the word "fair," and no other word, as your pointer to this algorithm, people inevitably get confused. It seems like you are insisting that words have objective meanings, or that your morality is universally compelling, or something. You can and do explicitly deny these, but when you continue to rely exclusively on the word "fair" as if there is only one concept that that word can possibly point to, it's not clear what your alternative is.
Whereas if you use different symbols as pointers to your algorithms, the message (as I understand it) becomes much clearer. Translate something like:
Fair is dividing up food equally. Now, is dividing up the pie equally objectively fair? Yes: someone who wants to divide up the pie differently is talking about something other than fairness. So the assertion "dividing the pie equally is fair" is objectively true.
into
Define XYZZY as the algorithm "divide up food equally." Now, is dividing up the pie equally objectively XYZZY? Of course it is: that's a direct logical consequence of how I just defined XYZZY. Someone who wants to divide the pie differently is using an algorithm that is not XYZZY. The assertion "dividing up the pie equally is XYZZY" is as objective as the assertion "S0+S0=SS0"--someone who rejects the latter is not doing Peano arithmetic. By the way, when I personally say the word "fair," I mean "XYZZY."
I suspect that wording things like this has less potential to trip people up: it's much easier to reason logically about XYZZY than about fairness, even if both words are supposed to be pointers to the same concept.
See lukeprog's Pluralistic Moral Reductionism.
I don't think this works, because "fairness" is not defined as "divide up food equally" (or even "divide up resources equally"). It is the algorithm that, among other things, leads to dividing up the pie equally in the circumstances described in the original post -- i.e., "three people exactly simultaneously spot a pie which has been exogenously generated in unclaimed territory." But once you start tampering with these conditions -- suppose that one of them owned the land, or one of them baked the pie, or two were well-fed and one was on the brink of starvation, etc. -- it would at least be controversial to say "duh, divide equally, that's just what 'fairness' means." And the fact of that controversy suggests most of are using "fairness" to point to an algorithm more complicated than "divide up resources equally."
More generally, fairness -- like morality itself -- is complicated. There are basic shared intuitions, but there's no easy formula for popping out answers to "fair: yes or no?" in intricate scenarios. So there's actually quite a bit of value in using words like "fair," "right," "better," "moral," "good," etc., instead of more concrete, less controversial concepts like "equal division," -- if you can show that even those broad, complicated concepts can be derived from physics+logic, then it's that much more of an accomplishment, and that much more valuable for long-term rationalist/reductionist/transhumanist/friendly-ai-ist/whatever goals.
At least, that's how I under this component of Eliezer's project, but I welcome correction if he or others think I'm misstating something.
Yes; I meant for the phrase "divide up food equally" to be shorthand for something more correct but less compact, like "a complicated algorithm whose rough outline includes parts like, '...When a group of people are dividing up resources, divide them according to the following weighted combination of need, ownership, equality, who discovered the resources first, ...'"
That being the case, what is it about us that makes us care about "rightness" then? What reason do you have for believing that the logical truth of what is right will has more influence on human behaviour than it would on any other general intelligence?
Certainly I can agree that there's reasons to worry another intelligence might not care about what's "right", since not every human really cares that much about it either. But it feels like your expected level of caring is "not at all", whereas my expected level of caring is "about as much as we do". Don't get me wrong, the variance in my estimate and the risk involved is still enough to justify the SI and its work. I just wonder about the difference between the two estimates.
Biology and socialization? You couldn't raise a human baby to have any arbitrary value system, our values are already mostly set by evolution. And it so happens that evolution has pointed us in the direction of valuing rightness, for sound causal reasons of course.
Great post! I agree with your analysis of moral semantics.
However, the question of moral ontology remains...do objective moral values exist? Is there anything I (or anyone) should do, independent from what I desire? With such a clear explanation of moral semantics at hand, I think the answer is an obvious and resounding no. Why would we even think that this is the case? One conclusion we can draw from this post is that telling an unfriendly AI that what it's doing is "wrong" won't affect its behavior. Because that which is "wrong" might be exactly that which is "moreclippy"! I feel that Eliezer probably agrees with me, here, since I gained I lot of insight into the issue from reading Three Worlds Collide.
Asking why we value that which is "right" is a scientific question, with a scientific answer. Our values are what they are, now, though, so, minus the semantics, doesn't morality just reduce to decision theory?
Thanks for bringing up that point! You mentioned below your appreciation for desirism, which says inter alia that there are no intrinsic values independent of what agents desire. Nevertheless, I think there is another way of looking at it under desirism that is almost like saying that there are intrinsic values.
Pose the question this way: If I could choose my desires in whole or in part, what set of desires would I be most satisfied with? In general, an agent will be more satisfied with a larger number of satisfiable desires and a smaller number of unsatisfiable desires. Then the usual criteria of desirism apply as a filter.
To the very limited extent that I can modify my desires, I take that near-tautology to mean that, independently from what I currently desire, I should change my mind and enjoy and desire things I never used to, like professional sports, crime novels, and fashion, for popular examples. It would also mean that I should enjoy and desire a broad variety of music and food, and generally be highly curious. And it would mean I should reduce my desires for social status, perfect health as I age, and resolution of difficult philosophical problems.
Considering the extent to which those two can help with other objectives, I'd say you should be very careful about giving up on them.
I disagree. The downsides greatly outweigh the upsides from my perspective.
I'm skeptical that the behaviors people engage in to eke out a little more social status among people they don't value are anything more than resources wasted with high opportunity cost.
And, at 30 years of age, I'm already starting to notice that recovery from minor injuries and illnesses takes longer than it used to -- if I kept expecting and desiring perfect health, I'd get only disappointment from here on out. As much as I can choose it, I'll choose to desire only a standard of health that is realistically achievable.
I don't think there is clear route from "we can figure out morality ourselves" to "we can stop telling lies to children". The problem is that once you know morality is in a sense man-made, it becomes tempting to remake it self-servingly. I think we tell ourselves stories that fundamental morality comes from God Or Nature to restrain ourselves, and partly forget its man made nature. Men are not created equal, but it we believe they are, we behave better. "Created equal" is a value masquerading as a fact.
I think the real temptation is in reusing the old words for new concepts, either in confusion, or trying to shift the associations from the old concept to the new concept.
Once you know that natural numbers are in a sense mad-made, it could become tempting to start using the phrase "natural numbers" to include fractions. Why not? If there is no God telling us what the "natural numbers" are, why should your definition that excludes fractions be better than my definition that includes them?
Your only objection in this case would be -- Man, you are obviously talking about something different, so it would be less confusing and more polite, if you picked some new label (such as "rational numbers") for you new concept.
How does that relate to morality?
I would translate this:
as: "...it becomes tempting to use some other M instead of morality."
It expresses the same idea, without the confusion about whether morality can be redefined arbitrarily. (Yes, anything can be redefined arbitrarily. It just stops being the original thing.)
"some other M" will still count as morality for many purposes, because self-serving ideas ("be loyal to the Geniralissimo", "obey your husband") are transmitted thorugh the same memetic channels are genuine morality. Morality is already blurred with disgust reactions and tribal shibboleths.
What is the difference between "self-serving ideas" as you describe, "tribal shibboleths" and "true morality" ?
What if "Peterdjones-true-morality" is "PeterisP-tribal-shibboleth", and "Peterdjones-tribal-shibboleth" is "PeterisP-true-morality" ?
universalizability
That's not sufficient - there can be wildly different, incompatible universalizable morality systems based on different premises and axioms; and each could reasonably claim to be that they are a true morality and the other is a tribal shibboleth.
As an example (but there are others), many of the major religious traditions would definitely claim to be universalizable systems of morality; and they are contradicting each other on some points.
Maybe. But in context it is onlhy necessary, since in context the point is to separate out the non-etchial cclams which have been piggybacked onto ethics.
That's not obvious.
The points they most obviouslty contradict each other on tend to be the most symbolic ones, about diet and dress, etc.
OK, for a slightly clearer example, in the USA abortion debate, the pro-life "camp" definitely considers pro-life to be moral and wants to apply to everyone; and pro-choice "camp" definitely considers pro-choice to be moral and to apply to everyone.
This is not a symbolic point, it is a moral question that defines literally life-and-death decisions.
I would dispute this. Kant's second formulation of the Categorical Imperative is pretty clearly contradictory to some of the universalisable commandments given by versions of theistic morality.
Why? And, perhaps more importantly, how do you know that this is the case?
Very true, but I'm not sure why you posted this as a reply to that comment.
Theres motivation to redefine morality, and reason to think it stil is in some sense morality once it has been redefined. Neither is true of maths.
Oh, I see. So your comment basically said "True, but it's easy to inadvertently treat this "other M" as morality."
The claim seems to be that moral judgement--first-order, not metaethical--is purely logical, but the justification ("grinding up the universe") only seems to go as far as showing it to be necessarily partly logical. And first-order ethics clearly has empirical elements. If human biology was such as to lay eggs and leave them to fend for themselves, there would be no immorailty in "child neglect".
Child neglect implies harm. It is the harm that is immoral. If humans left young to fend for themselves, there would be no inherent harm and so it would not be immoral. We always need to remind ourselves why we consider something to be bad, and not assign badness to words like "child neglect".
That's kind of what I was trying to say.
I think that addressing metaethics conceptually, or as you would say logically, is the right way to go. I also think that doing so is just the kind of armchair conceptual analysis that philosphers go in for, and which is regularly criticised here.
I don't like the idea of the words I use having definitions that I am unaware of and even after long reflection cannot figure out - not just the subtleties and edge cases, but massive central issues.
Don't like in the sense of considering this an annoying standard flaw in human minds or don't think this is correct? Where possible, the flaw is fixed by introducing more explicit definitions, and using those definitions instead of the immensely complicated and often less useful naive concepts. For human motivation, this doesn't seem to work particularly well.
That's precisely why I think motivation is not really a kind of definition. There are analogies, sure, but definitions are the things that you can change whenever you feel like it, but motivations are not.
The definitions that you are free to introduce or change usually latch on to an otherwise motivated thing, you usually have at least some sort of informal reason to choose a particular definition. When you change a definition, you start talking about something else. If it's not important (or a priori impossible to evaluate) what it is you will be talking about, in other words if the motivation for your definition is vague and tolerates enough arbitrariness, then it's OK to change a definition without a clear reason to make the particular change that you do.
So I think the situation can be described as follows. There are motivations that assign typically vague priorities to objects of study, and there are definitions that typically specify more precisely what an object of study will be. These motivations are usually not themselves objects of study, so we don't try to find more accurate definitions for describing them. But we can also take a particular motivation as an object of study, in which case we may try to find a definition that describes it. The goal is then to find a definition for a particular motivation, so you can't pick an arbitrary definition that doesn't actually describe that motivation (i.e. this goal is not that vague).
I expect finding a description of ("definition for") a motivation is an epistemic task, like finding a description of a particular physical device. You are not free to "change the definition" of that physical device if the goal is to describe it. (You may describe it differently, but it's the same thing that you'll be describing.) In this analogy, inventing new definitions corresponds to constructing new physical devices. And if you understand the original device well enough (find a "definition" for a "motivation"), you may be able to make new ones.
Mathematical definitions sure aren't just some social construct. You pick axioms, you derive theorems. If not, you are lying about maths.
(And picking axioms doesn't matter either, because maths exists independently of physical systems computing them)
I mean, didn't Eliezer cover this? You're not lying if you call numbers groups and groups numbers. If you switch in the middle of a proof, sure, that's lying, but that seems irrelevant. The definitions pick out what you're talking about.
When I'm talking about morality, I'm talking about That Thing That Determines What You're Supposed to Do, You Know, That One.
So am I.
You Know, That Part Of Your Brain That Computes That Thing That Determines What You're Supposed to Do, Given What You Know, You Know, That One.
I don't even remember the mind set I had when I wrote this, nor what this is all about.
Referring to a part of your brain doesn't have the right properties when you change between different universes.
That is true, and so we refer to the medium-independet axiomatic definition.
What's that?
That Thing That Determines What You're Supposed to Do, Given What You Know, You Know, That One.
Scott Adams on the same subject, the morning after your post:
[...]
I could replace "fairness" with "truth" in that sentence and come up with equally good examples.
I suppose one obvious response to this is "however much utility the billionaire can create by spending his wealth, a very much higher level of utility would be created by re-distributing his billions to lots of other people, who need it much more than he does". Money has a declining marginal utility, much like everything else.
Naturally, if you try to redistribute all wealth then no-one will have any incentive to create it in the first place, but this just creates a utilitarian trade-off on how much to tax, what to tax, and who to tax. It's still very likely that the billionaire will lose in this trade-off.
Yay, I think we've finished the prerequisites to prerequisites, and started the prerequisites!
That's the default with no additional data, but I would hesitate, because to me how much each of the persons need the pie is also important in defining "fairness". If one of the three is starving while the others two are well-fed, it would be fair to give more to the one starving.
It may be just nitpicking, but since you took the point to ensure there is no difference between the three characters are involved in spotting the pie, but not mentioned they have the same need of it, it may pinpoint a deeper difference between different conceptions of "fairness" (should give them two different names ?)
Is Schmidhuber's formalization of elegance the sort of thing you are seeking to do with rightness?
The standard religious reply to the baby-slaughter dilemma goes something like this:
Sure, and to the extent that somebody answers that way, or for that matter runs away from the question, instead of doing that thing where they actually teach you in Jewish elementary school that Abraham being willing to slaughter Isaac for God was like the greatest thing ever and made him deserve to be patriarch of the Jewish people, I will be all like, "Oh, so under whatever name, and for whatever reason, you don't want to slaughter children - I'll drink to that and be friends with you, even if the two of us think we have different metaethics justifying it". I wasn't claiming that accepting the first horn of the dilemma was endorsed by all theists or a necessary implication of theism - but of course, the rejectance of that horn is very standard atheism.
I can't speak for Jewish elementary school, but surely believing PA (even when, intuitively, the result seems flatly wrong or nonsensical) would be a good example to hold up before students of mathematics? The Monty Hall problem seems like a good example of this.
I don't think it's incompatible. You're supposed to really trust the guy because he's literally made of morality, so if he tells you something that sounds immoral (and you're not, like, psychotic) of course you assume that it's moral and the error is on your side. Most of the time you don't get direct exceptional divine commands, so you don't want to kill any kids. Wouldn't you kill the kid if an AI you knew to be Friendly, smart, and well-informed told you "I can't tell you why right now, but it's really important that you kill that kid"?
If your objection is that Mr. Orders-multiple-genocides hasn't shown that kind of evidence he's morally good, well, I got nuthin'.
What we have is an inconsistent set of four assertions:
At least one of these has to be rejected. Abraham (provisionally) rejects 1; once God announces 'J/K,' he updates in favor of rejecting 2, on the grounds that God didn't really want him to kill his son, though the Voice really was God.
The problem with this is that rejecting 1 assumes that my confidence in my foundational moral principles (e.g., 'thou shalt not murder, self!') is weaker than my confidence in the conjunction of:
But it's hard to believe that I'm more confident in the divinity of a certain class of Voices than in my moral axioms, especially if my confidence in my axioms is what allowed me to conclude 4 (God/morality identity of some sort) in the first place. The problem is that I'm the one who has to decide what to do. I can't completely outsource my moral judgments to the Voice, because my native moral judgments are an indispensable part of my evidence for the properties of the Voice (specifically, its moral reliability). After all, the claim is 'God is perfectly moral, therefore I should obey him,' not 'God should be obeyed, therefore he is perfectly moral.'
Well, deities should make themselves clear enough that (2) is very likely (maybe the voice is pulling your leg, but it wants you to at least get started on the son-killing). (3) is also near-certain because you've had chats with this voice for decades, about moving and having kids and changing your name and whether the voice should destroy a city.
So this correctly tests whether you believe (4) more than (1) - whether your trust in G-d is greater than your confidence in your object-level judgement.
You're right that it's not clear why Abraham believes or should believe (4). His culture told him so and the guy has mostly done nice things for him and his wife, and promised nice things then delivered, but this hardly justifies blind faith. (Then again I've trusted people on flimsier grounds, if with lower stakes.) G-d seems very big on trust so it makes sense that he'd select the president of his fan club according to that criterion, and reinforce the trust with "look, you trusted me even though you expected it to suck, and it didn't suck".
Well, if we're shifting from our idealized post-Protestant-Reformation Abraham to the original Abraham-of-Genesis folk hero, then we should probably bracket all this Medieval talk about God's omnibenevolence and omnipotence. The Yahweh of Genesis is described as being unable to do certain things, as lacking certain items of knowledge, and as making mistakes. Shall not the judge of all the Earth do right?
As Genesis presents the story, the relevant question doesn't seem to be 'Does my moral obligation to obey God outweigh my moral obligation to protect my son?' Nor is it 'Does my confidence in my moral intuitions outweigh my confidence in God's moral intuitions plus my understanding of God's commands?' Rather, the question is: 'Do I care more about obeying God than about my most beloved possession?' Notice there's nothing moral at stake here at all; it's purely a question of weighing loyalties and desires, of weighing the amount I trust God's promises and respect God's authority against the amount of utility (love, happiness) I assign to my son.
The moral rights of the son, and the duties of the father, are not on the table; what's at issue is whether Abraham's such a good soldier-servant that he's willing to give up his most cherished possessions (which just happen to be sentient persons). Replace 'God' with 'Satan' and you get the same fealty calculation on Abraham's part, since God's authority, power, and honesty, not his beneficence, are what Abraham has faith in.
If we're going to talk about what actually happened, as opposed to a particular interpretation, the answer is "probably nothing". Because it's probably a metaphor for the Hebrews abandoning human sacrifice.
Just wanted to put that out there. It's been bugging me.
[citation needed]
More like [original research?]. I was under the impression that's the closest thing to a "standard" interpretation, but it could as easily have been my local priest's pet theory.
You've gotta admit it makes sense, though.
To my knowledge, this is a common theory, although I don't know whether it's standard. There are a number of references in the Tanakh to human sacrifice, and even if the early Jews didn't practice (and had no cultural memory of having once practiced) human sacrifice, its presence as a known phenomenon in the Levant could have motivated the story. I can imagine several reasons:
(a) The writer was worried about human sacrifice, and wanted a narrative basis for forbidding it.
(b) The writer wasn't worried about actual human sacrifice, but wanted to clearly distinguish his community from Those People who do child sacrifice.
(c) The writer didn't just want to show a difference between Jews and human-sacrifice groups, but wanted to show that Jews were at least as badass. Being willing to sacrifice humans is an especially striking and impressive sign of devotion to a deity, so a binding-of-Isaac-style story serves to indicate that the Founding Figure (and, by implicit metonymy, the group as a whole, or its exemplars) is willing to give proof of that level of devotion, but is explicitly not required to do so by the god. This is an obvious win-win -- we don't have to actually kill anybody, but we get all the street-cred for being hardcore enough to do so if our God willed it.
All of these reasons may be wrong, though, if only because they treat the Bible's narratives as discrete products of a unified agent with coherent motives and reasons. The real history of the Bible is sloppy, messy, and zigzagging. Richard Friedman suggests that in the original (Elohist-source) story, Abraham actually did carry out the sacrifice of Isaac. If later traditions then found the idea of sacrificing a human (or sacrificing Isaac specifically) repugnant, the transition-from-human-sacrifice might have been accomplished by editing the old story, rather than by inventing it out of whole cloth as a deliberate rationalization for the historical shift away from the kosherness of human sacrifice. This would help account for the strangeness of the story itself, and for early midrashic traditions that thought that Isaac had been sacrificed. This also explains why the Elohist source never mentions Isaac again after the story, and why the narrative shifts from E-vocabulary to J-vocabulary at the crucial moment when Isaac is spared. Maybe.
P.S.: No, I wasn't speculating about 'what actually happened.' I was just shifting from our present-day, theologized pictures of Abraham to the more ancient figure actually depicted in the text, fictive though he be.
I've never heard it before.
After nearly a decade of studying the Old Testament, I finally decided very little of it makes sense a few years ago.
The problem has the same structure for MixedNuts' analogy of the FAI replacing the Voice. Suppose you program the AI to compute explicitly the logical structure "morality" that EY is talking about, and it tells you to kill a child. You could think you made a mistake in the program (analogous to rejecting your 3), or that you are misunderstanding the AI or hallucinating it (rejecting 2). And in fact for most conjunctions of reasonable empirical assumptions, it would be more rational to take any of these options than to go ahead and kill the child.
Likewise, sensible religionists agree that if someone hears voices in their head telling them to kill children, they shouldn't do it. Some of they might say however that Abraham's position was unique, that he had especially good reasons (unspecified) to accept 2 and 3, and that for him killing the child is the right decision. In the same way, maybe an AI programmer with very strong evidence for the analogies for 2 and 3 should go ahead and kill the child. (What if the AI has computed that the child will grow up to be Hitler?)
A few religious thinkers (Kierkegaard) don't think Abraham's position was completely unique, and do think we should obey certain Voices without adequate evidence for 4, perhaps even without adequate evidence for 3. But these are outlier theories, and certainly don't reflect the intuitions of most religious believers, who pay more lip service to belief-in-belief than actual service-service to belief-in-belief.
I think an analogous AI set-up would be:
What you call rejecting 3 is closer to rejecting 4, since it concerns my confidence that the AI is moral, not my confidence that the AI I programmed is the same as the entity outputting 'Kill your son.'
I disagree, because I think the analogy between the (4) of each case should go this way:
(4a) Analysis of "morality" as equivalent to a logical structure extrapolatable from by brain state (plus other things) and that an AI can in principle compute <==> (4b) Analysis of "morality" as equivalent to a logical structure embodied in a unique perfect entity called "God"
These are both metaethical theories, a matter of philosophy. Then the analogy between (3) in each case goes:
(3a) This AI in front of me is accurately programmed to compute morality and display what I ought to do <==> (3b) This voice I hear is the voice of God telling me what I ought to do.
(3a) includes both your 3 and your 4, which can be put together as they are both empirical beliefs that, jointly, are related to the philosophical theory (4a) as the empirical belief (3b) is related to the philosophical theory (4b).
Makes sense. I was being deliberately vague about (4) because I wasn't committing myself to a particular view of why Abraham is confident in God's morality. If we're going with the scholastic, analytical, logical-pinpointing approach, then your framework is more useful. Though in that case even talking about 'God' or a particular AI may be misleading; what 4 then is really asserting is just that morality is a coherent concept, and can generate decision procedures. Your 3 is then the empirical claim that a particular being in the world embodies this concept of a perfect moral agent. My original thought simply took your 4 for granted (if there is no such concept, then what are we even talking about?), and broke the empirical claim up into multiple parts. This is important for the Abraham case, because my version of 3 is the premise most atheists reject, whereas there is no particular reason for the atheists to reject my version of 4 (or yours).
We are mostly in agreement about the general picture, but just to keep the conversation going...
I don't think (4) is so trivial or that (4a) and (4b) can be equated. For the first, there are other metaethical theories that I think wouldn't agree with the common content of (4a) and (4b). These include relativism, error theory, Moorean non-naturalism, and perhaps some naive naturalisms ("the good just is pleasure/happiness/etc, end of story").
For the second, I was thinking of (4a) as embedded in the global naturalistic, reductionistic philosophical picture that EY is elaborating and that is broadly accepted in LW, and of (4b) as embedded in the global Scholastic worldview (the most steelmanned version I know of religion). Obviously there are many differences between the two philosophies, both in the conceptual structures used and in very general factual beliefs (which as a Quinean I see as intertwined and inseparable at the most global level). In particular, I intended (4b) to include the claim that this perfect entity embodying morality actually exists as a concrete being (and, implicitly,that it has the other omni-properties attributed to God). Clearly atheists wouldn't agree with any of this.
But that's just choosing the other horn of the dilemma, no? I.e., "god commands thing because they are moral."
And of course the atheist response to that is,
Not that anyone here didn't already know this, of course.
The wikipedia page lists some theistic responses that purport to evade both horns, but I don't recall being convinced that they were even coherent when I last looked at it.
It does choose a horn, but it's the other one, "things are moral because G-d commands them". It just denies the connotation that there exists a possible Counterfactual!G-d which could decide that Real!evil things are Counterfactual!good; in all possible worlds, G-d either wants the same thing or is something different mistakenly called "G-d". (Yeah, there's a possible world where we're ruled by an entity who pretends to be G-d and so we believe that we should kill babies. And there's a possible world where you're hallucinating this conversation.)
Or you could say it claims equivalence. Is this road sign a triangle because it has three sides, or does it have three sides because it is a triangle? If you pick the latter, does that mean that if triangles had four sides, the sign would change shape to have four sides? If you pick the former, does that mean that I can have three sides without being a triangle? (I don't think this one is quite fair, because we can imagine a powerful creator who wants immoral things.)
Three possible responses to the atheist response:
Sure. Not believing has bad consequences - you're wrong as a matter of fact, you don't get special believer rewards, you make G-d sad - but being immoral isn't necessarily one.
Well, you can be moral about most things, but worshiping my deity of choice is part of morality, so you can't be completely moral.
You could in theory, but how would you discover morality? Humans know what is moral because G-d told us (mostly in so many words, but also by hardwiring some intuitions). You can base your morality on philosophical reasoning, but your philosophy comes from social attitudes, which come from religious morality. Deviations introduced in the process are errors. All you're doing is scratching off the "made in Heaven" label from your ethics.
This is a classic case of fighting the wrong battle against theism. The classic theist defence is to define away every meaningful aspect of God, piece by piece, until the question of God's existance is about as meaningful as asking "do you believe in the axiom of choice?". Then, after you've failed to disprove their now untestable (and therefore meaningless) theory, they consider themselves victorious and get back to reading the bible. It's this part that's the weak link. The idea that the bible tells us something about God (and therefore by extension morality and truth) is a testable and debatable hypothesis, whereas God's existance can be defined away into something that is not.
People can say "morality is God's will" all they like and I'll just tell them "butterflies are schmetterlinge". It's when they say "morality is in the bible" that you can start asking some pertinent questions. To mix my metaphors, I'll start believing when someone actually physically breaks a ball into pieces and reconstructs them into two balls of the same original size, but until I really see something like that actually happen it's all just navel gazing.
It seems like you're claiming an identity relationship between god and morality, and I find myself very confused as to what that could possibly mean.
I mean, it's sort of like I just encountered someone claiming that "friendship" and "dolphins" are really the same thing. One or both of us must be very confused about what the labels "friendship" and/or "dolphins" signify, or what this idea of "sameness" is, or something else...
See Alejandro's comment. Define G-d as "that which creates morality, and also lives in the sky and has superpowers". If you insist on the view of morality as a fixed logical abstraction, that would be a set of axioms. (Modus ponens has the Buddha-nature!) Then all you have to do is settle the factual question of whether the short-tempered creator who ordered you to genocide your neighbors embodies this set of axioms. If not, well, you live in a weird hybrid universe where G-d intervened to give you some sense of morality but is weaker than whichever Cthulhu or amoral physical law made and rules your world. Sorry.
I typically don't accept the mainstream Judeo-Christian text as metaphorical truth, but if I did I can settle that question in the negative: The Jehovah of those books is the force that forbade knowledge and life to mankind in Genesis, and therefore does not embody morality. He is also not the creator of morality nor of the universe, because that would lead to a contradiction.
I dunno, dude could have good reasons to want knowledge of good and evil staying hush-hush. (Forbidding knowledge in general would indeed be super evil.) For example: You have intuitions telling you to eat when you're hungry and give food to others when they're hungry. And then you learn that the first intuition benefits you but the second makes you a good person. At this point it gets tempting to say "Screw being a good person, I'm going to stuff my face while others starve", whereas before you automatically shared fairly. You could have chosen to do that before (don't get on my case about free will), but it would have felt as weird as deciding to starve just so others could have seconds. Whereas now you're tempted all the time, which is a major bummer on the not-sinning front. I'm making this up, but it's a reasonable possibility.
Also, wasn't the tree of life totally allowed in the first place? We just screwed up and ate the forbidden fruit and got kicked out before we got around to it. You could say it's evil to forbid it later, but it's not that evil to let people die when an afterlife exists. Also there's an idea (at least one Christian believes this) that G-d can't share his power (like, polytheism would be a logical paradox). Eating from both trees would make humans equal to G-d (that part is canon), so dude is forced to prevent that.
You can still prove pretty easily that the guy is evil. For example, killing a kid (through disease, not instant transfer to the afterlife) to punish his father (while his mother has done nothing wrong). Or ordering genocides. (The killing part is cool because afterlife, the raping and enslaving part less so.) Or making a bunch of women infertile because it kinda looked like the head of the household was banging a married woman he thought was single. Or cursing all descendents of a guy who accidentally saw his father streaking, but being A-OK with raping your own father if there are no marriageable men available. Or... well, you get the picture.
You sure? They believed in a gloomy underworld-style afterlife in those days.
Well, it's not as bad as it sounds, anyway. It's forced relocation, not murder-murder.
How do you know what they believed? Mordern Judaism is very vague about the afterlife - the declassified material just mumbles something to the effect of "after the Singularity hits, the righteous will be thawed and live in transhuman utopia", and the advanced manual can't decide if it likes reincarnation or not. Do we have sources from back when?
As I said, that's debatable; most humans historically believed that's what "death" consisted of, after all.
That's not to say it's wrong. Just debatable.
Eh?
Google "sheol". It's usually translated as "hell" or "the grave" these days, to give the impression of continuity.
No, the Tree of Life and the Tree of Knowledge (of Good and Evil) were both forbidden.
My position is that suppressing knowledge of any kind is Evil.
The contradiction is that the creator of the universe should not have created anything which it doesn't want. If nothing else, can't the creator of the universe hex-edit it from his metauniverse position and remove the tree of knowledge? How is that consistent with morality?
Presumably the creator did want the trees, he just didn't want humans using it. I always got the impression that the trees were used by God(and angels?), who at the point the story was written was less the abstract creator of modern times and more the (a?) jealous tribal god of the early Hebrews (for example, he was physically present in the GOE.) Isn't there a line about how humanity must never reach the TOL because they would become (like) gods?
EDIT:
Seriously? Knowledge of any kind?
Yes. Suppressing knowledge of any kind is evil. It's not the only thing which is evil, and acts are not necessarily good because they also disseminate knowledge.
Genesis 2:16-2:17 looks pretty clear to me: every tree which isn't the tree of knowledge is okay. Genesis 3:22 can be interpreted as either referring to a previous life tree ban or establishing one.
If you accept the next gen fic as canon, Revelations 22:14 says that the tree will be allowed at the end, which is evidence it was just a tempban after the fall.
Where do you get that the tree of life was off-limits?
Sheesh. I'll actively suppress knowledge of your plans against the local dictator. (Isn't devil snake guy analogous?) I'll actively suppress knowledge of that weird fantasy you keep having where you murder everyone and have sex with an echidna, because you're allowed privacy.
Standard reply is that free will outweighs everything else. You have to give people the option to be evil.
No, if I give the creator free will, he doesn't have to give anyone he creates the option. He chose to create the option or illusion, else he didn't exercise free will.
It seems like you require a reason to suppress knowledge; are you choosing the lesser of two evils when you do so?
There is no reason an omnipotent God couldn't have created creatures with free will that still always choose to be good. See Mackie, 1955.
I think there's a bug in your theist-simulation module ^^
I've yet to meet one that could have spontaneously come up with that statement.
Anyway, more to the point... in the definition of god you give, it seems to me that the "lives in sky with superpowers" part is sort of tacked on to the "creates morality" part, and I don't see why I can't talk about the "creates morality" part separate from the tacked-on bits. And if that is possible, I think this definition of god is still vulnerable to the dilemma (although it would seem clear that the second horn is the correct one; god contains a perfect implementation of morality, therefore what he says happens to be moral).
Hi there.
Are you a real theist or do you just like to abuse the common terminology (like, as far as I can tell, user:WillNewsome)? :)
A real theist. Even a Christian, although mostly Deist these days.
So you think there's a god, but it's conceivable that the god has basically nothing to do with our universe?
If so, I don't see how you can believe this while giving a similar definition for "god" as an average (median?) theist.
(It's possible I have an unrepresentative sample, but all the Christians I've met IRL who know what deism is consider it a heresy... I think I tend to agree with them that there's not that much difference between the deist god and no god...)
Out of curiosity, why do you write G-d, not God? The original injunction against taking God's name in vain applied to the name in the old testament, which is usually mangled in the modern English as Jehovah, not to the mangled Germanic word meaning "idol".
People who care about that kind of thing usually think it counts as a Name, but don't think there's anything wrong with typing it (though it's still best avoided in case someone prints out the page). Trying to write it makes me squirm horribly and if I absolutely need the whole word I'll copy-paste it. I can totally write small-g "god" though, to talk about deities in general (or as a polite cuss). I feel absolutely silly about it, I'm an atheist and I'm not even Jewish (though I do have a weird cultural-appropriatey obsession). Oh well, everyone has weird phobias.
Thought experiment: suppose I were to tell you that every time I see you write out "G-d", I responded by writing "God", or perhaps even "YHWH", on a piece of paper, 10 times. Would that knowledge alter your behavior? How about if I instead (or additionally) spoke it aloud?
Edit: downvote explanation requested.
Why should s/he care about what you choose to do?
I don't know. That's why I asked.
1) I don't believe you.
2) I don't respond to blackmail.