Joi Ito said several things that are unpleasant but are probably believed by most people, and so I am glad for the reminder.

JOI ITO: This may upset some of my students at MIT, but one of my concerns is that it’s been a predominately male gang of kids, mostly white, who are building the core computer science around AI, and they’re more comfortable talking to computers than to human beings. A lot of them feel that if they could just make that science-fiction, generalized AI, we wouldn’t have to worry about all the messy stuff like politics and society. They think machines will just figure it all out for us.

Yes, you would expect non-white, older, women who are less comfortable talking to computers to be better suited dealing with AI friendliness! Their life experience of structural oppression helps them formally encode morals!

ITO: [Temple Grandin] says that Mozart and Einstein and Tesla would all be considered autistic if they were alive today. [...] Even though you probably wouldn’t want Einstein as your kid, saying “OK, I just want a normal kid” is not gonna lead to maximum societal benefit.

I should probably get a good daily reminder most people would not, in fact, want their kid to be as smart, impactful and successful in life as Einstein, and prefer "normal", not-too-much-above-average kids.

This is an explanation of Yudkowsky's idea from the metaethics sequence. I'm just trying to make it accessible in language and length with lots of concept handles and examples.

Technically, you could believe that people are equally allowed to be enslaved. All people equal + it's wrong to make me a slave = it's wrong to make anyone a slave.

"All men are created equal" emerges from two or more basic principles people are born with. You might say: "Look, you have value, yah? And your loved ones? Would they stop having value if you forgot about them? No? They have value whether or not you know them? How did you conclude they have value? Could that have happened with other people, too? Would you then think they had value? Would they stop having value if you didn't know them? No? Well, you don't know them; do they have value?

You take "people I care about have value" (born with it) and combine it with "be consistent" (also born with), and you get "everyone has value."

That's the idea in principle, anyway. You take some things people are all born with, and they combine to make the moral insights people can figure out and teach each other, just like we do with math.

You're right; I've provided no evidence.

Do you think the idea is sufficiently coherent and non-self-contradictory that the way to find out if it's right or wrong is to look for evidence?

If it was incoherent or contradicted itself, it wouldn't even need evidence to be disproven; we would already know it's wrong. Have I avoided being wrong in that way?

(by the way, understanding slavery might be necessary, but not sufficient to get someone to be against it. They might also need to figure out that people are equal, too. Good point, I might need to add that note into the post).

What?? Weird!

Maybe it was lost when i edited the draft.

I just deleted the link post and made a better one: http://lesswrong.com/r/discussion/lw/o0i/barack_obamas_opinions_on_nearfuture_ai_fixed/

I don't see a link. Was it lost like in my link post on a different subject? I still don't know how to post links correctly.

Without commenting on whether this presentation matches the original metaethics sequence (with which I disagree), this summary argument seems both unsupported and unfalsifiable.

1. No evidence is given for the central claim, that humans can and are converging towards a true morality we would all agree about if only we understood more true facts.
2. We're told that people in the past disagreed with us about some moral questions, but we know more and so we changed our minds and we are right while they were wrong. But no direct evidence is given for us being more right. The only way to judge who's right in a disagreement seems to be "the one who knows more relevant facts is more right" or "the one who more honestly and deeply considered the question". This does not appear to be an objectively measurable criterion (to say the least).
3. The claim that ancients, like Roman soldiers, thought slavery was morally fine because they didn't understand how much slaves suffer is frankly preposterous. Roman soldiers (and poor Roman citizens in general) were often enslaved, and some of them were later freed (or escaped from foreign captivity). Many Romans were freedmen or their descendants - some estimate that by the late Empire, almost all Roman citizens had at least some slave ancestors. And yet somehow these people, who both knew what slavery was like and were often in personal danger of it, did not think it immoral, while white Americans in no danger of enslavement campaigned for abolition.

Nobody is calling “a universal decision theory a moral theory”. According to hedonistic utilitarianism, and indeed all consequentialism, all actions are morally significant.

‘Moral’ means regarding opinions of which actions ought to be performed.

The author is far from alone in his view that both a complete rightness criterion and a consistent decision method must be required of all serious moral theories.

Among hedonistic utilitarians it's quite normal to demand both completeness, to include all (human) situations, and consistency, to avoid contradictions. The author simply describes what's normal among consequentialists, who, after all, are more or less the rational ones. ;-) There's one interesting exception though! The demand to include all situations, including the non-human ones, is radical, and quite hard a challenge for hedonistic utilitarians, who do have problems with the bloodthirsty predators of the jungle.