Without commenting on whether this presentation matches the original metaethics sequence (with which I disagree), this summary argument seems both unsupported and unfalsifiable.

1. No evidence is given for the central claim, that humans can and are converging towards a true morality we would all agree about if only we understood more true facts.
2. We're told that people in the past disagreed with us about some moral questions, but we know more and so we changed our minds and we are right while they were wrong. But no direct evidence is given for us being more right. The only way to judge who's right in a disagreement seems to be "the one who knows more relevant facts is more right" or "the one who more honestly and deeply considered the question". This does not appear to be an objectively measurable criterion (to say the least).
3. The claim that ancients, like Roman soldiers, thought slavery was morally fine because they didn't understand how much slaves suffer is frankly preposterous. Roman soldiers (and poor Roman citizens in general) were often enslaved, and some of them were later freed (or escaped from foreign captivity). Many Romans were freedmen or their descendants - some estimate that by the late Empire, almost all Roman citizens had at least some slave ancestors. And yet somehow these people, who both knew what slavery was like and were often in personal danger of it, did not think it immoral, while white Americans in no danger of enslavement campaigned for abolition.

"Don't wanna", shading into "Make Me" if they press. Anyone trying to tell you what to do isn't your Real Dad! (Unless they are, in which case maybe try and figure out what's going on.)

grumble grumble...

Look, I'm not pro-"Kill All Humans", but I don't think that last step is correct.

Bob can prefer that the human race die off and the earth spin uninhabited forever. It makes him evil, but there's no "logic error" in that, any more than there is in Al's preference that humanity spread out throughout the stars. They both envision future states and take actions that they believe will cause those states.

Yes, those with my values will live here, in Gondor. Your folks can live other there, in Mordor. Our citizens will no longer come into contact and conflict with one another, and peace will reign forever.

What, these segregated regions THEMSELVES come into conflict? Absurd. What would you even call a conflict that was between large groups of people? That could never happen. Everyone who shares my value system knows that lots of people would die, and we all agree that nothing could be worth that.

This is about as weak as an argument can possibly get.

The idea that slavery/segregation/discrimination in America has had an effect is not in dispute. The argument is regarding it's significance.

[Your hypothetical case study] is not evidence.

I fully agree. I was trying to distill the issue into simple terms. I would argue it's nearly self evident that opportunity is passed on over generations, and that a head start for a group of people based on race could be persistent over multiple generations.

[The study you linked which shows resumes with black sounding names are less likely to receive callbacks for job opportunities than white sounding names] does not demonstrate irrational discrimination. They did not consider the possibility that a person's race actually gives you useful information about them.

Are you saying it is appropriate for employers to discriminate based on race?

Jesse Jackson says something about his contrasting intuitions about black and white people...

Per capita murder rates are no doubt higher among blacks. The question is what caused this.

Perhaps you have heard the saying "rags to riches to rags in three generations". When I look at my family tree I see this happening many times.

You are not being discriminated against (or segregated) as a minority race.

Where I live a lot of the local whites are descended from prisoners who were slaves. They do not form an underclass in any way shape or form. In fact it is high status to have convict ancestry.

They are not being discriminated against (or segregated) as a minority race.

Edit: Regarding evidence slavery having an effect on current day conditions... Here is a study showing "the 1860 slave concentration is related to contemporary black-white inequality in poverty, independent of contemporary demographic and economic conditions, racialized wealth disparities and racial threat. [This] research suggests the importance of slavery for shaping existing U.S. racial inequality patterns."

The headline is misleading. I don't think there is an Apollo-style funding plan; i think Obama just thinks it'd be a good idea.

So "morals" is used to mean the same as "values" or "goals" or "preferences". It's not how I'm used to encountering the word, and it's confusing in comparison to how it's used in other contexts. Humans have separate moral and a-moral desires (and beliefs, emotions, judgments, etc) and when discussing human behavior, as opposed to idealized or artificial behavior, the distinction is useful.

Of course every field or community is allowed to redefine existing terminology, and many do. But now, whenever I encounter the word "moral", I'll have to remind myself I may be misunderstanding the intended meaning (in either direction).

If I re-formulate your question to "is there any rebuttal to why we don't donate way more to charity than we currently do" then the answer depends on your belief system.

(And also on how much money you currently donate to charity.)

"There is only one problem that we really care about. Optimization." That may be what you care about, but it is not what I care about, and it was not what I was talking about, which is intelligence. You cannot argue that we only care about optimization, and therefore intelligence is optimization, since by that argument dogs and cats are optimization, and blue and green are optimization, and everything is optimization, since otherwise we would be "debating definitions, which is not productive". But that is obvious nonsense.

In any case, it is plain that most of the human ability to accomplish things comes from the use of language, as is evident by the lack of accomplishment by normal human beings when they are not taught language. That is why I said that knowing language is in fact a sufficient test of intelligence. That is also why when AI is actually programmed, people will do it by trying to get something to understand language, and that will in fact result in the kind of AI that I was talking about, namely one that aims at vague goals that can change from day to day, not at paperclips. And this has nothing to do with any "homunculus." Rocks don't have any special goal like paperclips when they fall, or when they hit things, or when they bounce off. They just do what they do, and that's that. The same is true of human beings, and sometimes that means trying to have kids, and sometimes it means trying to help people, and sometimes it means trying to have a nice day. That is seeking different goals at different times, just as a rock does different things depending on its current situation. AIs will be the same.

Nobody is calling “a universal decision theory a moral theory”. According to hedonistic utilitarianism, and indeed all consequentialism, all actions are morally significant.

‘Moral’ means regarding opinions of which actions ought to be performed.

Two different actions don’t produce exactly the same utility, but even if they did it wouldn’t be any problem. To say that you may chose any one of two actions when it doesn’t matter which one you chose since they have the same value, isn’t to give “no guidance”. Consequentialists want to maximize the intrinsic value, and both these actions do just that.

Of course hedonistic utilitarianism doesn’t require completeness, which, by the way, isn’t one of its tenets either. But since it is complete, which of course is better than being incomplete, it’s normal for hedonistic utilitarianists to hold the metaethical view that a proper moral theory should answer all of the question: “Which actions ought to be performed?” What could be so good with answering it incompletely?

AIXI-tl is very inefficient. Practical AIs will not be like that, and they will not be limited to one rigid goal like that.

Your second claim doesn't follow from the first. Practical AIs will of course be different. But the basic structure of AIXI, reinforcement learning, is agnostic to the model used. It just requires some algorithm to do learning/prediction. As prediction algorithms get better and better, they will still suffer the same problems as AIXI. Unless you are proposing some totally different model of AI than reinforcement learning, that somehow doesn't suffer from these problems.

And even if you find my theory of intelligence unconvincing, one that implies that evolution is intelligent is even less convincing, since it does not respect what people actually mean by the word.

Now we are debating definitions, which is not productive.

Evolution is not typically thought of as intelligent because it's not an agent. It doesn't exist in an environment, make observations, and adjust it's model of the world, etc. I accept that evolution is not an agent. But that doesn't matter.

There is only one problem that we really care about. Optimization. This is the only thing that really matters. The advantage humans have in this world is our ability to solve problems and develop technology. The risk and benefit of superintelligent AI comes entirely from its potential to solve problems and engineer technologies better than humans.

And that's exactly what evolution does. It's an algorithm that can solve problems and design very sophisticated and efficient machines. It does the thing we care about, despite not being an agent. Whether it meets the definition of "intelligence" or not, is really irrelevant. All that matters is if it's an algorithm that can solve the types of problems we care about. There is no reason that solving problems or designing machines should require an algorithm to be an agent.

If it doesn't fundamentally change quantum mechanics as a theory, is the picture likely to turn out fundamentally different from MWI?

CI/OR is a different picture to MWI, yet neither change QM as a number-crunching theory. You have hit on the fundamental problems of empiricism: the correct interpretation of a data is underdetermined by data, and interpretations can differ radically with small changes in data or no changes in data.

We've already got a number of problems with MW -- see Dowker and Kent's paper.

The question is whether there is anything better. To go back to my original question, EY appears not to have heard of QBism, RQM, and other interpretations that aren't mentioned in The Fabric of Reality.

These are difficult question because we are speculating about future mathematics / physics.

First of all, there's the question of how much of the quantum framework will survive the unification with gravity. Up until now, all theories that worked inside it have failed; worse, they have introduced black-hole paradoxes: most notably, thunderbolts and the firewall problem. I'm totally in the dark if a future unification will require a modification of the fundamental mathematical structure of QM. Say, if ER = EPR, and entanglement can be explained with a modified geometry of space-time, does it mean that superposition is also a geometrical phoenomenon that doesn't require multiple worlds? I don't really know.

But more on the point, I think (hope?) that future explorations of the quantum framework will yield an expanded landscape, where interpretations will be seen as the surface phoenomenon of something deeper: for example, something akin to what happens in classical mechanics with the Hamiltonian / Lagrangian formulations.

On a side note, I've read only the Wikipedia article on QBism and my impression was that it had an epistemological leaning, not ontological: if you use only SIC-POVMs, you can explain all quantum quirks with the epistemology of probability distributions. I might be very wrong, though.

Who says we need to hardcode human values though? Any reasonable solution will involve an AI that learns what human values are. Or some other method to the control problem that makes AIs that don't want to harm or defy their creators.

But the problem is it's not just by values. It's also by wealth and intelligence and education. If you have half of the world that is really poor, and anyone that is intelligent or wealthy automatically leaves, then they will probably stay poor forever.

How about a lollipop? It's almost the same thing, and since inspector Kojak it's become much more socially acceptable, even cool, if you pull it off well.
If you are a woman, though, you'll likely suffer some sexual objectification (what a news!).

it seems indisputable to me

This is about as weak as an argument can possibly get.

<hypothetical case study that did not actually happen>

Again this is not evidence.

this study

Does not demonstrate irrational discrimination. They did not consider the possibility that a person's race actually gives you useful information about them.

Consider the following example:

There is nothing more painful to me at this stage in my life than to walk down the street and hear footsteps and start thinking about robbery. Then look around and see somebody white and feel relieved.... After all we have been through. Just to think we can't walk down our own streets, how humiliating.

Remarks at a meeting of Operation PUSH in Chicago (27 November 1993). Quoted in "Crime: New Frontier - Jesse Jackson Calls It Top Civil-Rights Issue" by Mary A. Johnson, 29 November 1993, Chicago Sun-Times (ellipsis in original). Partially quoted in "In America; A Sea Change On Crime" by Bob Herbert, 12 December 1993, New York Times.

I have looked at the study before it is well known.

And if there are other factors. I'd love to hear some of your evidence for other factors.

IQ is known to be highly heritable and highly correlated with many measures of success. As are other psychological dimensions such as the Big 5. Source: any psychology textbook.

Perhaps you have heard the saying "rags to riches to rags in three generations". When I look at my family tree I see this happening many times.

Where I live a lot of the local whites are descended from prisoners who were slaves. They do not form an underclass in any way shape or form. In fact it is high status to have convict ancestry.

Or consider Jews, against whom there was massive discrimination until very recently. They have been very successful.

I mean, Laffer Curve-type reasons if nothing else.

Citation required. What is strange about this is that when you go looking, you don't see good studies that track people through generations and show that this is in fact the case.

The idea that slavery/segregation/discrimination has created a very significant deficit for blacks seems beyond dispute in my view. The words "very significant" could be disputed based on how we defined them, but that's a technicality. I'm honestly shocked to hear this idea challenged...

I've cited this study.

It's stated that "African-Americans are twice as likely as whites to be unemployed and they earn nearly 25 percent less when they are employed." The study itself shows significant discrimination based on race in the beginning stages of the hiring process.

Lumifer seemed to accept the basic premise, but was nonetheless skeptical and too uninterested to look into the study. I'd be interested to know what you think.

Regardless, it is evidence that employers discriminate against blacks. And employment is tied to income...and wealth...and opportunity. And that is passed on generation after generation.

This idea "slavery is the cause" seems not to be an actual active idea but only functions as a thought terminating cliche. It could have been slavery so it was.

Again, it seems indisputable to me that slavery has an effect. Segregation and discrimination, too. I honestly don't understand how it couldn't. The only question that is left is in regard to the significance of the effect. And if there are other factors. I'd love to hear some of your evidence for other factors.

And as for this...

It reminds me of religious apologists talking about the problem of evil...

I strongly disagree. People being enslaved based on race for hundreds of years, segregated for a hundred more, and then discriminated against until the present day, and that leading to some problems within that race has zero, and I mean zero, to do with the concocted, magical-causal "explanations" of religion.

How about this...

Man A is freed from slavery at 40 with no skills, no education, no family and no professional or network.

Also at 40, man B has a small fortune, an education, is skilled in a trade, has a large family, a good reputation, and a wide network of business and social contacts.

Assuming the offspring of each man—A1 and B1—has identical DNA, which offspring has the highest probability of graduating from an elite university?

Which—A1 or B1—will be more likely to have a successful career?

Which will pass on the largest inheritance to A2 and B2?

Why?

And what do you expect to change in subsequent generations?

(One thing that could change are laws eliminating discrimination...)

I'm saying slavery/segregation/discrimination has created a very significant deficit for blacks

Citation required. What is strange about this is that when you go looking, you don't see good studies that track people through generations and show that this is in fact the case.

This idea "slavery is the cause" seems not to be an actual active idea but only functions as a thought terminating cliche.

It could have been slavery so it was.

It reminds me of religious apologists talking about the problem of evil, and how it 'could' be caused by man's sin (causing human evil) and possibly by Satan's sin (causing natural evil), which is required if we are to have free will. There is zero, I mean zero, interest in exploring just how 'sin' causes all the various forms of evil. How does sin cause our flawed DNA which allows cancer? <crickets> Etc.

As an example of how such discrimination can be rational and indeed reasonable...

You have a resume. It provides some noisy data about someone. Including that person's race. Let's trim it down. You have an IQ test result and the person's race. Let's say that two candidates has the same IQ in the test, but one came from a group known to have a significantly lower IQ on average.

If we assume that an IQ test result has any measurement noise - and they do - then the Bayesian conclusion is the candidate from the group with higher average IQ is likely to actually have a higher IQ.

Now resumes constitute very noisy data. People often even lie in their resumes. There are large differences between groups in the US. The dispute is about the reasons for the differences not whether they exist.

A study would need to overcome these effects to demonstrate irrational discrimination. They would need to show that e.g. there was consistent out-performance for the group discriminated against post recruitment.

Book "Without Conscience" by Robert Hare who is a real psychologist has simple tips on recognizing them. Not purely by photographic appearance but it is not too hard. Example with eye contact they tend to stare too long.

If it doesn't fundamentally change quantum mechanics as a theory, is the picture likely to turn out fundamentally different from MWI? Roger Penrose, a vocal MWI critic, seems to wholeheartedly agree that QM implies MWI; it's just that he thinks that this means the theory is wrong. David Deutsch, I believe, has said that he's not certain that quantum mechanics is correct; but any modification of the theory, according to him, is unlikely to do away with the parallel universes.

QBism, too, seems to me to essentially accept the MWI picture as the underlying ontology, but then says that we should only care about the worlds that we actually observe (Sean Carroll has presented criticism similar to this, and mentioned that it sounds more like therapy to him), although it could be that I've misunderstood something.

People certainly have a bias towards those-like-me, but it's not specifically anti-black, it's against anyone who looks/feels/smells different.

It's debatable whether or not it's specifically anti-black. Or anti-some-other-group. At any rate, a bias against those-not-like-me would be sufficient in this case to cause blacks a significant deficit in opportunity for employment in a historically majority white nation.

Um, the IQ would be different?...

As usual, I phrased my comment poorly. Let me try a different tack...

You are saying black Americans have a genetic deficit in the form of lower average IQ. Because IQ is heritable and very important toward social "success", this is a (or even the?) major factor in why they lag behind in certain social metrics (avg. income/wealth, crime rates, etc.) in American society.

I'm saying slavery/segregation/discrimination has created a very significant deficit for blacks to overcome in America, to the extent that we would expect to see something like we see in terms of the disparity in avg. income/wealth, crime rates, etc. I'd hypothesize slavery/segregation/discrimination has been consequential to the extent that even if blacks had a higher average IQ than whites, they would still be in a similar situation. (i.e. the discrimination is that bad and that significant.)

Plainly, advanced IQ (or other genetic advantages) aren't enough to overcome significant discrimination in all cases. The disadvantages can be too steep in a given society.

I'd propose a good portion of the U.S. is a bit more racist than I think you are taking into consideration. And this may have caused a deeper deficit for blacks than you are appreciating.

As to similarities, I was about to write that the discriminated-against will never rise to the highest positions in the society, but oh look! there is that Barack Hussain fellow...

Things can change. Slowly.

Ah I did (at the time), but forgot it was you that made those comments. So I should direct my question to Jacobian, not you.

In any case I'm certainly not a "save the world" type of person, and find myself thoroughly confused by those who profess to be and enter into self-destructive behavior as a result.

...and did you read my comments in the thread?

Read it already. Let's be clear: you think the mother should push her baby in front of a trolley to save five random strangers? If so, why? If not, why not? I don't consider this a loaded question -- it falls directly out of the utilitarian calculus and assumed values that leads to "donate 100% to charities."

[Let's assume the strangers are also same-age babies, so there's no weasel ways out ("baby has more life ahead of it", etc.)]

"Utilitarianism is a theory in normative ethics holding that the best moral action is the one that maximizes utility." -Wikipedia

The very next sentence starts with "Utility is defined in various ways..." It is entirely possible for there to be utility functions that treat sentient beings differently. John Stuart Mill may have phrased it as "the greatest good for the greatest number" but the clutch is in the word "good" which is left undefined. This is as opposed to, say, virtue ethics which doesn't care per se about the consequences of actions.

Is this sort of discrimination not consequential in your view?

I don't know about the study, I have a generic suspicion of social sciences studies, especially ones which come to highly convenient conclusions, and hey! they happen to have a what's politely called "replication crisis". I am not interested enough to go read the study and figure out if it's valid, but on my general priors, I believe that people with black names will get less callbacks. However it seems to me that people with names like Pham Ng or Li Xiu Ying will also get less callbacks. People certainly have a bias towards those-like-me, but it's not specifically anti-black, it's against anyone who looks/feels/smells different.

can you imagine a scenario in a society where a high IQ group of people was discriminated against to the extent where they couldn't overcome the discrimination, despite their advanced higher IQ?

Sure.

How would the circumstances be different than what blacks have faced in the U.S.?

Um, the IQ would be different? It's not a mystical inner quality that no one can fathom. It's measurable and on the scale of large groups of people the estimates gets pretty accurate.

On the clearly visible level there would be very obvious discrimination -- quotas on admissions to universities, for examples. These discriminated-against people would be barred from reaching high positions, but at the level they would be allowed to reach they would be considered very valuable. Even if, for example, such people could not make it into management, managers would try to hire as many of them as possible because they are productive and solve problems.

As to similarities, I was about to write that the discriminated-against will never rise to the highest positions in the society, but oh look! there is that Barack Hussain fellow...

I don't see any ongoing segregation

Not backed by the gov't through the present day but, as you mentioned, since WW2 and certainly long after slavery ended.

But discrimination based on race is still very common. I cited the study showing resumes with black sounding names receive significantly fewer callbacks than resumes with white sounding names...

You've not mentioned this study in your replies—Is this sort of discrimination not consequential in your view?

IQ is really really important.

As a bit of a thought experiment, can you imagine a scenario in a society where a high IQ group of people was discriminated against to the extent where they couldn't overcome the discrimination, despite their advanced higher IQ?

How would the circumstances be different than what blacks have faced in the U.S.? How would they be similar?

I don't see any ongoing segregation (though, interestingly enough, some Black movements nowadays are trying to revive it, in some places even successfully).

I've mentioned Jews upthread -- they were very consistently discriminated against until after the WW2. Did they have similar outcomes?

On the other hand you have SubSaharan Africa which is doing pretty badly by pretty much any criterion. That includes countries which were colonies only for a very very short period (such as Ethiopia, which is also mostly Christian and the former Emperor of which traced a direct lineage line to King Solomon and Queen of Sheba).

Do tell: What is the most important factor? Why?

Genetics, in particular IQ. Why? IQ is really really important.

Hey, look here, you totally should. All that emotional empathy just gets in the way.

It's not hard to find people whose ancestors 150 years ago were poor, uneducated, lacking skills and access to social networks... I think you're just describing an average peasant. And yet, there are different outcomes.

Ongoing segregation and discrimination against blacks in America since slavery doesn't seem to be making it into your math here. Why? It's significant and should be considered.

And it's not hard to imagine how "peasants" might do well when compared to former slaves...(1) being poor and being a slave are very different (2) It's much tougher to segregate and discriminate when everybody looks basically the same. It's easy when their skin color is different.

As I mentioned in my post upthread, I agree it's a factor. I just don't think it's the sole factor or even the most important factor.

Do tell: What is the most important factor? Why?

Er... no. Utilitarianism prohibits that exact thing by design. That's one of its most important aspects.

Read the definition. This is unambiguous.

In my view, segregating the world by values would actually be really good. People who have very different belief systems should not try or be forced to live in the same country.

A mother that followed that logic would push her own baby in front of a trolley to save five random strangers. Ask yourself if that is the moral framework you really want to follow.

Nonsense. I believe my life and the lives of people close to me are more important than someone starving in a place whose name I can't pronounce. I just don't assign the same weight to all people. That is perfectly consistent with utilitarianism.

I am confused as to whether I should upvote for "get a job at Google" or downvoter for "prostitute yourself".

Rejoice because the end is near.

Maybe buy Google stock?

*Southern California

It's socially acceptable to twirl and manipulate small objects in your hands, from pens to stress balls. If you need to get your mouth involved, it's mostly socially acceptable to chew on pens. Former smokers used to hold empty pipes in their mouths, just for comfort, but it's hard to pull off nowadays unless you're old or a fully-blown hipster.

Partly because of material wealth, partly because of availability of education and the opportunity to learn marketable skills, partly because of access to social and professional networks

It's not hard to find people whose ancestors 150 years ago were poor, uneducated, lacking skills and access to social networks... I think you're just describing an average peasant. And yet, there are different outcomes.

you discount the idea slavery, segregation and discrimination has had ill effects for African Americans in the U.S. up to the present day...Why is that?

As I mentioned in my post upthread, I agree it's a factor. I just don't think it's the sole factor or even the most important factor.

I agree with gjm that evil does not necessarily require coercion. Contemplate, say, instigating a lynching.

The reason EAs don't do any coercion is because they don't have any power. But I don't see anything in their line of reasoning which would stop them from coercing other people in case they do get some power. They are not libertarians.

Could chewing gum serve as a suitable replacement for you?

We don't know what an AI which maximizes human values is because we don't know what human values are at the necessary level of precision. Not to mention the assumption that the AI will be a maximizer and that values can be maximized.

Among hedonistic utilitarians it's quite normal to demand both completeness

Utilitarianism provides no guidance on many decisions: any decision where both actions produce the same utility.

Even if it is a complete theory, I don't think that completeness is demanded of the theory; rather it's merely a tenet of it. I can't think of any good a priori reasons to expect a theory to be complete in the first place.

Is there any product like an adult pacifier that is socially acceptable to use?

I am struggling with self-control to not interrupt people and am afraid for my job.

EDIT: In the meantime (or long-term if it works) I'll use less caffeine (currentlly 400mg daily) to see if that helps.

The question needs to cover how one should act in all situations, simply because we want to answer the question. Otherwise we’re left without guidance and with uncertainty.

Well first, we normally don't think of questions like which clothes to wear as being moral. Secondly, we're not left without guidance when morality leaves these issues alone: we have pragmatic reasons, for instance. Thirdly, we will always have to deal with uncertainty due to empirical uncertainty, so it must be acceptable anyway.

There is one additional issue I would like to highlight, an issue which rarely is mentioned or discussed. Commonly, normative ethics only concerns itself with human actions. The subspecies homo sapiens sapiens has understandably had a special place in philosophical discussions, but the question is not inherently only about one subspecies in the universe. The completeness criterion covers all situations in which somebody should perform an action, even if this “somebody” isn’t a human being. Human successors, alien life in other solar systems, and other species on Earth shouldn’t be arbitrarily excluded.

I'd agree, but accounts of normativity which are mind- or society-dependent, such as constructivism would have reason to make accounts of ethics for humanity different from accounts of ethics for nonhumans.

It seems like an impossible task for any moral theory based on virtue or deontology to ever be able to fulfil the criteria of completeness and consistency

I'm not sure I agree there. Usually these theories don't because the people who construct them disagree with some of the criteria, especially #1. But it doesn't seem difficult to make a complete and demanding form of virtue ethics or deontology.

https://www.quora.com/How-can-I-get-Wi-Fi-for-free-at-a-hotel/answer/Yishan-Wong

Want free wifi when staying at an hotel? Ask for it. Of course!, Duh, seems so obvious now that I think about it.

In certain cases people can pattern-match sociopath by looking at someone's face.

Do you have any links, because this is interesting if true. Kinda like human lie detectors. But I am skeptical, because how would such a thing arise?

Why would sociopaths have distinguishing facial markers and what are they?

In certain cases people can pattern-match sociopath by looking at someone's face. I didn't mean to suggest the average person can do it on a consistent basis.

100% doesn't work because then you starve. If I re-formulate your question to "is there any rebuttal to why we don't donate way more to charity than we currently do" then the answer depends on your belief system. If you are utilitarian, the answer is definitive no. You should spend way more on charity.

In hope that he will stop creating AI? But in 6 years it will be Microsoft.

100%? Well, your future charitable donations will be markedly curtailed after you starve to death.

Some argument along these lines may work; but I don't believe that doing evil requires coercion.

Suppose that for some reason I am filled with malice against you and wish to do you harm. Here are some things I can do that involve no coercion.

I know that you enjoy boating. I drill a small hole in your boat, and the next time you go out on the lake your boat sinks and you die.

I know that you are an alcoholic. I leave bottles of whisky around places you go, in the hope that it will inspire you to get drunk and get your life into a mess.

The law where we live is (as in many places) rather overstrict and I know that you -- like almost everyone in the area -- have committed a number of minor offences. I watch you carefully, make notes, and file a report with the police.

I get to know your wife, treat her really nicely, try to give her the impression that I have long been nursing a secret yearning for her. I hope that some day if your marriage hits an otherwise-navigable rocky patch, she will come to me for comfort and (entirely consensually) leave you for me.

I discover your political preferences and make a point of voting for candidates whose values and policies are opposed to them.

I put up posters near where you live, accusing you of horrible things that you haven't in fact done.

I put up posters near where you live, accusing you of horrible things that you have in fact done.

None of these involves coercion unless you interpret that word very broadly. Several of them don't, so far as I can see, involve coercion no matter how broadly you interpret it.

So if you want to be assured of not doing evil, you probably need more firebreaks besides "no coercion".

She could read "The Basic AI Drives" to him at night.

Some possible argument against charities. Personally I think that it is normal to donate around 1 per cent of income in form of charity support.

1. Some can't survive on less or have other obligations that looks like charity (child support)
2. We would have less initiative to earn more
3. It would hurt our economy, as it is consumer driven. We must buy Iphones
4. I do many useful things which intended on helping other people, but I need pleasures to recreate my commitments, so I spend money on myself.
5. I pay taxes and it is like charity.
6. I know better how to spent money on my needs.
7. Human psychology is about summing different values in one brain, so I could spent only part of my energy on charity.
8. If I buy goods, my money goes to working people, so it is like charity for them. If I stop buying goods, they will be jobless and will need charity money for survive. So the more I give for charity, the more people need it.
9. If you overdonate, you could flip-flop and start to hate the thing. Especially if you find that your money was not spent effectively.
10. Donating 100 per cent will make you look crazy in views of some, and their will to donate diminish.
11. If you spent more on yourself, you could ask higher salary and as result earn more and donate more. Only a homeless and jobless person could donate 100 per cent.

Oh! This point had evaded me: I thought x encoded the program and the input, not just the entire history.
So U, instead of executing, just locates the last thing written on tape according to x and repeat it.
Well, I'm disappointed... at U and at myself.

My idea is more on the line of "in the future we are going to grasp a conceptual frame that would make sense of all interpretations" (or explain them away) rather than pointing to a specific interpretation.

What makes you think that people can pattern-match sociopathy by looking at someone's face? Sociopathy usually doesn't lead to low charisma and people getting the sense not to interact with the person.

How about: doing evil (even inadvertently) requires coercion. Slavery, Nazis, tying a witch to a stake, you name it. Nothing effective altruists currently do is coercive (except to mosquitoes), so we're probably good. However, if we come up with a world improvement plan that requires coercing somebody, we should A) hear their take on it and B) empathize with them for a bit. This isn't a 100% perfect plan, but it seems to be a decent framework.

Is there a good rebuttal to why we don't donate 100% of our income to charity? I mean, as an explanation tribality / near - far are ok, but is there a good justification post-hoc?

And it's likely to cause a temporary inflammatory effect as lots of cells (including ones with no viral infection or latent viruses that everyone and I mean everyone has) blow up.

Getting rid of all latent viruses might be a feature and not a bug.

I'm confused. Is it normal to regard all possible acts and decisions as morally significant, and to call a universal decision theory a moral theory?

What meaning does the word "moral" even have at that point?

Do you think that we're likely to find something in those directions that would give a reason to prefer some other interpretation than MWI?

I agree. FAI somehow should use human upload or human-like architecture for its value core. In this case values will be presented in it in complex and non-ortogonal ways, and at least one human-like creature will survive.

Yes. I think that we need not only workable solution, but also implementable. If someone create 800 pages pdf starting with new set theory, solution of Lob theorem problem etc and come to Google with it and say: "Hi, please, switch off all you have and implement this" - it will not work.

But MIRI added in 2016 the line of research for machine learning.

Quoting https://en.wikipedia.org/wiki/Kleene%27s_T_predicate:

The ternary relation T1(e,i,x) takes three natural numbers as arguments. The triples of numbers (e,i,x) that belong to the relation (the ones for which T1(e,i,x) is true) are defined to be exactly the triples in which x encodes a computation history of the computable function with index e when run with input i, and the program halts as the last step of this computation history.

In other words: If someone gives you an encoding of a program, an encoding of its input and a trace of its run, you can check with a primitive recursive function whether you have been lied to.

Many people who delve into the deep parts of analytical philosophy will end up feeling at times like they can't justify anything, that definite knowledge is impossible to ascertain, and so forth. It's a classic trend. Hume is famous for being a "skeptic", although almost everyone seems to misunderstand what that means within the context of his philosophical system.

See here for a post I wrote which I could have called The Final Antidote to Skepticism.

I looked into some of the most obvious objections. Some have reasonable answers (why not just kill yourself?), some others are based on a (to me) crazy assumption: that the original state of the biosphere pre-humans somehow is more valuable than the collective experience of the human race.
To which I don't just disagree, but think it's a logic error, since values exist only in the mind of those who can compute it, whatever it is.

The author is far from alone in his view that both a complete rightness criterion and a consistent decision method must be required of all serious moral theories.

Among hedonistic utilitarians it's quite normal to demand both completeness, to include all (human) situations, and consistency, to avoid contradictions. The author simply describes what's normal among consequentialists, who, after all, are more or less the rational ones. ;-) There's one interesting exception though! The demand to include all situations, including the non-human ones, is radical, and quite hard a challenge for hedonistic utilitarians, who do have problems with the bloodthirsty predators of the jungle.

And how she will use this relation to make safer AI?

Get a job at Google or seek to influence the people developing the AI. If, say, you were a beautiful woman you could, probably successfully, start a relationship with one of Google's AI developers.

Eliezer and E.T. Jaynes strongly urge seeing probabilities as subjective degrees of certainty that follow fixed laws (an extension of logic). If QBism is supposed to be compatible with this view - and yet not a form of MWI - then where do the complex numbers come from? Do they represent the map or the territory?

https://www.ted.com/talks/amy_webb_how_i_hacked_online_dating

this is a relevant ted talk. She does a similar process to what I worked through.

Worth noting that if I understand the mode of action of these proteins and have correctly read the tested dosages in the papers out thus far, a treatment would be more along the lines of the antibody immunotherapy infusions that people get for some types of cancer than a pill for a systemic effect, with localized versions like a nebulizer applied to things like a strong lung infection. And it's likely to cause a temporary inflammatory effect as lots of cells (including ones with no viral infection or latent viruses that everyone and I mean everyone has) blow up. You probably don't take this for a cold.

I think my sentence is true with both definitions of success.

Of course not. Then you meant simply the success of the goals of the group's creators?