Transhumanist

Transhumanist

I am agreeing with that claim. Maybe what I'm really disagreeing with is the implication that the shortfall is some kind of issue?

Since they are getting contributions, gifts, and grants, I don't see that as a shortfall. That's like saying the SI has a huge shortfall because they get all their support through contributions. Many supporters of cryonics are ultra-wealthy. The second the material existence of Alcor is threatened, they'll contribute. I understand that you are following Alcor's public materials, but try attending a cryonics conference.

Gwern -- Alcor is not "run at an annual loss of hundreds of thousands". I can't imagine where you heard this.

Here.

Which predictions are very obvious?

The only part I disagree with strongly is the language of the last point. Referring to CEV as "THE morally right thing to do" makes it seem as if it were set in stone as the guaranteed best path to creating FAI, which it isn't. EY argues that building Friendly AI instead of just letting the chips fall where they may is the morally right thing to do, and I'd agree with that, but not that CEV specifically is the right thing to do.

One general goal point for FAI is to target outcomes "at least as good" as those which would be caused by benevolent human mind upload(s). So, the kind of "moral development" that a community of uploads would undergo should be encapsulated within a FAI. In fact, any beneficial area of the moral state space that would be accessible starting from humans or any combination of humans and tools should be accessible by a good FAI design. CEV is one such proposal towards such a design.

As I understand it, yes, the thinking is to optimize for our terminal values instead of this hypothetical alien species or some compromise of the two. However, if values among different intelligent species converge given greater intelligence, knowledge, and self-reflection, then we would expect our FAI to have goals that converge with the alien FAI. If values do not converge, then we would suppose our FAI to have different values than alien FAIs.

A "terminal value" might include carefully thinking through philosophical questions such as this and designing the best goal content possible given these considerations. So, if there are hypothetical alien values that seem "correct" (or simply sufficiently desirable from the subjective perspective) to extrapolated humanity, these values would be integrated into the CEV-output.

I think it depends on what is meant by "values". I would say that the values change while the fundamental motivations are fixed, though Vladimir's response makes me unsure about this. Another way of saying it is that supergoals are fixed but the "Friendliness content" changes. (Though I haven't seen the phrase "Friendliness content" around much lately, perhaps it's being discarded in favor of more formal terms.)

Maybe another useful distinction would be between Friendliness structure and content (see the CFAI entry on the wiki).