Comment on Open Thread June 2010, Part 3

Wed, 16 Jun 2010 16:56:26 GMT

We should assign some substantial probability to getting some weighting of our preference (from bargaining with transparency, acausal trade, altruistic brain emulations, etc). If a moderate weighting of our preferences gets most of the potential utility, then the expected utility of inhuman AIs getting powerful won't be astronomically less than the expected utility of, e.g. a 'Friendly AI' acting on idealized human preferences.

Discuss