Comment on Building toward a Friendly AI team

Thu, 07 Jun 2012 21:55:36 GMT

various game-theoretical concerns about cooperation

Most people have some pre-theoretic intuitions about cooperation, which game theory may merely formalize. It's not clear to me that familiarity with such theoretical concerns implies one ought to be more "altruistic" than average.

the surprisingly large apparent asymmetry between the size of altruistic returns available vs. the size of egoistic returns available

If someone is altruistic because they've maxed out their own egoistic values (or has gotten to severely diminishing returns), I certainly wouldn't count that against their rationality. But if "egoistic returns" include abstract values that the rest of humanity doesn't necessarily share, "large apparent asymmetry" is unclear to me.

as I've been saying earlier, the topic is weirdly neglected here for one with such high import

Where did you say that? (I wrote Shut Up and Divide? which may or may not be relevant depending on what you mean by "the topic".)

Surely it depends on how much more altruistic and how much more rational.

Why "surely", given that I'm not a random member of humanity, and may have more values in common with a less altruistic candidate than a more altruistic candidate?

Discuss