rwallace comments on Transparency and Accountability - Less Wrong

16 Post author: multifoliaterose 21 August 2010 01:01PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (141)

You are viewing a single comment's thread. Show more comments above.

Comment author: Larks 21 August 2010 06:03:55PM *  4 points [-]

SIAI does not presently exhibit high levels of transparency and accountability... For this reason together with the concerns which I express about Existential Risk and Public Relations, I believe that at present GiveWell's top ranked charities VillageReach and StopTB are better choices than SIAI

  • Suppose SIAI were a thousand times less accountable than VillageReach.
  • Suppose this made SIAI a million times less effective than it could be.
  • Suppose that even the most efficient Existential Risk charity could only reduce P(uFAI|AGI) by 10^-9
  • Suppose the odds of an AI singularity or 'foom' were only 10^-9.
  • Suppose a negative singularity only set mankind back by 100 years, rather than paperclipping the entire light cone and destroying all human value together.*

Even then the expected lives saved by SIAI is ~10^28.

  • It's patently obvious that SIAI has an annual income of less than $10^6.

  • Suppose the marginal dollar is worth 10^3 times less than the average dollar.

Even then a yearly donation of $1 saves an expected10^18 lives.

A yearly donation of £1 to VillageReach saves 100,000,000,000,000,000,000 fewer people.

*Sorry Clippy, but multifoliaterose is damaging you here; SIAI is a lot more amenable to negotiation than anyone else.

Comment author: rwallace 22 August 2010 03:41:24AM 14 points [-]

A problem with Pascal's Mugging arguments is that once you commit yourself to taking seriously very unlikely events (because they are multiplied by huge potential utilities), if you want to be consistent, you must take into account all potentially relevant unlikely events, not just the ones that point in your desired direction.

To be sure, you can come up with a story in which SIAI with probability epsilon makes a key positive difference, for bignum expected lives saved. But by the same token you can come up with stories in which SIAI with probability epsilon makes a key negative difference (e.g. by convincing people to abandon fruitful lines of research for fruitless ones), for bignum expected lives lost. Similarly, you can come up with stories in which even a small amount of resources spent elsewhere, with probability epsilon makes a key positive difference (e.g. a child saved from death by potentially curable disease, may grow up to make a critical scientific breakthrough or play a role in preserving world peace), for bignum expected lives saved.

Intuition would have us reject Pascal's Mugging, but when you think it through in full detail, the logical conclusion is that we should... reject Pascal's Mugging. It does actually reduce to normality.