GiveWell interview with major SIAI donor Jaan Tallinn

jsalvatier

GiveWell recently release notes from their interview with Jaan Tallinn, Skype co-founder and a major SIAI donor, about SIAI (link). Holden Karnofsky says

[M]y key high-level takeaways are that

I appreciated Jaan's thoughtfulness and willingness to engage in depth. It was an interesting exchange.

I continue to disagree with the way that SIAI is thinking about the "Friendliness" problem.

It seems to me that all the ways in which Jaan and I disagree on this topic have more to do with philosophy (how to quantify uncertainty; how to deal with conjunctions; how to act in consideration of low probabilities) and with social science-type intuitions (how would people likely use a particular sort of AI) than with computer science or programming (what properties has software usually had historically; which of these properties become incoherent/hard to imagine when applied to AGI)

GiveWell recently release notes from their interview with Jaan Tallinn, Skype co-founder and a major SIAI donor, about SIAI (link). Holden Karnofsky says

[M]y key high-level takeaways are that

I appreciated Jaan's thoughtfulness and willingness to engage in depth. It was an interesting exchange.

I continue to disagree with the way that SIAI is thinking about the "Friendliness" problem.

It seems to me that all the ways in which Jaan and I disagree on this topic have more to do with philosophy (how to quantify uncertainty; how to deal with conjunctions; how to act in consideration of low probabilities) and with social science-type intuitions (how would people likely use a particular sort of AI) than with computer science or programming (what properties has software usually had historically; which of these properties become incoherent/hard to imagine when applied to AGI)

I continue to be impressed by Holden's thoughtfulness and rigor. If people want charity evaluators to start rating the effectiveness of x-risk-reducing organizations, then those organizations need to do a better job with (1) basic org effectiveness and transparency - publishing a strategic plan, whistleblower policy, and the stuff Charity Navigator expects - and with (2) making the case for the utility of x-risk reduction more clearly and thoroughly.

Luckily I am currently helping the Singularity Institute with both projects, and there are better reasons to do (1) and (2) than 'looking good to charity evaluators'. That is a side benefit, though.

25

GiveWell interview with major SIAI donor Jaan Tallinn

25

25

25

GiveWell interview with major SIAI donor Jaan Tallinn

25

25