Mark Neyer

Message

You are Underestimating The Likelihood That Convergent Instrumental Subgoals Lead to Aligned AGI

This post is an argument for the Future Fund's "AI Worldview" prize. Namely, I claim that the estimates given for the following probability are too high: > P(misalignment x-risk|AGI)”: Conditional on AGI being developed by 2070, humanity will go extinct or drastically curtail its future potential due to loss of...

Sep 26, 20223

Questions about ''formalizing instrumental goals"

Epistemic Status: Autodidact outsider who suspects he has something to add to the conversation about AI risk. Abstract This essay raises questions about the methodology, and thus conclusions, reached in the paper “Formalizing Convergent Instrumental Goals.” This paper concluded that convergent instrumental goals common to any AGI would likely lead...

Apr 1, 20227

LESSWRONG
LW

LESSWRONG
LW

Mark Neyer

Mark Neyer

Mark Neyer

Mark Neyer

You are Underestimating The Likelihood That Convergent Instrumental Subgoals Lead to Aligned AGI

Questions about ''formalizing instrumental goals"

You are Underestimating The Likelihood That Convergent Instrumental Subgoals Lead to Aligned AGI

Questions about ''formalizing instrumental goals"