Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

AnnaSalamon comments on Lost Purposes - Less Wrong

68 Post author: Eliezer_Yudkowsky 25 November 2007 09:01AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (73)

Sort By: Old

You are viewing a single comment's thread.

Comment author: AnnaSalamon 17 November 2016 05:23:31AM 2 points [-]

"More than anything, you must be thinking of carrying your movement through to cutting him. You must thoroughly research this." (Emphasis added.)

I wish I could upvote this post multiple times. Especially the "You must thoroughly research this" part, although also all the examples of institutions failing to accomplish much of anything, even when you might naively have really thought they would (e.g., individual health procedures work; how could large-scale institutions for doing those procedures not work?). I read this post back in 2008 and was like "oh, cool, it's a nice handle for a common little bug that happens to me and other people". I reread it just now, and was more like "Shit, good point, we actually do need to 'thoroughly research' how to avoid lost purposes, or how to carry our movements through to cutting the enemy. For lack of this, the EA and AI Safety movements have perhaps mostly wasted their last several years." Recommending the "thoroughly research" part to anyone who wants to carry on a complex task across a set of tens or hundreds (or thousands) of people without having all the value dissipate into noise.