Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Brickman comments on That Tiny Note of Discord - Less Wrong

17 Post author: Eliezer_Yudkowsky 23 September 2008 06:02AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (34)

Sort By: Old

You are viewing a single comment's thread.

Comment author: Brickman 27 July 2011 01:07:43AM 0 points [-]

Despite having seen you say it in the past, it wasn't until reading this article that in sunk in for me just how little danger we were actually in of Eliezer1997 (or even Eliezer2000) actually making his AI. He had such a poor understanding of the problem, I don't see how he could've gotten there from here without having to answer the question of "Ok, now what do I tell the AI to do?" The danger was in us almost never getting Eliezer2008, or in Eliezer2000 wasting a whole bunch of future-minded peoples' money getting to the point where he realized he was stuck.

Except I suppose he did waste a lot of other people's money and delay present-you by several years. So I guess that danger wasn't entirely dodged after all. And maybe you did have something you planned to tell the AI to do anyways, something simple and useful sounding in and of itself with a tangible result. Probably something it could do "before" solving the question of what morality is, as a warmup. That's what the later articles in this series suggest, at least.

I also peeked at the Creating Friendly AI article just to see it. That, unlike this, looks like the work of somebody who is very, very ready to turn the universe into paperclips. There was an entire chapter about why the AI probably won't ever learn to "retaliate", as if that was one of the most likely ways for it to go wrong. I couldn't even stand to read more than half a chapter and I'm not you.

"To the extent that they were coherent ideas at all" you've said of half-baked AI ideas in other articles. It's nice to finally understand what that means.