You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

XiXiDu comments on [Link] A review of proposals toward safe AI - Less Wrong Discussion

7 Post author: XiXiDu 05 April 2011 01:49PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (10)

You are viewing a single comment's thread. Show more comments above.

Comment author: XiXiDu 05 April 2011 04:20:36PM *  2 points [-]

Here is an interesting interview between Hugo de Garis and Ben Goertzel:

Gut feeling: I’d probably sacrifice myself to create a superhuman artilect, but not my kids…. I do have huge ambitions and interests going way beyond the human race – but I’m still a human.

[...]

And the better an AGI theory we have, the more intelligently we’ll be able to bias the odds. But I doubt we’ll be able to get a good AGI theory via pure armchair theorizing. I think we’ll get there via an evolving combination of theory and experiment – experiment meaning, building and interacting with early-stage proto-AGI systems of various sorts.

Comment author: Normal_Anomaly 05 April 2011 10:17:39PM 0 points [-]

experiment meaning, building and interacting with early-stage proto-AGI systems of various sorts.

I'm not very familiar with Goertzel's ideas. Does he recognize the importance of not letting the proto-AGI systems self-improve while their values are uncertain?

Comment author: benelliott 06 April 2011 02:16:13PM 2 points [-]

From what I've gathered Ben thinks that these experiments will reveal that friendliness is impossible, that 'be nice to humans' is not a stable value. I'm not sure why he thinks this.