Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Ben_Jones comments on Recursive Self-Improvement - Less Wrong

14 Post author: Eliezer_Yudkowsky 01 December 2008 08:49PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (54)

Sort By: Old

You are viewing a single comment's thread.

Comment author: Ben_Jones 02 December 2008 03:29:16PM 1 point [-]

Eliezer: part of the AIXI sequence, which I don't think I'll end up writing.

Ahh, that's a shame, though fully understood. Don't suppose you (or anyone) can link to some literature about AIXI? Haven't been able to find anything comprehensive yet comprehensible to an amateur.

Tim Tyler: Brainpower went into making new brains historically - via sexual selection. Feedback from the previous generation of brains into the next generation has taken place historically.

Tim, Dawkins has a nice sequence in The Blind Watchmaker about a species of bird in which the female began selecting for big, lustrous tails. This led to birds with tails so big they could barely fly to escape predators. While selecting for intelligence in a partner is obviously plausible, I'd have to see very compelling evidence that it's leading to continuously smarter people, or even that it ever could. Possibly a loop of some description there, but K definitely < 1.

However!

Owing to our tremendous lack of insight into how genes affect brains

So what happens when we start to figure out what those genes do? and then start switching them on and off, and gaining more knowledge and insight into how the brain attacks problems? As we've read recently, natural selection increased brainpower (insight) massively through blind stumbling onto low-hanging fruit in a relatively small amount of time. Why would we suppose it reached any sort of limit - or at least a limit we couldn't surmount? The 18 years to maturity thing is pretty irrelevant here, as long as, say, 5% compound insight can be gained per generation. You're still looking at exponential increases, and you might only need a handful of generations before the FOOM itself switches medium.