Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Will_Pearson comments on Surprised by Brains - Less Wrong

22 Post author: Eliezer_Yudkowsky 23 November 2008 07:26AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (25)

Sort By: Old

You are viewing a single comment's thread.

Comment author: Will_Pearson 23 November 2008 10:11:24AM 3 points [-]

Believer: The search space is compressible -

The space of behaviors of Turing Machines is not compressible, sub spaces are, but not the whole lot. What space do you expect the SeedAIs to be searching? If you could show that it is compressible and bound to have an uncountable number of better versions of the SeedAI, then you could convince me that I should worry about Fooming.

As such when I think of self-modifiers I think of them searching the space of Turing Machines, which just seems hard.

Comment author: DilGreen 11 October 2010 02:02:43AM 2 points [-]

The space of possible gene combinations is not compressible - under the evolutionary mechanism.

The space of behaviours of Turing machines is not compressible, in the terms in which that compression has been envisaged.

The mechanism that compresses search space that Believer posits is something to do with brains; something to do with intelligence. And it works - we know it does; Kekule works on the structure of benzene without success; sleeps, dreams of a serpent biting its own tail, and waking, conceives of the benzene ring.

The mechanism (and everyone here believes that it is a mechanism) is currently mysterious. AI must possess this mechanism, or it will not be AI.