Eliezer_Yudkowsky comments on Information theory and FOOM - Less Wrong

6 Post author: PhilGoetz 14 October 2009 04:52PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (93)

You are viewing a single comment's thread. Show more comments above.

Comment author: PhilGoetz 15 October 2009 03:52:22PM *  5 points [-]

Actually, that's based on the mistaken belief that selection can provide only 1 bit of information per generation. If you'll look down to the end of the original 2007 post, you'll see I gave the correct (and now Eliezer-approved) formulation, which is:

If you take a population of organisms, and you divide it arbitrarily into 2 groups, and you show the 2 groups to God and ask, "Which one of these groups is, on average, more fit?", and God tells you, then you have been given 1 bit of information.

But if you take a population of organisms, and ask God to divide it into 2 groups, one consisting of organisms of above-average fitness, and one consisting of organisms of below-average fitness, that gives you a lot more than 1 bit. It takes n lg(n) bits to sort the population; then you subtract out the information needed to sort each half, so you gain n lg(n) - 2(n/2)lg(n/2) = n[lg(n) - lg(n/2)] = nlg(2) = n bits.

If you do tournament selection, you have n/2 tournaments, each of which gives you 1 bit, so you get n/2 bits per generation.

ADDED: This doesn't immediately get you out of the problem, as n bits spread out among n genomes gives you 1 bit per genome. That doesn't mean, though, that you've gained only 1 bit for the species as a whole. The more-important observation in that summary is that organisms with more mutations are more likely to die, eliminating > 1 mutation per death on average.

Comment author: Eliezer_Yudkowsky 15 October 2009 05:38:27PM 1 point [-]

That's not exactly Eliezer-approved, because now the real problem is to tell what the conditions are more like in nature - Worden or MacKay or somewhere in between. That's what I put up on the Wiki as summary of the state of information. Mathematical assumptions are cheaper than empirical truths.