I'm getting the feeling that Eliezer is starting to get overly eager to attack semantic stopsigns. I recommend magic oil in the evening and emergent phenomena in the morning.

My impression of "emergence" was that it's closely related to pattern recognition. You have atoms A, B..ZZZZZZZZZZ, and you recognize that these atoms form a certain pattern. So you say that a supercluster of galaxies/bar stool/intelligence "emerges" from a bunch of atoms.

I once had a prolonged debate with an anticognitivist. He, as usual, argued that no matter what kind of AI you build, if you take it apart, it's just "switches flipping". In that debate, I maintained that intelligence does not require anything else - it emerges from switches flipping the same way that Firefox emerges from switches flipping. Both are just human names for patterns that arise in a sea of subatomic particles.

[That was one frustrating debate... Both of us were equally bewildered that the other refuses to get it.]