"as long as the differences in the new situation are things that were originally allowed to vary"

And all the things that were fixed are still present of course! (since these are what we are presuming are the causal factors)

'How many new assumptions, exactly, are fatal? How many new terms are you allowed to introduce into an old equation before it becomes "unvetted", a "new abstraction"?'

Every abstraction is made by holding some things the same and allowing other things to vary. If it allowed nothing to vary it would be a concrete not an abstraction. If it allowed everything to vary it would be the highest possible abstraction - simply "existence." An abstraction can be reapplied elsewhere as long as the differences in the new situation are things that were originally allowed to vary.

That's not to say this couldn't be a black swan, there's no guarantees, but going purely on evidence what other choice do you have except to do it this way.

"So if you suppose that a final set of changes was enough to produce a sudden huge leap in effective intelligence, it does beg the question of what those changes were."

Perhaps the final cog was language. The original innovation is concepts: the ability to process thousands of entities at once by forming a class. Major efficiency boost. But chimps can form basic concepts and they didn't go foom.

Because forming a concept is not good enough - you have to be able to do something useful with it, to process it. Chimps got stuck there, but we passed abstractions through our existing concrete-only processing circuits by using a concrete proxy (a word).

How clear is the distinction between knowledge and intelligence really? The whole genius of the digitial computer is that programs are data. When a human observes someone else doing something, they can copy that action: data seems like programs there too.

And yet "cognitive" is listed as several levels above "knowledge" in the above post, and yesterday CYC was mocked as being not much better than a dictionary. Maybe cognition and knowledge are not so separate, but two perspectives on the same thing.

"Recursion that can rewrite the cognitive level is worth distinguishing."

Eliezer, would a human that modifies the genes that control how his brain is built qualify as the same class of recursion (but with a longer cycle-time), or is it not quite the same?

'I found my most productive fifteen minutes were when a friend said, out of nowhere, "want to see who can do the most work in 15 minutes?"'

That's interesting, because historically great works have been accomplished when a group of really talented people get together in the same place (e.g. Florence, Silicon Valley, Manhattan Project).

The Internet is great in that it enables you to find like minded people and bounce ideas of them. But that's only half the achievement puzzle. The other half is pestering each other to work, which the Internet is not so good for.

Many rationalists (not saying Eli is one) are of the opinion that introspection is worthless (or at least suspect), so not surprising that trying to predict certain things doesn't occur to us.

While I totally agree with the sentiment of Eliezer's prayer, I don't think saying a prayer on Thanksgiving makes you religious or even implies a belief in God - it's just tradition. It's harmless to follow traditions as long as you are epistemologically strong enough not to be in any danger of confusing reality and myth. Just like it's safe for a person with very strong reason to read a lot of fiction.

Robin's concept of "Singularity" may be very broad, but your concept of "Optimization Process" is too.

I agree. Creativity is not just being random. The old masters used measurement and perspective when painting their masterpieces, they didn't just sit there and hum and at the sky and wait for inspiration to strike them.

I think the idea that creativity is somehow mystical comes from a religious model of the human body. If you think your body has causal flesh and a supernatural/acausal soul, and that creativity comes from your soul (the part that is "you") then it follows that creativity comes from the acausal.

"So do we reason that the most unexpected events, convey the most information, and hence the most surprising acts are those that give us a pleasant shock of creativity - the feeling of suddenly absorbing new information?"

This is very cool.