"It seems unlikely that you could have a genetic algorithm operate on a population of code and end up with a program that passes the Turing test"
Well, we have one case of it working, and that wasn't even with the process being designed with the "pass the Turing test" specifically as a goal.
"because at each step the genetic algorithm (as an optimization procedure) needs to have some sense of what is more or less likely to pass the test."
Having an automated process for determining with certainty that something passes the Turing test is quite stronger than merely having nonzero information. Suppose I'm trying to use a genetic algorithm to create a Halting Tester, and I have a Halting Tester that says that a program doesn't halt. If I know that the program does, in fact, not halt after n steps (by simply running the program for n steps), that provides nonzero information about the efficacy of my Halting Tester. This suggests that I could create a genetic algorithm for creating Halting Testers (obviously, I couldn't evolve a perfect Halting Tester, but perhaps I could evolve one that is "good enough", given some standard). And who knows, maybe if I had such a genetic algorithm, not only would my Halting Testers evolve better Halting Testing, but since they are competing against each other, they would evolve better Tricking Other Halting Testers, and maybe that would eventually spawn AGI. I don't find that inconceivable.
Well, we have one case of it working, and that wasn't even with the process being designed with the "pass the Turing test" specifically as a goal.
Are you referring to the biological evolution of humans, or stuff like this?
Having an automated process for determining with certainty that something passes the Turing test is quite stronger than merely having nonzero information.
Right; how did you interpret "some sense of what is more or less likely to pass the test"?
Folks here should be familiar with most of these arguments. Putting some interesting quotes below:
http://aeon.co/magazine/being-human/david-deutsch-artificial-intelligence/
"Creative blocks: The very laws of physics imply that artificial intelligence must be possible. What's holding us up?"
He also says confusing things about induction being inadequate for creativity which I'm guessing he couldn't support well in this short essay (perhaps he explains better in his books). Not quoting here. His attack on Bayesianism as an explanation for intelligence is valid and interesting, but could be wrong. Given what we know about neural networks, something like this does happen in the brain, and possibly even at a concept level.
His final conclusions are disagreeable. He somehow concludes that the principal bottleneck in AGI research is a philosophical one.
In his last paragraph, he makes the following controversial statement:
This would be false if, for example, the mother controls gene expression while a foetus develops and helps shape the brain. We should be able to answer this question definitively once we can grow human babies completely in vitro. Another problem would be the impact of the cultural environment. A way to answer this question would be to see if our Stone Age ancestors would be classified as AGIs under a reasonable definition