When you're talking about passing a Turing test, power is absolutely key.
So, if we made a program that beat the Turing test, but the hardware consumed a lot of power, it would be a failure, but if we ran the program on different hardware with the exact same specs, except it was more energy efficient, it would be a success?
You're ignoring fundamental limits of computing efficiency here. You can't have the same specs if you have many orders of magnitude more energy efficiency. Something's got to give. At the transistor level you can't preserve the same amount of computation for vastly less power. This is why a petaflops human brain is not the same as a petaflops super computer. Computation is represented differently because the power constraints of a human brain force it to be. You cannot do the same amount of processing with a brain that you can do with a modern petaflops cl...
Some interesting numbers to contextualize IBM’s Watson:
To put this in perspective, a conservative upper bound for a human being standing still is at most about 150w — less than 1/10 of 1% of Watson — and the person just holds the buzzer and operates it with a muscular control system.
Each of the servers generates a maximum of 6,649 BTU/hour. Watson overall would generate about 600,000 BTU/hour and require massive amounts of air conditioning. I don’t know a good estimate on heat removal, but it would up Watson’s energy cost significantly.
I don’t mean to criticize Watson unduly; it certainly is an impressive engineering achievement and has generated a lot of good publicity and public interest in computing. The engineering feat is impressive if for no other reason than that it is the first accomplishment of this scale, and pioneering is always hard… future Watsons will be cheaper, faster, and more effective because of IBM’s great work on this.
But at the same time, the amazing power and storage costs for Watson really kind of water it down for me. I’m not surprised that if you throw power and hardware and memory at a problem, you can use rather straightforward machine learning methods to solve it. I feel similarly about Deep Blue and chess.
A Turing test that would be more impressive to me would be building something like Watson or Deep Blue that is not allowed to consume more power than an average human, and has comparable memory and speed. The reason this would be impressive is that in order to build it, you’d have to have some way of representing data and reasoning in the system that is efficient to a similar degree that human minds are. One thing you could not do is simply concatenate an unreasonable number of large feature vectors together and overfit a machine learning model. Since this is an important open problem with lots of implications, we should use funding and publicity to drive research organizations like IBM towards that goal. Maybe building Watson is a first step and now the task is to miniaturize Watson, and in doing so, we’ll be forced to learn about efficient brain architectures along the way.
Note: I gathered the numbers above by looking here and then scouring around for various listings of specific hardware specs. I'm willing to believe some of my numbers might be off, but probably not significantly.