I think I remember one particular prominent intellectual who, decades ago, essentially declared that when chess could be played better by a computer than a human, the problem of AI would be solved.

Hofstadter, in Godel, Escher, Bach?

Maybe you're one of those Cartesian dualists who thinks humans have souls that don't exist in physical reality and that's how they do their thinking

Not at all. Brains are complicated, not magic. But complicated is bad enough.

Would you consider the output of a regression a black box?

In the sense that we don't understand why the coefficients make sense; the only way to get that output is feed a lot of data into the machine and see what comes out. It's the difference between being able to make predictions and understanding what's going on (e.g. compare epicycle astronomy with the Copernican model. Equally good predictions, but one sheds better light on what's happening).

What's your machine learning background like, by the way?

One semester graduate course a few years ago.

It seems like you are counting it as a point against chess programs that we know exactly how they work, and a point against Watson that we don't know exactly how it works.

The goal is to understand intelligence. We know that chess programs aren't intelligent; the state space is just luckily small enough to brute force. Watson might be "intelligent", but we don't know. We need programs that are intelligent and that we understand.

My impression is that many, if not most, experts in AI see human intelligence as essentially algorithmic and see the field of AI as making slow progress towards something like human intelligence

I agree. My point is that there isn't likely to be a simple "intelligence algorithm". All the people like Hofstadter who've looked for one have been floundering for decades, and all the progress has been made by forgetting about "intelligence" and carving out smaller areas.