brazil84 comments on The $125,000 Summer Singularity Challenge - Less Wrong

20 Post author: Kaj_Sotala 29 July 2011 09:02PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (259)
Tags: charity siai

Comments (259)

You are viewing a single comment's thread. Show more comments above.

Comment author: brazil84 30 July 2011 11:08:11PM 5 points [-]

Further to my previous comment, I found the second final Jeopardy puzzle to be instructive. The category was "US Cities" and the clue was this:

Its largest airport was named for a World War II hero; its second largest, for a World War II battle.

A reasonably smart human will come up with an algorithm on the fly for solving this, which is to start thinking of major US cities (likely to have 2 or more airports); remember the names of their airports, and think about whether any of the names sound like a battle or a war hero. The three obvious cities to try are Los Angeles, New York, and Chicago. And "Midway" definitely sounds like the name of a battle.

But Watson was totally clueless. Even though it had the necessary information, it had to rely on pre-programmed algorithms to access that information. It was apparently unable to come up with a new algorithm on the fly.

Probably Watson relies heavily on statistical word associations. If the puzzle has "Charles Shulz" and "This Dog" in it, it will probably guess "Snoopy" without really parsing the puzzle. I'm just speculating here, but my impression is that AI has a long way to go.

Comment author: Sniffnoy 31 July 2011 02:01:59AM 4 points [-]

A reasonably smart human will come up with an algorithm on the fly for solving this, which is to start thinking of major US cities (likely to have 2 or more airports); remember the names of their airports, and think about whether any of the names sound like a battle or a war hero. The three obvious cities to try are Los Angeles, New York, and Chicago. And "Midway" definitely sounds like the name of a battle.

But Watson was totally clueless. Even though it had the necessary information, it had to rely on pre-programmed algorithms to access that information. It was apparently unable to come up with a new algorithm on the fly.

This isn't meaningful. Whatever method we use to "come up with algorithms on the fly" is itself an algorithm, just a more complicated one.

Probably Watson relies heavily on statistical word associations. If the puzzle has "Charles Shulz" and "This Dog" in it, it will probably guess "Snoopy" without really parsing the puzzle.

This isn't true. You know, a lot of the things you're talking about here regarding Watson aren't secret...

Comment author: brazil84 31 July 2011 02:14:25AM 1 point [-]

This isn't meaningful. Whatever method we use to "come up with algorithms on the fly" is itself an algorithm, just a more complicated one

Then why wasn't Watson simply programmed with one meta-algorithm rather than hundreds of specialized algorithms?

This isn't true. You know, a lot of the things you're talking about here regarding Watson aren't secret.

FWIW, the wiki article indicates that Watson would "parse the clues into different keywords and sentence fragments in order to find statistically related phrases." Would you mind giving me some links which show that Watson doesn't rely heavily on statistical word associations?

Comment author: Sniffnoy 31 July 2011 12:58:03PM 1 point [-]

Then why wasn't Watson simply programmed with one meta-algorithm rather than hundreds of specialized algorithms?

I don't have a clue what you're talking about. Where are you getting this claim that it was programmed with "hundreds of specialized algorithms"? And how is that really qualitatively different from what we do?

Would you mind giving me some links which show that Watson doesn't rely heavily on statistical word associations?

I never said it didn't. I was contradicting your statement that relied on that without any parsing.

Comment author: brazil84 31 July 2011 03:52:56PM *  1 point [-]

I don't have a clue what you're talking about. Where are you getting this claim that it was programmed with "hundreds of specialized algorithms"?

For one thing, the Wiki article talks about thousands of algorithms. My common sense tells me that many of those algorithms are specialized for particular types of puzzles. Anyway, why didn't Watsons creators program Watson with a meta-algorithm to enable it to solve puzzles like the Airport puzzle?

And how is that really qualitatively different from what we do?

For one thing, smart people can come up with new algorithms on the fly. For example an organized way of solving the airport puzzle. If that were just a matter of making a more complicated computer program, then why didn't Watson's creators do it?

I was contradicting your statement that relied on that without any parsing

My statement was speculation. So if you are confident that it is wrong, then presumably you must have solid evidence to believe so. If you don't know one way or another, then we are both in the same boat.

Comment author: handoflixue 02 August 2011 06:22:40PM 1 point [-]

For one thing, smart people can come up with new algorithms on the fly. For example an organized way of solving the airport puzzle. If that were just a matter of making a more complicated computer program, then why didn't Watson's creators do it?

That's like asking why a human contestant failed to come up with a new algorithm on the fly. Or, put simply: no one is perfect. Not the other players, not Watson, and not Watson's creators. While you've certainly identified a flaw, I'm not sure it's really quite as big a deal as you make it out to be. I mean, Watson did beat actual humans, so clearly they managed something fairly robust.

I don't think Watson is anywhere near an AGI, but the field of AI development seems to mostly include "applied-AI" like Deep Blue and Watson, and failures, so I'm going to go ahead and root for the successes in applied-AI :)

Comment author: brazil84 03 August 2011 12:57:51AM *  0 points [-]

That's like asking why a human contestant failed to come up with a new algorithm on the fly.

I disagree. A human contestant who failed to come up with a new algorithm was perhaps not smart enough, but is still able to engage in the same kind of flexible thinking under less challenging circumstances. I suspect Watson cannot do so under any circumstances.

I mean, Watson did beat actual humans, so clearly they managed something fairly robust.

Without it's super-human buzzer speed, I doubt Watson would have won.

Comment author: gwillen 08 August 2011 06:38:36PM 2 points [-]

I believe that the way things were designed, Ken Jennings was probably at least as good as Watson on buzzer speed. Watson presses the buzzer with a mechanical mechanism, to give it a latency similar to a finger; and Watson doesn't start going for the buzzer until it sees the 'buzzer unlocked' signal. By contrast, Ken Jennings has said that he starts pressing the buzzer before the signal, relying on his intuitive sense of the typical delay between the completion of a question and the buzzer-unlock signal.

Comment author: brazil84 08 August 2011 08:49:35PM 5 points [-]

Here's what Ken Jennings had to say:

Watson does have a big advantage in this regard, since it can knock out a microsecond-precise buzz every single time with little or no variation. Human reflexes can't compete with computer circuits in this regard. But I wouldn't call this unfair ... precise timing just happens to be one thing computers are better at than we humans. It's not like I think Watson should try buzzing in more erratically just to give homo sapiens a chance.

Here's what Wikipedia says:

The Jeopardy! staff used different means to notify Watson and the human players when to buzz, which was critical in many rounds. The humans were notified by a light, which took them tenths of a second to perceive. Watson was notified by an electronic signal and could activate the buzzer within about eight milliseconds. The humans tried to compensate for the perception delay by anticipating the light, but the variation in the anticipation time was generally too great to fall within Watson's response time. Watson did not operate to anticipate the notification signal.

Comment author: gwillen 08 August 2011 11:29:27PM 0 points [-]

Interesting, thanks. Upvote for doing some actual research. ;-)

Comment author: Sniffnoy 31 July 2011 07:21:14PM 1 point [-]

For one thing, the Wiki article talks about thousands of algorithms. My common sense tells me that many of those algorithms are specialized for particular types of puzzles. Anyway, why didn't Watsons creators program Watson with a meta-algorithm to enable it to solve puzzles like the Airport puzzle?

Er... they did? The whole thing ultimately had to produce one answer, after all. It just wasn't good enough.

Comment author: brazil84 31 July 2011 08:19:58PM -1 points [-]

The whole thing ultimately had to produce one answer, after all. It just wasn't good enough.

Ok, then arguably it's not so simple to create an algorithm which is "just more complicated." I mean, one could say that an ICBM is just like a Quassam rocket, but just more complicated.

Comment author: Clippy 01 August 2011 02:59:07PM 4 points [-]

An ICBM is "just" a bow-and-arrow system with a more precise guidance system, more energy available to spend reaching its destination, and a more destructive payload.

Comment author: brazil84 01 August 2011 07:38:50PM 1 point [-]

Right, and it's far more difficult to construct. It probably took thousands of years between the first missile weapons and modern ICBMs. I doubt that it will take thousands of years to create general AI, but it's still the same concept.

The first general AI will probably be "just" an algorithm running on a digital computer.

Comment author: Sniffnoy 31 July 2011 08:26:09PM 0 points [-]

This comment doesn't appear to have any relevance. Where did anyone suggest that the way to make it better is to just make it more complicated? Where did anyone suggest that improving it would be simple? I am completely baffled.

Comment author: brazil84 31 July 2011 08:38:20PM 1 point [-]

Earlier, we had this exchange:

Me:

But Watson was totally clueless. Even though it had the necessary information, it had to rely on pre-programmed algorithms to access that information. It was apparently unable to come up with a new algorithm on the fly.

You:

Whatever method we use to "come up with algorithms on the fly" is itself an algorithm, just a more complicated one.

So you seemed to be saying that there's no big deal about the human ability to come up with a new algorithm -- it's just another algorithm. Which is technically true, but this sort of meta-algorithm obviously would require a lot more sophistication to create.

Comment author: Sniffnoy 31 July 2011 08:51:39PM 0 points [-]

Well, yes. Though probably firstly should note that I am skeptical that what you are talking about -- the process of answering a Final Jeopardy question -- could actually be described as coming up with new algorithms on the fly in the first place. Regardless, if we do accept that, my point that there is no meaningful distinction between relying on pre-programmed algorithms, and (algorithmically) coming up with new ones on the fly, stands. There's plenty of ways in which our brains are more sophisticated than Watson, but that one isn't a meaningful distinction. Perhaps you mean something else.

Powered by Reddit Powered by Reddit