LESSWRONG
LW

Comment Permalink

I propose a game where there are resources to be identified (using these DNN computer vision algorithms), collected, and deposited at drop-off points. To advance embodied cognition, players get small robot drones of some sort, perhaps like a roomba with a robot arm attached.

The resources include dirty socks and plates, and the game is called "tidy skeptical_lurker's house, because he can't be bothered"

Lumifer9y-20

Just outsource it to Pentagon.

See in context

5 After Go, what games should be next for DeepMind?

by InquilineKea

10th Mar 2016

1 min read

5

So chess and Go are both games of perfect information. How important is it for the next game that DeepMind is trained on to be a game of perfect information?

How would the AI perform on generalized versions of both chess and Go? What about games like poker and Magic the Gathering?

How realistic do you think it's possible to train DeepMind on games of perfect information (full-map-reveal) against top-ranked players on games like Starcraft, AOE2, Civ, Sins of a Solar Empire, Command and Conquer, and Total War, for example? (in all possible map settings, including ones people don't frequently play at - e.g. start at "high resource" levels). How important is it for the AI to have a diverse set/library of user-created replays to test itself against, for example?

I'm also thinking... Shitty AI has always held back both RTS and TBS games.. Is it possible that we're only a few years away from non-shitty AI in all RTS and TBS games? Or is the AI in many of these games too hard-coded in to actually matter? (e.g. I know some people who develop AI for AOE2, and there are issues with AI behavior in the game being hard-coded in - e.g. villagers deleting the building they're building if you simply attack them).

Personal Blog

5

After Go, what games should be next for DeepMind?

3compartmentalization

New Comment

73 comments, sorted by

top scoring

Click to highlight new comments since: Today at 10:08 PM

Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings

[-]Gurkenglas9y120

Foldit.

[-]RaelwayScot9y120

Demis Hassabis has already announced that they'll be working on a Starcraft bot in some interview.

[-]Kaj_Sotala9y210

This interview, dated yesterday, doesn't go quite that far - he mentions Starcraft as a possibility, but explicitly says that they won't necessarily pursue it.

If the series continues this way with AlphaGo winning, what’s next — is there potential for another AI-vs-game showdown in the future?
I think for perfect information games, Go is the pinnacle. Certainly there are still other top Go players to play. There are other games — no-limit poker is very difficult, multiplayer has its challenges because it’s an imperfect information game. And then there are obviously all sorts of video games that humans play way better than computers, like StarCraft is another big game in Korea as well. Strategy games require a high level of strategic capability in an imperfect information world — "partially observed," it’s called. The thing about Go is obviously you can see everything on the board, so that makes it slightly easier for computers.
Is beating StarCraft something that you would personally be interested in?
Maybe. We’re only interested in things to the extent that they are on the main track of our research program. So the aim of DeepMind is not just to beat games, fun and excitin

... (read more)

[-]CronoDAS9y70

The game I'd like to see an AI for is Diplomacy. ;)

4Algernoq9y

Oh no! The AI would make us hate each other before betraying us.

[-]moridinamael9y70

Almost any game that their AI can play against itself is probably going to work. Except stuff like Pictionary where it's really important how a human, specifically, is going to interpret something.

I know a little bit about training neural networks, and I think it would be plausible to train one on a corpus of well-played StarCraft games to give it an initial sense of what it's supposed to do, and then having achieved that, let it play against itself a million times. But I don't think there's any need to let it watch how humans play. If it plays enough gam... (read more)

0Gurkenglas9y

It could become better than people at playing Pictionary, by drawing images that are most likely to be correctly recognized rather than the human way of translating the model in its head into a picture, and by analyzing what models are most likely to produce a picture rather than the human way of translating the picture into a model in its head. Except if you mean that it playing against itself would make it diverge into its own language of pictures. Although it might optimize in a direction that doesn't follow the spirit of the game, anologous to writing out the name of its task. Actually that could be interesting - could it invent a language that is maximally efficient at communicating concepts? To your last one, you might enjoy a MOBA where individual players have only information about stuff in their line of sight, but there's an extra player whose job it is to see everything and give "orders". I think there was one like that...

-2V_V9y

Demis Hassabis mentioned StarCraft as something they might want to do next. Video.

[-]Lumifer9y70

RTS is a bit of a special case because a lot of the skill involved is micromanagement and software is MUCH better at micromanagement than humans.

I don't expect to see highly sophisticated AI in games (at least adversarial, battle-it-out games) because there is no point. Games have to be fun which means that the goal of the AI is to gracefully lose to the human player after making him exert some effort.

You might be interested in Angband Borg.

[-]Torchlight_Crimson9y120

I don't expect to see highly sophisticated AI in games (at least adversarial, battle-it-out games) because there is no point. Games have to be fun which means that the goal of the AI is to gracefully lose to the human player after making him exert some effort.

I'm not sure about that. A common complaint about these kinds of games is that the AI's blatantly cheat, especially on higher difficulty levels. I could very well see a market for an AI that could give the human a challenge without cheating.

3Vaniver9y

Several years ago, Backgammon AI was at the point where it could absolutely demolish humans without cheating. My impression is that people hated it, and even if they rolled the dice for the AI and input the results themselves they were pretty sure that it had to be cheating somehow.

[-]Kaj_Sotala9y100

May have been a vocal minority. You get some people incorrectly complaining about AI cheating in any game that utilizes randomness (Civilization and the new XCOMs are two examples I know of); usually this leads to somebody running a series of tests or decompiling the source code to show people that no, the die rolls are actually fair or (as is commonly the case) actually actively biased in the human player's favor.

This never stops some people from complaining nonetheless, but a lot of others find the evidence convincing enough and just chalk it up to their own biases (and are less likely to suspect cheating when they play the next game that has random elements).

4[anonymous]9y

The Civ 5 AI does cheat insofar as it doesn't have to deal with the fog of war, IIRC. The XCOM AI seems to cheat because they've don't report the actual probability.

4Torchlight_Crimson9y

Not just that, especially on higher difficulty levels.

1Kaj_Sotala9y

Right, I meant that Civ doesn't cheat when it comes to die rolls - e.g. if it displays a 75% chance for the player to win a battle, then the probability really is at least 75%. It does cheat in a number of other ways.

5Torchlight_Crimson9y

That's why I said "AI that could give the human a challenge" not "AI that would demolish a human". Better yet, have the game difficulty setting actually control the intelligence of the AI, rather than how much the AI cheats.

-5Lumifer9y

6richard_reitz9y

And yet, humans currently have the edge in Brood War. Humans are probably doomed once StarCraft AIs get AlphaGo-level decision-making, but flawless micro—even on top of flawless* macro—won't help you if you only have zealots when your opponent does a muta switch. (Zealots can only attack ground and mutalisks fly, so zealots can't attack mutalisks; mutalisks are also faster than zealots.) *By flawless, I mean macro doesn't falter because of micro elsewhere; often, even at the highest levels, players won't build new units because they're too busy controlling a big engagement or heavily multitasking (dropping at one point, defending a poke elsewhere, etc). If you look at it broadly, making the correct units is part of macro, but that's not what I'm talking about when I say flawless macro.

2skeptical_lurker9y

Zealots/muta/dragoons/Hydralisks is just a standard rock/paper/scissors game theory thing, and it shouldn't be too hard to calculate an approximate nash equlibrium. The problem is that there is micro, macro, game theory, imperfect information, and an AI has to tie all these different aspects together (as well as perhaps some perceptual chunking to reduce the complexity) so its a real challange for combining different cognitive modules. This is too close to AGI for comfort IMO.

7Kaj_Sotala9y

Pretty sure it's still comfortably narrow AI. People used to think that chess required AGI-levels of intelligence, too.

-2Lumifer9y

Nobody said that flawless micro is sufficient and figuring out the rock/paper/scissors dynamic is not hard. Plus, given that it has enough "attention" for everything, an AI is likely to keep a dancing scout or two around the enemy base and see those mutalisks early enough.

5skeptical_lurker9y

The problem is that most RTS games stand no chance against me or any other half-descent player, unless they are cheating. And when they cheat, the game is very much brute force vs strategy. I've been playing "Ultimate general: Gettysburg", which was touted as having put a lot off effort into AI, and which paid off - when I play it on the highest difficulty settings, I can still win convincingly, but it does feel like I am playing an incompitant human, rather than an artificial stupidity. Its far more enjoyable to play.

5Lumifer9y

Sure. Consider that the game has to run on your sucky home computer (or, forbid, a console), most likely without a GPU. The strategy/tactics/behaviour code has to share the CPU cycles with a large variety of things including the uninteresting but vital functions like pathfinding and it has to make its decisions within the tick time which is a fraction of second. AND many players prefer the AI to be a pushover, anyway.

0skeptical_lurker9y

I think gaming machines generally do have GPUs... Of course, the GPU is also running the graphics, but the computer doesn't need to play well enough to beat world champions - I'm pretty sure that Alpha Go running on one CPU+GPU could play at a strong amateur level.

0Lumifer9y

Of course, but mass-market games like Starcraft are designed to perform decently on the run-of-the-mill machines with integrated graphics.

3Furcas9y

The micro capabilities of the AI could be limited so they're more or less equivalent to a human pro gamer's, forcing the AI to win via build choice and tactics.

5Lumifer9y

It's going to be a mess. Even if you, say, limit the AI's click-per-minute rate, it still has serious advantages. It knows how many fractions of a second can these units stay in the range of enemy artillery and still be able to pull back to recover. It knows whether those units will arrive in time to reinforce the defense or they'll be too late and should do something else instead. Build choice is not all that complicated and with tactics you run right into micro.

[-]CronoDAS9y140

Make the AI control a robot that looks at a physical screen and operates a physical mouse. Then it will be fair. ;)

-4Lumifer9y

The point of the exercise is NOT to devise a handicapping system which will produce a fair match.

2Furcas9y

Human-like uncertainty could be inserted into the AI's knowledge of those things, but yeah, as you say, it's going to be a mess. Probably best to pick another kind of game to beat humans at.

3skeptical_lurker9y

Or the game could be played on its slowest mode.

0DanArmak9y

RTS is special because it's realtime. An AI that's only 'good enough' in terms of strategy or tactics could still win by being far better at parallelizing and reaction speed. The bigger the game world, the more this is true. Human Starcraft players need to have a basic skill of taking hundreds of actions per minute before they can bring their superior strategy or tactics into play.

3compartmentalization9y

Something like this?

2Lumifer9y

Most games are real-time: FPSes, MMORGs, MOBAs, etc. Right.

0DanArmak9y

I just meant that if it wasn't realtime but turn-based, AIs would lose their advantage. And in all of these, AFAIK, when AI is better than humans, it's because it can do things humans simply can't: perfect aiming and movement (of the kind that's considered cheating when humans use software aids to achieve it in FPSs), coordinating a team that can't see each other because sharing info digitally over the 'chat' channel is very efficient, remembering perfectly a very complex maze, etc. Micromanagement is another of these. That computers are much better at some things than humans isn't a surprise. It's very important, but it's hard to compare it directly to games like Go or chess.

0Lumifer9y

Humans also can't run massive searches on deep trees or hold a huge library of opening moves in their memory. AIs solve problems differently from humans. Software is much better at some things (from micromanagement to aimbotting to doing things quickly) and is much worse, so far, at other things. The interesting place is the edge -- where software and human capabiilties are currently of the same magnitude. That's why aimbots are boring and a machine playing Go is oh so cool.

[-]moridinamael9y40

Is Alphabet stock a good proxy for owning a piece of DeepMind? Alphabet hasn't gained much at all since AlphaGo started winning. Maybe a few percent, but within the normal fluctuations. Of course this might be because all the smart money knew AlphaGo was going to win.

9gwern9y

If there was any movement in Alphabet, it should've been in January when the news came out. Markets don't move on anticipated events but unexpected events, and judging from the various betting markets an Alphago victory was not that surprising; the victory also didn't mean much because the widely held opinion was that Alphago can be expected to improve steadily over time and so even if Lee Sedol won, he would lose in the coming months (I believe Sedol said something like that before the games started, and Ke Jie has also revised his earlier comments and is now saying that he would lose to Alphago in a few months too), in which case the meaning of the match is reduced to a slight shift in the improvement rate - along the lines of 'Alphago didn't improve quite as fast as Deepmind expected'. Which is not something which is meaningful to Google's bottom line. (The real point of the match was to prove a point to the muggles and AI-deniers and get good publicity, of course.)

2Petter9y

It is not a good proxy. Deepmind is a small team and there are many more teams within Alphabet doing machine learning. Remember that the market cap of Goog is $500 billion. (Although if one wants to invest in AI in general I think it is a cheap stock)

[-]skeptical_lurker9y40

The resources include dirty socks and plates, and the game is called "tidy skeptical_lurker's house, because he can't be bothered"

-2Lumifer9y

Just outsource it to Pentagon.

[-]SquirrelInHell9y40

Why isn't it obvious?

I know what I'd do.

Run the algorithm on the Bitcoin market, and then on the stock market.

6[anonymous]9y

That's pretty darn far from perfect information.

2dxu9y

Even so, I highly doubt the best human traders are anywhere close to optimal. It'd be interesting to see how much better a machine-learning approach would fare.

6Vaniver9y

Many of the successful trading firms are powered by ML, of both the price-watching and NLP news-watching variety. I don't think Deepmind has a comparative advantage against them, but I do expect that people at those firms are trying out deep learning approaches.

0Lumifer9y

Here you go

0Houshalter9y

As if there aren't tons of other people using neural nets on the stock market.

-2Lumifer9y

Yeah, that.

6dxu9y

http://lesswrong.com/lw/k9/the_logical_fallacy_of_generalization_from/

-8Lumifer9y

[-]HungryHobo9y40

They've successfully trained related AI's to play retro games, I believe including some with non-perfect information.

links to code etc in the youtube video description.

https://www.youtube.com/watch?v=V1eYniJ0Rnk

2Douglas_Knight9y

The video games are far more interesting than just violating perfect information: the AI has to figure out the rules of the game. (Actually, they probably don't violate perfect information, which refers to the two players having access to different information and only makes sense when you think of both players as optimizing agents.)

[-]CronoDAS9y30

Computers can play one-on-one Limit Hold 'em pretty close to "perfectly"; a very good approximation to the Nash equilibrium strategy has been computed, and computers can follow it. The standard tournament game of no-limit 8-player Hold 'Em is a lot more computationally intensive to solve, though, and I don't think computers are especially good at it.

[-]skeptical_lurker9y20

What about chess? See if a DNN based AI beats a conventional chess AI running on the same processor power. Many people are interested in chess, and if it could push forwards chess theory, then that would be very interesting.

0TRIZ-Ingenieur9y

Why not check out the AGI capabilities of Alphago... It might be possible to train chess without architectural modifications. Each chessboard square could be modelled by a 2x2 three-state Go field storing information about chess figure type. How good can Alphago get? How much of its Go playing abilities will it loose?

2gjm9y

This isn't at all the same thing, but it might amuse you: Gess the game.

0gwern9y

http://arxiv.org/abs/1509.01549 is relevant.

[-]CronoDAS9y20

Contract Bridge is one of the big human strategy games - how good are AIs at that?

3Douglas_Knight9y

That isn't a formally specified game. For example, it is illegal to make up complicated ("synthetic") bidding systems.

[-]DanielVarga9y10

Here is something I'd like to see: You give the machine the formally specified ruleset of a game (go, chess, etc), wait while the reinforcement learning does its job, and out comes a world-class computer player.

[-]NancyLebovitz9y10

How about Risk?

1boatner9y

Oooh. If we're going to look at boardgames, the best one ever designed (IMO, of course) is : 1817

0[anonymous]9y

This I would love to see.

[-]WalterL9y10

Collectible card games is interesting to me. You get the imperfect information of poker, as well as a deckbuilding component that it seems like the AI should be good at (build a bunch of decks, play itself a few million times).

[-]The_Messenger9y00

Personally, I''m waiting for an AI that can outperform experts in Fantasy Football.

No small feat either. The sheer amount of data that needs to be processed is tremendous (think about all of the physical possibilities across all the football teams/games). Humans have the benefit of heuristics. Chess and Go are one thing. But being able to draft a winning fantasy team is a lot harder than it seems.

0ChristianKl9y

I would be very of that prediction. Do you know how the best AI's perform at Fantasy Football?

0The_Messenger9y

To my knowledge there hasn't been much involvement in AI Fantasy Football. However, I would imagine that existing AIs perform fairly poorly. They could probably beat your average player, but not a seasoned football fan who religiously follows the entire league. I could be wrong though. If there are any examples of AIs performing well at Fantasy Football I'd love to see them!

0ChristianKl9y

People who create AI for Fantasy Football that perform fairly poorly are quite free at being open about their AI. On the other hand why should someone who has a well-performing AI at Fantasy Football be public about the fact that he has the AI? That person could lose a lot of money by being open.

[-]Lumifer9y00

Random link: AI in Minecraft.

[-]Coacher9y00

I'll be scared, when they do Counter Strike.

Moderation Log