There have been a couple of brief discussions of this in the Open Thread, but it seems likely to generate more so here's a place for it.
The original paper in Nature about AlphaGo.
Google Asia Pacific blog, where results will be posted. DeepMind's YouTube channel, where the games are being live-streamed.
Discussion on Hacker News after AlphaGo's win of the first game.
'Yeah, we could maybe have AlphaGo learn everything totally from scratch and reach a superhuman level of knowledge just by playing itself, not using any human games for training material. Of course, reinventing everything that humanity has figured out while playing Go for the last 2,500 years, that's going to take quite a bit of time. Like a few months or so.'
Actually, the AlphaGo algorithm, this is something we’re going to try in the next few months — we think we could get rid of the supervised learning starting point and just do it completely from self-play, literally starting from nothing. It’d take longer, because the trial and error when you’re playing randomly would take longer to train, maybe a few months. But we think it’s possible to ground it all the way to pure learning.
http://www.theverge.com/2016/3/10/11192774/demis-hassabis-interview-alphago-google-deepmind-ai
We accidentally had a meetup as the game was ending. For the first time in my life - got to walk in to a room and say; "Who's been watching the big game". It was great, and then about 10mins later the resignation happened. was pretty exciting!
When I started hearing about the latest wave of results from neural networks, I thought to myself that Eliezer was probably wrong to bet against them. Should MIRI rethink its approach to friendliness?
Compared to its competition in the AGI race, MIRI was always going to be disadvantaged by both lack of resources and the need to choose an AI design that can predictably be made Friendly as opposed to optimizing mainly for capability. For this reason, I was against MIRI (or rather the Singularity Institute as it was known back then) going into AI research at all, as opposed to pursuing some other way of pushing for a positive Singularity.
In any case, what other approaches to Friendliness would you like MIRI to consider? The only other approach that I'm aware of that's somewhat developed is Paul Christiano's current approach (see for example https://medium.com/ai-control/alba-an-explicit-proposal-for-aligned-ai-17a55f60bbcf), which I understand is meant to be largely agnostic about the underlying AI technology. Personally I'm pretty skeptical but then I may be overly skeptical about everything. What are your thoughts? I don't recall seeing you having commented on them much.
Are you aware of any other ideas that MIRI should be considering?
It seems to be a combination of all of these.
The trainers are responsible for getting M to do what the trainers want, and the user trusts the trainers to do what the user wants.
In that case, there would be severe principle-agent problems, given the disparity between power/intelligence of the trainer/AI systems and the users. If I was someone who couldn't directly control an AI using your scheme, I'd be very concerned about getting uneven trades or having my property expropriated outright by individual AIs or AI conspiracies, or just ignored and left behind in the race to capture the cosmic commons. I would be really tempted to try another AI design that does purport to have the AI serve my interests directly, even if that scheme is not as "safe".
If I imagine an employee who sucks at philosophy but thinks 100x faster than me, I don't feel like they are going to fail to understand how to defer to me on philosophical questions.
If an employee sucks at philosophy, how does he even recognize philosophical problems as problems that he needs to consult you for? Most people have little idea that they should feel confused and uncertain about things like epistemology, decision theory, and ethics. I suppose it might be...
EY was influenced by E.T. Jaynes, who was really against neural networks, in favor of bayesian networks. He thought NNs were unprincipled and not mathematically elegant, and bayes nets were. I see the same opinions in some of EY's writings, like the one you link. And the general attitude that "non-elegant = bad" is basically MIRI's mission statement.
I don't agree with this at all. I wrote a thing here about how NNs can be elegant, and derived from first principles. But more generally, AI should use whatever works. If that happens to be "scruffy" methods, then so be it.
But more generally, AI should use whatever works. If that happens to be "scruffy" methods, then so be it.
This seems like a bizarre statement if we care about knowable AI safety. Near as I can tell, you just called for the rapid creation of AGI that we can't prove non-genocidal.
I think that MIRI did a mistake than decided not be evolved in actual AI research, but only in AI safety research. In retrospect the nature of this mistake is clear: MIRI was not recognised inside AI community, and its safety recommendations are not connected with actual AI development paths.
It is like a person would decide not to study nuclear physics but only nuclear safety. It even may work until some point, as safety laws are similar in many systems. But he will not be the first who will learn about surprises in new technology.
I think that MIRI did a mistake than decided not be evolved in actual AI research [...] MIRI was not recognised inside AI community
Being involved in actual AI research would have helped with that only if MIRI had been able to do good AI research, and would have been a net win only if MIRI had been able to do good AI research at less cost to their AI safety research than the gain from greater recognition in the AI community (and whatever other benefits doing AI research might have brought).
I think you're probably correct that MIRI would be more effective if it did AI research, but it's not at all obvious.
They may be used to create complex but boring part of the real AI like image recognition. DeepMind is no where near to NN, it combines several architectures. So NNs are like ToolAIs inside large AI system: they do a lot of work but on low level.
I found this interesting: AlphaGo's internal statistics predicted victory with high confidence at about three hours into the game (Lee Sedol resigned at about three and a half hours):
...For me, the key moment came when I saw Hassabis passing his iPhone to other Google executives in our VIP room, some three hours into the game. From their smiles, you knew straight away that they were pretty sure they were winning – although the experts providing live public commentary on the match weren’t clear on the matter, and remained confused up to the end of the game just before Lee resigned.
Hassabis’s certainty came from Google’s technical team, who pore over AlphaGo’s evaluation of its position, information that isn’t publicly available. I’d been asking Silver how AlphaGo saw the game going, and he’d already whispered back: “It’s looking good”.
And I realised I had a lump in my throat. From that point on, it was crushing for me to watch Lee’s struggle.
Towards the end of the match, Michael Redmond, an American commentator who is the only westerner to reach the top rank of 9 dan pro, said the game was still “very close”. But Hassabis was frowning and shaking his head – he knew that AlphaGo was d
I watched the whole of both games played so far. In the first game, Redmond definitely thought that Lee Sedol was winning, and at a point close to the end, he said, "I don't think it's going to be close," and I am fairly confident he meant that Lee Sedol would win by a substantial margin. Likewise, he definitely showed real surprise when the resignation came: even at that point, he expected a human victory.
In the second game, he was more cautious and refused to commit himself, but still seemed to think there were points where Lee Sedol had the advantage. However, in this one he did end up admitting that AlphaGo was winning long before the end came.
In particular, I thought Redmond's handling of the top right corner was striking. He identified it as a potential attack for white several times before white's actual attack, and then afterwards thought that a move was 'big' that AlphaGo ignored; later, on calculation, he realized that it was only (if I recall correctly) a one point move.
It looked to me like an example of the human bias towards the corners and walls, combined with his surprise at some of AlphaGo's moves that made significant changes in the center.
Myungwan Kim seemed to be quicker to reach the right conclusion - IIRC, by the time that the fighting in the lower right corner ended, he was pretty sure of AlphaGo winning, to the extent of guessing that a move that lost AlphaGo around 1.5 points down there was because AG would win anyway.
https://gogameguru.com/alphago-defeats-lee-sedol-game-1/ has some (non-video) comments on the game, and promises more detailed commentary later.
9p Myungwan Kim's commentary (I much preferred this over the official commentary; he's also commenting tomorrow, so recommend following his stream then, though he might start an hour delayed like he did today).
Fun comment from him: "[AlphaGo] play like a god, a god of Go".
He did...but...like, you can't really trust that. He'd have said that (or similar) no matter what. It isn't game commentary, its signalling.
There's a sort of humblebrag attitude that permeates all of Go. Every press conference is the same. Your opponent was very strong, you were fortunate, you have deep respect for your opponent and thank him for the opportunity.
In the game commentary you get the real dish. They stop using names and use "White/Black" to talk about either side. There things are much more honest.
Ignoring psychology and just looking at the results:
Delta-function prior at p=1/2 -- i.e., completely ignore the first two games and assume they're equally matched. Lee Sedol wins 12.5% of the time.
Laplace's law of succession gives a point estimate of 1/4 for Lee Sedol's win probability now. That means Lee Sedol wins about 1.6% of the time. [EDITED to add:] Er, no, actually if you're using the rule of succession you should apply it afresh after each game, and then the result is the same as with a uniform prior on [0,1] as in #3 below. Thanks to Unnamed for catching my error.
Uniform-on-[0,1] prior for Lee Sedol's win probability means posterior density is f(p)=3(1-p)^2, which means he wins the match exactly 5% of the time.
I think most people expected it to be pretty close. Take a prior density f(p)=4p(1-p), which favours middling probabilities but not too outrageously; then he wins the match about 7.1% of the time.
So ~5% seems reasonable without bringing psychological factors into it.
For me the most interesting part of this match was the part where one of the DeepMind team confirmed that because AlphaGo optimizes for probability of winning rather than expected score difference, games where it has the advantage will look close. It changes how you should interpret the apparent closeness of a game
Qiaochu Yuan, or him quoting someone.
Several things I thought were interesting:
The commentator (on the Deepmind channel) calling out several of AlphaGo's moves as conservative. Essentially, it would play an additional stone to settle or augment some group that he wouldn't necessarily have played around. What I'm curious about is how much this reflects an attempt by AlphaGo to conserve computational resources. "I think move A is a 12 point swing, and move B is a 10 point swing, but move B narrows the search tree for future moves in a way that I think will net me at least 2 more points." (It wouldn't be verbalized like that, since it's not thinking verbally, but you can get this effect naturally from the tree search and position evaluator.)
Both players took a long time to play "obvious" moves. (Typically, by this I mean something like a response to a forced move.) 이 sometimes didn't--there were a handful of moves he played immediately after AlphaGo's move--but I was still surprised by the amount of thought that went into some of the moves. This may be typical for tournament play--I haven't watched any live before this.
AlphaGo's willingness to play aggressively and get involved in big fights with 이, and then not lose. I'm not sure that all the fights developed to AlphaGo's advantage, but evidently enough of them did by enough.
I somewhat regret 이 not playing the game out to the end; it would have been nice to know the actual score. (I'm sure estimates will be available soon, if not already.)
What I'm curious about is how much this reflects an attempt by AlphaGo to conserve computational resources.
If I understand correctly, at least according to the Nature paper, it doesn't explicitly optimize for this. Game-playing software is often perceived as playing "conservatively", this is a general property of minimax search, and in the limit the Nash equilibrium consists of maximally conservative strategies.
but I was still surprised by the amount of thought that went into some of the moves.
Maybe these obvious moves weren't so obvious at that level.
I'm quite interested in how many of the methods employed in this AI can be applied to more general strategic problems.
From talking to a friend who did quite a bit of work in machine composition, he was of the opinion that tools for handling strategy tasks like go would also apply strongly to many design tasks like composing good music.
Go champion Lee Se-dol strikes back to beat Google's DeepMind AI for first time in forth game 3:1 http://www.theverge.com/2016/3/13/11184328/alphago-deepmind-go-match-4-result
Discussion on FiveThirtyEight about experts discussing consequences from the AlphaGo-Lee Sedol match.
It's worth noting that the match is under Chinese rules and not the more popular Japanese style rules (Korean rules are also Japanese style). That's because Chinese rules are easier for computers.
It would be interesting to have another match played on Japanese style rules.
I listened to the ending press conference. Interestingly, Demis Hassabis discusses AI ethics twice, saying that development will be largely open-sourced to ensure that AI "is for the many, not just the few." So, this gives the impression that Google AI ethics is more thinking along the lines of 'AI based economy renders many unemployed' rather than 'hard takeoff destroys humanity', or at least that is what they are publicly discussing at this time.
On a lighter note, one reporter asked IIRC "How many versions of alphago are there, and how lon...
It is also interesting to know the size of Alphago.
Wiki says: "The distributed version in October 2015 was using 1,202 CPUs and 176 GPUs (and was developed by teem of 100 scientists). Assuming that it was best GPU on the market in 2015, with power around 1 teraflop, total power of AlphaGO was around 200 teraplop or more. (I would give it 100 Teraflop - 1 Petaflop with 75 probability estimate). I also think that the size of the program is around terabytes, but only conclude it from the number of computers in use.
This could provide us with minimal size...
Does anyone know the current odds being given of Lee Sedol winning any of the three remaining games against AlphaGo? I'm curious if at this point is AlphaGo likely possible to beat by a human player better than Sedol (assuming there are any) or if we're looking at an AI player that is better than a human can be.
Lee Sedol isn't at the top of Go ratings. How would Ke Jie fare against AlphaGo? A match against the best human player would be a better test of AlphaGo capabilities.
I thought my excerpt answered that, but maybe that was illusion of transparency speaking. In particular, this paragraph:
To rephrase: the main trend is history has been to automate everything that can be automated, both to reduce costs and because machines can do things better than humans do. This isn't going to stop: I've already seen articles calling for both company middle managers, as well as government bureaucrats, to be replaced with AIs. If you have any kind of a business, you could potentially make it run better by putting a sufficiently sophisticated AI in charge - because it can think faster and smarter, deal with more information at once, and not have the issue of self-interest leading to office politics leading to many employees acting suboptimally from the company's point of view, that you'd get if you had a thousand human employees rather than a single AI.
This trend has been going on throughout history, doesn't show any signs of stopping, and inherently involves giving the AI systems whatever agency they need in order to run the company better.
And if your competitors are having AIs run their company and you don't, you're likely to be outcompeted, so you'll want to make sure your AIs are smarter and more capable of acting autonomously than the competitors. These pressures aren't just going to vanish at the point when AIs start approaching human capability.
The same considerations also apply to other domains than business - like governance - but the business and military domains are the most likely to have intense arms race dynamics going on.
Yes, illusion of transparency at work here. That paragraph has always been so clearly wrong to me that I wrote it off as the usual academic prose fluff, and didn't realize it was in fact the argument being made. Here is the issue I take with that:
You can find instances where industry is clamoring to use AI to reduce costs / improve productivity. For example, Uber and self-driving cars. However in these cases there are a combination of two factors at work: (1) the examples are necessarily specialized narrow AI, not general decision making; and/or (2) the co... (read more)