Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Comment author: ChristianKl 26 June 2017 01:07:34PM 0 points [-]

Are you sure that Google's 2016 net uses graphic cards? I would think that they use their Tensor Flow ASICs. The switch from graphic cards to ASICs is part of what allowed them huge performance improvements in a short time frame. I don't think that they will continue to improve much better than Moore's law.

Comment author: turchin 26 June 2017 01:22:59PM 1 point [-]

"We trained our models using TensorFlow (Abadi et al., 2016) on clusters containing 16-32 Tesla K40 GPUs" https://arxiv.org/pdf/1701.06538.pdf

So they did it before they implement Tensorflow hardware or didn't use it.

Current price of such Tesla cluster is around 50-100 K USD

Comment author: turchin 26 June 2017 10:40:07AM 2 points [-]

Yesterday I spent some time looking at the recent changes in the size and effectiveness of neural nets.

The size (number of parameters, or connections, roughly equal to synapses) of the cat recogniser by Google in 2012 was 1 billion.

Later private work was done mostly on graphic cards and the size of parameters was limited by the size of the memory of graphic cards, which recently up to 12 GB. For example, Karpathy's RNN has only 3 million parameters but was able to generate grammatically correct text. http://karpathy.github.io/2015/05/21/rnn-effectiveness/

However, latest work by Google created in 2016 a neural net with the size of 130 billion parameters, and they now use it in Google translate. They showed that quality is growing with the size of the net, but some diminishing returns are observed. https://arxiv.org/pdf/1701.06538.pdf So the number of parameters in the best neural nets by Google grew 100 times for 5 years, and they are planning trillion parameters net soon.

The human brain has around 100 trillion synapses. If the speed of growth of best neural nets continues, 100 trillion net is 5 years from now, or somewhere in 2022. (By saying "best nets" I exclude some useless very large simulations which were already done.)

However, there is a problem of educating such big nets, as difficulty is growing as a square of the number of parameters. And there is a limitation of the memory graphic cards. However, OpenAI found a solution which is easily scalable by changing the way the net is educated. It is not the backpropagation, but gradient descent in very large parameters space. https://blog.openai.com/evolution-strategies/

If we look on the performance side of neural nets, it is doubling every year, and in 2016-2017 start to demonstrate superhuman performance on many recognition tasks. However, there are always some other tasks, where humans are better, and that is why it is not easy to say how far is the human level performance. https://srconstantin.wordpress.com/2017/01/28/performance-trends-in-ai/

Hence infrahuman AI is possible around 2022. Saying “infrahuman" I mean that it will do the most thing that humans are able to do, but it still be not genius, not conscious, not Einstein, etc, but probably good robot brain. From this point, it could start help researchers to make research, which could be the next discontinuity. One of the main features of such AI will be that it will be able to understand most of the human language.

Comment author: dogiv 23 June 2017 02:29:17PM 0 points [-]

I like the first idea. But can we really guarantee that after changing its source code to give itself maximum utility, it will stop all other actions? If it has access to its own source code, what ensures that its utility is "maximum" when it can change the limit arbitrarily? And if all possible actions have the same expected utility, an optimizer could output any solution--"no action" would be the trivial one but it's not the only one.

An AI that has achieved all of its goals might still be dangerous, since it would presumably lose all high-level executive function (its optimization behavior) but have no incentive to turn off any sub-programs that are still running.

Both proposals have the possible failure mode that the AI will discover or guess that this mechanism exists, and then it will only care about making sure it gets activated--which might mean doing bad enough things that humans are forced to open the box and shut it down.

Comment author: turchin 23 June 2017 03:24:37PM 0 points [-]

The idea is a not intended to be used as a primary way of the AI control but as the last form of AI turn off option. I describe it in the lengthy text, where all possible ways of AI boxing are explored, which I am currently writing under the name "Catching treacherous turn: confinement and circuit breaker system to prevent AI revolt, self-improving and escape".

It also will work only if the reward function is presented not as plain text in the source code, but as a separate black box (created using cryptography or physical isolation). The stop code is, in fact, some solution of complex cryptography used in this cryptographic reward function.

I agree that running subagents may be a problem. We still don't have a theory of AI halting. It probably better to use such super reward before many subagents were created.

The last your objection is more serious as it shows that such mechanism could turn safe AI into dangerous "addict".

Comment author: ChristianKl 22 June 2017 04:09:29PM 0 points [-]

Bruce Bueno de Mesquita seems to be of the opinion that even 20 years ago computer models outperformed humans once the modeling is finished but modeling seems crucial.

In his 2008 book, he advocates that the best move for Israel/Palestine would be to make a treaty that requires the two countries to share tourism revenue which each other. That's not the kind of move that an AI like DeepMind would produce without a human coming up with the move beforehand.

Comment author: turchin 22 June 2017 04:31:52PM 0 points [-]

So it looks like that if model creation job could be at least partly automated, it would give a strategic advantage in business, politics and military planning.

Comment author: ChristianKl 22 June 2017 04:01:54PM *  0 points [-]

The fact that H lost?

That argument feels circular in nature. You believe that Trump won because of a powerful computer model, simply because Trump won and he was supported by a computer model.

One the other hand, you have a tech billionaire who's gathering top programmers to fight. On the other hand, you have a company that has to be told by the daughter of that tech-billionaire what software they should use.

Who's press person said they worked for the leave-campaign and who's CEO is currently on the record for never having worked for the leave-campaign, neither paid nor unpaid.

From a NYTimes article:

But Cambridge’s psychographic models proved unreliable in the Cruz presidential campaign, according to Rick Tyler, a former Cruz aide, and another consultant involved in the campaign. In one early test, more than half the Oklahoma voters whom Cambridge had identified as Cruz supporters actually favored other candidates. The campaign stopped using Cambridge’s data entirely after the South Carolina primary.

There's a lot of irony in the fact that Cambridge Analytica seems to be better at telling spin about its amazing abilities of political manipulation in an untargeted way, than they are actually at helping political campaign.

I just saw on scout.ai's about page that they see themselves as being in the science fiction business. Maybe I should be less hard on them.

Comment author: turchin 22 June 2017 04:18:01PM 0 points [-]

I want to underline again that the fact that I discuss a possibility doesn't mean that I believe in it. The winning is evidence of intelligent power but given prior about its previous failures, it may be not strong evidence.

Comment author: ChristianKl 22 June 2017 03:18:26PM 0 points [-]

Geopolitical forecasting requires you to build a good model of the conflict that you care about. Once you do have a model you can feed the model into a computer like the Bruce Bueno de Mesquita does and the computer might do better at calculating the optimal move. I don't think that current existing AI system are up to the task of modeling a complicated geopolitical event.

Comment author: turchin 22 June 2017 03:40:07PM 0 points [-]

I also don't think that it is now possible to model full geopolitics, but if some smaller but effective model of it will be created by humans, it may be used by AI.

Comment author: ChristianKl 22 June 2017 02:59:03PM 0 points [-]

On the Go side, the program with the superhuman performance is run by Eric Schmidt's company.

What makes you think that Eric Schmidt's people aren't the best in the other domain as well?

Comment author: turchin 22 June 2017 03:06:03PM 0 points [-]

The fact that H lost?

But in fact, I don't want to derail the discussion about AI's possible decisive advantage in the future in the conspiracy looking discussion about past elections, which I mentioned as a possible example of strategic games, but not as the fact proving that such AI actually exists.

Comment author: ChristianKl 22 June 2017 02:50:11PM 0 points [-]

There was some speculation about it: https://scout.ai/story/the-rise-of-the-weaponized-ai-propaganda-machine

When I read "Cambridge Analytica isn’t the only company that could pull this off -- but it is the most powerful right now." I immediately think "citation needed".

Eric Schmidt funded multiple companies to provide technology to get Hillary elected.

Comment author: turchin 22 June 2017 02:52:31PM 0 points [-]

There are many programs which play Go, but only one currently with superhuman performance.

Comment author: dogiv 22 June 2017 01:46:09PM 1 point [-]

AI is good at well-defined strategy games, but (so far) bad at understanding and integrating real-world constraints. I suspect that there are already significant efforts to use narrow AI to help humans with strategic planning, but that these remain secret. For an AGI to defeat that sort of human-computer combination would require considerably superhuman capabilities, which means without an intelligence explosion it would take a great deal of time and resources.

Comment author: turchin 22 June 2017 02:29:14PM 0 points [-]

If AI will be able to use humans as outsourced form of intuition like in Mechanical Turk, it may be able to play such games with much less own intelligence.

Such game may resemble Trump's election campaign, where cyberweapons, fake news and internet memes was used by some algorithm. There was some speculation about it: https://scout.ai/story/the-rise-of-the-weaponized-ai-propaganda-machine

We already see superhuman performance in war-simulating games, but nothing like it in AI self-improving.

Mildly superhuman capabilities may be reached without intelligence explosion by the low-level accumulation of hardware, training and knowledge.

Comment author: turchin 22 June 2017 02:15:32PM 2 points [-]

Wireheading bomb – a putative mechanism to stop dangerous AI.

If a reward function of an AI is presented openly in its source code, any attempt to self-improve by AI will result in its own immediate wireheading, as when it reaches its own source code, it will become able to modify it in order to get maximum reward. So we could create an AI architecture in the way that as soon as it gets access to its own source code, it stops, and use it as a way of reaching passive safety and self-limited self-improving capacity.

We also could do exactly opposite, and put a reward function into the remote impenetrable cryptographically protected box, so AI will not be able to wirehead itself in an unexpected moment. However, we could keep some solutions of the reward functions as a string of numbers, which are equal to maximum reward for this black box. If AI finds this string of numbers, it will reach its maximum utility and stop. The benefit of this stop switch is that AI will not be against it, as it would give it infinite reward. So it would actively cooperate in an attempt to stop it, if it will know that such stop-code exists.

View more: Next