[Link]"Neural Turing Machines"

Prankster

Discusses the technical aspects of one of Googles AI projects. According to a pcworld the system "apes human memory and programming skills" (this article seems pretty solid, also contains link to the paper).

The abstract:

We extend the capabilities of neural networks by coupling them to external memory resources, which they can interact with by attentional processes. The combined system is analogous to a Turing Machine or Von Neumann architecture but is differentiable end-to-end, allowing it to be efficiently trained with gradient descent. Preliminary results demonstrate that Neural Turing Machines can infer simple algorithms such as copying, sorting, and associative recall from input and output examples.

(First post here, feedback on the appropriateness of the post appreciated)

The paper.

The abstract:

We extend the capabilities of neural networks by coupling them to external memory resources, which they can interact with by attentional processes. The combined system is analogous to a Turing Machine or Von Neumann architecture but is differentiable end-to-end, allowing it to be efficiently trained with gradient descent. Preliminary results demonstrate that Neural Turing Machines can infer simple algorithms such as copying, sorting, and associative recall from input and output examples.

(First post here, feedback on the appropriateness of the post appreciated)

Some good comments on related work in https://www.reddit.com/r/MachineLearning/comments/2kth6d/googles_secretive_deepmind_startup_unveils_a/

(After reading the paper, I still don't really get how you wire some RAM into a neural network. Maybe it makes more sense to others.)

It's not really RAM, but rather a tape. (like a doubly linked list) The LSTM controller can't specify any location in logarithmic space / time. They add multiple tape readers at one point though.

1IlyaShpitser12y

A neural network is just a function of some sort (inputs to outputs). So is a finite state machine (which is what a regular Turing machine uses). Put that way, nothing conceptually interesting is going on. As usual, the devil is in the details here.

4[anonymous]12y

I would naively assume similar to the way we interface our own, with some kind of mapping of a subset at a time of the memory to a sensory input to the network, which then just needs to remember itself how to look up the relevant subsets.

23

[Link]"Neural Turing Machines"

23

23

23

[Link]"Neural Turing Machines"

23

23