[Link]"Neural Turing Machines"

Prankster

Discusses the technical aspects of one of Googles AI projects. According to a pcworld the system "apes human memory and programming skills" (this article seems pretty solid, also contains link to the paper).

The abstract:

We extend the capabilities of neural networks by coupling them to external memory resources, which they can interact with by attentional processes. The combined system is analogous to a Turing Machine or Von Neumann architecture but is differentiable end-to-end, allowing it to be efficiently trained with gradient descent. Preliminary results demonstrate that Neural Turing Machines can infer simple algorithms such as copying, sorting, and associative recall from input and output examples.

(First post here, feedback on the appropriateness of the post appreciated)

The paper.

The abstract:

We extend the capabilities of neural networks by coupling them to external memory resources, which they can interact with by attentional processes. The combined system is analogous to a Turing Machine or Von Neumann architecture but is differentiable end-to-end, allowing it to be efficiently trained with gradient descent. Preliminary results demonstrate that Neural Turing Machines can infer simple algorithms such as copying, sorting, and associative recall from input and output examples.

(First post here, feedback on the appropriateness of the post appreciated)

If you want to put it that way, nothing conceptually interesting is going on in any neural network paper - we already know they're universal.

My problem is the details: I visualize neural networks as like pachinko machines or Galton's quincunx - you drop a bunch of bits (many balls) into the top (bottom) layer of neurons (pins) and they cascade down to the bottom based on the activation functions (spacing of pins & how many balls hit a pin simultaneously), and at the bottom (top) is emitted a final smaller output (1 ball somewhere). I don't get the details of what it means to add a memory to this many-to-one function.

I think another way to look at neural networks is they are nested non-linear regression models.

I am probably in the minority here, but I don't think the stuff in the OP is that interesting.

2V_V12y

The same way you add memory to many-to-one boolean functions: feedback loops.

6DanB12y

Recurrent Neural Networks feed their outputs back into their inputs, and run continuously. Instead of visualizing a river flowing directly downstream, visualize a river with a little cove off to one side, where vortices swirl around.

23

[Link]"Neural Turing Machines"

23

23

23

[Link]"Neural Turing Machines"

23

23