2018 DynamicNeuralTuringMachinewithC

From GM-RKB
Jump to navigation Jump to search

Subject Headings: Dynamic Neural Turing Machine, Neural Turing Machine.

Notes

Previous version, also referred as:

Cited By

Quotes

Abstract

We extend the neural Turing machine (NTM) model into a dynamic neural Turing machine (D-NTM) by introducing trainable address vectors. This addressing scheme maintains for each memory cell two separate vectors, content and address vectors. This allows the D-NTM to learn a wide variety of location-based addressing strategies, including both linear and nonlinear ones. We implement the D-NTM with both continuous and discrete read and write mechanisms. We investigate the mechanisms and effects of learning to read and write into a memory through experiments on Facebook bAbI tasks using both a feedforward and GRU controller. We provide extensive analysis of our model and compare different variations of neural Turing machines on this task. We show that our model outperforms long short-term memory and NTM variants. We provide further experimental results on the sequential MNIST, Stanford Natural Language Inference, associative recall, and copy tasks.

References

;

 AuthorvolumeDate ValuetitletypejournaltitleUrldoinoteyear
2018 DynamicNeuralTuringMachinewithCYoshua Bengio
Kyunghyun Cho
Caglar Gulcehre
Sarath Chandar
Dynamic Neural Turing Machine with Continuous and Discrete Addressing Schemes10.1162/neco_a_010602018