2017 DyNetTheDynamicNeuralNetworkToo
- (Neubig et al., 2017) ⇒ Graham Neubig, Chris Dyer, Yoav Goldberg, Austin Matthews, Waleed Ammar, Antonios Anastasopoulos, Miguel Ballesteros, David Chiang, Daniel Clothiaux, Trevor Cohn, Kevin Duh, Manaal Faruqui, Cynthia Gan, Dan Garrette, Yangfeng Ji, Lingpeng Kong, Adhiguna Kuncoro, Gaurav Kumar, Chaitanya Malaviya, Paul Michel, Yusuke Oda, Matthew Richardson, Naomi Saphra, Swabha Swayamdipta, and Pengcheng Yin. (2017). “DyNet: The Dynamic Neural Network Toolkit.” In: ePrint: arXiv:1701.03980.
Subject Headings: Natural Language Generation Task.
Notes
- Version(s) and Link(s):
- Repository
Cited By
- Google Scholar: ~ 152 Citations.
- Semantic Scholar: ~ 253 Citations.
- MS Academic: ~ 231 Citations.
Quotes
Abstract
We describe DyNet, a toolkit for implementing neural network models based on dynamic declaration of network structure. In the static declaration strategy that is used in toolkits like Theano, CNTK, and TensorFlow, the user first defines a computation graph (a symbolic representation of the computation), and then examples are fed into an engine that executes this computation and computes its derivatives. In DyNet's dynamic declaration strategy, computation graph construction is mostly transparent, being implicitly constructed by executing procedural code that computes the network outputs, and the user is free to use different network structures for each input. Dynamic declaration thus facilitates the implementation of more complicated network architectures, and DyNet is specifically designed to allow users to implement their models in a way that is idiomatic in their preferred programming language (C + + or Python). One challenge with dynamic declaration is that because the symbolic computation graph is defined anew for every training example, its construction must have low overhead. To achieve this, DyNet has an optimized C + + backend and lightweight graph representation. Experiments show that DyNet's speeds are faster than or comparable with static declaration toolkits, and significantly faster than Chainer, another dynamic declaration toolkit. DyNet is released open-source under the Apache 2.0 license and available at https://github.com/clab/dynet
References
BibTeX
@article{2017_DyNetTheDynamicNeuralNetworkToo,
author = {Graham Neubig and
Chris Dyer and
Yoav Goldberg and
Austin Matthews and
Waleed Ammar and
Antonios Anastasopoulos and
Miguel Ballesteros and
David Chiang and
Daniel Clothiaux and
Trevor Cohn and
Kevin Duh and
Manaal Faruqui and
Cynthia Gan and
Dan Garrette and
Yangfeng Ji and
Lingpeng Kong and
Adhiguna Kuncoro and
Gaurav Kumar and
Chaitanya Malaviya and
Paul Michel and
Yusuke Oda and
Matthew Richardson and
Naomi Saphra and
Swabha Swayamdipta and
Pengcheng Yin},
title = {DyNet: The Dynamic Neural Network Toolkit},
journal = {CoRR},
volume = {abs/1701.03980},
year = {2017},
url = {http://arxiv.org/abs/1701.03980},
archivePrefix = {arXiv},
eprint = {1701.03980},
}