2017 ConvolutionalSequencetoSequence

• (Gehring et al., 2017) ⇒ Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N. Dauphin. (2017). “Convolutional Sequence to Sequence Learning.” In: International Conference on Machine Learning.

Subject Headings: Convolutional Network-based Sequence-to-Sequence Model.

Notes

• There is an implementation in FAIR Fairseq Toolkit.

Cited By

• http://scholar.google.com/scholar?q=%222017%22+Convolutional+Sequence+to+Sequence+Learning

Quotes

Abstract

The prevalent approach to sequence to sequence learning maps an input sequence to a variable length output sequence via recurrent neural networks. We introduce an architecture based entirely on convolutional neural networks. Compared to recurrent models, computations over all elements can be fully parallelized during training and optimization is easier since the number of non-linearities is fixed and independent of the input length. Our use of gated linear units eases gradient propagation and we equip each decoder layer with a separate attention module. We outperform the accuracy of the deep LSTM setup of Wu et al. (2016) on both WMT'14 English-German and WMT'14 English-French translation at an order of magnitude faster speed, both on GPU and CPU.

References

;

	Author	volume	Date Value	title	type	journal	titleUrl	doi	note	year
2017 ConvolutionalSequencetoSequence	Yann N. Dauphin Jonas Gehring Michael Auli David Grangier Denis Yarats			Convolutional Sequence to Sequence Learning