Sequence-to-Sequence Model Training Algorithm

A Sequence-to-Sequence Model Training Algorithm is a model training algorithm that can be implemented by a sequence-to-sequence model training system (to solve a sequence-to-sequence training task to produce a trained sequence-to-sequence model).

• Context:
  • …
• Example(s):
  • a Neural seq2seq Training Algorithm.
  • …
• Counter-Example(s):
  • a Sequence-to-Label Training Algorithm.
• See: Model Training Algorithm, BIO Labeling.

---

---

References

2017

• (Gehring et al., 2017) ⇒ Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N. Dauphin. (2017). “Convolutional Sequence to Sequence Learning.” In: International Conference on Machine Learning.
  • QUOTE: ... The prevalent approach to sequence to sequence learning maps an input sequence to a variable length output sequence via recurrent neural networks. We introduce an architecture based entirely on convolutional neural networks. ...

2016

• (Luong et al., 2016) ⇒ Minh-Thang Luong, Quoc V. Le, Ilya Sutskever, Oriol Vinyals, and Lukasz Kaiser. (2016). “Multi-task Sequence to Sequence Learning.” In: Proceedings of 4th International Conference on Learning Representations (ICLR-2016).
  • QUOTE: ... for dealing with variable-length inputs and outputs. seq2seq learning, at its core, uses recurrent neural networks to map variable-length input sequences to variable-length output sequences. While relatively new, the seq2seq approach has achieved state-of-the-art results in not only its original application – machine translation – (Luong et al., 2015b; Jean et al., 2015a; Luong et al., 2015a; Jean et al., 2015b; Luong & Manning, 2015), but also image caption generation (Vinyals et al., 2015b), and constituency parsing (Vinyals et al., 2015a).

---