Attention-based Encoder-Decoder Neural Network

A Attention-based Encoder-Decoder Neural Network is an Encoder-Decoder Neural Network that includes an Attention Mechanism.

• AKA: Attention-based Encoder-Decoder Neural Network.
• Context:
  • …
• Example(s):
  • a Attention Seq2Seq Neural Network.
  • a Attention-based Encoder-Decoder RNN (RNN model).
  • an Attention-based Encoder-Decoder Transformer Network.
  • …
• Counter-Example(s):
  • a Attention-based Encoder-Only Neural Network (encoder-only neural network).
  • a Attention-based Decoder-Only Neural Network (decoder-only neural network).
• See: Content-Based Attention Network, Neural NLP.

---

---

References

2016a

• (Schmaltz et al., 2016) ⇒ Allen Schmaltz, Yoon Kim, Alexander M Rush, and Stuart Shieber. (2016). “Sentence-Level Grammatical Error Identification As Sequence-to-Sequence Correction.” In: Proceedings of the 11th Workshop on Innovative Use of NLP for Building Educational Applications.
  • QUOTE: We demonstrate that an attention-based encoder-decoder model can be used for sentence-level grammatical error identification for the Automated Evaluation of Scientific Writing (AESW) Shared Task 2016. …

2016b

• (Xie et al., 2016) ⇒ Ziang Xie, Anand Avati, Naveen Arivazhagan, Dan Jurafsky, and Andrew Y. Ng. (2016). “Neural Language Correction with Character-Based Attention.” In: CoRR, abs/1603.09727.

2015

• (Bahdanau et al., 2015) ⇒ Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. (2015). “Neural Machine Translation by Jointly Learning to Align and Translate.” In: Proceedings of the Third International Conference on Learning Representations, (ICLR-2015).

---