Pointer Network (Ptr-Net)

A Pointer Network (Ptr-Net) is a Sequence-to-Sequence Neural Network With Attention that produces output sequences using a content-based attention mechanism over input sequences.

• AKA: Pointer-Generator Network.
• Context:
  • It was first introduced by Vinyals et al. (2015).
• Example(s):
  • that described in Vinyals et al. (2015).
  • …
• Counter-Example(s):
  • a Pointer-Generator Network (See et al., 2017).
  • a DeepFix Seq2Seq Neural Network (Gupta et al., 2017),
  • a Bidirectional Associative Memory (BAM) Network,
  • a LSTM-based Encoder/Decoder Neural Network,
  • a Transformer Neural Network.
• See: Sequence-to-Sequence Model, Neural Machine Translation, Encoder-Decoder Neural Network, Artificial Neural Network, Natural Language Processing Task, Language Model, Summarization NLP Task.

---

---

References

2017

• (See et al., 2017) ⇒ Abigail See, Peter J. Liu, and Christopher D. Manning. (2017). “Get To The Point: Summarization with Pointer-Generator Networks.” In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). DOI:10.18653/v1/P17-1099.
  • QUOTE: The pointer network (Vinyals et al., 2015) is a sequence-to-sequence model that uses the soft attention distribution of Bahdanau et al. (2015) to produce an output sequence consisting of elements from the input sequence. The pointer network has been used to create hybrid approaches for NMT (Gulcehre et al., 2016), language modeling (Merity et al., 2016), and summarization (Gu et al., 2016; Gulcehre et al., 2016; Miao and Blunsom, 2016; Nallapati et al., 2016; Zeng et al., 2016).

2015

• (Vinyals et al., 2015) ⇒ Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly. (2015). "Pointer Networks". In: Proceedings of the Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems (NIPS 2015).
  • QUOTE: The proposed approach is depicted in Figure 1.

Figure 1:(a) Sequence-to-Sequence - An RNN (blue) processes the input sequence to create a code vector that is used to generate the output sequence (purple) using the probability chain rule and another RNN. The output dimensionality is fixed by the dimensionality of the problem and it is the same during training and inference in Sutskever et al.(2014). (b) Ptr-Net - An encoding RNN converts the input sequence to a code (blue) that is fed to the generating network (purple). At each step, the generating network produces a vector that modulates a content-based attention mechanism over inputs (Bahdanau et al., 2015, Graves et al., 2014). The output of the attention mechanism is a softmax distribution with dictionary size equal to the length of the input.

---