Neural Sequence Learning Task

A Neural Sequence Learning Task is a Sequence Learning Task that requires a pre-trained Deep Learning Neural Network to produce a Vectorized Word Representation for Natural Language Modeling.

• AKA: Deep Sequence Learning Task.
• Context:
  • It can be solved by a Neural Sequence Learning System (that implements a Neural Sequence Learning Algorithm).
• Example(s):
  • a Sequence-to-Sequence Learning Task.
  • …
• Counter-Example(s):
  • a Sequence Classification Task,
  • a Sequence Segmentation Task,
  • a Sequence-Member Tagging Task.
• See: Word Sense Disambiguation, Natural Language Processing Task, Sentiment Analysis, LSTM, Recurrent Neural Network, Associative Reinforcement Learning, Active Learning, Bidirectional Long Short-Term Memory.

---

---

References

2018

• (Liao et al., 2018) ⇒ Binbing Liao, Jingqing Zhang, Chao Wu, Douglas McIlwraith, Tong Chen, Shengwen Yang, Yike Guo, and Fei Wu. (2018). “Deep Sequence Learning with Auxiliary Information for Traffic Prediction.” In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ISBN:978-1-4503-5552-0 doi:10.1145/3219819.3219895
  • QUOTE: Therefore a hybrid model based on deep sequence learning with auxiliary information for traffic prediction is proposed. In this model, offline geographical and social attributes, spatial dependencies and online crowd queries are integrated. The contribution of this paper can be summarised as follows:
    • We release a large-scale traffic prediction dataset with offline and online auxiliary information including map crowd search queries, road intersections and geographical and social attributes.
    • We integrate the sequence to sequence deep neural networks with geographical and social attributes via a wide and deep manner.
    • To incorporate the spatial dependencies within local road network, we utilise the graph convolution neural network to embed the traffic speed of neighbouring road segments.
    • We quantify the potential influence and devise a query impact algorithm to calculate the impact that online crowd queries have on the road segments.
    • We propose a hybrid Seq2Seq model which incorporates the offline geographical and social attributes, spatial dependencies and online crowd queries with a deep fusion.

2017

• (Raganato et al., 2017) ⇒ Alessandro Raganato, Claudio Delli Bovi, and Roberto Navigli. (2017). “Neural Sequence Learning Models for Word Sense Disambiguation.” In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing.
  • QUOTE: In this paper we adopted a new perspective on supervised WSD, so far typically viewed as a classification problem at the word level, and framed it using neural sequence learning. To this aim we defined, analyzed and compared experimentally different end-to-end models of varying complexities, including augmentations based on an attention mechanism and multitask learning.

    Unlike previous supervised approaches, where a dedicated model needs to be trained for every content word and each disambiguation target is treated in isolation, sequence learning approaches learn a single model in one pass from the training data, and then disambiguate jointly all target words within an input text. The resulting models consistently achieved state-of-the-art (or statistically equivalent) figures in all benchmarks for all-words WSD, both fine-grained and coarse-grained, effectively demonstrating that we can overcome the so far undisputed and long-standing word-expert assumption of supervised WSD, while retaining the accuracy of supervised word experts.

---