Neural Network-based Text Classification Algorithm

A Neural Network-based Text Classification Algorithm is a supervised text classification algorithm that is a neural classification algorithm.

• Context:
  • It can be implemented by a Neural Text Classification System (to solve a neural text classification task).
  • It can range from (typically) being a Fully-Supervised Neural Text Classification Algorithm to being a Semi-Supervised Neural Text Classification Algorithm.
  • It can range from being a Neural Binary Text Classification Algorithm to being a Neural Multiclass Text Classification Algorithm.
  • It can range from being a Neural Unilabel Text Classification Algorithm to being a Neural Multilabel Text Classification Algorithm.
• Example(s):
  • a Neural Document Classification Algorithm, Neural Sentence Classification Algorithm, Neural Word-Mention Classification Algorithm, ...
  • a Neural Character-level Text Classification Algorithm, such as: (Zhang, Zhao and LeCun, 2015).
  • a Neural Word Sense Disambiguation Algorithm.
  • …
• Counter-Example(s):
  • an SVM-based Text Classification.
• See: Text Item Embedding, Word Embedding.

---

---

References

2017a

• (Goldberg, 2017) ⇒ Yoav Goldberg. (2017). “Neural Network Methods for Natural Language Processing.” In: Synthesis Lectures on Human Language Technologies, 10(1). doi:10.2200/S00762ED1V01Y201703HLT037
  • QUOTE: … we will now explore examples of concrete NLP classification tasks, and suitable features for them. While the promise of neural networks is to alleviate the need for manual feature engineering, we still need to take these sources of information into consideration when designing our models: we want to make sure that the network we design can make effective use of the available signals, either by giving it direct access to them by use of feature-engineering; by designing the network architecture to expose the needed signals; or by adding them as an additional loss signals when training the models. ...

2017b

• James Vanneman. (2017). “Recurrent Convolutional Neural Networks for Text Classification." Blog Post - Aug 2, 2017

2016

• https://medium.com/@surmenok/character-level-convolutional-networks-for-text-classification-d582c0c36ace

2015a

• (Lai et al., 2015) ⇒ Siwei Lai, Liheng Xu, Kang Liu, and Jun Zhao. (2015). “Recurrent Convolutional Neural Networks for Text Classification.” In: Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence. ISBN:0-262-51129-0
  • QUOTE: … In contrast to traditional methods, we introduce a recurrent convolutional neural network for text classification without human-designed features. In our model, we apply a recurrent structure to capture contextual information as far as possible when learning word representations, which may introduce considerably less noise compared to traditional window-based neural networks. We also employ a max-pooling layer that automatically judges which words play key roles in text classification to capture the key components in texts. ...

2015b

• (Zhang, Zhao and LeCun, 2015) ⇒ Xiang Zhang, Junbo Zhao, and Yann LeCun. (2015). “Character-level Convolutional Networks for Text Classification.” In: Advances in Neural Information Processing Systems, pp. 649-657.
  • ABSTRACT: This article offers an empirical exploration on the use of character-level convolutional networks (ConvNets) for text classification. We constructed several large-scale datasets to show that character-level convolutional networks could achieve state-of-the-art or competitive results. Comparisons are offered against traditional models such as bag of words, n-grams and their TFIDF variants, and deep learning models such as word-based ConvNets and recurrent neural networks.

2014

• (Johnson & Zhang, 2014) ⇒ Rie Johnson, and Tong Zhang. (2014). “Effective Use of Word Order for Text Categorization with Convolutional Neural Networks.” arXiv preprint arXiv:1412.1058
  • ABSTRACT: Convolutional neural network (CNN) is a neural network that can make use of the internal structure of data such as the 2D structure of image data. This paper studies CNN on text categorization to exploit the 1D structure (namely, word order) of text data for accurate prediction. Instead of using low-dimensional word vectors as input as is often done, we directly apply CNN to high-dimensional text data, which leads to directly learning embedding of small text regions for use in classification. In addition to a straightforward adaptation of CNN from image to text, a simple but new variation which employs bag-of-word conversion in the convolution layer is proposed. An extension to combine multiple convolution layers is also explored for higher accuracy. The experiments demonstrate the effectiveness of our approach in comparison with state-of-the-art methods.

---