Neural Sequence Model
Jump to navigation
Jump to search
A Neural Sequence Model is a neural network model that is a sequence data model (designed to process sequence data).
- Context:
- It can (typically) be implemented using Neural Sequence Architecture.
- ...
- It can include Encoder-Decoder Sequence Models.
- It can be trained using a Neural Sequence Model Training System (that may implement Backpropagation Through Time (BPTT)).
- …
- Example(s):
- Neural Text-to-Text Model (text-to-text models), such as:
- A seq2seq Model which utilizes an encoder to process the input sequence and a decoder to generate the output sequence.
- A Recurrent Neural Network (RNN), Long Short-Term Memory (LSTM) Model, or Gated Recurrent Unit (GRU) Model
- An Attention-based Neural Sequence Model, which employs an attention mechanismto weight the significance of different parts of the input sequence differently.
- A Selective State Space Model (SSM), which combines traditional state space modeling with neural network components.
- …
- Neural Text-to-Text Model (text-to-text models), such as:
- Counter-Example(s):
- A Feedforward Neural Network used for image classification, which does not model sequence data.
- A Convolutional Neural Network (CNN) used for processing static images.
- See: Sequence-to-Sequence Learning Task, Neural Language Model, Time Series Analysis, Deep Learning in Natural Language Processing.
References
2023
- (Gu & Dao, 2023) ⇒ Albert Gu, and Tri Dao. (2023). “Mamba: Linear-Time Sequence Modeling with Selective State Spaces.” doi:10.48550/arXiv.2312.00752.