DAG-LSTM Neural Network
A DAG-LSTM Neural Network is a Bidirectional Recurrent Neural Network that consists of DAG-structured LSTM memory blocks.
- Example(s):
- Counter-Example(s):
- See: Recurrent Neural Network, Recursive Neural Network, Support Vector Machine, Directed Acyclic Graph, Long Short-Term Memory (LSTM) Network.
References
2016
- (Zhu et al.,2016) ⇒ Xiaodan Zhu, Parinaz Sobhani, and Hongyu Guo (2016). "Dag-structured long short-term memory for semantic compositionality". In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (pp. 917-926).
- QUOTE: Figure 1 depicts an example of DAG-structured LSTM (referred to as DAG-LSTM in the remainder of the paper) in modeling a sentence.
The proposed DAG-LSTM networks consist of four types of nodes, denoted in Figure 1 with different colors. The blue nodes (0, 1, 2, 6, and 7) correspond to normal chain-structured LSTM memory blocks. The yellow nodes (5 and 8) model noncompositional knowledge. The purple nodes (3 and 4), which we call fork blocks or fork nodes in this paper, are the modified versions of regular LSTM nodes, summarizing history for different types of outgoing blocks. The merging memory block is depicted in red (node 9), aiming at infusing information from multiple histories and deciding which sources will be considered more. Each category of these four types of memory blocks share its own parameters or weight matrices; e.g., the two yellow blocks share the same parameters.
Figure 1: An example of DAG-LSTM in modeling a sentence. Nodes with different colors contain different types of LSTM memory blocks.
- QUOTE: Figure 1 depicts an example of DAG-structured LSTM (referred to as DAG-LSTM in the remainder of the paper) in modeling a sentence.
