Hierarchical Recurrent Encoder-Decoder (HRED) Neural Network Training Algorithm: Difference between revisions
Jump to navigation
Jump to search
No edit summary |
m (Text replacement - ". " to ". ") |
||
Line 22: | Line 22: | ||
|[[File:2015_AHierarchicalRecurrentEncoderDe_Fig3.png|750px]] | |[[File:2015_AHierarchicalRecurrentEncoderDe_Fig3.png|750px]] | ||
|- | |- | ||
|+ align="bottom" style="caption-side:bottom;text-align:left;font-weight:normal; padding-left: 1em;padding-right: 1em; padding-top:0.2em;padding-bottom:0.2em;background-color:#F4F6F6 "|'''Figure 3:''' The [[hierarchical recurrent encoder-decoder (HRED) for query suggestion]]. </s> Each arrow is a [[non-linear transformation]]. </s> The [[user]] types <code>cleveland gallery</code> → <code>lake erie art</code>. </s> During [[training]], the [[model]] [[encode]]s <code>cleveland gallery</code>, [[update]]s the [[session-level recurrent state]] and [[maximize]] the [[probability]] of seeing the following [[query]] <code>lake erie art</code>. </s> The process is repeated for all [[queri]]es in the [[session]]. </s> During [[testing]], a [[contextual suggestion]] is generated by [[encoding]] the previous [[queri]]es, by updating the [[session-level recurrent state]]s accordingly and by [[sampling]] a new [[query]] from the last obtained [[session-level recurrent state]]. </s> In the example, the generated [[contextual suggestion]] is <code>cleveland indian art</code>. | |+ align="bottom" style="caption-side:bottom;text-align:left;font-weight:normal; padding-left: 1em;padding-right: 1em; padding-top:0.2em;padding-bottom:0.2em;background-color:#F4F6F6 "|'''Figure 3:''' The [[hierarchical recurrent encoder-decoder (HRED) for query suggestion]]. </s> Each arrow is a [[non-linear transformation]]. </s> The [[user]] types <code>cleveland gallery</code> → <code>lake erie art</code>. </s> During [[training]], the [[model]] [[encode]]s <code>cleveland gallery</code>, [[update]]s the [[session-level recurrent state]] and [[maximize]] the [[probability]] of seeing the following [[query]] <code>lake erie art</code>. </s> The process is repeated for all [[queri]]es in the [[session]]. </s> During [[testing]], a [[contextual suggestion]] is generated by [[encoding]] the previous [[queri]]es, by updating the [[session-level recurrent state]]s accordingly and by [[sampling]] a new [[query]] from the last obtained [[session-level recurrent state]]. </s> In the example, the generated [[contextual suggestion]] is <code>cleveland indian art</code>. | ||
|} | |} | ||
</div> | </div> |
Revision as of 12:24, 2 August 2022
A Hierarchical Recurrent Encoder-Decoder (HRED) Neural Network Training Algorithm is a feedforward NNet training algorithm that implements a hierarchical recurrent encoder-decoder neural network.
- Example(s):
- Counter-Example(s):
- See: Long Short-Term Memory, Recurrent Neural Network, Convolutional Neural Network, Gating Mechanism, Encoder-Decoder Neural Network.
References
2015
- (Sordoni et al., 2015) ⇒ Alessandro Sordoni, Yoshua Bengio, Hossein Vahabi, Christina Lioma, Jakob Grue Simonsen, and Jian-Yun Nie. (2015). “A Hierarchical Recurrent Encoder-Decoder for Generative Context-Aware Query Suggestion.” In: Proceedings of the 24th ACM International Conference on Information and Knowledge Management (CIKM 2015). DOI:10.1145/2806416.2806493. arXiv:1507.02221.
- QUOTE: Our hierarchical recurrent encoder-decoder (HRED) is pictured in Figure 3. Given a query in the session, the model encodes the information seen up to that position and tries to predict the following query. The process is iterated throughout all the queries in the session. In the forward pass, the model computes the query-level encodings, the session-level recurrent states and the log-likelihood of each query in the session given the previous ones. In the backward pass, the gradients are computed and the parameters are updated.
- QUOTE: Our hierarchical recurrent encoder-decoder (HRED) is pictured in Figure 3. Given a query in the session, the model encodes the information seen up to that position and tries to predict the following query. The process is iterated throughout all the queries in the session. In the forward pass, the model computes the query-level encodings, the session-level recurrent states and the log-likelihood of each query in the session given the previous ones. In the backward pass, the gradients are computed and the parameters are updated.
![]() |