Hierarchical Recurrent Encoder-Decoder (HRED) Neural Network Training Algorithm: Difference between revisions
Jump to navigation
Jump to search
m (Text replacement - ". ----" to ". ----") |
m (Text replacement - "([^\s])([\s]{1,7})\<P\>" to "$1 <P>") |
||
Line 16: | Line 16: | ||
=== 2015 === | === 2015 === | ||
* ([[2015_AHierarchicalRecurrentEncoderDe|Sordoni et al., 2015]]) ⇒ [[Alessandro Sordoni]], [[Yoshua Bengio]], [[Hossein Vahabi]], [[Christina Lioma]], [[Jakob Grue Simonsen]], and [[Jian-Yun Nie]]. ([[2015]]). “[https://arxiv.org/pdf/1507.02221.pdf A Hierarchical Recurrent Encoder-Decoder for Generative Context-Aware Query Suggestion].” In: [[Proceedings of the 24th ACM International Conference on Information and Knowledge Management (CIKM 2015)]]. [https://doi.org/10.1145/2806416.2806493 DOI:10.1145/2806416.2806493]. [http://arxiv.org/abs/1507.02221 arXiv:1507.02221]. | * ([[2015_AHierarchicalRecurrentEncoderDe|Sordoni et al., 2015]]) ⇒ [[Alessandro Sordoni]], [[Yoshua Bengio]], [[Hossein Vahabi]], [[Christina Lioma]], [[Jakob Grue Simonsen]], and [[Jian-Yun Nie]]. ([[2015]]). “[https://arxiv.org/pdf/1507.02221.pdf A Hierarchical Recurrent Encoder-Decoder for Generative Context-Aware Query Suggestion].” In: [[Proceedings of the 24th ACM International Conference on Information and Knowledge Management (CIKM 2015)]]. [https://doi.org/10.1145/2806416.2806493 DOI:10.1145/2806416.2806493]. [http://arxiv.org/abs/1507.02221 arXiv:1507.02221]. | ||
** QUOTE: Our [[hierarchical recurrent encoder-decoder (HRED)]] is pictured in [[#FIG3|Figure 3]]. </s> Given a [[query]] in the [[session]], the [[model]] [[encode]]s the [[information]] seen up to that [[position]] and tries to [[predict]] the following [[query]]. </s> The [[process]] is iterated throughout all the [[queri]]es in the session. </s> In the [[forward pass]], the [[model]] [[compute]]s the [[query-level encoding]]s, the [[session-level recurrent state]]s and the [[log-likelihood]] of each [[query]] in the session given the previous ones. </s> In the [[backward pass]], the [[gradient]]s are computed and the [[parameter]]s are [[updated]]. </s> | ** QUOTE: Our [[hierarchical recurrent encoder-decoder (HRED)]] is pictured in [[#FIG3|Figure 3]]. </s> Given a [[query]] in the [[session]], the [[model]] [[encode]]s the [[information]] seen up to that [[position]] and tries to [[predict]] the following [[query]]. </s> The [[process]] is iterated throughout all the [[queri]]es in the session. </s> In the [[forward pass]], the [[model]] [[compute]]s the [[query-level encoding]]s, the [[session-level recurrent state]]s and the [[log-likelihood]] of each [[query]] in the session given the previous ones. </s> In the [[backward pass]], the [[gradient]]s are computed and the [[parameter]]s are [[updated]]. </s> <P> | ||
<P> | |||
<div id="FIG3"> | <div id="FIG3"> | ||
{|style="border:2px solid #F4F6F6; text-align:center; vertical-align:center; border-spacing: 1px; margin: 1em auto; width: 80%" | {|style="border:2px solid #F4F6F6; text-align:center; vertical-align:center; border-spacing: 1px; margin: 1em auto; width: 80%" |
Revision as of 17:27, 16 August 2021
A Hierarchical Recurrent Encoder-Decoder (HRED) Neural Network Training Algorithm is a feedforward NNet training algorithm that implements a hierarchical recurrent encoder-decoder neural network.
- Example(s):
- Counter-Example(s):
- See: Long Short-Term Memory, Recurrent Neural Network, Convolutional Neural Network, Gating Mechanism,Encoder-Decoder Neural Network.
References
2015
- (Sordoni et al., 2015) ⇒ Alessandro Sordoni, Yoshua Bengio, Hossein Vahabi, Christina Lioma, Jakob Grue Simonsen, and Jian-Yun Nie. (2015). “A Hierarchical Recurrent Encoder-Decoder for Generative Context-Aware Query Suggestion.” In: Proceedings of the 24th ACM International Conference on Information and Knowledge Management (CIKM 2015). DOI:10.1145/2806416.2806493. arXiv:1507.02221.
- QUOTE: Our hierarchical recurrent encoder-decoder (HRED) is pictured in Figure 3. Given a query in the session, the model encodes the information seen up to that position and tries to predict the following query. The process is iterated throughout all the queries in the session. In the forward pass, the model computes the query-level encodings, the session-level recurrent states and the log-likelihood of each query in the session given the previous ones. In the backward pass, the gradients are computed and the parameters are updated.
![]() |