Hierarchical Recurrent Encoder-Decoder (HRED) Neural Network Training Algorithm: Difference between revisions

From GM-RKB
Jump to navigation Jump to search
m (Text replacement - ". ----" to ". ----")
m (Text replacement - "([^\s])([\s]{1,7})\<P\>" to "$1 <P>")
Line 16: Line 16:
=== 2015 ===
=== 2015 ===
* ([[2015_AHierarchicalRecurrentEncoderDe|Sordoni et al., 2015]]) ⇒ [[Alessandro Sordoni]], [[Yoshua Bengio]], [[Hossein Vahabi]], [[Christina Lioma]], [[Jakob Grue Simonsen]], and [[Jian-Yun Nie]]. ([[2015]]). &ldquo;[https://arxiv.org/pdf/1507.02221.pdf A Hierarchical Recurrent Encoder-Decoder for Generative Context-Aware Query Suggestion].&rdquo; In: [[Proceedings of the 24th ACM International Conference on Information and Knowledge Management (CIKM 2015)]]. [https://doi.org/10.1145/2806416.2806493 DOI:10.1145/2806416.2806493]. [http://arxiv.org/abs/1507.02221 arXiv:1507.02221].
* ([[2015_AHierarchicalRecurrentEncoderDe|Sordoni et al., 2015]]) ⇒ [[Alessandro Sordoni]], [[Yoshua Bengio]], [[Hossein Vahabi]], [[Christina Lioma]], [[Jakob Grue Simonsen]], and [[Jian-Yun Nie]]. ([[2015]]). &ldquo;[https://arxiv.org/pdf/1507.02221.pdf A Hierarchical Recurrent Encoder-Decoder for Generative Context-Aware Query Suggestion].&rdquo; In: [[Proceedings of the 24th ACM International Conference on Information and Knowledge Management (CIKM 2015)]]. [https://doi.org/10.1145/2806416.2806493 DOI:10.1145/2806416.2806493]. [http://arxiv.org/abs/1507.02221 arXiv:1507.02221].
** QUOTE: Our [[hierarchical recurrent encoder-decoder (HRED)]] is pictured in [[#FIG3|Figure 3]]. </s> Given a [[query]] in the [[session]], the [[model]] [[encode]]s the [[information]] seen up to that [[position]] and tries to [[predict]] the following [[query]]. </s> The [[process]] is iterated throughout all the [[queri]]es in the session. </s> In the [[forward pass]], the [[model]] [[compute]]s the [[query-level encoding]]s, the [[session-level recurrent state]]s and the [[log-likelihood]] of each [[query]] in the session given the previous ones. </s> In the [[backward pass]], the [[gradient]]s are computed and the [[parameter]]s are [[updated]]. </s>
** QUOTE: Our [[hierarchical recurrent encoder-decoder (HRED)]] is pictured in [[#FIG3|Figure 3]]. </s> Given a [[query]] in the [[session]], the [[model]] [[encode]]s the [[information]] seen up to that [[position]] and tries to [[predict]] the following [[query]]. </s> The [[process]] is iterated throughout all the [[queri]]es in the session. </s> In the [[forward pass]], the [[model]] [[compute]]s the [[query-level encoding]]s, the [[session-level recurrent state]]s and the [[log-likelihood]] of each [[query]] in the session given the previous ones. </s> In the [[backward pass]], the [[gradient]]s are computed and the [[parameter]]s are [[updated]]. </s>         <P>
<P>
<div id="FIG3">
<div id="FIG3">
{|style="border:2px solid #F4F6F6; text-align:center; vertical-align:center; border-spacing: 1px; margin: 1em auto; width: 80%"
{|style="border:2px solid #F4F6F6; text-align:center; vertical-align:center; border-spacing: 1px; margin: 1em auto; width: 80%"

Revision as of 17:27, 16 August 2021

A Hierarchical Recurrent Encoder-Decoder (HRED) Neural Network Training Algorithm is a feedforward NNet training algorithm that implements a hierarchical recurrent encoder-decoder neural network.



References

2015

2015 AHierarchicalRecurrentEncoderDe Fig3.png
Figure 3: The hierarchical recurrent encoder-decoder (HRED) for query suggestion. Each arrow is a non-linear transformation. The user types cleveland gallerylake erie art. During training, the model encodes cleveland gallery, updates the session-level recurrent state and maximize the probability of seeing the following query lake erie art. The process is repeated for all queries in the session. During testing, a contextual suggestion is generated by encoding the previous queries, by updating the session-level recurrent states accordingly and by sampling a new query from the last obtained session-level recurrent state. In the example, the generated contextual suggestion is cleveland indian art.