Epochwise Backpropagation Through Time (EBPTT) Algorithm

An Epochwise Backpropagation Through Time (EBPTT) Algorithm is a Backpropagation Through Time Algorithm in which the training data is segmented into epochs.

• Context:
  • It was initially developed by Williams & Peng, (1990).
• Example(s):
  • …
• Counter-Example(s):
  • Memory-Efficient Backpropagation Through Time Algorithm,
  • Real-Time Recurrent Learning (RTRL) Algorithm,
  • Stochastic Gradient Descent Backpropagation Algorithm,
  • Truncated Back-Propagation Through Time (TBPTT) Algorithm,
  • Unbiased Online Recurrent Optimisation (UORO) Algorithm.
• See: Recurrent Neural Network, Elman Networks, Jordan Networks, Gradient, Backpropagation Algorithm.

---

---

References

1990

• (Williams & Peng, 1990) ⇒ Ronald J. Williams, and Jing Peng (1990). "An Efficient Gradient-Based Algorithm for On-Line Training of Recurrent Network Trajectories". In: Neural Computation 2(4): 490-501.
  • QUOTE: The backpropagation-through-time approach can be derived by unfolding the temporal operation of a network into a multilayer feedforward network that grows by one layer on each time step. If the training stream is segmented into epochs, then one can derive the specific version that we will call epochwise backpropagation through time.

---