2014 AClockworkRNN

• (Koutnik et al., 2014) ⇒ Jan Koutnik, Klaus Greff, Faustino J. Gomez, and Jurgen Schmidhuber. (2014). “A Clockwork RNN.” In: Proceedings of the 31th International Conference on Machine Learning (ICML 2014).

Subject Headings: Clockwork RNN (CW-RNN), RNNs.

Cited By

• Google Scholar: ~ 382 Citations, Retrieved: 2020-12-5.

Quotes

Abstract

Sequence prediction and classification are ubiquitous and challenging problems in machine learning that can require identifying complex dependencies between temporally distant inputs. Recurrent Neural Networks (RNNs) have the ability, in theory, to cope with these temporal dependencies by virtue of the short-term memory implemented by their recurrent (feedback) connections. However, in practice they are difficult to train successfully when long-term memory is required. This paper introduces a simple, yet powerful modification to the simple RNN (SRN) architecture, the Clockwork RNN (CW-RNN), in which the hidden layer is partitioned into separate modules, each processing inputs at its own temporal granularity, making computations only at its prescribed clock rate. Rather than making the standard RNN models more complex, CW-RNN reduces the number of SRN parameters, improves the performance significantly in the tasks tested, and speeds up the network evaluation. The network is demonstrated in preliminary experiments involving three tasks: audio signal generation, TIMIT spoken word classification, where it outperforms both SRN and LSTM networks, and online handwriting recognition, where it outperforms SRNs.

BibTeX

@inproceedings{2014_AClockworkRNN,
  author    = {Jan Koutnik and
               Klaus Greff and
               Faustino J. Gomez and
               Jurgen Schmidhuber},
  title     = {A Clockwork RNN},
  booktitle = {Proceedings of the 31th International Conference on Machine Learning
               (ICML 2014)},
  address   = {Beijing, China},
  month     = {June},
  year      = {2014},
  series    = {JMLR Workshop and Conference Proceedings},
  volume    = {32},
  pages     = {1863--1871},
  publisher = {JMLR.org},
  url       = {http://proceedings.mlr.press/v32/koutnik14.html},
}

---

	Author	volume	Date Value	title	type	journal	titleUrl	doi	note	year
2014 AClockworkRNN	Jürgen Schmidhuber Jan Koutnik Klaus Greff Faustino J. Gomez			A Clockwork RNN						2014