2016 WideResidualNetworks

(Zagoruyko & Komodakis, 2016) ⇒ Sergey Zagoruyko, and Nikos Komodakis. (2016). “Wide Residual Networks.” In: Proceedings of the British Machine Vision Conference 2016 (BMVC 2016).

Subject Headings: Residual Neural Network; Deep Residual Neural Network, Wide Residual Network.

Notes

Version(s):
- https://arxiv.org/abs/1605.07146
Other Link(s):
- Online Repository: https://github.com/szagoruyko/wide-residual-networks

Cited By

Google Scholar: ~ 3,020 Citations, Retrieved:2021-01-24.

Quotes

Abstract

Deep residual networks were shown to be able to scale up to thousands of layers and still have improving performance. However, each fraction of a percent of improved accuracy costs nearly doubling the number of layers, and so training very deep residual networks has a problem of diminishing feature reuse, which makes these networks very slow to train. To tackle these problems, in this paper we conduct a detailed experimental study on the architecture of ResNet blocks, based on which we propose a novel architecture where we decrease depth and increase width of residual networks. We call the resulting network structures wide residual networks (WRNs) and show that these are far superior over their commonly used thin and very deep counterparts. For example, we demonstrate that even a simple 16-layer-deep wide residual network outperforms in accuracy and efficiency all previous deep residual networks, including thousand-layer-deep networks, achieving new state-of-the-art results on CIFAR-10, CIFAR-100 and SVHN. Our code is available at https://github.com/szagoruyko/wide-residual-networks

1. Introduction

2. Wide Residual Networks

Residual block with identity mapping can be represented by the following formula:

[math]\displaystyle{ \mathbf{x}_{l+1}=\mathbf{x}_{l}+\mathcal{F}\left(\mathbf{x}_{l}, \mathcal{W}_{l}\right) }[/math]

(1)

where $\mathbf{x}_{l+1}$ and $\mathbf{x}_{l}$ are input and output of the $l$-th unit in the network, $\mathcal{F}$ is a residual function and $\mathcal{W}_{l}$ are parameters of the block. Residual network consists of sequentially stacked residual block.

**Figure 1:** Various residual blocks used in the paper. Batch normalization and ReLU precede each convolution (omitted for clarity).

(...)

3. Experimental Results

4. Conclusions

References

BibTeX

@inproceedings{2016_WideResidualNetworks,
  author    = {Sergey Zagoruyko and
               Nikos Komodakis},
  editor    = {Richard C. Wilson and
               Edwin R. Hancock and
               William A. P. Smith},
  title     = {Wide Residual Networks},
  booktitle = {Proceedings of the British Machine Vision Conference 2016 (BMVC 2016)},
  publisher = {BMVA Press},
  year      = {2016},
  url       = {http://www.bmva.org/bmvc/2016/papers/paper087/index.html},
}

	Author	volume	Date Value	title	type	journal	titleUrl	doi	note	year
2016 WideResidualNetworks	Sergey Zagoruyko Nikos Komodakis			Wide Residual Networks						2016