Stochastic Feedforward Neural Network (SFNN)

A Stochastic Feedforward Neural Network is a Feed-Forward Neural Network that includes stochastic binary neurons as well as deterministic neurons.

• Example(s):
  • the SFNN proposed in Lee et al. (2017),
  • …
• Counter-Example(s):
  • a Multilayer Feedforward Neural Network,
  • a Single-Layer Feedforward Neural Network,
  • a Convolutional Neural Network,
  • a Radial Basis Function NNet.
  • a Recurrent Neural Network.
• See: Artificial Neural Network, Neural Network Topology, Backpropagation, Machine Learning, Learning Curve, Radial Basis Function Network, Deep Learning, Deep Learning Artificial Neural Network, Pattern Recognition, Feedforward Neural Network, Recurrent Network, Adaptive Resonance Theory, Directed Cycle, Backprop Algorithm, Perceptron Model.

---

---

References

2017

• (Lee et al., 2017) ⇒ Kimin Lee, Jaehyung Kim, Song Chong, and Jinwoo Shin (2017). "Simplified Stochastic Feedforward Neural Networks". preprint arXiv:1704.03188.
  • QUOTE: Stochastic feedforward neural network (SFNN) is a hybrid model, which has both stochastic binary and deterministic hidden units. We first introduce SFNN with one stochastic hidden layer (and without deterministic hidden layers) for simplicity. Throughout this paper, we commonly denote the bias for unit i and the weight matrix of the [math]\displaystyle{ \ell }[/math]-th hidden layer by [math]\displaystyle{ b^\ell_i }[/math] and [math]\displaystyle{ W^{\ell} }[/math], respectively. Then, the stochastic hidden layer in SFNN is defined as a binary random vector with <math>N^1 units ...

---