2018 SubsampledStochasticVarianceRed

• (Zou et al., 2018) ⇒ Difan Zou, Pan Xu, and Quanquan Gu. (2018). “Subsampled Stochastic Variance-Reduced Gradient Langevin Dynamics.” In: Proceedings of International Conference on Uncertainty in Artificial Intelligence.

Subject Headings: Stochastic Variance Reduced Gradient Langevin Dynamics Algorithm.

Cited By

• http://scholar.google.com/scholar?q=%222018%22+Subsampled+Stochastic+Variance-Reduced+Gradient+Langevin+Dynamics

Quotes

Abstract

Stochastic variance-reduced gradient Langevin dynamics (SVRG-LD) was recently proposed to improve the performance of stochastic gradient Langevin dynamics (SGLD) by reducing the variance of the stochastic gradient. In this paper, we propose a variant of SVRG-LD, namely SVRG-LD +, which replaces the full gradient in each epoch with a subsampled one. We provide a nonasymptotic analysis of the convergence of SVRG-LD + in 2-Wasserstein distance, and show that SVRG-LD + enjoys a lower gradient complexity1 than SVRG-LD, when the sample size is large or the target accuracy requirement is moderate. Our analysis directly implies a sharper convergence rate for SVRG-LD, which improves the existing convergence rate by a factor of κ 1/6n 1/6, where κ is the condition number of the log-density function and n is the sample size. Experiments on both synthetic and real-world datasets validate our theoretical results.

References

;

	Author	volume	Date Value	title	type	journal	titleUrl	doi	note	year
2018 SubsampledStochasticVarianceRed	Quanquan Gu Difan Zou Pan Xu			Subsampled Stochastic Variance-Reduced Gradient Langevin Dynamics