2018 SubsampledStochasticVarianceRed: Difference between revisions
m (Text replacement - "== ==" to "== ==") |
m (Text replacement - " ==" to " ==") |
||
Line 7: | Line 7: | ||
== Cited By == | == Cited By == | ||
* http://scholar.google.com/scholar?q=%222018%22+Subsampled+Stochastic+Variance-Reduced+Gradient+Langevin+Dynamics | * http://scholar.google.com/scholar?q=%222018%22+Subsampled+Stochastic+Variance-Reduced+Gradient+Langevin+Dynamics | ||
== Quotes == | == Quotes == |
Latest revision as of 18:11, 10 February 2021
- (Zou et al., 2018) ⇒ Difan Zou, Pan Xu, and Quanquan Gu. (2018). “Subsampled Stochastic Variance-Reduced Gradient Langevin Dynamics.” In: Proceedings of International Conference on Uncertainty in Artificial Intelligence.
Subject Headings: Stochastic Variance Reduced Gradient Langevin Dynamics Algorithm.
Notes
Cited By
Quotes
Abstract
Stochastic variance-reduced gradient Langevin dynamics (SVRG-LD) was recently proposed to improve the performance of stochastic gradient Langevin dynamics (SGLD) by reducing the variance of the stochastic gradient. In this paper, we propose a variant of SVRG-LD, namely SVRG-LD +, which replaces the full gradient in each epoch with a subsampled one. We provide a nonasymptotic analysis of the convergence of SVRG-LD + in 2-Wasserstein distance, and show that SVRG-LD + enjoys a lower gradient complexity1 than SVRG-LD, when the sample size is large or the target accuracy requirement is moderate. Our analysis directly implies a sharper convergence rate for SVRG-LD, which improves the existing convergence rate by a factor of κ 1/6n 1/6, where κ is the condition number of the log-density function and n is the sample size. Experiments on both synthetic and real-world datasets validate our theoretical results.
References
;
Author | volume | Date Value | title | type | journal | titleUrl | doi | note | year | |
---|---|---|---|---|---|---|---|---|---|---|
2018 SubsampledStochasticVarianceRed | Quanquan Gu Difan Zou Pan Xu | Subsampled Stochastic Variance-Reduced Gradient Langevin Dynamics |