Stochastic DCA with Variance Reduction and Applications in Machine Learning - Université de Lorraine Access content directly
Journal Articles Journal of Machine Learning Research Year : 2022

Stochastic DCA with Variance Reduction and Applications in Machine Learning

Abstract

We design stochastic Difference-of-Convex-functions Algorithms (DCA) for solving a class of structured Difference-of-Convex-functions (DC) problems. As the standard DCA requires the full information of (sub)gradients which could be expensive in large-scale settings, stochastic approaches rely upon stochastic information instead. However, stochastic estimations generate additional variance terms making stochastic algorithms unstable. Therefore, we integrate some novel variance reduction techniques including SVRG and SAGA into our design. The almost sure convergence to critical points of the proposed algorithms is established and the algorithms' complexities are analyzed. To study the efficiency of our algorithms, we apply them to three important problems in machine learning: nonnegative principal component analysis, group variable selection in multiclass logistic regression, and sparse linear regression. Numerical experiments have shown the merits of our proposed algorithms in comparison with other state-of-the-art stochastic methods for solving nonconvex large-sum problems.
No file

Dates and versions

hal-03748673 , version 1 (09-08-2022)

Licence

Attribution

Identifiers

  • HAL Id : hal-03748673 , version 1

Cite

Hoai An Le Thi, Hoang Phuc Hau Luu, Hoai Minh Le, Tao Pham Dinh. Stochastic DCA with Variance Reduction and Applications in Machine Learning. Journal of Machine Learning Research, 2022, 23 (206), pp.1-44. ⟨hal-03748673⟩
49 View
0 Download

Share

Gmail Facebook Twitter LinkedIn More