Skip to Main content Skip to Navigation
Theses

Advanced Difference of Convex functions Algorithms for some topics of Machine Learning with Big Data

Abstract : Big Data has become gradually essential and ubiquitous in all aspects nowadays. Therefore, there is an urge to develop innovative and efficient techniques to deal with the rapid growth in the volume of data. This dissertation considers the following problems in Big Data: group variable selection in multi-class logistic regression, dimension reduction by t-SNE (t-distributed Stochastic Neighbor Embedding), and deep clustering. We develop advanced DCAs (Difference of Convex functions Algorithms) for these problems, which are based on DC Programming and DCA – the powerful tools for non-smooth non-convex optimization problems. Firstly, we consider the problem of group variable selection in multi-class logistic regression. We tackle this problem by using recently advanced DCAs -- Stochastic DCA and DCA-Like. Specifically, Stochastic DCA specializes in the large sum of DC functions minimization problem, which only requires a subset of DC functions at each iteration. DCA-Like relaxes the convexity condition of the second DC component while guaranteeing the convergence. Accelerated DCA-Like incorporates the Nesterov's acceleration technique into DCA-Like to improve its performance. The numerical experiments in benchmark high-dimensional datasets show the effectiveness of proposed algorithms in terms of running time and solution quality. The second part studies the t-SNE problem, an effective non-linear dimensional reduction technique. Motivated by the novelty of DCA-Like and Accelerated DCA-Like, we develop two algorithms for the t-SNE problem. The superiority of proposed algorithms in comparison with existing methods is illustrated through numerical experiments for visualization application. Finally, the third part considers the problem of deep clustering. In the first application, we propose two algorithms based on DCA to combine t-SNE with MSSC (Minimum Sum-of-Squares Clustering) by following two approaches: “tandem analysis” and joint-clustering. The second application considers clustering with auto-encoder (a well-known type of neural network). We propose an extension to a class of joint-clustering algorithms to overcome the scaling problem and applied for a specific case of joint-clustering with MSSC. Numerical experiments on several real-world datasets show the effectiveness of our methods in rapidity and clustering quality, compared to the state-of-the-art methods.
Complete list of metadata

Cited literature [143 references]  Display  Hide  Download

https://hal.univ-lorraine.fr/tel-02557281
Contributor : Thèses Ul <>
Submitted on : Tuesday, April 28, 2020 - 4:17:55 PM
Last modification on : Wednesday, April 29, 2020 - 1:22:36 AM

File

DDOC_T_2019_0255_TRAN.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : tel-02557281, version 1

Collections

Citation

Bach Tran. Advanced Difference of Convex functions Algorithms for some topics of Machine Learning with Big Data. Computer Science [cs]. Université de Lorraine, 2019. English. ⟨NNT : 2019LORR0255⟩. ⟨tel-02557281⟩

Share

Metrics

Record views

176

Files downloads

523