, The material of this chapter is developed from the following works
Stochastic DCA for the large-sum of non-convex functions problem. Application to group variables selection in multiclass logistic regression, International Conference on Machine Learning ICML, pp.3394-3403, 2017. ,
URL : https://hal.archives-ouvertes.fr/hal-01664761
Stochastic DCA for Sparse Multiclass Logistic Regression, Advances in Intelligent Systems and Computing, vol.629, pp.1-12, 2017. ,
Novel DCA Based Algorithms for Minimizing the Sum of a Nonconvex Function and Composite Functions with Applications in Machine learning ,
Stochastic DCA for minimizing a large sum of DC functions and its application in Multi-class Logistic Regression ,
, The material of this chapter is developed from the following works
A DCA-based approach for Joint Clustering and Dimensional Reduction by t-SNE ,
Deep Clustering with Spherical Distance in Latent Space, Advanced Computational Methods for Knowledge Engineering. ICCSAMA 2019. Advances in Intelligent Systems and Computing ,
Customer Clustering of French Transmission System Operator (RTE) Based on Their Electricity Consumption, Optimization of Complex Systems: Theory, Models, Algorithms and Applications. WCGO 2019. Advances in Intelligent Systems and Computing, vol.991, pp.893-905 ,
Time-series clustering -A decade review, Information Systems, vol.53, pp.16-38, 2015. ,
, Clustering with Deep Learning: Taxonomy and New Methods, 2018.
On the approximability of minimizing nonzero variables or unsatisfied relations in linear systems, Theor. Comput. Sci, vol.209, issue.1, pp.237-260, 1998. ,
, DC Programming and DCA, 2005.
Clustering and Unsupervised Anomaly Detection with 2 Normalized Deep Auto-Encoder Representations, 2018. ,
Logistic regression in the medical literature: Standards for use and reporting, with particular attention to one medical domain, Journal of Clinical Epidemiology, vol.54, issue.10, pp.979-985, 2001. ,
Advanced Methods of Marketing Research, 1994. ,
Evaluating trauma care: The TRISS method. Trauma Score and the Injury Severity Score, The Journal of Trauma, vol.27, issue.4, pp.370-378, 1987. ,
, Convex Optimization, 2004.
Feature selection via concave minimization and support vector machines, ICML, vol.98, pp.82-90, 1998. ,
Feature selection via concave minimization and support vector machines, Machine Learning Proceedings of the Fifteenth International Conference (ICML 1998), pp.82-90, 1998. ,
Dimension Reduction: A Guided Tour, FNT in Machine Learning, vol.2, pp.275-364, 2009. ,
Locally Consistent Concept Factorization for Document Clustering, IEEE Transactions on Knowledge and Data Engineering, vol.23, issue.6, pp.902-913, 2011. ,
Deep Clustering for Unsupervised Learning of Visual Features, Computer Vision -ECCV 2018, pp.139-156, 2018. ,
Nonlinear wavelet image processing: Variational problems, compression, and noise removal through wavelet shrinkage, IEEE Transactions on Image Processing, vol.7, issue.3, pp.319-335, 1998. ,
On Using Principal Components Before Separating a Mixture of Two Multivariate Normal Distributions, Journal of the Royal Statistical Society. Series C (Applied Statistics), vol.32, issue.3, pp.267-275, 1983. ,
Unsupervised Multi-Manifold Clustering by Learning Deep Representation, Workshops at the Thirty-First AAAI Conference on Artificial Intelligence, 2017. ,
Trading convexity for scalability, ICML '06: Proceedings of the 23rd International Conference on Machine Learning, pp.201-208, 2006. ,
The regression analysis of binary sequences (with discussion), J Roy Stat Soc B, vol.20, pp.215-242, 1958. ,
Customer Clustering of French Transmission System Operator (RTE) Based on Their Electricity Consumption, Optimization of Complex Systems: Theory, Models, Algorithms and Applications, pp.893-905, 2020. ,
Deep Representation Learning Characterized by Inter-Class Separation for Image Clustering, 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), pp.628-637, 2019. ,
K-means clustering in a low-dimensional Euclidean space, New Approaches in Classification and Data Analysis, Studies in Classification, Data Analysis, and Knowledge Organization, pp.212-219, 1994. ,
,
Maximum Likelihood from Incomplete Data Via the EM Algorithm, Journal of the Royal Statistical Society: Series B (Methodological), vol.39, issue.1, pp.1-22, 1977. ,
Concept Decompositions for Large Sparse Text Data Using Clustering, Machine Learning, vol.42, pp.143-175, 2001. ,
Efficient projections onto the l 1-ball for learning in high dimensions, Proceedings of the 25th International Conference on Machine Learning, pp.272-279, 2008. ,
, Deep k-Means: Jointly clustering with k-Means and learning representations, 2018.
Large-scale Bayesian logistic regression for text categorization, Technometrics, vol.49, issue.3, pp.291-304, 2007. ,
Deep clustering via joint convolutional autoencoder embedding and relative entropy minimization, Proceedings of the IEEE International Conference on Computer Vision, pp.5736-5745, 2017. ,
Understanding the difficulty of training deep feedforward neural networks, Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, pp.249-256, 2010. ,
On similarity queries for time-series data: Constraint specification and implementation, Principles and Practice of Constraint Programming -CP '95, pp.137-153, 1995. ,
, Deep Learning, 2016.
Deep Embedded Clustering with Data Augmentation, ACML, p.16, 2018. ,
Improved deep embedded clustering with local structure preservation, International Joint Conference on Artificial Intelligence (IJCAI-17), pp.1753-1759, 2017. ,
Stochastic neighbor embedding, Advances in Neural Information Processing Systems, pp.857-864, 2003. ,
Reducing the Dimensionality of Data with Neural Networks, Science, vol.313, issue.5786, pp.504-507, 2006. ,
Convex Analysis and Minimization Algorithms, Grundlehren Der Mathematischen Wissenschaften, Convex Analysis and Minimization Algorithms, 1993. ,
Deep Embedding Network for Clustering, 2014 22nd International Conference on Pattern Recognition, pp.1532-1537, 2014. ,
Deep Subspace Clustering Networks, NIPS, 2017. ,
A Gradient-Based Optimization Algorithm for LASSO, Journal of Computational and Graphical Statistics, vol.17, issue.4, pp.994-1009, 2008. ,
Logistic Regression in Rare Events Data, Political Analysis, vol.9, pp.137-163, 2001. ,
Adam: A Method for Stochastic Optimization, 2014. ,
Sparse semi-supervised support vector machines by DC programming and DCA, Neurocomputing, vol.153, pp.62-76, 2015. ,
URL : https://hal.archives-ouvertes.fr/hal-01634225
DeepVQ: A deep network architecture for vector quantization, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp.2579-2582, 2018. ,
Contributionà l'optimisation non convexe et l'optimisation globale: Théorie, algorithmes et applications, Habilitationà Diriger des Recherches, 1997. ,
A new efficient algorithm based on DC programming and DCA for clustering, Journal of Global Optimization, vol.37, issue.4, pp.593-608, 2007. ,
URL : https://hal.archives-ouvertes.fr/hal-01636755
A DC programming approach for feature selection in support vector machines learning, Advances in Data Analysis and Classification, vol.2, pp.259-278, 2008. ,
URL : https://hal.archives-ouvertes.fr/hal-01636751
New and efficient DCA based algorithms for minimum sum-of-squares clustering, Pattern Recognition, vol.47, issue.1, pp.388-401, 2014. ,
URL : https://hal.archives-ouvertes.fr/hal-01636670
Feature selection in machine learning: An exact penalty approach using a Difference of Convex function Algorithm, Mach Learn, vol.101, issue.1, pp.163-186, 2015. ,
URL : https://hal.archives-ouvertes.fr/hal-01636662
Binary classification via spherical separator by DC programming and DCA, J Glob Optim, vol.56, issue.4, pp.1393-1407, 2013. ,
URL : https://hal.archives-ouvertes.fr/hal-01636671
Stochastic DCA for the Large-sum of Non-convex Functions Problem and its Application to Group Variable Selection in Classification, Proceedings of the 34th International Conference on Machine Learning, vol.70, pp.3394-3403, 2017. ,
URL : https://hal.archives-ouvertes.fr/hal-01664761
A DCA-Like Algorithm and its Accelerated Version with Application in Data Visualization, 2018. ,
Stochastic DCA for sparse multiclass logistic regression, Advanced Computational Methods for Knowledge Engineering, pp.1-12, 2018. ,
Novel DCA Based Algorithms for Minimizing the Sum of a Nonconvex Function and Composite Functions. Applications in Machine learning, 2019. ,
Stochastic DCA for minimizing a large sum of DC functions with application to Multi-class Logistic Regression, 2019. ,
Long-Short Portfolio Optimization Under Cardinality Constraints by Difference of Convex Functions Algorithm, J Optim Theory Appl, vol.161, issue.1, pp.199-224, 2014. ,
URL : https://hal.archives-ouvertes.fr/hal-01636672
Self-organizing maps by difference of convex functions optimization, Data Min Knowl Disc, vol.28, issue.5, pp.1336-1365, 2014. ,
URL : https://hal.archives-ouvertes.fr/hal-01636658
DCA based algorithms for feature selection in multi-class support vector machine, Ann Oper Res, vol.249, issue.1, pp.273-300, 2017. ,
URL : https://hal.archives-ouvertes.fr/hal-01616963
A DC Programming Approach for Finding Communities in Networks, Neural Computation, vol.26, issue.12, pp.2827-2854, 2014. ,
URL : https://hal.archives-ouvertes.fr/hal-01636651
Solving a Class of Linearly Constrained Indefinite Quadratic Problems by D.C. Algorithms, Journal of Global Optimization, vol.11, issue.3, pp.253-285, 1997. ,
URL : https://hal.archives-ouvertes.fr/hal-01636781
The DC (Difference of Convex Functions) Programming and DCA Revisited with DC Models of Real World Nonconvex Optimization Problems, Annals of Operations Research, vol.133, issue.1, pp.23-46, 2005. ,
URL : https://hal.archives-ouvertes.fr/hal-01636759
Minimum sum-of-squares clustering by DC programming and DCA, International Conference on Intelligent Computing, pp.327-340, 2009. ,
DC programming and DCA: thirty years of developments, Mathematical Programming, pp.1-64, 2018. ,
DC approximation approaches for sparse optimization, European Journal of Operational Research, vol.244, issue.1, pp.26-46, 2015. ,
URL : https://hal.archives-ouvertes.fr/hal-01634220
Efficient approaches for 2 -0 regularization and applications to feature selection in SVM, Applied Intelligence, vol.45, issue.2, pp.549-565, 2016. ,
URL : https://hal.archives-ouvertes.fr/hal-01616985
DC Programming and DCA for Sparse Optimal Scoring Problem, Neurocomput, vol.186, issue.C, pp.170-181, 2016. ,
URL : https://hal.archives-ouvertes.fr/hal-01616991
DC Programming and DCA for Sparse Fisher Linear Discriminant Analysis, Neural Comput. Appl, vol.28, issue.9, pp.2809-2822, 2017. ,
URL : https://hal.archives-ouvertes.fr/hal-01616990
Efficient Nonnegative Matrix Factorization by DC Programming and DCA, Neural Comput, vol.28, issue.6, pp.1163-1216, 2016. ,
URL : https://hal.archives-ouvertes.fr/hal-01616987
Gradient-based learning applied to document recognition, Proceedings of the IEEE, vol.86, issue.11, pp.2278-2324, 1998. ,
Accelerated proximal gradient methods for nonconvex programming, Advances in Neural Information Processing Systems, pp.377-387, 2015. ,
Logistic regression for disease classification using microarray data: Model selection in a large p and small n case, Bioinformatics, vol.23, issue.15, pp.1945-1951, 2007. ,
Multicategory ?-Learning, Journal of the American Statistical Association, vol.101, issue.474, pp.500-509, 2006. ,
URL : https://hal.archives-ouvertes.fr/hal-01020707
Least squares quantization in PCM, IEEE Transactions on Information Theory, vol.28, issue.2, pp.129-137, 1982. ,
Matching Theory, 2009. ,
Accelerating t-sne using tree-based algorithms, Journal of machine learning research, vol.15, issue.1, pp.3221-3245, 2014. ,
Some methods for classification and analysis of multivariate observations, Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, vol.1, pp.281-297, 1967. ,
A Survey of Clustering With Deep Learning: From the Perspective of Network Architecture, IEEE Access, vol.6, pp.39501-39514, 2018. ,
Human-level control through deep reinforcement learning, Nature, vol.518, issue.7540, pp.529-533, 2015. ,
Joint Dimension Reduction and Clustering, Nonlinear Principal Component Analysis and Its Applications, pp.57-64, 2016. ,
Rectified Linear Units Improve Restricted Boltzmann Machines, Proceedings of the 27th International Conference on International Conference on Machine Learning, ICML'10, pp.807-814, 2010. ,
DCA Based Approaches for Mathematical Programs with Equilibrium Constraints, 2018. ,
URL : https://hal.archives-ouvertes.fr/tel-01886895
Updating Quasi-Newton Matrices with Limited Storage. Mathematics of Computation, vol.35, pp.773-782, 1980. ,
K-Shape: Efficient and Accurate Clustering of Time Series, Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data, pp.1855-1870, 2015. ,
Proximal algorithms. Found, Trends Optim, vol.1, issue.3, pp.127-239, 2014. ,
A Difference of Convex Functions Algorithm for Switched Linear Regression, IEEE Transactions on Automatic Control, vol.59, issue.8, pp.2277-2282, 2014. ,
URL : https://hal.archives-ouvertes.fr/hal-00931206
Convex analysis approach to dc programming: Theory, algorithms and applications, Acta Mathematica Vietnamica, vol.22, issue.1, pp.289-355, 1997. ,
A D. C. Optimization Algorithm for Solving the Trust-Region Subproblem, SIAM Journal of Optimization, vol.8, issue.2, pp.476-505, 1998. ,
URL : https://hal.archives-ouvertes.fr/hal-01636777
Recent Advances in DC Programming and DCA, Transactions on Computational Intelligence XIII, pp.1-37, 2014. ,
URL : https://hal.archives-ouvertes.fr/hal-01664024
Algorithms for Solving a Class of Nonconvex Optimization Problems. Methods of Subgradients, Mathematics for Optimization, vol.129, pp.249-271, 1986. ,
Accelerated Difference of Convex functions Algorithm and its Application to Sparse Binary Logistic Regression, Twenty-Seventh International Joint Conference on Artificial Intelligence, pp.1369-1375, 2018. ,
Sparse covariance matrix estimation by DCA-Based Algorithms, Neural Computation, vol.29, issue.11, pp.3040-3077, 2017. ,
URL : https://hal.archives-ouvertes.fr/hal-01769311
DC Proximal Newton for Nonconvex Optimization Problems, IEEE Transactions on Neural Networks and Learning Systems, vol.27, issue.3, pp.636-647, 2016. ,
URL : https://hal.archives-ouvertes.fr/hal-00952445
Searching and mining trillions of time series subsequences under dynamic time warping, Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining -KDD '12, p.262, 2012. ,
, Convex Analysis, 1970.
T-Distributed stochastic neighbor embedding spectral clustering, 2017 International Joint Conference on Neural Networks (IJCNN), pp.1628-1632, 2017. ,
Learning representations by back-propagating errors, Nature, vol.323, issue.6088, p.533, 1986. ,
, Deep Continuous Clustering, 2018.
Deep Discriminative Clustering Network, 2018 International Joint Conference on Neural Networks (IJCNN), pp.1-7, 2018. ,
An Introduction to the Conjugate Gradient Method Without the Agonizing Pain, 1994. ,
Auto-encoder Based Data Clustering, Progress in Pattern Recognition, pp.117-124, 2013. ,
Manifold clustering, Tenth IEEE International Conference on Computer Vision (ICCV'05, vol.1, pp.648-653, 2005. ,
The Challenges of Clustering High Dimensional Data, New Directions in Statistical Physics: Econophysics, Bioinformatics, and Pattern Recognition, pp.273-309, 2004. ,
Cluster Ensembles -A Knowledge Reuse Framework for Combining Multiple Partitions, Journal of Machine Learning Research, vol.3, pp.583-617, 2002. ,
Classification of EEG signals using neural network and logistic regression, Comput. Methods Programs Biomed, vol.78, issue.2, pp.87-99, 2005. ,
DeepCluster: A General Clustering Framework Based on Deep Learning, Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp.809-825, 2017. ,
Learning a parametric embedding by preserving local structure, RBM, vol.500, issue.500, p.26, 2009. ,
Visualizing Data using t-SNE, Journal of Machine Learning Research, vol.9, pp.2579-2605, 2008. ,
Dimensionality Reduction: A Comparative Review, J Mach Learn Res, vol.10, p.13, 2009. ,
Factorial k-means analysis for two-way data, Computational Statistics & Data Analysis, vol.37, issue.1, pp.49-64, 2001. ,
Sparse group lasso and high dimensional multinomial classification, Comput. Stat. Data Anal, vol.71, pp.771-786, 2014. ,
Partial-Hessian strategies for fast learning of nonlinear embeddings, 2012. ,
Ramp Loss Support Vector Data Description, Intelligent Information and Database Systems, vol.10191, pp.421-431, 2017. ,
Clustering by Orthogonal Non-negative Matrix Factorization: A Sequential Non-convex Penalty Approach, ICASSP 2019 -2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.5576-5580, 2019. ,
Clustering of time series data-a survey, Pattern Recognition, vol.38, pp.1857-1874, 2005. ,
,
Combined Single-Cell Functional and Gene Expression Analysis Resolves Heterogeneity within Stem Cell Populations, Cell Stem Cell, vol.16, issue.6, pp.712-724, 2015. ,
Penalized classification using Fisher's linear discriminant, Journal of the Royal Statistical Society: Series B, vol.73, issue.5, pp.753-772, 2011. ,
, Fashion-MNIST: A Novel Image Dataset for Benchmarking Machine Learning Algorithms, 2017.
Unsupervised deep embedding for clustering analysis, International Conference on Machine Learning, pp.478-487, 2016. ,
Document Clustering Based on Nonnegative Matrix Factorization, Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Informaion Retrieval, SIGIR '03, pp.267-273, 2003. ,
Learning from hidden traits: Joint factor analysis and latent clustering, IEEE Transactions on Signal Processing, vol.65, issue.1, pp.256-269, 2017. ,
Towards K-meansfriendly Spaces: Simultaneous Deep Learning and Clustering, International Conference on Machine Learning, pp.3861-3870, 2017. ,
Joint Unsupervised Learning of Deep Representations and Image Clusters, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.5147-5156, 2016. ,
Heavy-tailed symmetric stochastic neighbor embedding, Advances in Neural Information Processing Systems, pp.2169-2177, 2009. ,
Majorization-minimization for manifold embedding, Artificial Intelligence and Statistics, pp.1088-1097, 2015. ,
Manifold Clustering of Shapes, Sixth International Conference on Data Mining (ICDM'06), pp.1167-1171, 2006. ,
Minimization of 1?2 for Compressed Sensing, SIAM J. Sci. Comput, vol.37, issue.1, pp.536-563, 2015. ,
The Concave-Convex Procedure (CCCP), Advances in Neural Information Processing Systems, vol.14, pp.1033-1040, 2002. ,
Learning a mixture model for clustering with the completed likelihood minimum message length criterion, Pattern Recognition, vol.47, issue.5, pp.2011-2030, 2014. ,
Spectral Relaxation for K-means Clustering, Advances in Neural Information Processing Systems, vol.14, pp.1057-1064, 2002. ,
DRLnet: Deep Difference Representation Learning Network and An Unsupervised Optimization Framework, IJCAI, 2017. ,
Scalable Deep k-Subspace Clustering, Computer Vision -ACCV 2018, pp.466-481, 2019. ,
Neural Collaborative Subspace Clustering, Proceedings of the 36th International Conference on Machine Learning, vol.97, pp.7384-7393, 2019. ,
M-Isomap: Orthogonal Constrained Marginal Isomap for Nonlinear Dimensionality Reduction, IEEE Trans. Cybern, vol.43, issue.1, pp.180-191, 2013. ,