Optimization and statistical learning theory for piecewise smooth and switching regression

Fabien Lauer 1
1 ABC - Machine Learning and Computational Biology
LORIA - ALGO - Department of Algorithms, Computation, Image and Geometry
Abstract : This manuscript deals with several machine learning problems. More precisely, we study two regression problems involving multiple models: piecewise smooth regression and switching regression. Piecewise smooth regression refers to the case where the target function involves jumps (of values or derivatives) and is usually tackled by learning multiple smooth models and a classifier determining the active model on the basis of the input. Switching regression refers to the case where the target function switches between multiple behaviors arbitrarily (and thus independently of the input). The first part of the document focuses on optimization and computational complexity issues. Here, we try to characterize under which conditions it is possible to exactly minimize the empirical risk of these particular regression models. In the second part, we analyze the generalization performance of the models in the framework of statistical learning theory. The standard tools of this framework are introduced in a chapter dedicated to multi-category classification, which we encounter in piecewise smooth regression and which also shares a number of characteristic features with switching regression regarding the analysis in generalization. Technically, the first part contains proofs of NP-hardness, polynomial-time exact algorithms for fixed dimensions and a global optimization method with reasonable computing time for moderate dimensions. The second part derives risk bounds by relying on the estimation of Rademacher complexities, structural decomposition lemmas, chaining arguments and covering numbers. The obtained risk bounds are discussed with a particular emphasis on their dependency on the number of component models.
Document type :
Accreditation to supervise research
Complete list of metadatas

Cited literature [160 references]  Display  Hide  Download

https://hal.univ-lorraine.fr/tel-02307957
Contributor : Fabien Lauer <>
Submitted on : Tuesday, October 8, 2019 - 10:39:40 AM
Last modification on : Thursday, October 10, 2019 - 1:24:36 AM

File

HDR_F_LAUER.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : tel-02307957, version 1

Collections

Citation

Fabien Lauer. Optimization and statistical learning theory for piecewise smooth and switching regression. Machine Learning [cs.LG]. Université de Lorraine, 2019. ⟨tel-02307957⟩

Share

Metrics

Record views

85

Files downloads

13