Skip to Main content Skip to Navigation
Journal articles

A unified DC programming framework and efficient DCA based approaches for large scale batch reinforcement learning

Abstract : We investigate a powerful nonconvex optimization approach based on Difference of Convex functions (DC) programming and DC Algorithm (DCA) for reinforcement learning, a general class of machine learning techniques which aims to estimate the optimal learning policy in a dynamic environment typically formulated as a Markov decision process (with an incomplete model). The problem is tackled as finding the zero of the so-called optimal Bellman residual via the linear value-function approximation for which two optimization models are proposed: minimizing the ℓp-norm of a vector-valued convex function, and minimizing a concave function under linear constraints. They are all formulated as DC programs for which attractive DCA schemes are developed. Numerical experiments on various examples of the two benchmarks of Markov decision process problems—Garnet and Gridworld problems, show the efficiency of our approaches in comparison with two existing DCA based algorithms and two state-of-the-art reinforcement learning algorithms.
Document type :
Journal articles
Complete list of metadatas

https://hal.univ-lorraine.fr/hal-02502125
Contributor : Hoai An Le Thi <>
Submitted on : Monday, March 9, 2020 - 12:16:23 AM
Last modification on : Wednesday, May 6, 2020 - 2:34:53 PM

Identifiers

Citation

Le Thi Hoai An, Vinh Thanh Ho, Tao Pham Dinh. A unified DC programming framework and efficient DCA based approaches for large scale batch reinforcement learning. Journal of Global Optimization, Springer Verlag, 2019, 73 (2), pp.279-310. ⟨10.1007/s10898-018-0698-y⟩. ⟨hal-02502125⟩

Share

Metrics

Record views

73