Skip to Main content Skip to Navigation

Reconnaissance automatique des émotions par données multimodales : expressions faciales et des signaux physiologiques

Abstract : This thesis presents a generic method for automatic recognition of emotions from a bimodal system based on facial expressions and physiological signals. This data processing approach leads to better extraction of information and is more reliable than single modality. The proposed algorithm for facial expression recognition is based on the distance variation of facial muscles from the neutral state and on the classification by means of Support Vector Machines (SVM). And the emotion recognition from physiological signals is based on the classification of statistical parameters by the same classifier. In order to have a more reliable recognition system, we have combined the facial expressions and physiological signals. The direct combination of such information is not trivial giving the differences of characteristics (such as frequency, amplitude, variation, and dimensionality). To remedy this, we have merged the information at different levels of implementation. At feature-level fusion, we have tested the mutual information approach for selecting the most relevant and principal component analysis to reduce their dimensionality. For decision-level fusion we have implemented two methods; the first based on voting process and another based on dynamic Bayesian networks. The optimal results were obtained with the fusion of features based on Principal Component Analysis. These methods have been tested on a database developed in our laboratory from healthy subjects and inducing with IAPS pictures. A self-assessment step has been applied to all subjects in order to improve the annotation of images used for induction. The obtained results have shown good performance even in presence of variability among individuals and the emotional state variability for several days
Document type :
Complete list of metadata

Cited literature [219 references]  Display  Hide  Download
Contributor : Thèses UL Connect in order to contact the contributor
Submitted on : Thursday, March 29, 2018 - 11:53:10 AM
Last modification on : Monday, April 16, 2018 - 10:43:11 AM
Long-term archiving on: : Friday, September 14, 2018 - 1:40:04 AM


Files produced by the author(s)


  • HAL Id : tel-01748925, version 1



Faiza Abdat Khalfi. Reconnaissance automatique des émotions par données multimodales : expressions faciales et des signaux physiologiques. Autre. Université Paul Verlaine - Metz, 2010. Français. ⟨NNT : 2010METZ035S⟩. ⟨tel-01748925⟩



Record views


Files downloads